XCRI-CAPAggregator API & Tooling Overview

Ian IbbotsonKnowledge Integration Ltd

This work is licensed under a Creative Commons Attribution 3.0 Unported License.

XCRI-CAP Aggregator
What is it?

End to end way of collecting course data from any provider into a unified repository

Scheduled pull of XCRI-CAP documents

Checksums and date stamps

Indexing and update of changes in core aggregation database (Mongo, Elasticsearch, triplestore)

Optional API's for push submission

Api's for third party embedding / usage

Integrated with XCRI Validator web service

Project Status

Iteration 2 completed Core development and data ingest

Iteration 3 Work on extending APIs and adding access points

Iteration 4 Implementing Usability and other Feedback

Overall Architecture

1. Feed Manager

http://coursedata.k-int.com/FeedManager

Simple HTML Client for owners of XCRI-CAP documents who want them published

Upload / Schedule Collection (Pull)

Force re-harvest

View documents

Approve / Publish to discover app

Feed Status Page

Submission Console

Search Harvested Feed

Dashboard / Approve (wip)

Backlog includes...- Analysis of facets (Subjects)- Warnings / Errors- Change Frequency- Interaction with Terminology

2. Discovery Interface

http://coursedata.k-int.com/discover

Search all published feeds

Facet by subject / provider / TBA

Simple interface for practitioners

Clean URLs

http://coursedata.k-int.com/discover/?q=Science

Content Type Negotiation (accept header, format=json|xml|atom|rss) OpenSearch.

Search Interface

Results

Faceting

Details

3. Discover Rest Interface

http://coursedata.k-int.com/discover

&format=[xml|json|rss|atom]

q parameter Standard lucene query syntax, boolean operators, any field or free text.

Additional parameters location= gazetteer based searching in r3 (next week)

discover?q=science&format=rss

4. Direct Access
ElasticSearch / Triple Store

ES search now, ask for port opening

TripleStore R4 (3 weeks)

Direct ElasticSearch

http://coursedata.k-int.com:9200

/courses/course/_search?q=title:science

SOLR api in release 4 via ES Plugin

Remember no access to gazetteer or controlled term lookup via direct ES search. Facets will be codes (Free text atm) and not terms.

Some GUIs / other minted identifiers

Provider URI in standards, seldom in practice, go via Co-Reference Service to harmonise

ElasticSearch on :9200

http://coursedata.k-int.com:9200/courses/course/_searchq=title:science&pretty=true

5. Submission Interface

For institutions wanting to directly push XCRI documents into the aggregator

http://coursedata.k-int.com/repository/upload

BASIC Auth protected / https

Content Type Negotiation

Automated Submission

Automated Submission

http://coursedata.k-int.com/repository/uploadRecognises upload.json/upload.xml and Accept headers for content type negotiation. Err and warn response messages for validation problems.

Custom Handlers

Dynamically Discovered

Scriptable (groovy) Handlers::

getHandlerName, getRevision, getPreconditions, setup

Handlers are selected by precondition, with most specific being selected.

def getPreconditions() { [ 'p.rootElementNamespace=="http://xcri.org/profiles/catalog"' ] }

Add && provider==MyProviderId for more specific variants

REST API Summary

http://coursedata.k-int.com

/FeedManager

/discover?q=x&format=[atom|rss|json|xml]

/discover/course/[&format=xml|json]

Toolkits

Release 4 Code Client Libraries

Python, Java, Php, .net

Sourcecode

FeedManager, Discover App

https://github.com/k-int/XCRI-Aggregator

Core Aggregator Platform

Sourcecode

FeedManager, Discover App

https://github.com/k-int/XCRI-Aggregator

Core Aggregator Platformgit@github.com:k-int/AggregatorCore.git

Questions

Muokkaa otsikon tekstimuotoa napsauttamalla

Muokkaa jsennyksen tekstimuotoa napsauttamallaToinen jsennystasoKolmas jsennystasoNeljs jsennystasoViides jsennystasoKuudes jsennystasoSeitsems jsennystasoKahdeksas jsennystasoYhdekss jsennystaso