1. Executive Summary

The purpose of this document is to define architecture and design principles of the system to fulfill requirements (for example as defined in COMSODE Deliverable 2.1 ‘User requirements for the publication platform from target organizations, including the map of typical environments’).

The document describes architecture of a software solution for open data publication (Open Data Node) which was developed by COMSODE project and is basis also for methodologies for Open Data publication (which will also developed by COMSODE project). Architecture was developed based on identified user requirements (Use cases).

The system is split into modules that communicates using interfaces. Modules and interfaces are described in more details in particular sections. For each module it is described what is the role of the module and how it fits into context of overall system, including also description of its interactions with other modules. Some modules are describes also with alternatives that can be used to fulfill the functionality of the module.

Due to modularity of the system, it is possible to use the system in different deployment environments and also to use only some modules of the system and integrate with other modules and systems. Some deployment schemes are also described in the document.

Modules are licensed as Open Source, thus external contributors (essentially whoever) can improve the modules or extend the system. To facilitate that, also description of development model and environment is included  in the document.

This architecture was a base for initial implementation work of the COMSODE project. However, it may happen that some new component will be identified in the future, component which will better fulfill functionality of particular module. In that case, we will update the module description in this document accordingly, including explanation of the replacement.

2. Document context

2.1. Purpose of document

The research and development project COMSODE had as one of the main objectives within its 24 months of duration to create a publication platform called Open Data Node (ODN) that builds on results of previous research and development in the linked data field. Its mission was to bring results from research environment into real-world for people, SMEs and other organizations to use and reuse.

ODN is a foundation for a data integration platform based on Open Data which allows the reuse of data not only between public bodies and end-users but also among public bodies themselves: public bodies can exchange information by using the same infrastructure and tools as end-users which can decrease costs of exchanging the data and in most cases also enhance the quality and speed of the exchange.

This document represents the output of Task 2.2 - Architecture and design of the publication platform (ODN) of the COMSODE project and is maintained since then to reflect changes.

The purpose of the document is to provide overview of the architecture of Open Data Node, its modules, dependencies between modules and main communication interfaces.

For particular modules you can find description of their functionality and also overview of design of the modules.

Finally, it also describes possible deployment environments where ODN can be installed.

2.2. Related Documents

• COMSODE DOW, version date 2013-08-08, pages 7-8 and 42-43

• COMSODE Deliverable 2.1: User requirements for the publication platform from target organizations, including the map of typical environments

3. Methodology used

3.1. Methodology

The architecture of the system and design were created in an iterative way – from outlines in the OpenData.sk Wiki, through COMSODE (DoW, the meetings and discussions within consortium and with User Board, to Deliverable D2.3) to current form described in this document. We took into account also previous development of components of the system, as some of them existed before the COMSODE project started or evolved after COMSODE project ended.

The COMSODE project used internal collaborative space - called wiki - based on Atlassian Confluence technology with access of all members of the project team. There was established a dedicated space in the internal wiki for collection of inputs to the architecture and design from consortium members and from members of User Board. Also input (obtained by means other than wiki) from possible users of the platform was included in use cases. As part of final COMSODE outputs, ODN documentation was copied into this public Wiki hosted by OpenData.sk community.

ODN platform is planed to reuse, integrate and extend Open Source components, and architecture is based on the current status of those components and also on what additional features are possible to be included in those components.

It is expected, that the document will be updated when significant design changes are required due to changes in existing components or when a new component that better fits the project needs becomes available and can replace the currently selected component.

3.2. Partner contributions

The architecture and design document was prepared under management of COMSODE consortium member - EEA Ltd. Tomas Knap and Peter Hanečák are architects of ODN and UnifiedViews and DSE CUNI provided main input to Use cases. Project coordinator UNIMIB, DSE CUNI and ADDSEN have reviewed the pre-final version of the document.

After end of COMSODE project, EEA is helping to maintain this documentation.

ODN performs extraction and transformation (conversion, cleansing, anonymization, linking etc.) of data provided by governments and municipalities. ODN stores the result of data transformations in its own storage. ODN publishes the results of data transformations in open and machine readable formats to data consumers (citizens, companies, governments).

4.1. Actors

Main actors dealing with the system

• Data publishers
  government, municipalities, and other subjects providing data

• Data consumers
  government, municipalities, non-profit organizations (NGOs), citizens (general public), companies (SMEs), application developers consuming the transformed data

• Data administrators
  administrators, analysts, data curators, etc. (partially) responsible for configuring ODN - usually employees of data publisher

• Administrators
  IT staff responsible for installation, maintenance and (partially) configuration of ODN - usually employees of data publisher

ODN helps publishers with the complexity of source data and their transformations to open data and deliver easy-to-use and high quality open data to the data consumers.

ODN helps data consumers get the data easily and efficiently in open, machine readable formats.

4.2. Inputs

Input to the system is data of data publishers stored in heterogeneous environments, using wide variety of formats and employing a lot of different technologies to access and process that data

4.3. Outputs

Output of the system are published open data in various forms, as linked data or as tabular data. Also API access to the data is included. Data consumer may be provided with:

• RDF data as a result of export from the storage

• RDF data as a result of SPARQL Query

• CSV data as a result of export from the storage

• REST API to access data in the storage

All forms of the data shall be available for data consumers under an open license. For more details about the formats of published open data, please see Section 4.3.

4.4. Features for data publishers

• automated and repeatable data harvesting: extraction and transformation (conversion, cleansing, anonymization, etc.) of data, both:

• initial harvesting of whole datasets (first import)

• periodical harvesting of incremental updates

• integration tools for extracting data from publisher’s internal systems (e.g. databases, data in files, information systems with API, etc.)

• internal storage for data and metadata; the metadata format will be based on DCAT (http://www.w3.org/TR/vocab-dcat/)

• data publishing in open and machine readable formats to the general public and businesses including automated efficient distribution of updated data and metadata (dataset replication)

• integration with data catalogs (like CKAN) for automated publication and updating of dataset metadata

• internal data catalog of datasets for maintenance of dataset metadata

4.5. Features for data consumers

Features for data consumers are discussed separately for different types of data consumers.

4.5.1. Citizen, data analyst, etc.

• user is typically accessing ODN instance maintained by someone else, user it not running his own instance

• user may download data dumps and call APIs to get the data which he is interested in

• data dumps changes are advertised as Atom feeds

• user may access the data indirectly, for example via 3rd party data catalog, which - in order to show the user preview or visualization of data - has to first download that data (in a similar manner as if user was accessing it: i.e. downloading a dump or accessing an API from ODN instance maintained by someone else)

4.5.2. Aggregator of Open Data (public body, NGO, SME, etc.)

• the same as in Section 3.5.1

• aggregator may easily replicate content in another Open Data Node

• aggregator may automate data integration and linking

4.5.3. Application developer using Open Data (SME, NGO, etc., public body too)

• the same as in Section 3.5.2

• application developer has tools for preparing automated generation of API and custom API  of the published datasets

4.5.4. Data administrator

• possibility to set up the data extraction, transformation, and publication of open data

• possibility to monitor execution of data extraction and transformatio tasks

• possibility to debug data extraction and transformation

4.6. Use Cases

In this deliverable, we depict use cases for data publishers and data consumers. Full list of use cases, also with related scenarios and mock-ups can be found at  https://team.eea.sk/wiki/display/COMSODE/Use+Cases (note: This is the internal consortium wiki space mentioned in section “2.1. Methodology” - it is subject to change based on subsequent user requirements and will be most probably moved to public space once community around ODN grows).

For each use case below, we introduce its name, short description and modules in ODN participating on this use case (ODN/M = ODN/Management module, ODN/UV = ODN/UnifiedViews, ODN/P = ODN/Publication module, ODN/IC = ODN/InternalCatalog module, ODN/C = ODN/PublicCatalog module). For full details, please see https://team.eea.sk/wiki/display/COMSODE/Use+Cases .

4.6.1. Use Cases for Data Publisher

4.6.2. Use Cases for Data Consumers

4.7. Other inputs for Architecture decisions

This is additional list of inputs extending Deliverable 2.1: ‘User requirements for the publication platform from target organizations, including the map of typical environments’.

• The system must be extensible with new DPU on the data transformation (ETL) pipelines.

• The overhead of managing data transformation (ETL) tasks and managing outputs from/preparing inputs to DPUs must be reasonable - any execution of ETL task must not take more than 200% of the time needed for manual execution of the DPUs’ logic without the use of the ETL framework (under the condition than the pipeline is running alone)

• The system must be able to process big files - CSV files containing millions of rows, RDF files containing hundreds of millions of triples.

• Response time: The Web management GUI of ODN must respond in 99.9% cases in less than 1s (all components of ODN on one machine, client is connecting to server using 100Mb line at least).

• Target platform: Linux, Windows

• Preferred languages for the system implementation: Java, others only in case of reuse of existing components with sufficient added-value (for ODN and for ODN users)

• The internal format for all data being transformed during the ETL process is RDF, an universal machine readable data format.

5. Open Data Node Modules

Open Data Node consists of the following modules:

• ODN/UnifiedViews

• ODN/Storage

• ODN/Publication

• ODN/InternalCatalog

• ODN/PublicCatalog

• ODN/Management
  

Modules listed above are discussed in more detail in the following sections.

6. Development

The main target development platform is Java. ODN/Publication may use PHP tool for generating APIs. Similarly, ODN/InternalCatalog and ODN/PublicCatalog use already existing Python based tool CKAN to ensure the data catalog functionality.

ODN must be able to run on both Linux and Windows operating systems.

6.1. Development process

Source codes related to ODN are kept in github under github project https://github.com/OpenDataNode. Specific repositories are created for particular module.

• ODN/InternalCatalog for ODN specific extensions: https://github.com/OpenDataNode/InternalCatalog
• UnifiedViews plugins specific for ODN: https://github.com/OpenDataNode/UV-Plugins 

Certain modules are developed under a separated github projects and are reused in ODN. Any changes to that modules are requested using issues in particular project or as a pull request to proper repository on github.

• ODN/UnifiedViews https://github.com/UnifiedViews
• ODN/InternalCatalog and ODN/PublicCatalog core part and extensions: https://github.com/ckan

In exceptional case we can fork original repository (mainly in case when original repository doesn't accept pull request), but main goal will be to merge changes back later.

For the whole ODN, we will follow the guidelines for sustainable development proposed for developing ODN/UnifiedViews (https://grips.semantic-web.at/display/UDDOC/Guidelines+for+Contributors).

6.2. Used technologies

There are two main technological stacks in the ODN, derived from re-use of UnifiedViews and CKAN in the project:

1. Java based stack - for ODN/UnifiedViews, ODN/Management, etc.
   
2. Python based stack - for ODN/InternalCatalog and ODN/PublicCatalog
   

6.2.1. Java stack

Java based technologies are used for majority of ODN modules. Each module has its own sub-set of technologies, frameworks, libraries and tools. Some of those are mentioned here, but for more concrete and up-to-date information, please refer to homepages and documentation of upstream projects (TODO: add Wiki page listing all main upstram projects and link here).

6.2.1.1. Spring

Spring is an open source application framework and inversion of control container for Java platform. The core features of the Spring Framework can be used by any Java application, but there are extensions for building web applications on top of the Java EE (enterprise) platform. Spring will be used in all ODN modules implemented in Java.

6.2.1.2. Vaadin

Java open source framework for building web applications. This framework incorporates event-driven programming and widgets, which enables a programming model that is closer to GUI software development than traditional web development with HTML and JavaScript. Developers may write pure Java code and the framework ensures the communication between server side code (application server) and client side code (browser), also Java code is automatically translated to JavaScript for the client code. Vaadin will be used in ODN/UnifiedViews, ODN/Management, ODN/Publication for implementing Management GUI. ODN/InternalCatalog, ODN/PublicCatalog will be based on existing tool - DKAN - and its interface.

6.2.1.3. Sesame openRDF

Sesame openRDF is an open source framework for processing RDF data. This includes parsing, storing, inferencing and querying of/over such data. It offers an easy-to-use API that can be connected to all leading RDF storage solutions. It allows to connect to SPARQL endpoints and create applications that leverage the power of linked data and Semantic Web. OpenRDF is used in ODN/UnifiedViews, ODN/Storage, ODN/Publication to work with RDF database.

6.2.1.4. EclipseLink

EclipseLink is the open source Eclipse Persistence Services Project. The software provides an extensible framework that allows Java developers to interact with various data services, including databases, web services, Object XML mapping (OXM), and Enterprise Information Systems (EIS). EclipseLink is used to persist data objects in ODN/UnifiedViews.

6.2.1.5. XStream

XStream is a simple library to serialize objects to XML and back. XStream uses reflection to discover the structure of the object graph to serialize at run time and does not require modifications to objects. It can serialize internal fields, including private and final, and supports non-public and inner classes. XStream is used in ODN/UnifiedViews to store pipeline configurations.

6.2.1.6. KineticJS

KineticJS is an HTML5 Canvas JavaScript framework that enables high performance animations, transitions, node nesting, layering, filtering, caching, event handling for desktop and mobile applications.

You can draw things onto the stage, add event listeners to them, move them, scale them, and rotate them independently from other shapes to support high performance animations, even if your application uses thousands of shapes.

6.2.1.7. OSGI framework

ODN/UnifiedViews must support easy and smooth extension with custom DPUs added into the running application. Every DPUs may use its set of libraries. These libraries must not be in conflict. To ensure that, we use OSGI framework, in particular the Apache Felix implementation (http://felix.apache.org/).

6.2.2. Python stack

TODO: Add some more details about Python itself (version supported, etc.) and frameworks used in CKAN, etc.

6.3. Development tools

6.3.1. Maven

Maven (http://maven.apache.org/) is used for management of dependencies in the source code, portability between Java IDEs and easy application build.

6.3.2. Git + GitFlow

Git (http://git-scm.com/) is a version control system used for source code version control management and tracking changes.

GitFlow (https://github.com/nvie/gitflow) is a collection of Git extensions to provide high-level repository operations for Vincent Driessen's branching model (http://nvie.com/git-model).  Nice overview of whole worklow is available at http://danielkummer.github.io/git-flow-cheatsheet/

SourceTree (http://www.sourcetreeapp.com/) is recommended GUI client (available for MS Windows and MAC OSX) as it has good support for defined gitflow workflow. For linux, command line is necessary to use to follow gitflow workflow.

6.3.3. IDE for developers

Project is not strictly bounded to specific IDE, but Eclipse (http://www.eclipse.org/), Netbeans (https://netbeans.org/) or InteliJ Idea (http://www.jetbrains.com/idea/) are recommended IDEs for developing Java applications.

Eclipse and Netbeans have also plugins for development of other type of applications (for example php/Drupal based).

7. Deployment

Firstly, we describe basic deployment scenarios of Open Data Node, considering Open Data Node as a single unit. Afterwards, we discuss deployment of ODN’s modules.

7.1. Basic ODN Deployment Scenarios

Open Data Node can be deployed many times by many actors. ODN can help with needs specific to each particular actor:

• government organizations, municipalities, etc., want to publish majority of their information as Open Data

• other government bodies need to work with some data published by other government bodies

• non-profits and application developers want to run specific tasks using copies of official data, for example analytic and visualization applications, data integration, etc.

This section contains schemes for basic deployment options for ODN. Options are sorted based on achievable publishing quality from best (and most expensive) at the top to worst (and cheapest) at the bottom.

7.1.1. Tight integration, at the publisher's premises

Open Data Node is tightly integrated with publisher's internal application(s) - it has direct access to backend databases or is integrated into application(s) workflows via API. ODN is deployed alongside publisher's internal application(s), it has to respect network, security and other zones.

Typical scenario: 

• publisher wants to achieve high quality and efficiency and is willing to invest more

Prerequisites:

• publisher is willing to update existing workflows and applications

7.1.2. Loose integration

Open Data Node is integrated with publisher's application(s) in a loose way using some periodical data dumps or APIs. ODN can be deployed in several locations:

• at publisher's premises - access to the data is secured in a similar way as in the case of tight integration in Section 6.1.1.

• at collocated housing, for example data center shared by multiple government organizations - access to the source data secured for example using combination of IPsec, HTTPS and access controls (authentication and authorization)

• in the cloud - access to the source data secured with just HTTPS and access control (not suitable for sensitive data or sensitive internal systems)

Typical scenario:

• publisher wants to achieve high quality and efficiency but has limited resources so changes to existing infrastructure and applications have to be limited

• aggregator is affiliated with one or more publishers and is willing and able to invest into tighter integration with them (ministry aggregating data from municipalities or SME planning to make business using aggregated data)

Prerequisites:

• publisher is able to do minor modifications to existing workflows and applications

7.1.3. No integration, deployed at 3rd parties

Open Data Node is not explicitly integrated with publisher's applications, other existing means are used to get access to the data (either Open Data or other format or API, at worst case scrapping of data from website). ODN is deployed at 3rd party using their own hardware, collocated housing, or Cloud.

Typical scenario:

• publisher wants to publish Open Data but has severely limited resources or options so changes to existing infrastructure and applications have to be preferably none

• 3rd party aggregator or application developer wants to use data from one or more publishers but for some reason is not able or willing to implement tighter integration with them

Prerequisites:

• some usable form of access to source data is possible without changing existing workflows and applications

7.2. Deployment of ODN Modules

7.2.1. Single Machine

ODN modules may be deployed all on one machine as depicted below. The deployment requires:

• Application Server, which supports Java 7 EE applications. ODN will be tested on Apache Tomcat 7+. Application server is needed for management GUIs of ODN modules.

• Relational database management system (RDBMS) - Relational database is used mainly for storing configurations of the modules (definitions of data transformation and publishing tasks, configuration of data catalogs, etc.); in these cases, relational database is preferred, because the schema of the configurations is known in advance and should not change much in the future. Relational database is also used by ODN/Storage to store tabular data produced by ODN/UnifiedViews. Decision is still pending about the particular database management system we will use, however, due to object relational mapping frameworks, which abstract the underlying database system, changing the database system during the works on the project is easy and straightforward.

• RDF Storage - We may use any RDF store, which is supported by openRDF API (http://www.openrdf.org/), e.g., Openlink Virtuoso (http://virtuoso.openlinksw.com/rdf-quad-store/) or Sesame (http://www.openrdf.org/). RDF store is used by ODN/UnifiedViews to store intermediate results of the data transformations and also by ODN/Storage to store the RDF data produced by ODN/UnifiedViews.

• HTTP Server - ODN will be tested with Apache HTTP Server. HTTP Server is required for ODN/InternalCatalog, ODN/PublicCatalog and ODN/Publication.

7.2.2. Distributed Environment

ODN modules support distributed deployment to more physical devices. Basically every artifact depicted in Figure above may be deployed on a different device. If ODN/UnifiedViews - Engine and ODN/UnifiedViews - Management GUI is placed on different devices, administrator has to set up shared network file system both these artifacts may use.

Typically, we expect such deployment in large organizations with more complicated IT architecture driven by (among other things) security requirements. In this case:

• ODN/UnifiedViews is expected to be deployed withing internal, restricted segment along with ODN/Management,

• ODN/Publication is expected to be in DMZ segment accessible from the outside by general public (like a typical webserver) and

• ODN/Storage is expected to be in internal or other appropriate segment, reachable by both ODN/UnifiedViews and ODN/Publication

7.2.3. Custom Environment

As the design of ODN is modular (driven in particular by our vision, plans, engineering experience, etc., but in general also by best practises in software development and Open Source development) individual user will have possibility to not just move the individual modules between multiple machines, but also to skip (do not deploy) certain modules, if they do not need them.

For example, ODN/InternalCatalog and ODN/PublicCatalog may be skipped, in exchange for direct integration between ODN/UnifiedViews and ODN/Publication with the particular national data catalog.

Note: Given proper modular design and implementation, exact options and combinations are wide and depend strongly on particular user and his needs and use cases, so we are not able to (and will not) document all possible options here. Some most common cases will be later explained in ODN documentation and COMSODE Methodology.

8. Licensing

Open Data Node as a whole is Free and Open Source software (see https://en.wikipedia.org/wiki/Free_and_open-source_software).

As it re-uses many existing free and open source components, it is not governed by one single license. Majority of components are covered by three basic families of licenses: GPL, APL and BSD. More detailed licensing information about individual ODN modules follows:

8.1 ODN/UnifiedViews

UnifiedViews is licensed under the combination of GPLv3 and LGPLv3. For more information, please see https://github.com/UnifiedViews/Core#licenses .

8.2. ODN/Storage

In this module, following components are used:

• OpenRDF Sesame, which is licensed under BSD-style License but is reusing also other components with separate licenses, see https://bitbucket.org/openrdf/sesame/src/master/core/LICENSE.txt and https://bitbucket.org/openrdf/sesame/src/master/core/NOTICE.txt
• PostgreSQL, licensed under its own Open Source license similar to BSD and MIT licenses (see https://wiki.postgresql.org/wiki/FAQ#What_is_the_license_of_PostgreSQL.3F)
  
• Virtuoso Open Source, licensed under GPLv2, for more information please take a look at http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VOSLicense
  

8.3 ODN/InternalCatalog and ODN/PublicCatalog

CKAN (reused in modules ODN/InternalCatalog and ODN/PublicCatalog) is licensed under AGPLv3 (see https://github.com/ckan/ckan#copying-and-license).

8.4 ODN/Management

In this module, the principal components are:

• midPoint, which is licensed under APLv2 (see https://github.com/Evolveum/midpoint/blob/master/LICENSE)
• CAS, licensed under APLv2 (see https://www.apereo.org/content/cas-under-apache-20-license)
  

8.5 ODN/Publication

This module is primarily built on portion of Virtuoso Open Source (SPARQL endpoint and its GUI) - mentioned above in section "ODN/Storage" - and LDVMi.

LDVMi is licensed under under APLv2 (see https://www.apereo.org/content/cas-under-apache-20-license).

8.5 Other modules and components

Each module can be broken down further into even smaller components and libraries. Each such smaller part has the same or other (but compatible) Open Source license.

For example, UnifiedViews uses following components and libraries:

As it is hard to maintain complete component and licensing list down to the lowest level, thus we're maintaining only information about components directly utilized in ODN. Please consult documentation of each such major component (mentioned in previous sections) to obtain its more detailed information about its sub-components and licensing information.

9. Abbreviations

10. Glossary

Atom feed - a published list (or "feed") of recent articles or content in a standardized, machine readable format that can be downloaded by programs that use it (http://en.wikipedia.org/wiki/Atom_(standard)) . It is an alternative to RSS.

Data catalog - a database where are stored metadata about datasets.

Data catalog vocabulary (DCAT) - an RDF vocabulary designed to facilitate interoperability between data catalogs published on the Web.

Data mart - access layer of the data warehouse environment that is used to get data out to the users (http://en.wikipedia.org/wiki/Data_mart). In ODN it is the database, which is used by data catalogs to get dumps from, by ODN/Publication to provide REST APIgenerate data dumps, provide data as results of API calls.

Internal data mart - the database where the results of the ODN/UnifiedViews is stored when the pipeline finishes. see also Data mart

Data Processing Unit - see DPU

Dataset - one or more related input files (http://en.wikipedia.org/wiki/Data_set). In ODN, dataset is processed as input and also ODN produces datasets as output.

Dataset record - record about the dataset containing metadata of the dataset, associated publication pipelines and data catalog resources

Data catalog resource - the particular file (dump), REST API, or SPARQL Endpoint, visualization etc. associated with the given dataset record. 

DCAT Distribution - Basic classes in DCAT are "Catalog", "Dataset" and "Distribution". Distribution is equal to catalog resource. 

Data Publication Pipeline - oriented acyclic graph, where nodes represent DPU instances, and oriented edges represent data flow between these instances. Every pipeline consists of one or more DPUs. ODN/UnifiedViews supports exchange of RDF data between DPUs.

Data Publication Task - see Data Publication Pipeline

Data unit - an input to DPU or output from DPU is called data unit (DU). Every DPU may have more input data units and more output data units. DPUs may also use different types of DU, e.g., RDF data unit being able to read/write RDF data or File data unit being able to read/write generic files.

DPU - plugin on the data processing pipelines, which executes certain transformation, cleansing, quality assessment on top of the processed data. DPU encapsulates certain business logic needed when processing data (e.g., one DPU may extract data from a SPARQL endpoint or apply a SPARQL query). Every DPU must define its required/optional inputs and produced outputs.

DPU instance - placement of DPU on a pipeline. DPU instance is therefore created when DPU is placed on pipeline canvas.

DPU configuration - associative array of key-value pairs, which customize functionality of DPU instance.

External data catalog - for ODN it is data catalog that is not part of ODN but ODN can be integrated with it. see also Data catalog

Instance Configuration - configuration for specific DPU instance. It is created at the time of placing DPU on the pipeline (canvas) as a copy of Template Configuration

Internal Data catalog - Data catalog that is part of ODN and is visible to public. In ODN is also referenced as Data catalog.

Public Data catalog - Data catalog that is part of ODN and is visible to public. In ODN is also referenced as Data catalog.

RDF - standard model for data interchange on the Web. RDF has features that facilitate data merging even if the underlying schemas differ, and it specifically supports the evolution of schemas over time without requiring all the data consumers to be changed. (http://en.wikipedia.org/wiki/Resource_Description_Framework, http://www.w3.org/RDF/)

Resource Description Framework - see RDF

Representational state transfer - (REST) software architectural style consisting of a coordinated set of architectural constraints applied to components, connectors, and data elements, within a distributed hypermedia system and also applied to the development of web services. (https://en.wikipedia.org/wiki/Representational_State_Transfer)

SPARQL - (SPARQL Query Language for RDF) query language for databases, able to retrieve and manipulate data stored in RDF format. SPARQL allows for a query to consist of triple patterns, conjunctions, disjunctions, and optional patterns. (http://en.wikipedia.org/wiki/SPARQL, http://www.w3.org/2001/sw/wiki/SPARQL)

Staging database - the database used by ODN/UnifiedViews to store intermediate results of the DPUs on the pipeline

Template Configuration - previously called "default configuration", default DPU configuration associated with each DPU.

Transformation Pipeline - see Data Transformation Pipeline

11. References

1. http://www.comsode.eu/  main page of project COMSODE

2. https://github.com/UnifiedViews - source codes of ODN/UnifiedViews module

3. http://ckan.org/ - data catalog CKAN home page

4. http://nucivic.com/dkan/ - open data platform for cataloging home page

5. http://www.w3.org/2001/sw/DataAccess/tests/implementations - comparison of different implementations of SPARQL endpoint

6. http://www.openrdf.org/ - homepage of openRDF Sezame framework for SPARQL 

7. http://theodi.org/blog/git-data-publishing - ideas how to use GitHub for data publishing