Who can do this (User role)

Data Publisher

Description

I want to create a pipeline from scratch in ODN/UV canvas, so that it can be associated with a dataset in ODN/IC.

Preconditions

  • ODN is installed

How to do it:

  • DataProcessingUnits (building blocks of a pipeline) are available on the left side in the tree structure
  • the DPUs are orgaqnized in E, T, L subtrees in accordance with the purpose it serves in the transformation process
    • E - extractors retrieve source data from different sources (remote/local file storage, relational database, etc...)
    • T - transformers modify the structure and content of data input, transform data from one datatype to another, etc...)   
    • L - loaders load the transformed data to selected data storage, publish it in the ODN/IC as resource of a dataset 
  • use Drag & drop method to put a selected DPU from tree to canvas workspace
  • use doubleclick or hover over a DPU in workspace and select the gearwheel icon to open the configuration dialog of a DPU
    • each configuration dialog contains 'About' section which describes the DPU and provides help on how to configure 
  • hover over a DPU in workspace and select the arrow icon to create a conection between two DPUs
    • if the connection created is displayed in red color, it is either incorrectly configured or it is not possible to interconnect the selected DPUs at all
  • once the connection of two DPUs is created it can be configured by clicking the connection and selecting the gearwheel icon 
    • configuration dialog is opened where the output data unit of the source DPU can be mapped to input data unit of the target DPU
  • hover over a DPU in workspace and select the basket icon to delete the DPU from workspace (all connections from and to this DPU are deleted as well)
  • click the connection and select the basket icon to delete the selected connection between two DPUs
  • '+' '-' icons on the upper right corner of the screen for zooming the workspace in and out
  • the green arrow icon can be used for undo the last step performed
  • use 'Save' button to save the pipeline created 

 

  • before the pipeline is executed, it needs to be associated with a dataset in ODN/IC

Notes