Skip to main content

Pipelines

Pipeline is function to process data. It can be used to load, transform, enrich, filter, and route data. Pipelines are the core of the data processing in the system.

Create Pipeline​

1) Open Integration Detail Page 2) Open "Pipelines" tab and click on the "Create Pipeline" button create_pipeline_button 3) In the opened form, fill in the required fields:

  • Name: the name of the pipeline
  • Connector: select the connector that will be used in the pipeline. After selecting the connector, the list of fields that must be filled in will be displayed.
  • Connector fields: different for each connector. Fill in the required fields. Green "Create" buttons allow you to create new entities (e.g. view, logs entity, etc.) directly from the pipeline form. 4) Click on the "Create Pipeline" button

Pipeline List​

Pipelines are displayed on the "Pipelines" tab of the Integration Detail Page. Users can quickly see the name, related entities, last run, status, and actions for each pipeline. pipeline_list

  • Filter pipelines by name using the search bar.
  • Pagination to navigate through the list pages.
  • Click on a pipeline status to open logs for pipeline last run. If the pipeline failed, the logs will contain the error message.

Actions​

  • Edit: Modify the pipeline details.
  • Delete: Remove the pipeline from the system.