Records
Items in an Entity or View are displayed in the Records table. When you select a specific Entity or View, you are first presented with this table, which shows all items contained within it. You can view, edit, delete, and create new records.
Both Entity Records and View Records use the same query building mechanism, allowing you to filter, sort, and project data.
Key Characteristics
- JSON data: Records are JSON objects.
- Queryable: Records can be queried using a mongo-like query language.
- Data Manipulation: Records allow for standard CRUD (Create, Read, Update, Delete) operations on data records.
- Indexing: For improved query performance, data can be indexed on one or multiple fields.
- Views: Users can create data views that filter and display records according to specified criteria.
Record structure
Record is a single JSON object that represents a data record in an Entity or View. Each record has a unique identifier, which is automatically generated by the system, meta, data and pipelines fields.
{
"id": "5f7b1b3b-1b1b-4b1b-9b1b-1b1b1b1b1b1b",
"meta": {
"created_at": "2021-01-01T00:00:00Z",
"updated_at": "2021-01-01T00:00:00Z"
},
"data": {
"name": "John Doe",
"age": 30,
"email": "[email protected]"
},
"pipelines": {
"pipeline_id1": {
"processedAt": "2021-01-01T00:00:00Z"
}
}
}
Meta
- created_at: Timestamp when the record was created
- updated_at: Timestamp when the record was last updated
Data
"data" field contains the actual data of the record. When integration pipelines are used to populate entities or you create entities manually, the data field will contain the data you want to store.
Pipelines
Pipelines field contains information about the pipelines that processed the record. Each pipeline has a unique identifier and a processedAt field that contains the timestamp when the pipeline processed the record. We use this field to understand which records were processed by a specific pipeline and when. If record's processedAt value less than updated_at value, it means that this record is updated and must be reprocessed by the pipeline.
Add Record
To add record from console click on "New Record" button.
You will see a form where you can enter JSON object with data for new record.
View/Update Record
To view or update record click on the "Open" button in record row. You will see a form with record data. You can edit data and save changes by clicking "Save" button.
Delete records
To delete records you must select records by clicking on the checkbox in the first column of the table. After that click on the "Delete" button. You will see a confirmation dialog. Click "Delete" to confirm deletion.
You can select all records by clicking on the checkbox in the table header.
Records filtering
To filter records we use MongoDB-like query language. You can filter, project and sort records using this language. Detail information about query editor available on Query Editor Doc Page.
Select columns
You can visualy select columns that are displayed in the table. Click on the "Columns" button to open columns manager.
Columns manager window will be opened.
- Available fields (Left panel): This panel lists all the fields available in the records.
- Selected columns (Central panel): This panel lists the fields that will be displayed in the table.
- You can drag and drop fields from the Available fields list to the Selected columns list.
- You can rearrange the order of the selected columns by clicking and dragging the icon with six dots (⋮⋮) next to each field name.
- Field Removal: Users can remove a column from the selection by clicking the "×" icon next to the respective field.
- "Select All" button: This button selects all the fields available in the records. "Select All" button available only for records with less than 100 columns. If you have more than 100 columns, you must select columns manually.
- "Clear" button: This button clears all the selected columns.
- Result preview (Right panel): This panel displays a preview of the records table with the selected columns.
To apply changes click on the "Save and close" button.
Manage records table
- You can change size of each column by dragging the column border.
- You can change sorting by clicking in sort button in column header
- You can delete column from table by clicking on the cross button in column header
All changes will be saved in your browser local storage and will be restored when you open Records page next time.
Clear filter, columns and sorting
To clear filter, columns or sorting click on the "Trash" button near required button.
View from Query
You can save query as a View.
To do this click on the "Create view from query" button. You will see a form where you can enter View name and update pipeline. Click "Create view" to create new View based on your Query.