Entities

"Entity" represents a structural unit designed to organize and store persistent data. Analogous to an index in Elasticsearch, or a collection in MongoDB. You can think of an entity as a table in a relational database, or a document in a NoSQL database.

Key Characteristics

• JSON data: Entities are JSON objects.
• Queryable: Entities can be queried using a mongo-like query language.
• Data Manipulation: Entities allow for standard CRUD (Create, Read, Update, Delete) operations on data records.
• Indexing: For improved query performance, data can be indexed on one or multiple fields.
• Views: Users can create data views that filter and display records according to specified criteria.
• Logging: Entities can be used to store logs from integration pipelines or other processes.

Use Cases

Entities can be used for various purposes such as:

• Data Storage: Store and manage data records from external systems
• Logging Data: Store logs from integration pipelines or other processes
• Materialized Views: Persistently store the results of complex Views

Entity List

The Entity List page displays all entities that have been created. Users can quickly see the entity name and type.

From Entity List page user can perform the following actions on each entity:

• Open List of Entity Records by clicking on the entity name
• Create new Entity
• See the number of Views related to the Entity in the "Views" button
• Open List of Entity Views by clicking on the "Views" button
• Open Dropdown list with all available Views by clicking on the arrow in the "Views" button
• Edit Entity by clicking on the "Edit" button
• Delete Entity by clicking on the "Delete" button

For navigation purposes, users can:

• Filter entities by name using the search bar
• Use pagination to navigate through the list pages

Create Entity

To create an entity we must specify the following:

• Name: Name of the entity. String from 3 to 64 characters
• Type: We must select Regular or Logs entity type.
  • Regular entities are used to store data records
  • Logs entities are used to store logs from integration pipelines or other processes. It has additional indexes and fields for logging purposes.

Date Settings

Date Settings is optional settings for date fields. If the entity data contains date fields, you must specify the source field, target field, source type and source format. BuiltAPI will automatically convert the date fields to BuiltAPI date format.

• source path: Path to the original date field.
• target path: Path to the converted field in the entity record
• type: Type of date field:
  • ISO: ISO8601 format. Example: 2023-01-01T00:00:00Z
  • Seconds: UNIX timestamp in seconds. Example: 1609459200
  • Milliseconds: UNIX timestamp in milliseconds. Example: 1609459200000
  • Custom: If the date format is not one of the above, you can specify a custom format. The format must be a valid JS time format string.

Example:

#original record
{
    "name": "John Doe",
    "age": 30,
    "email": "[email protected]",
    "dob": "2021-01-01T00:00:00Z"
}

In the above example, the "dob" field is in ISO8601 format. If we want to use dob field in query we must specify the date settings as follows:

In this example we create new entity "user" with date settings for "dob" field. We specify that the source field is "dob", the target field is "dob_iso" and the type is "ISO". Each time a record is created or updated, BuiltAPI will automatically convert the "dob" field from ISO8601 format to "dob_iso" field in BuiltAPI date format.

You can set "source path" and "target path" to the same value if you want to overwrite the original field.

Entity records

An entity record is a single JSON object that represents a data record in an Entity. See details in the Records section.

Entity types

Regular

Regular Entities are used to store data records. They can be used to store data from external systems, or to store data generated by integration pipelines. Entities can be used to materialize the results of Views.

Logs

Logs Entities are used to store logs from integration pipelines or other processes. Logs Entities have additional indexes and fields for logging purposes. Logs Entities can be used to store logs from integration pipelines, or to store logs from other processes.

Logs records creates automatically when you create an Integration, or can be created manually and set as logs entity in "Create Integration" form.

Logs records example

{
  "level": "error",
  "timestamp": "2024-07-04T20:56:52.502Z",
  "workspaceId": "workspace-1234",
  "integrationId": "integration-5678",
  "pipelineId": "pipeline-91011",
  "integrationName": "Integration Name",
  "pipelineName": "Pipeline Name",
  "pipelineRunId": "run-1213",
  "message": "Request failed with status code 400",
  "details": {
    "id": "asset-1415",
    "error": {
      "name": "ConnectorError",
      "type": "RESPONSE",
      "details": {
        "data": {
          "errorMessages": "'type_of_use'"
        },
        "status": 400
      },
      "params": {
        "method": "POST",
        "baseUrl": "https://api.example.com",
        "path": "/services/endpoint/",
        "body": {
          "data": [
            {
              "asset": {
                "id": "asset-1415",
                "type_of_use": null
              },
              "settings": {
                "normilise_consumption": "Yes"
              }
            }
          ]
        },
        "allowStatuses": [
          202
        ]
      },
      "message": "Request failed with status code 400"
    }
  }
}

Logs records always have the following fields:

• level: Log level. Can be "info", "warning", "error", "debug"
• timestamp: Timestamp when the log was created
• workspaceId: Workspace ID
• integrationId: Integration ID
• pipelineId: Pipeline ID
• integrationName: Integration Name
• pipelineName: Pipeline Name
• pipelineRunId: Pipeline Run ID
• message: Log message
• details: Additional details about the log