For a list of all new, running, and completed ingestions navigate to Settings > Data Ingestion > Ingestions.
Ingestion details
The ingestion page offers the following information on individual ingestion jobs:
| Column | Description |
|---|---|
| ID | Unique ID of the ingestion. Click on it for a detailed log (see Ingestion log). |
| Data Source | Name of the data source for which the ingestion was running. |
| Connection | Connection used for ingestion. |
| Space | Target space for ingested data. |
| State | Current state of the ingestion. For a full list, see Ingestion states. |
| Started | When the Ingestion was started. |
| Finished | When the Ingestion was completed. |
| Data Source UUID | Unique ID of the related data source. You can use it as a reference to the Dawiso Integration Runtime instance for private connections. |
For manually initiated ingestions, it may take some time before they become visible in the list.
Ingestion states
An ingestion goes through several phases. Load To MR Succeeded is the final state of a successfully completed ingestion.
| State | Description |
|---|---|
| New | Ingestion was created and is waiting for processing. |
| Scheduled | Ingestion is scheduled for the next execution. |
| Ingestion Running | Data is being extracted from the source system. |
| Ingestion Succeeded | Extraction of data from the source has been completed. |
| Ingestion Failed | There was an error reading data from the source (e.g., timeout or invalid credentials). Review the detailed ingestion log and correct the error if possible. For assistance, contact your Account Manager. |
| Processing Running | Dawiso is cleaning, validating, and transforming the data obtained from the source. |
| Processing Succeeded | Processing has been completed. |
| Processing Failed | Processing has failed. The support team will be notified automatically and will get in touch with you. |
| Transferring Running | Ingested data is being moved to the staging area in your Dawiso instance. |
| Transferring Succeeded | Transfer of the data has been completed. |
| Transferring Failed | Transferring data has failed. The support team will be notified automatically and will get in touch with you. |
| Load To MR Running | Ingested data is being moved from the staging area to the target space (Metadata Repository). You can monitor the progress directly in the target space (see Monitoring the progress). |
| Load To MR Succeeded | Ingestion has been completed successfully. The ingested metadata is up-to-date with the source and ready for documentation. |
| Load To MR Failed | Ingested data cannot be moved to the target space. The support team will be notified automatically and will get in touch with you. |
Ingestion log
When you click on the ID of a selected ingesion, you will be redirected to its Ingestion log. Here, you will find all information, including error messages, for the ingestion.
Use this information to debug errors or optimize the speed and impact of the ingestion process.
The log provides the following information:
| Column | Description |
|---|---|
| ID | ID of the log record. |
| Timestamp | Data and time of the record. |
| Severity | 1 - Informative 2 - Warning 3 - Error |
| Message | Detailed information on specific actions. |
Monitoring the progress
If there is an ingestion running into your space, there will be a % number next to your application’s name. Once finished or failed, it will be replaced with a green or red mark and a message.
Hover over the message:
-
For a quick overview of the ingestion process, including information on the ingestion start and finish times, and how many objects were inserted, updated, or deleted.
-
At the bottom of the overview popup, click the:
- Show details button: For a detailed list of all ingestions.
- Ingest button: Easily ingest data from the overview page without going to the settings.