- Navigate to Settings > Data Ingestion > Data Sources.
- Click New data source.
- Select your system and the newly created connection.
- Name your source (and add a description) for easier identification in Dawiso.
[Optional] Select factories
In this step, you can filter which factories will be scanned. Provide your list in the following JSON format:
[{ "Factory": "MyFactory1", "Options": { "RunsHistoryInDays": 2 }}
]Properties
| Property | Description | Values | Mandatory |
|---|---|---|---|
Factory | Name of the factory. | string | Yes |
RunHistoryInDays | Defines the number of past days for which pipeline and trigger run history is loaded. The default setting is 2 days. | integer | Yes |
| By default, the scanner loads all Data Factories available for the Service Principal (based on the Reader access configuration in Azure). |
When the filter is applied, Dawiso first loads the list of all available factories, then runs a detailed scan only for the listed factories.
If a non-existent factory name is specified in the filter, Dawiso will return an error during ingestion. In that case, correct the filter and retry the ingestion.
Destination configuration
In the Destination configuration step:
-
Select into what space you want to store your ingested metadata. Keep in mind you can select only the spaces you have access to.
-
Select the workflow for ingested objects management. For more information on workflows, see the article on Workflow types.
-
[Optional steps]
Configuration Description Schedule To customize the regular automated ingestion, check the box next to Schedule and adjust the frequency. Optional Settings Specify options supported by the provider in JSON format. Option will be made available based on feedback. For more information, see Optional Settings. Additional Settings Add a JSON formatted list of additional settings that can be used to troubleshoot your custom scenarios. We recommend leaving this field empty unless our Dawiso support team suggests otherwise. Enable AI-Generated Descriptions Enable AI-generated descriptions for your ingested metadata. Descriptions will be generated according to prompts that are defined in packages. Each attribute type can have its own configurable prompt and any AI-generated content will be clearly marked with a banner the text is manually reviewed and saved. Descriptions will be generated on the following levels: database, schema, view, and table. 
-
Save your data source.
If you delete a data source and then create a new one to the same space, the original ingested data will remain and mix with the new data. To prevent duplicates, always delete the metadata application from the space (with the old source) before creating and ingesting a new one.
Run ingestion
Once you create your data source, you will be redirected to a list of your data sources.
- Click the three dots next to your newly created source, and select Run ingestion.
- Confirm to ingest your data.
You can find all new, running, or completed ingestions by navigating to Settings > Data Ingestion > Ingestions. For more information, see the article on Ingestions.
If you are ingesting data on-premises, refer to these articles instead Dawiso Integration Runtime (DIR).
Once data is ingested to metadata applications, manual changes to the object hierarchy are NOT allowed. Changes such as adding, deleting, or moving objects under different parents may cause data inconsistencies and break data scans.