If you’re new to working with packages, we recommend starting with the Writing an Application Package tutorials. This guide builds on those basics and focuses specifically on metadata ingestion.
A metadata scanning application is designed to automatically ingest metadata from a connected system. Dawiso currently supports ingestion from the numerous providers, including SQL Server, Google BigQuery, Keboola, and many more. See the full list with supported providers.
However, as each database has its own structure, naming conventions, and logic, it might necessary to adjust the application to your specific needs. To do so, you can:
- Define the metamodel: Establish the structure that represents how objects and attributes are organized in the source system.
- Configure the queries and map attributes: Specify how to extract the metadata and align it with your defined metamodel.
You can either scan data or import them using Excel files, not both. Because of this, data import should be disabled in the metadata package, ensuring that all content is sourced directly from the scanning process.
Tutorial Outline
To explain the process of creating a metadata application clearly, we will demonstrate the configuration using our example Recipe Metadata Ingestion app. Download the full examples we used in this tutorial to follow along:
- Recipe Manager (version data ingestion): Recipe Manager (Metadata Application).json
- SQL script with the example database: Recipe SQL Script.sql In this tutorial, we will walk through the following process:
| Step | Description |
|---|---|
| [Preparation] Analyze and configure base | Analyze the source structure and configure the base of the package that reflects the object types and attributes in your database. |
| 1. Define hierarchies | Define the native hierarchy (reflecting the real structure of the source system) and a hybrid hierarchy (a custom organization in Dawiso). |
| 2. Set up data source definition | Configure ingestion rules and query logic for metadata extraction. |
| 3. Map attributes | Map ingested fields to Dawiso attributes to ensure they are ingested correctly as the intended attribute types and to maintain consistency and usability across your catalog. |
| [Extra] Define object type templates | Object type templates can be defined either before or after configuring the ingestion. Usually, it’s done at the end. This step is not covered in this tutorial, refer to the Writing an Application Package tutorials instead. |
The steps outline a suggested order for setting up metadata application in Dawiso. The exact sequence is flexible and should be adapted based on your familiarity with packages and the size and complexity of your setup.