1. Navigate to Settings > Data Ingestion > Data Sources.
  2. Click New data source.
    • Create data source.png
  3. Select your system and the newly created connection.
  4. Name your source and add a description for easier identification in Dawiso.

Dawiso scans the database entered in the connection’s Database field. The data source has no list of databases or schemas to choose and no switches per object type: each ingestion reads every user schema that the Dawiso user can see. Which schemas, tables and procedures the user can see depends on its permissions, see Prepare for Vertica integration.

Dawiso always skips the Vertica system schemas (v_catalog, v_monitor, v_internal, v_internal_tables, v_txtindex, v_func, v_secret_managers), functions implemented in external libraries (UDx), including the ones that ship with Vertica, and the flex-table function public.isOrContains.

Destination configuration

In the Destination configuration step:

  1. Select the space where Dawiso stores your ingested metadata. You can select only the spaces you have access to.
  2. Select the workflow for managing ingested objects. For more information, see Workflow types.
  3. Optionally, configure:
    • Schedule: To customize the regular automated ingestion, check the box next to Schedule and adjust the frequency.
    • Optional Settings: Specify options supported by the provider in JSON format. For more information, see Optional Settings.
    • Additional Settings: Add a JSON formatted list of additional settings for troubleshooting your custom scenarios. Leave this field empty unless Dawiso support asks you to change it.
    • Enable AI-Generated Descriptions: Generate descriptions for your ingested metadata from predefined prompts. Each attribute type can have its own configurable prompt. AI-generated descriptions carry a banner until a user reviews and saves them.
      • Descriptions are generated at the following levels: database, schema, view, and table. Enable AI-Gen Descriptions.png
  4. Save your data source.
Warning

If you delete a data source and then create a new one for the same space, the original ingested data remains and mixes with the new data. To prevent duplicates, delete the metadata application from the space (with the old source) before creating and ingesting a new one.

Optional settings

In the data source configuration step, the Optional Settings field accepts a query transformation that adjusts queries dynamically based on predefined values.

Query transformations can access the following data source fields:

  • optionalSettings.dictionary: Define key-value pairs to reference in transformations.

    "dictionary":{ "schemas": "dbo, system, api", "databases": "Alpha, Bravo" }
  • datasource: Reference values directly from data source properties; these vary by provider.

    datasource.databases
  • optionalSettings: Reference values from optional settings in the data source configuration.

    { "databases": "Alpha, Bravo" }

Run ingestion

After you create your data source, Dawiso redirects you to the list of your data sources.

  1. Click the three dots next to your newly created source, and select Run ingestion.
  2. Confirm to ingest your data.

You can find all new, running, or completed ingestions by navigating to Settings > Data Ingestion > Ingestions.

Tip

For more information, see Ingestions.

If you ingest data on-premises, see Dawiso Integration Runtime (DIR) instead.

Troubleshooting

IssueSolution
No tables or columns are ingestedVertica’s catalog shows a user only the objects it may access, so read access to V_CATALOG alone returns no tables. Grant the permissions in Prepare for Vertica integration.
Procedures are ingested without their source codeVertica returns the source code of a procedure only to a user that may execute it. Run GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA schema_name TO dawiso_user; for each schema that contains procedures. This statement covers stored procedures as well; Vertica has no ALL PROCEDURES IN SCHEMA form. It applies only to procedures that exist when you run it, so run it again after new procedures are created. On Vertica 11.x, also grant USAGE on the schema.
Functions, procedures or sequences are missingWith per-schema permissions instead of the SYSMONITOR role, the user sees only the functions and procedures it may execute and the sequences it may select from. Grant the permissions described in Prepare for Vertica integration.
Warning

Once data is ingested into a metadata application, manual changes to the object hierarchy aren’t allowed. Adding, deleting, or moving objects under different parents can cause data inconsistencies and break data scans.