Databricks

Databricks is a unified analytics platform based on Apache Spark, streamlining data science and engineering workflows for scalable data processing, advanced analytics, and machine learning. Dawiso catalogs the full Unity Catalog surface (catalogs, schemas, tables, views, materialized views, metric views, columns, volumes, functions, ML models) together with the workspace consumption layer (SQL warehouses, jobs, pipelines, AI/BI dashboards and their datasets, Genie spaces) and governance metadata (tags, column masks, row filters, data classification, constraints).

Supported versions

Dawiso reads Databricks metadata through Unity Catalog system tables (SQL Statements API) and the Unity Catalog REST API. Requirements:

• Unity Catalog enabled in your Databricks workspace.
• A running SQL Warehouse the scanner can execute statements on.
• Access to Databricks system tables (system.information_schema, system.access, and optionally system.lakeflow, system.compute, system.data_classification).

Object hierarchy

Unity Catalog objects nest under the metastore; workspace objects (warehouses, jobs, pipelines, dashboards, Genie spaces) attach directly to the metastore level:

Scanned SQL and YAML definitions

Dawiso scans the source definition of query-defined objects and renders it as a code block in a dedicated Definition section on the object page:

• Views and Materialized Views: SQL definition from information_schema.views
• Functions: Routine body
• Metric Views: The YAML definition (dimensions, measures, source). Requires SELECT on the metric view. For more information, see the permission notes on Prepare for Databricks integration.
• Dashboard Datasets and Genie Sample Queries: The dataset/sample query SQL

Data lineage

Dawiso builds one lineage graph across Unity Catalog and the workspace layer. Two kinds of edges exist:

• Deterministic edges are derived from object definitions and always present after a scan.
• Activity-based edges come from system.access.table_lineage / column_lineage, which record executed reads and writes. They appear only for activity inside the configured lineage window (lineageDays, default 30 days) and can lag behind real time slightly (system-table latency).

Edge	Level	Source	Kind
Table → Table / View / Materialized View / Metric View	table	system.access.table_lineage	activity-based
Column → Column (incl. view and materialized-view columns)	column	system.access.column_lineage	activity-based
Base object → Metric View	table	metric-view definition (source:)	deterministic
Base column → Metric Column	column	metric-view dimension/measure expressions	deterministic
Foreign keys (Table → Table)	table	information_schema.referential_constraints	deterministic
Table / View / MV / Metric View → Dashboard	table	system.access.table_lineage (dashboard runs)	activity-based
Dashboard Dataset ← referenced tables/views	table	dataset SQL parsing	deterministic
Table / View / MV / Metric View → Job / Pipeline (reads), Job / Pipeline → outputs (writes)	table	system.access.table_lineage	activity-based
Volume → Table (loads), Table → Volume (exports)	table	system.access.table_lineage volume paths	activity-based
Genie Space ← attached tables	table	Genie space configuration	deterministic

Materialized-view refreshes are recorded by Databricks as column-level lineage, so materialized views participate in column lineage automatically. Metric views have no engine-recorded column lineage, which Dawiso derives from the YAML expressions instead, so it is present even without recent activity.

Object types

Dawiso ingests the following objects from Databricks:

Metastore

Maps to the Databricks Metastore object type.

Attribute	Key	Scanned (read-only)
Summary	core_summary	no (editable)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
ID — Native Databricks identifier	core_id	yes
Workspace ID	workspace_id	yes

Catalog

Maps to the Databricks Catalog object type.

Attribute	Key	Scanned (read-only)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
Owner	core_owner	yes
Created at	core_created_at	yes
Created by	core_created_by	yes
Modified at	core_modified_at	yes
Modified by	core_modified_by	yes
Web URL — Deep link — drives the “Open in Databricks” button	web_url	yes

Schema

Maps to the Databricks Schema object type.

Attribute	Key	Scanned (read-only)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
Owner	core_owner	yes
Catalog	core_catalog	yes
Created at	core_created_at	yes
Created by	core_created_by	yes
Modified at	core_modified_at	yes
Modified by	core_modified_by	yes
Web URL — Deep link — drives the “Open in Databricks” button	web_url	yes

Table

Maps to the Databricks Table object type.

Attribute	Key	Scanned (read-only)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
Owner	core_owner	yes
Type — Object kind as reported by Databricks	core_type	yes
Catalog	core_catalog	yes
Schema	schema	yes
Data Source Format	data_source_format	yes
Storage Path	storage_path	yes
Created at	core_created_at	yes
Created by	core_created_by	yes
Modified at	core_modified_at	yes
Modified by	core_modified_by	yes
Web URL — Deep link — drives the “Open in Databricks” button	web_url	yes

Table Column

Maps to the Databricks Table Column object type.

Attribute	Key	Scanned (read-only)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
Ordinal position	core_ordinal_position	yes
Full Data Type	full_data_type	yes
Data Type	core_data_type	yes
Nullable	core_is_nullable	yes
Default value	core_default_value	yes
Length	core_length	yes
Numeric precision	core_numeric_precision	yes
Numeric scale	core_numeric_scale	yes
Parent Table Type	parent_table_type	yes
Primary key	core_is_primary_key	yes
Is Foreign Key?	is_foreign_key	yes

View

Maps to the Databricks View object type.

Attribute	Key	Scanned (read-only)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
Owner	core_owner	yes
Type — Object kind as reported by Databricks	core_type	yes
Catalog	core_catalog	yes
Schema	schema	yes
Data Source Format	data_source_format	yes
Storage Path	storage_path	yes
Created at	core_created_at	yes
Created by	core_created_by	yes
Modified at	core_modified_at	yes
Modified by	core_modified_by	yes
Web URL — Deep link — drives the “Open in Databricks” button	web_url	yes
Definition — Scanned SQL / YAML definition (rendered as a code block)	core_definition	yes

View Column

Maps to the Databricks View Column object type.

Attribute	Key	Scanned (read-only)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
Ordinal position	core_ordinal_position	yes
Full Data Type	full_data_type	yes
Data Type	core_data_type	yes
Nullable	core_is_nullable	yes
Default value	core_default_value	yes
Length	core_length	yes
Numeric precision	core_numeric_precision	yes
Numeric scale	core_numeric_scale	yes
Parent Table Type	parent_table_type	yes

Materialized View

Maps to the Databricks Materialized View object type.

Attribute	Key	Scanned (read-only)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
Owner	core_owner	yes
Type — Object kind as reported by Databricks	core_type	yes
Catalog	core_catalog	yes
Schema	schema	yes
Data Source Format	data_source_format	yes
Storage Path	storage_path	yes
Created at	core_created_at	yes
Created by	core_created_by	yes
Modified at	core_modified_at	yes
Modified by	core_modified_by	yes
Web URL — Deep link — drives the “Open in Databricks” button	web_url	yes
Definition — Scanned SQL / YAML definition (rendered as a code block)	core_definition	yes

Materialized View Column

Maps to the Databricks Materialized View Column object type.

Attribute	Key	Scanned (read-only)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
Ordinal position	core_ordinal_position	yes
Full Data Type	full_data_type	yes
Data Type	core_data_type	yes
Nullable	core_is_nullable	yes
Default value	core_default_value	yes
Length	core_length	yes
Numeric precision	core_numeric_precision	yes
Numeric scale	core_numeric_scale	yes
Parent Table Type	parent_table_type	yes

Metric View

Maps to the Databricks Metric View object type.

Attribute	Key	Scanned (read-only)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
Owner	core_owner	yes
Type — Object kind as reported by Databricks	core_type	yes
Catalog	core_catalog	yes
Schema	schema	yes
Data Source Format	data_source_format	yes
Storage Path	storage_path	yes
Created at	core_created_at	yes
Created by	core_created_by	yes
Modified at	core_modified_at	yes
Modified by	core_modified_by	yes
Web URL — Deep link — drives the “Open in Databricks” button	web_url	yes
Definition — Scanned SQL / YAML definition (rendered as a code block)	core_definition	yes

Metric Column

Maps to the Databricks Metric Column object type.

Attribute	Key	Scanned (read-only)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
Type — Object kind as reported by Databricks	core_type	yes
Full Data Type	full_data_type	yes
Page Name	core_display_name	yes
Synonyms	core_synonyms	yes
Format JSON	format_json	yes

Volume

Maps to the Databricks Volume object type.

Attribute	Key	Scanned (read-only)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
Owner	core_owner	yes
Type — Object kind as reported by Databricks	core_type	yes
Catalog	core_catalog	yes
Schema	schema	yes
Storage Location	storage_location	yes
Created at	core_created_at	yes
Created by	core_created_by	yes
Modified at	core_modified_at	yes
Modified by	core_modified_by	yes
Web URL — Deep link — drives the “Open in Databricks” button	web_url	yes

Function

Maps to the Databricks Function object type.

Attribute	Key	Scanned (read-only)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
Owner	core_owner	yes
Type — Object kind as reported by Databricks	core_type	yes
Catalog	core_catalog	yes
Schema	schema	yes
Data Type	core_data_type	yes
Full Data Type	full_data_type	yes
Definition — Scanned SQL / YAML definition (rendered as a code block)	core_definition	yes
External Language	external_language	yes
Is deterministic	core_is_deterministic	yes
Created at	core_created_at	yes
Created by	core_created_by	yes
Modified at	core_modified_at	yes
Modified by	core_modified_by	yes
Web URL — Deep link — drives the “Open in Databricks” button	web_url	yes

ML Model

Maps to the Databricks ML Model object type.

Attribute	Key	Scanned (read-only)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
Owner	core_owner	yes
Catalog	core_catalog	yes
Schema	schema	yes
Storage Location	storage_location	yes
Created at	core_created_at	yes
Modified at	core_modified_at	yes
Web URL — Deep link — drives the “Open in Databricks” button	web_url	yes

External Location

Maps to the Databricks External Location object type.

Attribute	Key	Scanned (read-only)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
Owner	core_owner	yes
URL	core_url	yes
Credential Name	credential_name	yes
Is Read Only	core_is_read_only	yes
Created at	core_created_at	yes
Modified at	core_modified_at	yes
Web URL — Deep link — drives the “Open in Databricks” button	web_url	yes

Storage Credential

Maps to the Databricks Storage Credential object type.

Attribute	Key	Scanned (read-only)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
Owner	core_owner	yes
Created at	core_created_at	yes
Modified at	core_modified_at	yes
Web URL — Deep link — drives the “Open in Databricks” button	web_url	yes

SQL Warehouse

Maps to the Databricks SQL Warehouse object type.

Attribute	Key	Scanned (read-only)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
ID — Native Databricks identifier	core_id	yes
Type — Object kind as reported by Databricks	core_type	yes
Warehouse Channel	warehouse_channel	yes
Size	core_size	yes
Min Clusters	min_clusters	yes
Max Clusters	max_clusters	yes
Auto Stop Minutes	auto_stop_minutes	yes
Created by	core_created_by	yes
Modified at	core_modified_at	yes
Web URL — Deep link — drives the “Open in Databricks” button	web_url	yes

Job

Maps to the Databricks Job object type.

Attribute	Key	Scanned (read-only)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
ID — Native Databricks identifier	core_id	yes
Type — Object kind as reported by Databricks	core_type	yes
Paused	paused	yes
Timeout (seconds)	timeout_seconds	yes
Created by	core_created_by	yes
Run As User Name	run_as_user_name	yes
Created at	core_created_at	yes
Modified at	core_modified_at	yes
Web URL — Deep link — drives the “Open in Databricks” button	web_url	yes

Pipeline

Maps to the Databricks Pipeline object type.

Attribute	Key	Scanned (read-only)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
ID — Native Databricks identifier	core_id	yes
Type — Object kind as reported by Databricks	core_type	yes
Created by	core_created_by	yes
Run As	run_as	yes
Created at	core_created_at	yes
Modified at	core_modified_at	yes
Web URL — Deep link — drives the “Open in Databricks” button	web_url	yes

Dashboard

Maps to the Databricks Dashboard object type.

Attribute	Key	Scanned (read-only)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
ID — Native Databricks identifier	core_id	yes
Warehouse ID	warehouse_id	yes
Parent Path	parent_path	yes
State	core_state	yes
Path	core_path	yes
Created at	core_created_at	yes
Modified at	core_modified_at	yes
Web URL — Deep link — drives the “Open in Databricks” button	web_url	yes

Dashboard Dataset

Maps to the Databricks Dashboard Dataset object type.

Attribute	Key	Scanned (read-only)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
Internal Name	internal_name	yes
Definition — Scanned SQL / YAML definition (rendered as a code block)	core_definition	yes

Genie Space

Maps to the Databricks Genie Space object type.

Attribute	Key	Scanned (read-only)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
ID — Native Databricks identifier	core_id	yes
Warehouse ID	warehouse_id	yes
Parent Path	parent_path	yes
Run As Type	run_as_type	yes
Instructions	instructions	yes
Created at	core_created_at	yes
Modified at	core_modified_at	yes
Web URL — Deep link — drives the “Open in Databricks” button	web_url	yes

Genie Sample Query

Maps to the Databricks Genie Sample Query object type.

Attribute	Key	Scanned (read-only)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
Definition — Scanned SQL / YAML definition (rendered as a code block)	core_definition	yes

Tag

Maps to the Databricks Tag object type.

Attribute	Key	Scanned (read-only)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
Tag Name	tag_name	yes
Value	core_value	yes
Source View	source_view	yes

Column Mask

Maps to the Databricks Column Mask object type.

Attribute	Key	Scanned (read-only)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
Mask Name	mask_name	yes
Using Columns	using_columns	yes

Row Filter

Maps to the Databricks Row Filter object type.

Attribute	Key	Scanned (read-only)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
Filter Name	filter_name	yes
Target Columns	target_columns	yes

Data Classification

Maps to the Databricks Data Classification object type.

Attribute	Key	Scanned (read-only)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
Classification Tag	class_tag	yes
Confidence	confidence	yes
Frequency	frequency	yes
Data Type	core_data_type	yes
Latest Detected	latest_detected_time	yes
First Detected	first_detected_time	yes
Exclusion State	exclusion_state	yes

Constraint

Maps to the Databricks Constraint object type.

Attribute	Key	Scanned (read-only)
Description — Editable description maintained in Dawiso	core_description	no (editable)
Scanned Description — Comment / description scanned from Databricks	core_description_scanned	yes
Constraint Name	constraint_name	yes
Type — Object kind as reported by Databricks	core_type	yes

What changed compared to the previous Databricks integration

This integration replaces the previous Databricks metadata application. If you used the earlier version:

• New object types: Metric Views + Metric Columns, Materialized Views + Materialized View Columns as first-class types, Dashboards + Dashboard Datasets, Genie Spaces + Sample Queries, SQL Warehouses, Jobs, Pipelines, External Locations, Storage Credentials, Tags, Column Masks, Row Filters, Data Classification, Constraints.
• Scanned definitions: SQL for views, materialized views and functions; YAML for metric views; dataset SQL for dashboards and Genie.
• Lineage: dashboards, jobs, pipelines and volumes participate in the lineage graph; metric views get deterministic base and column lineage.
• Read-only friendly: the scan works at the metadata-only BROWSE privilege tier (see Prepare for Databricks integration); higher tiers unlock definitions and additional objects.
• Attribute harmonization: shared attributes (owner, created/modified timestamps, data type, ordinal position, definition, …) use Dawiso core attribute keys, so cross-technology filters treat Databricks objects and other scanners’ objects uniformly.

Limitations

• Lineage from system.access is activity-based. This means objects with no executed reads/writes inside the lineage window show no activity-based edges (deterministic edges are unaffected).
• Notebooks and legacy DBSQL dashboards are not catalogued; lineage events attributed to them are not shown.
• Metric-view YAML definitions and metric-view base lineage require SELECT on the metric views; at the BROWSE-only tier the metric-view columns are still scanned.
• Storage credentials require metastore-admin privileges and are disabled by default.
• Primary/foreign-key constraint details depend on USE CATALOG on the in-scope catalogs.