Google BigQuery

Google BigQuery is a fully managed, serverless data warehouse that enables fast, SQL-based analysis of large datasets. It offers scalable and cost-effective data processing, with built-in machine learning capabilities and integration with various Google Cloud services.

Default hierarchy

In the chart below, you will find the organizational structure of Google BigQuery objects. Objects are in represented in green and folders are in yellow.

Data lineage

Dawiso reads lineage for Google BigQuery from Google’s Data Lineage API, which records how BigQuery jobs move data. Lineage is ingested only when Scan lineage is enabled on the data source, and it is scanned in a separate pass after the objects.

Dawiso ingests two kinds of edges:

• Table to table: The upstream and downstream tables of copy jobs, load jobs, and query jobs.
• Column to column: The source columns behind each target column.

The Data Lineage API reports lineage only for jobs that ran after it was enabled on the project, so a newly enabled project starts without history. For the API and role setup, see Prepare for Google BigQuery integration.

What the Data Lineage API does not record

Dawiso shows the lineage that Google records. The following gaps come from the API, not from the scan:

• Nested fields. Column lineage covers top-level columns only. Fields inside a STRUCT or JSON column are not reported, so the whole struct appears as a single column.
• Load jobs. They produce table-level lineage, but no column lineage.
• Routines. Stored procedures and functions produce no direct lineage.
• Large jobs. A job that creates more than 1,500 column-level links produces table-level lineage only.
• External tables. Upstream column-level lineage is not collected for them.
• Scheduled transfers. Recurring load jobs from the BigQuery Data Transfer Service are not recorded automatically.

For the full list, see Google’s data lineage documentation.

Object types

Dataset

Maps to Google BigQuery Dataset object type.

Source Attribute	Target Attribute in Dawiso
catalog_name	Catalog
creation_time	Created
ddl	DDL
default_collation_name	Collation
last_modified_time	Last Modified
location	Location
schema_owner	Schema Owner

Function

Maps to Google BigQuery Function object type.

Source Attribute	Target Attribute in Dawiso
created	Created
data_type	Data Type
ddl	DDL
external_language	External Language
is_deterministic	Is Deterministic
last_altered	Last Altered
routine_body	Language
routine_catalog	Catalog
routine_definition	Definition
routine_schema	Schema
routine_type	Type
security_type	Security Type

Function Parameter

Maps to Google BigQuery Function Parameter object type.

Source Attribute	Target Attribute in Dawiso
data_type	Data Type
is_aggregate	Is Aggregate
is_result	Is Result
ordinal_position	Ordinal Position
parameter_default	Default
parameter_mode	Mode

Procedure

Maps to Google BigQuery Procedure object type.

Source Attribute	Target Attribute in Dawiso
created	Created
data_type	Data Type
ddl	DDL
external_language	External Language
is_deterministic	Is Deterministic
last_altered	Last Altered
routine_body	Language
routine_catalog	Catalog
routine_definition	Definition
routine_schema	Schema
routine_type	Type
security_type	Security Type

Procedure Parameter

Maps to Google BigQuery Procedure Parameter object type.

Source Attribute	Target Attribute in Dawiso
data_type	Data Type
is_aggregate	Is Aggregate
is_result	Is Result
ordinal_position	Ordinal Position
parameter_default	Default
parameter_mode	Mode

Table

Maps to Google BigQuery Table object type.

Source Attribute	Target Attribute in Dawiso
base_table_catalog	Base Table Catalog
base_table_name	Base Table Name
creation_time	Created
ddl	DDL
default_collation_name	Collation
is_change_history_enabled	Is Change History Enabled
is_insertable_into	Is Insertable Into
is_typed	Is Typed
replica_source_catalog	Replica Source Catalog
replica_source_name	Replica Source Name
replica_source_schema	Replica Source Schema
replication_error	Replication Error
replication_status	Replication Status
snapshot_time_ms	Snapshot Time MS
table_catalog	Catalog
table_schema	Schema
table_type	Type
upsert_stream_apply_watermark	Upsert Stream Apply Watermark

Table Column

Maps to Google BigQuery Table Column object type.

Source Attribute	Target Attribute in Dawiso
clustering_ordinal_position	Clustering Ordinal Option
collation_name	Collation
column_default	Default
data_type	Data Type
generation_expression	Generation Expression
is_generated	Is Generated
is_hidden	Is Hidden
is_nullable	Is Nullable
is_partitioning_column	Is Partitioning Column
is_stored	Is Stored
is_system_defined	Is System Defined
is_updatable	Is Updatable
ordinal_position	Ordinal Position
rounding_mode	Rounding Mode
table_catalog	Catalog
table_schema	Schema

View

Maps to Google BigQuery View object type.

Source Attribute	Target Attribute in Dawiso
check_option	Check Option
table_catalog	Catalog
table_schema	Schema
use_standard_sql	Use Standard SQL
view_definition	Definition

View Column

Maps to Google BigQuery View Column object type.

Source Attribute	Target Attribute in Dawiso
clustering_ordinal_position	Clustering Ordinal Option
collation_name	Collation
column_default	Default
data_type	Data Type
generation_expression	Generation Expression
is_generated	Is Generated
is_hidden	Is Hidden
is_nullable	Is Nullable
is_partitioning_column	Is Partitioning Column
is_stored	Is Stored
is_system_defined	Is System Defined
is_updatable	Is Updatable
ordinal_position	Ordinal Position
rounding_mode	Rounding Mode
table_catalog	Catalog
table_schema	Schema

Relations

From Object Type	Relation Type	To Object Type	Note
Table	foreign key parent / foreign key child	Table	Foreign keys between tables.