Package Installation Pipeline

When you install a package, Dawiso transforms its JSON configuration into live metadata entities. Understanding this pipeline helps diagnose installation failures and structure packages that validate on the first attempt.

Version States

Every package version passes through four states in strict sequence:

State	Trigger	What happens
CREATED	Upload or save	Raw JSON stored. No validation yet.
VALIDATED	Validate action	Schema checks, reference resolution, dependency verification. Errors block progression.
VERSIONED	Create version	Immutable snapshot created. Version number assigned.
INSTALLED	Install action	Loaders execute sequentially. Assets become live metadata entities in the database.

A package cannot skip states. Each state transition runs its own checks before allowing progression to the next.

CREATED

The entry state. Dawiso stores the raw package JSON without interpreting it. The package appears in the management UI but has no effect on the running system. Editing the JSON is allowed in this state.

VALIDATED

The validation engine performs three categories of checks:

• Schema validation — Verifies the JSON structure matches the expected format for each asset type. Missing required fields, incorrect data types, and malformed references produce errors.
• Reference resolution — Resolves all cross-package references ({packageKey}#{assetKey}) against installed packages. Unresolvable references produce errors.
• Dependency verification — Confirms all entries in dependsOn point to packages in INSTALLED state. Missing or non-installed dependencies produce errors.

Validation errors must be resolved before the package can progress. Warnings do not block progression but indicate potential issues.

VERSIONED

Creates an immutable snapshot of the validated JSON. The version number follows semantic versioning. Once versioned, the JSON cannot be edited — any changes require a new version.

INSTALLED

The installation engine executes loaders against the versioned snapshot. Each loader reads its asset type from the JSON and writes the corresponding database entities. This state transition is the only one that modifies the live metadata model.

The Loader System

Installation executes 240+ sequential loaders, each responsible for one asset type. Loaders run in a fixed order defined by the system — not by the order assets appear in the package JSON.

Each loader performs three operations:

1. Read the asset definition from the package JSON.
2. Map the asset key to a database entity (create or update).
3. Register the key → database ID mapping for downstream loaders.

This sequential design means later loaders can reference entities created by earlier loaders. It also means loader order violations cause predictable failures.

Loader Execution Flow

For each asset in its scope, a loader follows this sequence:

1. Lookup — Check if an entity with this key already exists in the database (from a previous installation or a base package).
2. Create or update — If the entity exists, update its properties. If not, create a new entity and assign a database ID.
3. Register mapping — Store the {packageKey}#{assetKey} → database ID pair in the installation registry.
4. Cache update — Call UpdateCacheData to propagate the change to the in-memory cache.

If any step fails, the loader logs the error and — depending on severity — either skips the asset or aborts the entire installation.

Loader Groups

Loaders are organized into five groups that execute in the order listed below.

Codetable Loaders

Codetable loaders create foundational visual and UI assets: icons, colors, components, pages, and templates. These assets have no dependencies on structural metadata, so they load safely at the start of the sequence.

Codetable assets include:

• Icons — SVG definitions referenced by object types and UI components.
• Colors — Named color values for status indicators, badges, and chart elements.
• Components — Reusable UI building blocks (e.g., core#ownership_generic).
• Pages — Full-page layout definitions for navigation targets.
• Templates — main and objectDetail templates that define entity detail views.

Structural Loaders

Structural loaders create the core metadata model: objectTypes, attributeTypes, relationTypes, workflows, workflowStates, and domainTypes. Each structural asset receives a database ID that downstream loaders reference.

They process assets in dependency order within the group. Object types load before attribute types because attributes reference their parent object type. Similarly, workflows load before workflow states.

Relationship Loaders

Relationship loaders wire structural assets together: objectType-attribute mappings, objectType-relation mappings, feature attachments, and conditions. These loaders depend on structural assets existing in the database, which is why they run after structural loaders.

Key relationship assets:

• Object type-attribute mappings — Attach attribute types to object types, defining which fields appear on each entity.
• Object type-relation mappings — Define which relation types connect which object types.
• Feature attachments — Enable optional features (versioning, comments, lineage) on specific object types.
• Conditions — Define visibility and validation rules for attributes based on field values.

Integration Loaders

Integration loaders configure external connectivity: external data sources, API mappings, and query sets. These loaders reference both structural assets (for schema alignment) and codetable assets (for UI presentation).

These assets connect Dawiso to external systems — data warehouses, BI tools, API endpoints. Each integration asset maps external schema elements to internal object types and attribute types.

Translation Loaders

Translation loaders apply language-specific labels to every translatable asset created by previous loaders. Translation loaders run last because they reference assets from all other groups.

A translation entry for a nonexistent asset key fails silently — the label is stored but never displayed. This behavior is by design: it allows translation packages to ship labels for assets that may or may not be installed.

Translatable assets include object type names, attribute labels, workflow state names, domain type titles, page titles, and template section headers.

Asset Key Mapping

Every asset in a package has a string key (e.g., business_term, core#description). During installation, each key gets mapped to a numeric database ID. Dawiso maintains this mapping in a registry that serves three purposes:

• Dependency tracking — The system knows which database entities belong to which package version.
• Cleanup on deletion — Uninstalling a package removes all entities mapped to its keys.
• Duplicate detection — Installing a package that declares a key already owned by another package raises a conflict error.

Cross-package references use the {packageKey}#{assetKey} syntax. The loader resolves the package key first, then looks up the asset key within that package’s mapping.

Key Uniqueness Rules

Asset keys must be unique within their asset type and package scope. Two different packages can define an attribute type with the same local key (e.g., description), but cross-package references disambiguate them (core#description vs. cust_erp#description).

Within a single package, duplicate keys within the same asset type cause a validation error. Duplicate keys across different asset types are allowed — an object type key document and an attribute type key document can coexist.

Post-Installation Steps

Installation writes entities to the database, but three subsystems require separate updates before changes are fully visible.

Elasticsearch Reindex

New or modified search indexes do not return results until Elasticsearch reindexes the affected entity types. Reindex is not triggered automatically by installation. Navigate to the search administration section and trigger reindex for the relevant entity types.

The reindex operation processes all entities of the specified type. For large deployments, schedule reindex during low-traffic periods to avoid performance degradation.

In-Memory Cache Refresh

Each loader calls UpdateCacheData after writing its entities. This handles most cache invalidation. Edge cases — particularly around template changes and workflow state transitions — may require a manual application restart.

Cache entries have no automatic TTL. Until explicitly refreshed, stale entries persist indefinitely. The UpdateCacheData mechanism targets specific cache keys, but complex template changes sometimes affect cache keys that the loader does not anticipate.

User Space Permissions Recalculation

New object types and domain types may not appear in user catalogs until the permissions engine recalculates access. This recalculation runs on a schedule, not on-demand after installation. Administrators can trigger manual recalculation from the permissions administration section.

Permissions recalculation evaluates each user’s role assignments against the updated metadata model. New object types inherit default visibility rules, but custom permission configurations require explicit setup.

Troubleshooting Common Failures

Validation Fails with “Reference Not Found”

The most common validation error. Occurs when an asset references another asset that either does not exist or belongs to a package not listed in dependsOn. To diagnose:

1. Identify the failing reference from the validation error message (e.g., core_glossary#business_term).
2. Verify the referenced package (core_glossary) is listed in dependsOn.
3. Verify the referenced package is in INSTALLED state.
4. Verify the asset key (business_term) exists in the referenced package.

Installation Hangs or Times Out

Large packages with hundreds of assets may exceed default timeout thresholds. Each loader writes to the database and updates the cache sequentially. Packages with 500+ assets can take several minutes to install.

If installation times out, check the server logs for the last successfully processed loader. The installation can be retried — loaders use upsert logic, so re-running does not create duplicate entities.

Partial Installation

If the process crashes mid-installation, some loaders will have completed while others have not. The package remains in VERSIONED state (it does not transition to INSTALLED). Retrying the installation re-runs all loaders from the beginning. Completed loaders update existing entities rather than creating duplicates.

Gotchas

Post-Installation Checklist

1. Verify the package version state is INSTALLED in the package management UI.
2. Check the installation log for warnings or skipped assets.
3. Trigger an Elasticsearch reindex for any new or modified object types.
4. Confirm new object types appear in the catalog (if they don’t, trigger permissions recalculation).
5. Open one instance of each new object type and verify the template renders correctly.
6. Test search against new entity types to confirm index availability.
7. Verify translations by switching the UI language and checking labels on new assets.
8. If workflow states were added or modified, create a test entity and walk through each state transition.
9. Review the key-to-ID mapping in the package registry for unexpected conflicts or missing entries.
10. Confirm that relation types display correctly on both source and target object type pages.