Sync Environments

Many teams run more than one Entropy Data instance — for example a test instance where changes are curated and validated, and a production instance that serves consumers. This guide shows how to promote the portable organization state from one instance to another with the Entropy Data CLI, using a test → prod promotion as the running example.

The CLI offers three ways to move organization state:

• sync — copy selected resources directly from a source instance to a target instance in one command.
• export dir + apply dir — the same operation in two steps, with a reviewable YAML tree in between (handy for GitOps or air-gapped promotion). apply dir works like kubectl apply -f <dir>.
• import zip — import a zip produced by the in-app Settings → Export page.

Every write is an idempotent PUT-by-id, so a run converges and is safe to repeat. Only portable, declarative state is copied — never secrets, telemetry, or environment-specific identity (see What is not synced).

Prerequisites

• Entropy Data CLI installed via uv tool install entropy-data.
• An API key with organization scope on both the source and the target instance.
• Both instances on a recent Entropy Data version — sync uses the organization-features and semantics-ontology endpoints, so a self-hosted target should be on the latest release.

Sync test to prod with sync

1. Configure a connection for each instance

Add a named connection per environment so you can address them by name:

entropy-data connection add test    # prompts for the test instance API key + host
entropy-data connection add prod    # prompts for the prod instance API key + host

Connections are stored in ~/.entropy-data/config.toml:

[connections.test]
api_key = "ed_test_..."
host = "https://test.entropy-data.com"

[connections.prod]
api_key = "ed_prod_..."
host = "https://app.entropy-data.com"

2. Choose what to sync

sync copies nothing by default — you name the resources to copy with --include (a run without it fails and lists the available resources). This keeps every sync scoped to exactly what you intend to promote. See What can be synced for the full list of resource names.

3. Preview the change

--dry-run exports the source and prints the per-resource create/update counts it would apply, without writing anything to the target:

entropy-data sync --source test --target prod \
  --include teams,policies,datacontracts,dataproducts --dry-run

Dry run — planned changes:
  teams: create=13, update=0
  policies: create=9, update=0
  datacontracts: create=24, update=0
  dataproducts: create=28, update=0

4. Sync

entropy-data sync --source test --target prod \
  --include teams,policies,datacontracts,dataproducts

The CLI exports the selected resources from test, then applies them to prod in dependency order, and prints a summary:

Summary: 171 succeeded, 3 failed (0 export failures)

Run it again any time test changes — existing resources are updated in place, new ones are created, and unchanged ones are re-sent harmlessly.

5. Mirror deletions with --prune (optional)

By default sync only creates and updates. To make the target an exact mirror of the source for the included resources — also deleting target resources that no longer exist on the source — add --prune. Deletions run in reverse dependency order and are confirmation-gated unless you pass --yes:

entropy-data sync --source test --target prod --include datacontracts --prune

Useful options

sync exits non-zero if any export or apply write fails, so it is safe to gate a CI/CD pipeline on.

What can be synced

sync, export dir, and apply dir move the portable declarative state of an organization, applied in dependency order so references always resolve:

Three resources are handled specially:

• Teams — applied parent-first, with members stripped (the export keeps them so the artifact stays a faithful snapshot; they are just not re-created on the target).
• Semantics — copied as one OSI-compliant ontology YAML document per namespace, which the target imports in correct internal order (groups before members, concepts before relationships) so the graph round-trips losslessly.
• Organization features — an org-level singleton applied last, so a restrictive policy it carries cannot reject earlier resources. It is never pruned.

What is not synced

Only state reachable through the public /api/** API and portable across instances is copied. The CLI never moves:

• Secrets and credentials — API keys, git credentials, integration and connector connection secrets.
• Users and team members — identities are per-instance; stripped on apply.
• Telemetry and history — usage, costs, events, ingestion runs, lineage, audit fields.
• Environment-specific identity — hostnames and the like.

The following portable configuration is not supported yet, but may be added later: organization customization/branding, SCIM mapping, the team-roles configuration, notification channels, connectors, and integrations.

Alternative: export, review, then apply

For GitOps or air-gapped promotion, run the two halves separately with a reviewable YAML tree in between:

# On (or with access to) the source instance
entropy-data --connection test export dir ./org-state

# Review or commit ./org-state to git, then, against the target
entropy-data --connection prod apply dir ./org-state

apply dir works like kubectl apply -f <dir>: it reconciles the instance to match the directory. This is exactly what sync does internally — export to a staging directory, then apply — so the artifacts are interchangeable. It supports the same --include, --exclude, --dry-run, and --prune flags. Unlike sync, apply dir applies the whole tree by default (the directory is your selection).

The directory layout (folder-as-kind)

export dir writes, and apply dir reads, a tree where the folder name is the resource kind and each YAML file below it is one resource:

org-state/
  teams/                          # folder name = resource kind
    marketing.yaml                # one file per resource (filename is cosmetic)
    finance--legal.yaml
  policies/
    pii-policy.yaml
  datacontracts/
    orders-1-orders.yaml
  semantic-ontology/
    main.yaml                     # document resources: one YAML doc per namespace
  organization-features/
    organization-features.yaml    # singleton: <name>/<name>.yaml

A few properties of this convention:

• The folder is authoritative for the kind. Unlike a kubectl manifest, the files carry no kind: field — a file only means what its enclosing folder says. Moving datacontracts/x.yaml into policies/ would apply it as a policy.
• A resource's identity comes from its body (id, externalId, or namespace), not the filename. The filename is only there to keep the tree readable.
• Unknown folders are ignored. apply dir walks the known resource kinds and skips any folder that is not one of them, so an extra README/ or notes/ folder is harmless.

The recognized folder names (resource kinds), in dependency order, are:

teams  tags  definitions  policies  sourcesystems  certifications
classification-schemes  assets  datacontracts  dataproducts  example-data
access  semantic-namespaces  semantic-ontology  organization-features

The CLI is the source of truth for this list — run entropy-data apply dir --help (or entropy-data sync --help) to print the recognized kinds from the version you have installed, in case this page has drifted.

This layout matches the app's own organization export, so an export dir tree and an in-app Settings → Export zip are interchangeable.

Alternative: import an in-app export zip

The in-app Settings → Export page (organization owners only) downloads a zip of the organization's resources as YAML. Import it into any instance with import zip:

entropy-data --connection prod import zip export.zip

The CLI handles dependency ordering, team hierarchy resolution, and member stripping automatically.

Known issues

• Team members — exported teams may reference users (email addresses) that don't exist on the target. The CLI strips members automatically on apply.
• Access agreements — some agreements may reference deleted data products (marked 'unknown') and will fail on apply. These are typically rejected/cancelled agreements that can be safely skipped; the run continues and reports them in the summary.