Skip to content

Data Quality

Entropy Data treats data quality as part of the data contract: the contract defines the rules, a test engine checks them against the real data, and the results show up wherever people judge whether data can be trusted.

Define

Quality rules live in the data contract, attached to a schema or a field, following the Open Data Contract Standard (ODCS). They range from library checks such as row counts, uniqueness, and not-null to custom SQL checks. Give each rule a dimension so its results count towards the right quality dimension.

Test

Results always belong to a data contract and one of its servers. There are four ways to produce them:

OptionWhen to use it
Run Checks in the UIEntropy Data runs the contract's checks itself, on demand or on a schedule for contracts carrying the configured tags. See Test a data contract.
Data Contract CLI (docs)datacontract test --publish runs the checks locally, in a CI/CD pipeline, or in Databricks, and uploads the results.
Data Contract CLI with dbtdatacontract dbt sync turns the contract's rules into dbt tests in your dbt project, and datacontract dbt test --publish runs them and uploads the results. See Sync with dbt.
External tools via the APIAny tool that already checks your data, such as Soda, Great Expectations, or an in-house framework, can publish its results with POST /api/test-results.

Monitor

  • Check results on the contract's Data Quality section show which checks passed or failed, per server and run.
  • The Data Quality Score sums up results as a mark out of 100 for a contract, data product, team, domain, or the whole organization, broken down by dimension.
  • Notifications alert owners when a check fails, keeps failing, or recovers.
  • Insights show how many data products have test results, and the contract coverage across your organization.
  • The Data Product Score includes a rule that every data contract of a product passes its latest test run.
  • The TestResultsCreatedEvent event lets your own automation react to every published run.