Marquez is an open-source metadata server that collects, stores and visualises lineage metadata through OpenLineage. It is the reference implementation of OpenLineage. Data platform teams send metadata from running jobs to its OpenLineage-compatible endpoint and use the web interface to view a graph of dependencies between datasets and jobs, with execution details and performance metrics. Its lineage API supports tasks such as backfills and root-cause analysis. Integrations exist for Apache Airflow, Spark, Flink, dbt and Dagster. You run Marquez yourself, with Docker, with the included Helm chart on Kubernetes, or from source (Java 17 and PostgreSQL 14).
WeWork released Marquez as open source; it is now a graduated project of the LF AI & Data Foundation, part of the Linux Foundation, a non-profit based in the United States. The code is licensed under Apache 2.0. Because you host it yourself, metadata and lineage stay in your own environment. By default the API has no authentication or authorisation, so you need to put these in place yourself.