All books

Professional

Apps About Coach Log in Start reading

Quantitative Finance · Glossary

What is Data lineage, provenance record?

Also known as: data lineage · provenance record

Definition 7.6 Research, Data and Risk Platforms · Chapter 7 — Data Quality and Lineage

The data lineage of a result is the graph of dataset versions and transformations it was derived from, back to its raw inputs. A provenance record is the stored description of how one dataset version was produced: the versions of its inputs (by content hash), the code and its version, and when and by whom it ran.

The lineage of the day’s results. The vendor’s correction of the close makes every version built from the old close stale (red): the marks, the P&L and the research features; the minute bars, built from the trades alone, are not affected. Data: pl_dataqual.correction_and_lineage.
Figure 7.4. The lineage of the day’s results. The vendor’s correction of the close makes every version built from the old close stale (red): the marks, the P&L and the research features; the minute bars, built from the trades alone, are not affected. Data: pl_dataqual.correction_and_lineage.
Read in context →