All books

Professional

Apps About Coach Log in Start reading

Quantitative Finance · Glossary

What is Anomaly flag, quarantine?

Also known as: anomaly flag · quarantine

Definition 7.2 Research, Data and Risk Platforms · Chapter 7 — Data Quality and Lineage

An anomaly flag is a record attached to a row or a partition that names the rule it failed, the severity and the detail, leaving the row itself unchanged; consumers decide whether to exclude flagged rows. A quarantine is the state of a partition that failed an error-severity rule: it is withheld from readers until a person releases it, with a reason, or it is rebuilt.

The data-quality layer: every partition runs the rule suite; flags are recorded beside the rows, never in place of them; a partition with an error-severity flag is quarantined until released; a vendor’s correction arrives as a new version of the data, and the rules run again.
Figure 7.1. The data-quality layer: every partition runs the rule suite; flags are recorded beside the rows, never in place of them; a partition with an error-severity flag is quarantined until released; a vendor’s correction arrives as a new version of the data, and the rules run again.
SIM2’s trades on the defective copy of seed 0, on a log scale: two decimal slips (one ten times too high, one ten times too low) stand out and are flagged; the genuine 3% jump late in the day is invisible at this scale and is not flagged by the aware rule. Data: fig_dataqual.py.
Figure 7.2. SIM2’s trades on the defective copy of seed 0, on a log scale: two decimal slips (one ten times too high, one ten times too low) stand out and are flagged; the genuine 3% jump late in the day is invisible at this scale and is not flagged by the aware rule. Data: fig_dataqual.py.
Read in context →