All books

Professional

Apps About Coach Log in Start reading

Quantitative Finance · Glossary

What is Flat file format, schema version?

Also known as: flat file format · schema version

Definition 4.2 Research, Data and Risk Platforms · Chapter 4 — A Tick Store on Open Formats

A flat file format stores records of one fixed width one after another, after a header that states their layout, so that record ii is at a computable offset and nothing needs to be parsed to reach it. The header’s schema version identifies the layout; a reader refuses a version it does not know.

The tick store’s flat file: a header of multiple of 64 bytes (magic, version, record width, the layout as JSON), then fixed-width records appended as they arrive. A reader maps the file; the operating system reads a page from disk the first time a record in it is touched.
Figure 4.1. The tick store’s flat file: a header of multiple of 64 bytes (magic, version, record width, the layout as JSON), then fixed-width records appended as they arrive. A reader maps the file; the operating system reads a page from disk the first time a record in it is touched.
Read in context →