QVD File Format Explained: Internal Structure, Symbol Tables and Bit-Packed Data

What Is a QVD File?

A QVD file is Qlik's native data file format. It stores one logical table, including field metadata and the data needed to reconstruct every row. Qlik products use QVD files heavily because they are compact, typed, and efficient for repeated reloads.

This article is not a generic Qlik marketing article. If you need a product overview, Qlik has its own QVD files documentation. Here we care about the file format: what sits on disk, how a reader finds values, and why a record usually stores indexes instead of repeating the actual values.

Understanding the QVD file format is useful when you build a QVD file reader, create QVD output, convert QVD to CSV, inspect broken files, or integrate QVD into an ETL process outside Qlik. You do not need this level of detail to click a button. You do need it when the button stops clicking back.

For interactive inspection, see the Q-Eye QVD editing tutorial. For automated conversion, see the QVD to CSV tutorial for Advanced ETL Processor.

High-Level QVD File Structure

A QVD file can conceptually be divided into three main parts:

1. XML header
2. Per-field symbol tables containing unique values
3. Bit-packed record/index section

XML Header
+----------+
     |
     v
00 terminator
+----------+
     |
     v
Field 1 Symbol Table
Field 2 Symbol Table
Field 3 Symbol Table
...
+----------+
     |
     v
Bit-Packed Record Index

The important idea is that records normally store indexes into symbol tables. They do not repeat the full string, integer, double, date, timestamp, or dual representation for every row. That is dictionary encoding, and it is one reason QVD files can be compact when fields contain repeated values.

Imagine a `Country` field with millions of rows but only a few countries. Storing `UK` once and then referring to symbol index `0` is much cheaper than writing `UK` again and again until the disk starts making disapproving noises.

The XML header describes the table before the binary data starts

A QVD file starts with UTF-8 XML. The XML header is followed by a zero byte. That zero byte marks the end of the text header and the beginning of the binary sections.

The XML contains table-level metadata and one `QvdFieldHeader` block per field. It gives the reader enough information to locate symbol tables and decode the packed record section.

Important table-level fields include:

A small illustrative header fragment looks like this:

<QvdTableHeader>
  <QvBuildNo>12345</QvBuildNo>
  <TableName>Sales</TableName>
  <NoOfRecords>500000</NoOfRecords>
  <RecordByteSize>4</RecordByteSize>
  <Fields>
    <QvdFieldHeader>
      <FieldName>Country</FieldName>
      <BitOffset>0</BitOffset>
      <BitWidth>3</BitWidth>
      <Bias>0</Bias>
      <NoOfSymbols>5</NoOfSymbols>
      <Offset>0</Offset>
      <Length>42</Length>
    </QvdFieldHeader>
  </Fields>
</QvdTableHeader>

Real QVD headers can contain more metadata. A reader should parse the XML properly rather than searching for text with hope and string offsets. Hope is not a parser. It is barely a project plan.

QvdFieldHeader tells the reader how each field is stored

Each field has its own `QvdFieldHeader`. This block connects logical metadata to the binary parts of the file.

`Offset` and `Length` are especially important when reading symbol tables. They tell the reader where the unique values for a field are stored. Without them, the binary section is just a queue of bytes wearing a false moustache.

Symbol tables store unique values once

QVD symbol tables use dictionary encoding. Each field has a list of unique values. Records then store an index into that list.

Source values for a `Country` field might be:

UK
France
UK
Germany
France

The symbol table can store only the unique values:

The records then store:

0, 1, 0, 2, 1

This can dramatically reduce storage for low-cardinality fields. Countries, statuses, flags, product categories, month names, and small code lists are good examples. A field with millions of unique long strings will not benefit in the same way. Compression is clever, but it is not magic. If every value is unique, the dictionary still has to remember every awkward guest at the party.

To inspect QVD metadata interactively, the QVD field types tutorial for Q-Eye is a useful companion.

QVD symbol physical types are not the same as logical field types

In the QVD files handled by our implementation, each symbol starts with a physical type identifier. That identifier tells the reader how the symbol value is physically represented in the symbol table.

These are physical symbol representations handled by our QVD implementation. Logical field types come from the XML metadata, especially `NumberFormat`.

Types 5 and 6 are commonly understood as dual-style values: a numeric representation plus text. That matters when a value has both a display form and a numeric meaning. Dates and timestamps are the classic place where this distinction keeps everyone honest.

Logical field types come from NumberFormat metadata

A physical symbol type tells you how bytes are stored. A logical field type tells you how the value should be interpreted. These are related, but they are not the same thing.

Logical `NumberFormat` values may include:

For example, a physical double can represent a plain numeric value, a date, a time, or a timestamp. The XML metadata gives the reader the context. Without that context, `45123.5` is just a number looking suspiciously pleased with itself.

QVD dates and timestamps are numeric values with calendar meaning

QVD numeric dates use OLE Automation date representation. The integer part counts days relative to 30 December 1899. Fractional values represent time. Microsoft documents the same base-date behaviour in its DateTime.FromOADate reference.

That means a date or timestamp may be stored physically as a double, while `NumberFormat` tells the reader to interpret that double as `DATE`, `TIME`, or `TIMESTAMP`.

Keep this distinction simple when implementing a reader. First decode the physical value. Then apply the logical interpretation. Trying to infer everything from the raw bytes alone is how date bugs acquire a second career.

The record index stores symbol references, not repeated values

The record section is a fixed-size sequence of packed records. Each record contains packed values for the fields. Those values are used to find entries in the field symbol tables.

Consider a small table:

The symbol tables might be:

The packed record indexes then represent:

The actual byte layout is packed at bit level, but the lookup idea is straightforward: decode packed integer, apply bias, look up symbol.

BitWidth controls how many bits each field needs

`BitWidth` is approximately `ceil(log2(number of required values))`. It tells the reader how many bits to extract for a field in each record.

Null and bias handling can affect the final required width. If an extra packed value is needed to represent a null-like state, the writer must reserve enough bit space for that too.

This is one of the practical reasons QVD storage can stay compact. A status field with two values needs one bit per record, not a full byte or repeated text. It is frugal in a way only binary formats and people who keep every cable since 2004 truly understand.

BitOffset lets fields share bytes without wasting space

`BitOffset` is the field's starting bit inside the packed record. Fields are packed consecutively at bit level. They do not need to start on byte boundaries.

Example:

Record bits:
0         4            12   15
|---------|-------------|----|
 Field A     Field B     Field C
 width 4     width 8     width 3

Field A: BitOffset 0,  BitWidth 4
Field B: BitOffset 4,  BitWidth 8
Field C: BitOffset 12, BitWidth 3

Because fields can start in the middle of a byte, the reader must extract arbitrary bit ranges. A byte-oriented read is not enough. You need bit operations that can handle offsets and widths cleanly across byte boundaries.

Bias changes the stored index and matters for nulls

The packed value is not always the final symbol index. Bias must be applied:

SymbolIndex = RawPackedValue + Bias

Bias is important for null handling. In our implementation, a resulting index outside the valid symbol-table range is treated as null or empty. Our writer uses bias `-2` when a field contains null values.

That statement is deliberately phrased as implementation behaviour. It is not presented here as an official Qlik specification. When writing your own reader, handle the metadata in the file you are given, validate the calculated symbol index, and treat out-of-range indexes carefully.

The practical rule is simple: never use the raw packed value directly as the symbol index unless the bias is zero and you have verified the field definition. Assumptions are where ETL jobs go to develop hobbies.

How a QVD record is decoded

A QVD record decoder follows a predictable sequence:

1. Read `RecordByteSize` bytes.
2. For each field, use `BitOffset` and `BitWidth`.
3. Extract the packed integer.
4. Apply `Bias`.
5. Validate the resulting index.
6. Retrieve `SymbolTable[SymbolIndex]`.

Concise pseudocode:

RawValue = ExtractBits(Record, BitOffset, BitWidth)
SymbolIndex = RawValue + Bias

if SymbolIndex is valid:
    Value = Symbols[SymbolIndex]
else:
    Value = NULL

Repeat this for every field in every record and you reconstruct the logical table. The field order, symbol table offsets, bit offsets, bit widths, bias values, and record byte size all need to agree. If one of them is wrong, the decoded rows will look like someone sorted the alphabet with boxing gloves on.

Complete QVD file layout

The full conceptual layout looks like this:

QVD FILE
|-- XML Header
|   |-- Table metadata
|   |-- QvdFieldHeader: Field 1
|   |-- QvdFieldHeader: Field 2
|   |-- QvdFieldHeader: ...
|
|-- 00 terminator
|
|-- Symbol Table Field 1
|-- Symbol Table Field 2
|-- Symbol Table Field 3
|-- ...
|
`-- Bit-Packed Records

That layout is the safest way to reason about a QVD parser. Start with metadata. Use metadata to read symbols. Use metadata and symbols to decode records. Do not jump straight into the binary section and hope the structure announces itself. Binary rarely does small talk.

Why QVD storage is efficient

QVD storage is efficient because it combines several practical ideas:

• Unique values are stored once in per-field symbol tables.
• Records use dictionary indexes rather than repeated values.
• Low-cardinality fields benefit strongly from symbol tables.
• The minimum practical number of bits is used for indexes.
• The record/index section has a fixed size per row.
• Sequential reads are efficient once the header and symbols are loaded.

There is no need to exaggerate this. QVD is efficient because it avoids obvious waste and gives the reader strong metadata. That is enough. Good engineering often looks boring from the outside, which is how you know it might actually work.

For a broader comparison with QVX, see our QVD vs QVX guide. If you need to import QVD into a database, the QVD to SQL Server import guide shows the workflow side.

Implementing a QVD reader means handling four stages

A practical QVD reader has four main stages:

1. Read and parse the XML header.
2. Build field definitions from `QvdFieldHeader` metadata.
3. Load and decode symbol tables for every field.
4. Read and decode bit-packed records.

Each stage depends on the previous one. The XML tells you where the symbols are. The symbols tell you what indexes mean. The bit-packed records tell you which symbols appear in each row.

If you only need a one-off inspection, do not write a reader from scratch. Use Q-Eye or another existing tool. Writing binary parsers for occasional curiosity is how weekends disappear. If the process must run on a schedule, use an ETL tool and keep your weekends for something healthier, like arguing about tabs versus spaces.

Advanced ETL Processor includes QVD support for automated workflows, and Q-Eye is available for manual QVD and QVX file inspection. The Q-Eye product page covers the interactive tool, while Advanced ETL Processor Enterprise is the better fit for scheduled imports, conversions, validation, and reporting workflows.

Conclusion

The most useful way to understand QVD is:

XML metadata
     |
     v
Symbol dictionaries
     |
     v
Bit-packed indexes

The conceptual formula is:

Value = SymbolTable[ExtractBits(Record) + Bias]

That formula hides plenty of careful implementation work, but it captures the core idea. Read the metadata, decode the symbols, extract the packed indexes, apply bias, and reconstruct the values.

Need to read, convert or automate QVD files? Advanced ETL Processor provides visual data transformation and automation tools for QVD, QVX, Excel, CSV, databases, Parquet and many other formats. Use it when QVD processing needs to be repeatable, validated, scheduled, and logged. If you only need to open one file and have a quick look, use Q-Eye first. Sensible tools for sensible jobs. Terrible database jokes available separately, but usually included by accident.