This document is for anyone embedding VDB in a product or service, whether through the Python API or by querying the SQLite files directly.
If you are coming from 6.x, read MIGRATING_TO_V7.md first. The database file names, the published image URLs, three environment variables and the version comparison API all changed, and v7 does not read v6 databases.
As a Python library, the only dependency is Python 3.10 or newer. If you query the SQLite files directly, use a SQLite library newer than 3.45.2.
The database is two SQLite files that are always used together.
data.index.vdb7 the search side. Small, random-access, hot.
cve_index: one row per (cve_id, vers, purl_prefix)
cve_metadata, cve_metadata_text: optional metadata
data.vdb7 the payload side. Large, sequential, cold.
cve_source_data: each CVE 5.2 source blob stored once
cve_data: package locator rows referencing those blobs
The split exists because the two have opposite access patterns. You resolve a
package against cve_index first, which is cheap and indexed, and only then
hydrate the matching rows from data.vdb7. Source blobs are stored once and
referenced by hash, so a CVE affecting 300 packages costs one blob rather than
300 copies.
A search against a database with zero rows exits 1 with an explicit empty-database message rather than reporting “no results”. Treat that as scanner down.
A database directory is either a full build or a type shard. A shard’s
vdb.meta declares completeness: "partial" and lists the purl types it
serves.
Five shards are structural: deb, rpm, apk, app (the application
ecosystem union), and cpe, which is the complement holding every purl-prefix
type no other shard claims. The rest are per-type, one for each purl-spec type
except deb, rpm, apk and generic. That gives 43 possible shard names,
and the splitter emits only the ones the build has rows for, which was 26 on a
recent app+OS build. A shard’s own vdb.meta (artifact.types and
siblings.available) is the authority on what exists and what it serves, so do
not hard-code a shard list.
Per-type ecosystem shards deliberately overlap the group shards rather than
partitioning them. The npm shard and the app shard both serve npm. They are
separate views, not a partition, which is why removing a type from cpe would
strand it for a store that only carries the group shards.
This matters most to a direct-SQL integration. On a shard, cve_index holds
rows only for the shard’s types, so a purl of an uncovered type resolves to
nothing, and that nothing means “not checked”, not “clean”. The library
surfaces the distinction: single-purl searches raise ShardCoverageError,
batch searches attach coverage_gap markers and a coverage companion. Plain
SQL surfaces nothing. If you run your own queries, either use a full database
or check coverage through vdb db status --json (missing_group_shards,
per-shard stale) before answering. vdb db status and vdb db refresh are
the supported way to fetch, refresh and inspect both the full image and shards.
Builds can populate two additional tables in the index database:
cve_metadata holds compact package-level metadata for source, severity,
dates, malware state, version range and purl prefix.
cve_metadata_text holds the long CVE-level text once per CVE: aliases,
references, descriptions and affected symbols.
Default public databases leave both empty to keep the index small, and v7
publishes no pre-built metadata artifact. If your integration needs full text,
alias, reference, package-name, symbol, source, severity, date or
malware-aware search, build with vdb --cache --include-metadata (or
--cache-os --include-metadata), or point VDB_DATABASE_URL at an internally
published metadata artifact.
One caveat if you read cve_metadata_text directly: it is keyed on cve_id
alone, so when several feeds describe one CVE, every row for that CVE renders a
single feed’s advisory text. On a recent build, all 1,451 CVEs carried by both
an Alpine and an Azure Linux locator held Azure-only text. Use it for search
and for human context, not as the authoritative description of the specific
row you matched. That row’s own advisory is in data.vdb7.
CREATE TABLE if not exists cve_index(
cve_id TEXT NOT NULL,
type TEXT NOT NULL,
namespace TEXT,
name TEXT NOT NULL,
vers TEXT NOT NULL,
purl_prefix TEXT NOT NULL,
PRIMARY KEY(cve_id, vers, purl_prefix) ON CONFLICT IGNORE
) WITHOUT ROWID
CREATE TABLE if not exists cve_source_data(
source_data_hash TEXT PRIMARY KEY,
source_data BLOB NOT NULL
) WITHOUT ROWID
CREATE TABLE if not exists cve_data(
cve_id TEXT NOT NULL,
type TEXT NOT NULL,
namespace TEXT,
name TEXT NOT NULL,
override_data BLOB,
source_data_hash TEXT NOT NULL,
vers TEXT NOT NULL,
purl_prefix TEXT NOT NULL
)
CREATE TABLE if not exists cve_metadata(
cve_id TEXT NOT NULL,
vers TEXT NOT NULL,
purl_prefix TEXT NOT NULL,
source TEXT NOT NULL,
severity TEXT,
score REAL,
published TEXT,
updated TEXT,
is_malware INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY(cve_id, vers, purl_prefix)
) WITHOUT ROWID
CREATE TABLE if not exists cve_metadata_text(
cve_id TEXT NOT NULL PRIMARY KEY,
aliases TEXT,
reference_text TEXT,
description_text TEXT,
affected_symbols TEXT
) WITHOUT ROWID
ON CONFLICT IGNORE on cve_index means the first writer of a
(cve_id, vers, purl_prefix) wins. Source ingestion order therefore decides
which feed’s row survives when two feeds state the same bound for the same
package. That is deliberate and stable, but it means reordering sources changes
data, not just performance.
Convert the purl string into a purl prefix. In most cases the prefix is
everything before the @, so purl_str.split("@")[0] is enough. The robust
version parses the purl properly:
Set purl_prefix to "pkg:" + purl_obj["type"]. Append the namespace after a
slash if there is one. For Linux distros, if there is a distro_name
qualifier, append that after a slash too. Append the name after a slash.
SELECT cve_id, type, namespace, name, vers, purl_prefix
FROM cve_index
WHERE purl_prefix = ?;
Then check the concrete version against each matched vers. Through the Python
library connection, vers_compare(version, vers) is registered as a SQLite
user-defined function. Outside it, implement equivalent VERS range containment.
Two things will cost you matches if you skip them. Locators are lowercased, so
lowercase your query prefix. And feeds publish some names percent-encoded
(libcrypto%2b%2b, gcc-c%2b%2b, npm scopes as %40scope/name), so probe the
encoded spelling as well as the decoded one. For names outside ASCII there is a
third spelling to probe: the store lowercases the locator after encoding it,
which folds an escape’s hex digits but not the character itself, so a name
containing an uppercase non-ASCII character is stored with that character’s
original-case bytes. Encode the ASCII-only fold as well as the full Unicode
fold. This is not a corner case in the malware feeds, where mixed-script
homoglyph names are the attack.
Parse the CPE to extract vendor, product and version:
import re
CPE_FULL_REGEX = re.compile(
"cpe:?:[^:]+:(?P<cve_type>[^:]+):(?P<vendor>[^:]+):(?P<package>[^:]+):(?P<version>[^:]+):(?P<update>[^:]+):(?P<edition>[^:]+):(?P<lang>[^:]+):(?P<sw_edition>[^:]+):(?P<target_sw>[^:]+):(?P<target_hw>[^:]+):(?P<other>[^:]+)"
)
In cve_index, vendor maps to namespace and product maps to name. VDB uses
a UNION so SQLite can use an index for both the namespace/name and type/name
forms:
SELECT cve_id, type, namespace, name, vers, purl_prefix
FROM cve_index
WHERE namespace = ? AND name = ?
UNION
SELECT cve_id, type, namespace, name, vers, purl_prefix
FROM cve_index
WHERE type = ? AND name = ?;
Be aware of what a CPE-derived row is. A CPE vendor becomes the purl type, so
cpe:2.3:a:f5:big_ip_... is stored as pkg:f5/big-ip. Those locators are
purl-shaped strings that are not registered purl types, and the CPE
components the purl form cannot hold (part, target_sw, edition,
target_hw) are not stored. If you need those as query keys, CPE rows are not
the right source today.
CPE lookups need the full database. The cpe shard holds the complement types,
but a CPE query is not confined to them.
See the vers specification
for the containment algorithm. A vers value in cve_index carries at most
two constraints, one lower and one upper.
Two behaviours to implement rather than discover:
An empty vers matches every version. Records with no version range genuinely
apply to all versions, so treating an empty range as matching nothing
under-reports.
A non-numeric version sorts above numeric ones. This is what stops an npm
@latest dist-tag from matching every advisory with an upper bound. Coercing
it to zero, as VDB 6.x did, put it below every fixed version.
Search cve_index first for matching cve_id, vers and purl_prefix, then
hydrate:
SELECT d.cve_id, d.type, d.namespace, d.name, d.source_data_hash,
json(s.source_data), d.override_data, d.vers, d.purl_prefix
FROM cve_data d
JOIN cve_source_data s ON d.source_data_hash = s.source_data_hash
WHERE d.cve_id = ? AND d.vers = ? AND d.purl_prefix = ?
ORDER BY d.cve_id DESC;
Deduplicate on source_data_hash. The same CVE can match through several
vers and purl prefixes, and those rows share one blob.
For most new integrations, prefer these over direct SQL:
search.search_packages([...])
search.search_packages_batched([...])
search.search_bom_summary(bom)
search.search_bom_detailed(bom)
search.search_by_alias(alias_id)
search.search_by_reference(reference)
search.search_by_package_name(name)
search.search_by_symbol(symbol)
search.search_full_text(query)
They accept filters for severity threshold, source, date ranges, malware-only
and exclude-malware, package scope (app_only, os_only), package ecosystem,
and pagination (page, page_size, max_results).
Purl, CPE, CVE-ID and git-URL lookups all work on a normal full database. On a
type shard, the shard-unsafe APIs (search_by_cpe_like, search_by_cve,
search_by_alias, search_by_reference, search_full_text,
search_by_symbol, latest_malware) raise PartialDatabaseError because they
would silently under-report. Metadata-dependent APIs and filters need populated
cve_metadata rows; on a default database they fail safe by returning no
metadata matches rather than synthesising anything.
Search results are dictionaries. The fields most integrations use:
cve_id the vulnerability identifier
source_data the CVE 5.2 record as a plain dict, ready for json.dumps
vers the version range that matched
fix_version the version the issue is resolved in, when known
purl_prefix the locator the row is stored under
matched_by which candidate spelling matched
severity, score, published, updated, is_malware from cve_metadata
source_data is a dict. It is not a Pydantic model and has no .model_dump().
Snippets from the 6.x era that call res["source_data"].model_dump(mode="json")
raise AttributeError on v7.
For large SBOMs, search_by_cdx_bom returns a generator of batches so peak
memory stays bounded:
for batch in search.search_by_cdx_bom("bom.json", with_data=True):
for res in batch:
...
Set VDB_HOME before importing vdb.lib.config. Path and build-scope
variables are read at import time, so setting them afterwards in a long-running
process has no effect.
For read-only deployments where the .vdb7 files are not modified while the
process runs, set VDB_SQLITE_IMMUTABLE=true to open search databases with
SQLite’s immutable URI option.
Point VDB_DATABASE_URL or VDB_APP_ONLY_DATABASE_URL at artifacts your own
workflow produces, so your application instances do not depend on a third-party
refresh job for security-relevant data.