What a flag is
A flag is recorded strictly as(column, category, confidence) with no example
value, and that triple is what propagates downstream into emitted dbt. Detection
runs on column names and aggregate shape, never by inspecting values, so a
column named code full of email addresses is not flagged anywhere in dex.
A flag is never removed by evidence. Value-shape statistics computed during
profiling move its confidence in both directions, and they fail closed: when the
evidence is missing or ambiguous the name-derived confidence stands, because
absence of evidence never weakens a flag. Only a human decision removes a flag,
and only through the config entries below.
Presence versus threshold
This is the distinction that decides every surface’s behavior, and it is why two commands can disagree about the same column without either being wrong. Some consumers act on a flag’s presence at any confidence; others compare its confidence against the blocking threshold.
A weak flag therefore still suppresses min and max and still stamps
meta,
while allowing a query that projects the column. What a weak flag means is the
consumer’s decision, and the consumers differ on purpose.
The stamp is contains_pii: true on the model, and contains_pii: true plus
pii_category: <category> on each flagged column, written under config.meta
because dbt deprecated a top-level meta and dbt Fusion rejects it. The key names
are unchanged from earlier releases, because tools outside dex read them. Readers
accept a stamp in either location, preferring config.meta, so a project
scaffolded by an older dex keeps working without a rewrite.
The threshold
A flag at confidence 0.5 or above blocks projection. The threshold is a hard-coded engine constant, uniform across categories, and deliberately not configurable: a configurable threshold would let a one-line config edit quietly widen the PII boundary. At today’s base confidences everything blocks, so only a flag de-rated by value-shape evidence at profile time falls below. This section is the only place in the repository that states the number.The firewall rule
Output may carry values only from profiled columns whose flag is absent or below the threshold. Every value path from a blocking column must pass through a measuring aggregate (COUNT, APPROX_COUNT_DISTINCT, AVG, SUM, STDDEV, and the like). Value-carrying aggregates (MIN, MAX, ANY_VALUE, STRING_AGG) do not qualify, unknown functions fail closed, andSELECT * is refused when the
expansion includes a blocking column.
Filters, join conditions, GROUP BY and ORDER BY are unrestricted: values flow
in, not out. A count projects no column, which is why a filter over a flagged
column is still attributable.
Projecting a column whose flag sits below the threshold runs, with an envelope
warning naming the column, category, and confidence. Treat that warning as
information for the user, not an error to fix.
Unnested JSON and array outputs inherit the source column’s flags.
The stricter surface
Where a command’s result is the values rather than an aggregate those values slice, the gate moves from threshold to presence and from dropping a column to refusing the command. A metric query returns aggregates that a dimension merely groups, so a flagged dimension can be dropped from the grouping and the query still answers something. Listing a dimension’s values cannot degrade that way, so any flag refuses the command outright. The refusal names both clearing routes. Evidence rules run in both directions: a dimension whose name reads innocuous is refused when its resolved column is flagged, and a profiled, cleared column is not re-blocked by a PII-shaped name. Where a dimension resolves to no physical column, or to a relation that was never profiled, the name heuristic is the fail-closed floor, so silence never clears. A result that ran on the floor alone says so. Which evidence each backend can reach, and why a computed expression resolves to no column, is in �0 and �1.Clearing a flag
Two durable routes, both reviewable in git and both re-applied on every profile so they survive re-profiling:- A
pii_overridesentry in.dex/config.yml. config: {meta: {pii: false}}on the dimension in the project, for a semantic layer.
dex config set pii_overrides '<list>' writes the first route as an
in-place edit, setting the list whole (existing entries included). It answers
with the diff and writes only on --confirm, and that confirmation is the
human’s decision to clear the flag, never an agent’s. Never hand-edit .dex/cache.json to clear a flag, and never
suggest weakening detection: neither survives the next profile, and neither is
reviewable.
A pii_overrides entry takes one of two mutually exclusive shapes, enforced at
load:
explore profile and explore map emit
this warning with the same text, so a scheduled map is enough to catch a
rename that left an override pointing at nothing.
Where the policy is enforced
�0 carries the surfaces that gate a statement: the query firewall, the auto-profile that makes a column adjudicable at all, the seed header gate on values entering git, and row-attribution counts. Profiling’s own suppression and the diagram renderer are in �1,meta stamping on emitted models is in
�3, and the per-backend request gate is in
�4.
Source: references/pii-policy.md (edit on main). This page is generated; change the source, not the site.