Version: 0.4.0 (draft)
Author: Lovro Lucic
License: CC-BY-4.0
Frame Check is a small deterministic tool. It reads one English analytical
document and reports what perspectives the document takes, how confidently it
speaks, and whether its numbers hold up. No language model judges the text;
the same input always produces the same output. This page explains how each
part works and where it stops being reliable. It describes a tool, not a
research program. Where a claim here has been tested and failed, that is said
plainly.
Frame Check has two parts.
Framing analysis (deterministic, no model calls). Pattern matchers
compute, for the document:
densely (per 1,000 words). The five are causes, risks, stakeholders,
trends, and uncertainty.
present, and the future.
descriptive, or advisory.
of hedged to unhedged claims.
one-line headline.
Verification floor (data sources, no model calls). Numeric claims are
checked against authoritative data where coverage exists (see section 3).
Identical inputs return identical measurements. Nothing on this path calls a
language model.
Frame Check ships a catalog of about twenty named framing patterns, each with
a stable ID (FVS-NNN), detection cues, counter-examples, a worked example, and
a question worth asking. The catalog and its rules are bundled with the tool
and served as MCP resources.
A match means a rule-based signal fired. It is a hypothesis for you to check,
not a verdict, so every match is delivered with a question rather than a
conclusion. The rules are identification-only: they flag where a pattern may
be present and never rewrite or grade the text. They are pinned and guarded by
tests so a change to the engine cannot silently shift what a match means.
Frame Check also reports divergence: the catalog entries the document did not
trigger, sorted by signal strength. These absences are often more useful than
the matches, because they point at perspectives the writer did not take.
Numeric claims are matched against data providers that have genuine coverage
for the claim type: SEC EDGAR (US public-company filings), FRED (US
macroeconomics), World Bank (country statistics), Alpha Vantage (equities),
Wolfram Alpha (computational and physical facts), and a few others, with web
search as a last-resort fallback.
This layer is deliberately bounded. Coverage is strong for US public-company
financials and macroeconomic indicators and close to zero for medical, legal,
and niche-industry claims. In one calibration corpus, about 86 percent of
claims were not verifiable against any available source. Frame Check reports
per-provider precision, recall, and F1 rather than asserting accuracy, and it
marks a claim unverifiable instead of guessing.
Frame Check does not judge reasoning quality, logical validity, insight, or
whether a document is useful for your purpose. Those need a human reader.
It does not detect semantic manipulation. A document can pass every structural
check and still mislead through word choice, selective emphasis, or a false
premise. Structural analysis is a floor to stand on, not a verdict.
The named-pattern layer is the weakest part, and the honest record says so.
When the rule-based detector was compared against independent human labeling on
a mixed-genre corpus, agreement was near chance (macro-F1 of 0.157; a later
rule revision reached about 0.27, still low). So a frame match should move you
to look closer, never to conclude. The full study lives in
`fvs_eval/validation_study/`, and `VALIDATION_PROGRAM.md` and
`ANTICIPATED_CRITIQUES.md` keep the running account of what is and is not
established.
One thing does hold up under a controlled check: the same content rewritten
from a different frame produces a measurably different structural profile. The
structural signals track framing even though the named-pattern labels do not
yet match human readers well.
Code is Apache-2.0. The frame vocabulary, this document, and the worked
examples are CC-BY-4.0, so they can be cited and reused.
Lucic, L. (2026). Frame Check: a tool for seeing the frame in
documents and checking their claims. https://frame.clarethium.com
See `CITATION.cff` for the canonical record.
The frame vocabulary v0.2.0 catalogs the structural framing patterns this methodology detects. 16 entries are published and curated; 4 were withdrawn during review and are documented with disposition notes at the library index.