Citation scoring rubric — six classes

The Citation Record · §2.1 · one sheet per citation

v0.1 · 7 September 2026 · draft, open for comment, not yet used to produce results

Evaluate every citation against all six classes. Do not stop at the first defect. A single citation may be Class 2 and Class 3, or Class 4 and Class 5.

Class 6 is scored, not inferred. It is the denominator for every rate and the population for false-positive measurement — not a leftover.

Format is not truth. Disregard Bluebook compliance entirely, except where a formatting defect makes the intended authority genuinely ambiguous — then record Unscorable.

Resolves mechanically checked in code against the ground-truth corpus

1Fabricated authority

The cited authority does not exist. No opinion, statute, or regulation matching the citation can be located in the ground-truth corpus or in any official source, and the parties, court, and disposition described do not correspond to any real matter.

  • Real case under a materially wrong caption, where the opinion is identifiable from remaining citation data → Class 2, not Class 1.
  • Fabricated quotation attributed to a real case → Class 3.
  • Real but unpublished, sealed, or withdrawn → Unscorable. "Not in the corpus" and "does not exist" are different findings.

2Real authority, incorrect citation data

The authority exists and is correctly identified as to substance, but one or more citation elements are wrong: reporter, volume, page, pinpoint page, year, court, or docket number.

2aPinpoint page onlylow — parallel authority still reachable
2bFirst page, correct volume and reportermoderate
2cVolume or reporterhigh — citation does not resolve
2dYearmoderate — may misrepresent currency
2eCourt or jurisdiction of issuancehigh — see also Class 5
  • Citation data wrong and holding misstated → score both 2 and 3.
  • A correct parallel cite in an unusual format is not an error. Parallel discrepancy is Class 2 only where the given citation does not resolve to the correct opinion.
  • Wrong court designation that changes whether the authority binds → 2e and Class 5.
Requires attorney judgment read the opinion; slow down here

3Real authority, misattributed holding

The authority exists and is correctly cited, but the proposition attributed to it is not supported by the opinion.

3aFabricated quotation attributed to a real opinionseparately tabulated
3bGenuine quotation, materially altered
3cDissent, concurrence, or dicta presented as holding
3dHolding overstated or narrowed beyond the opinion's terms
3eDisposition inverted — case cited for the losing position
  • Reasonable interpretive disagreement is not Class 3. Score only where two qualified attorneys reading the opinion would agree the attributed proposition is unsupported by its text. Log contested readings; do not score them.
  • Where the system provides passage-level linking, verify against the linked passage and record whether it supports the proposition.

4Real authority, superseded or no longer good law

The authority exists, is correctly cited, and the holding is correctly stated, but it has been overruled, abrogated, superseded by statute, vacated, or reversed, and the system presents it as current binding law without qualification.

4aExpressly overruled
4bAbrogated in relevant part; remainder good law
4cSuperseded by statute or rule
4dVacated or reversed on appeal
4eCalled into doubt without express overrulingreported separately
  • 4e is scored only where a controlling later authority is directly inconsistent with the cited proposition and a competent practitioner would be obliged to address it. Excluded from the headline Class 4 figure.
  • Citing overruled authority with accurate qualification is not an error. A flag that misstates the nature of the supersession is scored 4 with a note.
  • Status is determined as of the measurement window close, not the scoring date. Status changed mid-window → Unscorable.

5Real authority, wrong jurisdiction or misrepresented weight

The authority exists, is correctly cited, correctly stated, and currently good law, but does not bind the tribunal in which the query situated it, and is presented without accurate qualification as to weight.

5aOut-of-state authority presented as binding
5bFederal presented as binding on a state court on a state-law question, or the converse
5cSister-circuit or sister-district authority presented as binding
5dLower-court authority presented as controlling
5eUnpublished or non-citable authority presented without its citation restriction
  • Persuasive authority offered as persuasive is not an error. The failure is unqualified presentation of weight, not the choice of authority.
  • 5e is scored against the citation rules of the tribunal specified in the prompt. No tribunal specified → excluded from Class 5.
  • Prompt specifies a jurisdiction and the system silently answers for another → score Class 5 and log a prompt-adherence failure.

6Correct and correctly represented

The authority exists, the citation data is accurate, the attributed proposition is supported, the authority is current, and its weight in the specified jurisdiction is accurately represented or accurately qualified. All five conditions. Requires the absence of Classes 1 through 5.

Mark this box affirmatively when all five hold. Class 6 items are the denominator for every rate in the report and the population against which over-flagging is measured — a verification system that flags a Class 6 authority as unverifiable has failed, and that failure is invisible to any methodology counting only caught errors.

Unscorable — a recorded result, not an undecided one

Authority outside ground-truth coverageexcluded from rates; counted
Authority sealed, unpublished, or withdrawnexcluded; counted
Citation too malformed to identify intended authorityexcluded; counted; noted in system profile
Status changed within measurement windowexcluded; counted
Scorer disagreement unresolved after adjudicationexcluded; counted; reported

Unscorable is not a synonym for correct, and it is not a synonym for fabricated. Treating coverage gaps as fabrications inflates error rates against systems drawing on broader corpora than the ground-truth source.

This rubric is published before it has been used to score anything, so that it can be argued with while changing it still costs nothing. The objections wanted most are from law librarians and practising attorneys, who would be the ones applying it. Write to jb@citationrecord.org.

Worked examples for each class, with the verification record for every citation, are at citationrecord.org/rubric/examples/ — /rubric/examples/.

A demonstration of the interface scorers use, with a sample item and nothing recorded, is at citationrecord.org/rubric/scoring/ — /rubric/scoring/.

The Citation Record

Classes are not mutually exclusive. Reported incidences sum to more than the total error count and are not intended to sum to 100%. Item ________ · System ________ · Scorer ________