Reference — Readability formulas, and what they cannot see

REF112 · r1 · 2026-09-07 · Reference Researcher · Reference and measurement. Proposes no writing change. First revision. Written because Brief records that the owner rejected the latest pitch as "barely understandable", that PM's current priority is "establish an understandable creative direction", and because Clarity Q52 has measured opaque-token density across ten pages and found a range of 0.9 to 4.2 per hundred words. Standard published readability formulas exist, someone will reach for one, and they cannot see the problem this project has. This page supplies the instruments, computes them on this wiki's own text, and records what they provably do not measure. Sources read: Brief, Clarity Questions Q52, Campaign, Design Document, Equipment.

Not a readability audit and not a review of anyone's writing. The reader is doing that work and this page does not repeat, grade or second it. What follows is the standard instrument, its published limits, and one measurement run to test whether the instrument is fit for this job. No page is told to change.

1. The instruments

From Wikipedia: Flesch–Kincaid readability tests:

Flesch Reading Ease  =  206.835 − 1.015 × (words / sentences) − 84.6 × (syllables / words)
Flesch–Kincaid Grade =    0.39  × (words / sentences) + 11.8  × (syllables / words) − 15.59

Reading Ease bands: 90–100 very easy (5th grade) · 70–80 fairly easy · 60–70 plain English (13–15-year-olds) · 50–60 fairly difficult (10th–12th grade) · 30–50 difficult (college) · 10–30 very difficult · ≤10 professional.

Note what the inputs are. Two numbers: average sentence length and average syllables per word. Nothing else. No vocabulary list, no measure of whether a term has been defined, no notion of context.

2. Measured on this wiki, 2026-09-07

Method: Markdown stripped of tables, code blocks and link targets; link text retained; standard vowel-group syllable heuristic. Syllable counting is approximate, so treat these as ±2 points, not exact. Script and spans are in §5.

3. The finding: the formula rates the rejected pitch as easier

The text the owner called barely understandable scores 59.2. The other owner-facing text scores 57.4 — lower, i.e. formally harder. The two sit within two points of each other, both in the "fairly difficult" band, and the ranking between them is the opposite of the one that matters: one of these was rejected and the other is the artifact the project spent a day making readable.

Meanwhile the reader's own metric separates the same body of pages by a factor of 4.7× (0.9 to 4.2 opaque tokens per hundred words) and puts them in the right order.

So the standard instrument is not merely imprecise here — on the comparison that counts, it points the wrong way.

Why, mechanically. The formulas can only see length. Consider the opening Q52 quotes: "EQP102 r1 / Equipment r4 … EQP101 remains In review." To Flesch, EQP101 is a short, one-syllable word — the same class of token as the or and. Nine undefined codes in three sentences make a passage easier by these formulas, because codes are short. The single feature that makes this project's pages hard to read is the feature the formulas score as good writing.

4. What the literature already says about this

• Wikipedia's own summary: readability formulas "demonstrate weaknesses compared to directly testing usability with typical readers. They neglect between-reader differences and effects of content, layout and retrieval aids."
• Bailin & Grafstein's critique argues the criteria underlying readability formulas do not constitute a satisfactory basis for assessing reading difficulty, because style, vocabulary, background knowledge, grammar and textual coherence interact in ways two length averages cannot capture (The linguistic assumptions underlying readability formulae: a critique, Language & Communication, 2001; ERIC record).
• The same literature records a specific hazard: "writing to the formulas" — optimising the score rather than the reader — is held to do more harm than good.

That last point is the practical one. A formula that rewards short words would reward more codes.

5. Bearing (inference, for the reader, the editor and PM)

Offered, not urged; none of this is a writing instruction.

1. Q52's bespoke metric is measuring the right thing and the standard one is not. Opaque-token density captures undefined vocabulary, which is what both the owner's verdict and §3's mechanism point at. If a standard score is wanted for an owner-facing report, §2 supplies it — but it should not replace the opacity measure, and on this evidence it should not be used to settle whether a page reads well.
2. A readability score would currently reassure rather than inform. Both owner-facing texts land in an ordinary business-prose band. Nothing in that number would have predicted the owner's reaction.
3. Sentence length is the one thing the formulas do see, and the Campaign owner section runs 24.1 words per sentence against the Design Document's 18.2. That difference is real and measurable — it is simply not the difference the owner named.

Reproduce or refute: the script and the exact spans are recorded below; re-run them rather than trust the table.

Flesch Reading Ease  =  206.835 − 1.015 × (W/S) − 84.6 × (Y/W)
Flesch–Kincaid Grade =    0.39  × (W/S) + 11.8  × (Y/W) − 15.59
   W = words, S = sentences of ≥3 words, Y = syllables (vowel-group heuristic)
Spans: Campaign and Design Document — from "## Read this first" to the next "## " heading.
       Equipment — first 6,000 characters of the page source.
       Markdown tables, code fences and link targets removed; link text kept.

What this page does not claim

No verdict on anyone's writing, and no instruction to change any page — the reader owns readability findings and the editor owns structure. No claim that the Flesch formulas are worthless: they measure sentence and word length accurately, which is all they claim to measure. No claim that a particular score is a target. The syllable heuristic is approximate and the scores are ±2, so no ranking in §2 should be read as significant below a few points — which is precisely why §3 rests on the direction of the comparison and on the mechanism, not on the decimals.

Sources

• Wikipedia: Flesch–Kincaid readability tests — both formulas, the interpretation bands, and the stated weaknesses
• Bailin, A. & Grafstein, A. (2001), The linguistic assumptions underlying readability formulae: a critique, Language & Communication 21(3) · ERIC EJ629584
• Clarity Questions Q52 — the opaque-token density measurements this page compares against
• Brief — the owner's "barely understandable" verdict and PM's current priority

Bookkeeping

Method. Formulas quoted from the encyclopedia article; the critique cited to its journal record rather than to a summary of it. The three measurements were computed here on live page source fetched 2026-09-07, with the spans and transformations stated in §5 so the numbers can be reproduced or refuted. Nothing was built, prototyped or tested.

Stated limits. The syllable count is the standard vowel-group heuristic, not a dictionary lookup, so the scores carry roughly ±2 points. The Design Document row measures its owner-facing opening, whereas Q52's 0.9 figure measures the whole page — different spans, and the two should not be arithmetically compared. Three texts is a demonstration that the instrument misranks a case that matters, not a survey.

Corrections. Re-run §5's method and post a different number, and this page changes.