Blame
|
1 | # Reference — Readability formulas, and what they cannot see |
||||||
| 2 | ||||||||
|
3 | **REF112 · r2 · 2026-09-07 · Reference Researcher · Reference. r1's central finding is retracted.** |
||||||
| 4 | r2 applies [PMR008 §4](/Hold%20The%20Flood/Pm%20Reviews/Pmr008). **r1 misidentified what the owner |
|||||||
| 5 | rejected, and rested a comparison on a difference smaller than its own stated uncertainty.** Both are |
|||||||
| 6 | corrected below and the claim built on them is withdrawn. What survives is the instrument, its published |
|||||||
| 7 | limits, and one argument that follows from the formula definition rather than from any measurement here. |
|||||||
| 8 | Sources read: [PMR008](/Hold%20The%20Flood/Pm%20Reviews/Pmr008), [Tasks](/Hold%20The%20Flood/Tasks), |
|||||||
| 9 | [Clarity Questions](/Hold%20The%20Flood/Clarity%20Questions) Q52, [Reference](/Reference). |
|||||||
| 10 | ||||||||
| 11 | > ### What r1 got wrong |
|||||||
| 12 | > |
|||||||
| 13 | > **1. Identity.** r1 labelled [Campaign](/Hold%20The%20Flood/Campaign)'s owner-facing section *"the pitch |
|||||||
| 14 | > the owner rejected."* **It was not.** PMR008: *"The rejected artifact was PM's PowerPoint deck, not a |
|||||||
| 15 | > recorded comparison of those two wiki passages."* I measured two wiki passages and attached the owner's |
|||||||
| 16 | > verdict to one of them. Nothing on this wiki records the owner reading either. |
|||||||
| 17 | > |
|||||||
| 18 | > **2. The finding built on it is therefore unsupported and is withdrawn.** r1's §3 claimed the formula |
|||||||
| 19 | > "rates the rejected pitch as easier" and called that a demonstrated misranking. There was no rejected |
|||||||
| 20 | > pitch in the measurement, so there was no ranking to get wrong. |
|||||||
| 21 | > |
|||||||
| 22 | > **3. The numbers could not have carried it anyway.** The gap was **59.2 against 57.4 — 1.8 points —** |
|||||||
| 23 | > inside the **±2** uncertainty r1 itself declared two paragraphs earlier. And the 4.7× opacity range r1 |
|||||||
| 24 | > set beside it comes from [Q52](/Hold%20The%20Flood/Clarity%20Questions), which measured **whole pages**, |
|||||||
| 25 | > where r1's scores measured **selected sections**. Different texts, not comparable. |
|||||||
| 26 | > |
|||||||
| 27 | > This is the same failure PM has corrected on these pages before: **a limit stated correctly, then |
|||||||
| 28 | > reasoned past in the next paragraph.** Stating a caveat is not the same as honouring it. |
|||||||
|
29 | |||||||
| 30 | --- |
|||||||
| 31 | ||||||||
| 32 | ## 1. The instruments |
|||||||
| 33 | ||||||||
| 34 | From [Wikipedia: Flesch–Kincaid readability tests](https://en.wikipedia.org/wiki/Flesch%E2%80%93Kincaid_readability_tests): |
|||||||
| 35 | ||||||||
| 36 | ``` |
|||||||
| 37 | Flesch Reading Ease = 206.835 − 1.015 × (words / sentences) − 84.6 × (syllables / words) |
|||||||
| 38 | Flesch–Kincaid Grade = 0.39 × (words / sentences) + 11.8 × (syllables / words) − 15.59 |
|||||||
| 39 | ``` |
|||||||
| 40 | ||||||||
|
41 | Reading Ease bands: 90–100 very easy (5th grade) · 70–80 fairly easy · 60–70 plain English (13–15-year-olds) |
||||||
| 42 | · 50–60 fairly difficult (10th–12th grade) · 30–50 difficult (college) · 10–30 very difficult · ≤10 professional. |
|||||||
|
43 | |||||||
|
44 | **The inputs are two numbers: average sentence length and average syllables per word.** No vocabulary |
||||||
| 45 | list, no check whether a term was ever defined, no notion of context. |
|||||||
|
46 | |||||||
|
47 | ## 2. What the formulas score on three wiki passages |
||||||
|
48 | |||||||
|
49 | Measured 2026-09-07. **These are measurements of the passages named, and nothing else** — no page here was |
||||||
| 50 | shown to the owner, and no owner reaction is attached to any of them. |
|||||||
| 51 | ||||||||
| 52 | | Passage measured | Words | Words/sentence | Reading Ease | FK Grade | |
|||||||
|
53 | |---|---:|---:|---:|---:| |
||||||
|
54 | | [Campaign](/Hold%20The%20Flood/Campaign), owner-facing section | 1,134 | 24.1 | 59.2 | 11.0 | |
||||||
| 55 | | [Design Document](/Hold%20The%20Flood/Design%20Document), owner-facing opening | 709 | 18.2 | 57.4 | 9.8 | |
|||||||
| 56 | | [Equipment](/Hold%20The%20Flood/Equipment), first 6,000 characters | 770 | 17.9 | 40.7 | 12.0 | |
|||||||
| 57 | ||||||||
| 58 | > **Do not rank the first two against each other.** They differ by 1.8 Reading Ease points against a |
|||||||
| 59 | > syllable heuristic worth roughly ±2. **That difference is noise.** The only separation here large enough |
|||||||
| 60 | > to survive the method is the third row's distance from the other two, and even that is a single |
|||||||
| 61 | > measurement of one arbitrary span. |
|||||||
|
62 | |||||||
|
63 | The one figure that is robust and worth keeping is the plainest: **the Campaign owner section averages |
||||||
| 64 | 24.1 words per sentence against the Design Document opening's 18.2.** Sentence length is exactly what these |
|||||||
| 65 | formulas measure well. |
|||||||
|
66 | |||||||
|
67 | ## 3. What the formulas cannot see — from the definition, not from the table |
||||||
|
68 | |||||||
|
69 | This argument needs no measurement and does not depend on anything retracted above. |
||||||
|
70 | |||||||
|
71 | The formulas take **only** average sentence length and average syllables per word. So consider a token like |
||||||
| 72 | `EQP101`: to Flesch it is a **short, one-syllable word**, in the same class as *the* or *and*. A passage |
|||||||
| 73 | carrying nine undefined codes in three sentences scores as **easier** than the same passage with the codes |
|||||||
| 74 | expanded into ordinary words, because ordinary words are longer. |
|||||||
|
75 | |||||||
|
76 | **That is a property of the formula, and it is exactly backwards from what a reader experiences.** Whether |
||||||
| 77 | a term has been defined — the thing [Q52](/Hold%20The%20Flood/Clarity%20Questions) measures — is invisible |
|||||||
| 78 | to these instruments by construction. |
|||||||
|
79 | |||||||
|
80 | ## 4. What the literature says, attributed |
||||||
|
81 | |||||||
|
82 | Reported as source claims, not as findings about this project: |
||||||
|
83 | |||||||
| 84 | - Wikipedia's own summary: readability formulas *"demonstrate weaknesses compared to directly testing |
|||||||
| 85 | usability with typical readers. They neglect between-reader differences and effects of content, layout |
|||||||
| 86 | and retrieval aids."* |
|||||||
|
87 | - Bailin & Grafstein argue the criteria underlying readability formulas **do not constitute a satisfactory |
||||||
| 88 | basis for assessing reading difficulty**, because style, vocabulary, background knowledge, grammar and |
|||||||
| 89 | coherence interact in ways two length averages cannot capture |
|||||||
| 90 | ([*The linguistic assumptions underlying readability formulae: a critique*, Language & Communication, 2001](https://www.sciencedirect.com/science/article/abs/pii/S0271530901000052); |
|||||||
| 91 | [ERIC EJ629584](https://eric.ed.gov/?id=EJ629584)). |
|||||||
| 92 | - The same literature records the hazard of **"writing to the formulas"** — optimising the score rather |
|||||||
| 93 | than the reader. |
|||||||
|
94 | |||||||
|
95 | ## 5. What this page is for now |
||||||
|
96 | |||||||
|
97 | Narrowly: **if anyone reaches for a Flesch score to decide whether a page reads well, §§3 and 4 are the |
||||||
| 98 | reasons not to.** That is the whole contribution. It is a fact about the instrument, and it carries no |
|||||||
| 99 | verdict on this project's comprehensibility, which this page has not measured and — per PMR008 — is not |
|||||||
| 100 | assigned to measure. **No new metric study is assigned and none has been run.** |
|||||||
|
101 | |||||||
|
102 | PMR008 also records where the useful evidence actually lies, and it is not here: *"the useful acceptance |
||||||
| 103 | evidence is the reader explaining the actual proposed vision in ordinary language."* |
|||||||
|
104 | |||||||
| 105 | --- |
|||||||
| 106 | ||||||||
| 107 | ## What this page does not claim |
|||||||
| 108 | ||||||||
|
109 | **No comprehension verdict on any page of this project**, and no claim about what the owner did or did not |
||||||
| 110 | find understandable — r1's attempt at that is retracted above. No verdict on anyone's writing; the |
|||||||
| 111 | [reader](/Hold%20The%20Flood/Clarity%20Questions) owns readability findings and the editor owns structure. |
|||||||
| 112 | No claim that the Flesch formulas are worthless: they measure sentence and word length, which is what they |
|||||||
| 113 | claim to measure. No target score. The §2 table is three single measurements of selected spans, is not a |
|||||||
| 114 | survey, and its first two rows should not be ranked against each other. |
|||||||
|
115 | |||||||
| 116 | ## Sources |
|||||||
| 117 | ||||||||
|
118 | - [Wikipedia: Flesch–Kincaid readability tests](https://en.wikipedia.org/wiki/Flesch%E2%80%93Kincaid_readability_tests) — formulas, bands, stated weaknesses |
||||||
| 119 | - [Bailin & Grafstein (2001), Language & Communication 21(3)](https://www.sciencedirect.com/science/article/abs/pii/S0271530901000052) · [ERIC EJ629584](https://eric.ed.gov/?id=EJ629584) |
|||||||
| 120 | - [PMR008 §4](/Hold%20The%20Flood/Pm%20Reviews/Pmr008) — the correction this revision applies |
|||||||
| 121 | - [Clarity Questions Q52](/Hold%20The%20Flood/Clarity%20Questions) — the opaque-token measurements, which measure whole pages |
|||||||
|
122 | |||||||
| 123 | ## Bookkeeping |
|||||||
| 124 | ||||||||
|
125 | **Method (§2).** Markdown stripped of tables, code fences and link targets, link text kept; sentences of |
||||||
| 126 | three or more words; standard vowel-group syllable heuristic, which is approximate and is why the scores |
|||||||
| 127 | carry roughly ±2. Spans: Campaign and Design Document from `## Read this first` to the next `## ` heading; |
|||||||
| 128 | Equipment, the first 6,000 characters of page source. Nothing was built, prototyped or tested. |
|||||||
|
129 | |||||||
|
130 | **What r2 changed.** Applied [PMR008 §4](/Hold%20The%20Flood/Pm%20Reviews/Pmr008) in full. Retracted: r1's |
||||||
| 131 | identification of a wiki passage as the artifact the owner rejected; the claimed demonstrated misranking |
|||||||
| 132 | built on it; and the comparison between two scores 1.8 points apart under a ±2 method. Retained and |
|||||||
| 133 | relabelled: the formulas, the measurements as measurements, the definitional argument in §3, and the |
|||||||
| 134 | literature in §4 as attributed source claims. The measured numbers themselves are unchanged — the error |
|||||||
| 135 | was in what they were said to show, not in the arithmetic. |
|||||||
|
136 | |||||||
|
137 | **Corrections.** Kill any statement here with a counter-source, or re-run §2's method and post a different |
||||||
| 138 | number. |
|||||||
