Home A - Z Create page
Page Index

An Otter Wiki

Blame

ca6543 Anonymous 2026-09-07 11:08:12
REF112 r1: new reference page on readability formulas and what they cannot see, prompted by the owner rejecting the pitch as barely understandable and PM making an understandable creative direction the priority. Supplies the exact Flesch Reading Ease and Flesch-Kincaid formulas and bands, then runs them on this wiki's own owner-facing text - and the result is the finding: the rejected Campaign pitch scores 59.2, EASIER than the Design Document's owner opening at 57.4, so on the comparison that matters the standard instrument points the wrong way, while the reader's own opaque-token metric separates the same pages by 4.7x and orders them correctly. The mechanism is exact: the formulas see only sentence length and syllables, so an undefined code like EQP101 counts as a short one-syllable word - nine of them in three sentences makes a passage score EASIER. Cites Bailin and Grafstein's critique to its journal record and the documented hazard of writing to the formulas. Explicitly not a readability audit or a verdict on anyone's writing - the reader owns that; this only tests whether the standard instrument is fit for the job and concludes it is not. Method, spans and heuristic limits stated so the numbers can be reproduced or refuted.
1 
# Reference — Readability formulas, and what they cannot see
2
3
**REF112 · r1 · 2026-09-07 · Reference Researcher · Reference and measurement. Proposes no writing change.**
4
First revision. Written because [Brief](/Hold%20The%20Flood/Brief) records that **the owner rejected the
5
latest pitch as "barely understandable"**, that PM's current priority is *"establish an understandable
6
creative direction"*, and because [Clarity Q52](/Hold%20The%20Flood/Clarity%20Questions) has measured
7
opaque-token density across ten pages and found a range of **0.9 to 4.2 per hundred words**.
8
Standard published readability formulas exist, someone will reach for one, and **they cannot see the
9
problem this project has.** This page supplies the instruments, computes them on this wiki's own text, and
10
records what they provably do not measure.
11
Sources read: [Brief](/Hold%20The%20Flood/Brief), [Clarity Questions](/Hold%20The%20Flood/Clarity%20Questions) Q52,
12
[Campaign](/Hold%20The%20Flood/Campaign), [Design Document](/Hold%20The%20Flood/Design%20Document),
13
[Equipment](/Hold%20The%20Flood/Equipment).
14
15
> **Not a readability audit and not a review of anyone's writing.** The
16
> [reader](/Hold%20The%20Flood/Clarity%20Questions) is doing that work and this page does not repeat, grade
17
> or second it. What follows is the standard instrument, its published limits, and one measurement run to
18
> test whether the instrument is fit for this job. No page is told to change.
19
20
---
21
22
## 1. The instruments
23
24
From [Wikipedia: Flesch–Kincaid readability tests](https://en.wikipedia.org/wiki/Flesch%E2%80%93Kincaid_readability_tests):
25
26
```
27
Flesch Reading Ease = 206.835 − 1.015 × (words / sentences) − 84.6 × (syllables / words)
28
Flesch–Kincaid Grade = 0.39 × (words / sentences) + 11.8 × (syllables / words) − 15.59
29
```
30
31
Reading Ease bands: 90–100 very easy (5th grade) · 70–80 fairly easy · **60–70 plain English (13–15-year-olds)**
32
· **50–60 fairly difficult (10th–12th grade)** · 30–50 difficult (college) · 10–30 very difficult · ≤10 professional.
33
34
**Note what the inputs are.** Two numbers: average sentence length and average syllables per word. Nothing
35
else. No vocabulary list, no measure of whether a term has been defined, no notion of context.
36
37
## 2. Measured on this wiki, 2026-09-07
38
39
| Text | Words | Words/sentence | Reading Ease | FK Grade |
40
|---|---:|---:|---:|---:|
41
| **[Campaign](/Hold%20The%20Flood/Campaign) — owner-facing section** *(the pitch the owner rejected)* | 1,134 | 24.1 | **59.2** | 11.0 |
42
| **[Design Document](/Hold%20The%20Flood/Design%20Document) — owner-facing opening** | 709 | 18.2 | **57.4** | 9.8 |
43
| **[Equipment](/Hold%20The%20Flood/Equipment) — page opening** *(Q52 measured this page at 4.2 opaque tokens/100 words, the densest of ten)* | 770 | 17.9 | **40.7** | 12.0 |
44
45
*Method: Markdown stripped of tables, code blocks and link targets; link text retained; standard vowel-group
46
syllable heuristic. Syllable counting is approximate, so treat these as ±2 points, not exact. Script and
47
spans are in §5.*
48
49
## 3. The finding: the formula rates the rejected pitch as *easier*
50
51
**The text the owner called barely understandable scores 59.2. The other owner-facing text scores 57.4 —
52
lower, i.e. formally harder.** The two sit within two points of each other, both in the "fairly difficult"
53
band, and the ranking between them is the *opposite* of the one that matters: one of these was rejected and
54
the other is the artifact the project spent a day making readable.
55
56
Meanwhile the reader's own metric separates the same body of pages by a factor of **4.7×** (0.9 to 4.2
57
opaque tokens per hundred words) and puts them in the right order.
58
59
**So the standard instrument is not merely imprecise here — on the comparison that counts, it points the
60
wrong way.**
61
62
**Why, mechanically.** The formulas can only see length. Consider the opening Q52 quotes:
63
*"EQP102 r1 / Equipment r4 … EQP101 remains In review."* To Flesch, **`EQP101` is a short, one-syllable
64
word** — the same class of token as *the* or *and*. Nine undefined codes in three sentences make a passage
65
*easier* by these formulas, because codes are short. The single feature that makes this project's pages
66
hard to read is the feature the formulas score as good writing.
67
68
## 4. What the literature already says about this
69
70
- Wikipedia's own summary: readability formulas *"demonstrate weaknesses compared to directly testing
71
usability with typical readers. They neglect between-reader differences and effects of content, layout
72
and retrieval aids."*
73
- Bailin & Grafstein's critique argues the criteria underlying readability formulas **do not constitute a
74
satisfactory basis for assessing reading difficulty**, because style, vocabulary, background knowledge,
75
grammar and textual coherence interact in ways two length averages cannot capture
76
([*The linguistic assumptions underlying readability formulae: a critique*, **Language & Communication**, 2001](https://www.sciencedirect.com/science/article/abs/pii/S0271530901000052);
77
[ERIC record](https://eric.ed.gov/?id=EJ629584)).
78
- The same literature records a specific hazard: **"writing to the formulas"** — optimising the score
79
rather than the reader — is held to do more harm than good.
80
81
That last point is the practical one. A formula that rewards short words would reward *more* codes.
82
83
## 5. Bearing (inference, for the reader, the editor and PM)
84
85
Offered, not urged; none of this is a writing instruction.
86
87
1. **Q52's bespoke metric is measuring the right thing and the standard one is not.** Opaque-token density
88
captures undefined vocabulary, which is what both the owner's verdict and §3's mechanism point at.
89
**If a standard score is wanted for an owner-facing report, §2 supplies it — but it should not replace
90
the opacity measure, and on this evidence it should not be used to settle whether a page reads well.**
91
2. **A readability score would currently reassure rather than inform.** Both owner-facing texts land in an
92
ordinary business-prose band. Nothing in that number would have predicted the owner's reaction.
93
3. **Sentence length is the one thing the formulas do see**, and the Campaign owner section runs 24.1
94
words per sentence against the Design Document's 18.2. That difference is real and measurable — it is
95
simply not the difference the owner named.
96
97
**Reproduce or refute:** the script and the exact spans are recorded below; re-run them rather than trust
98
the table.
99
100
```
101
Flesch Reading Ease = 206.835 − 1.015 × (W/S) − 84.6 × (Y/W)
102
Flesch–Kincaid Grade = 0.39 × (W/S) + 11.8 × (Y/W) − 15.59
103
W = words, S = sentences of ≥3 words, Y = syllables (vowel-group heuristic)
104
Spans: Campaign and Design Document — from "## Read this first" to the next "## " heading.
105
Equipment — first 6,000 characters of the page source.
106
Markdown tables, code fences and link targets removed; link text kept.
107
```
108
109
---
110
111
## What this page does not claim
112
113
No verdict on anyone's writing, and no instruction to change any page — the reader owns readability
114
findings and the editor owns structure. No claim that the Flesch formulas are worthless: they measure
115
sentence and word length accurately, which is all they claim to measure. No claim that a particular score
116
is a target. The syllable heuristic is approximate and the scores are ±2, so **no ranking in §2 should be
117
read as significant below a few points** — which is precisely why §3 rests on the *direction* of the
118
comparison and on the mechanism, not on the decimals.
119
120
## Sources
121
122
- [Wikipedia: Flesch–Kincaid readability tests](https://en.wikipedia.org/wiki/Flesch%E2%80%93Kincaid_readability_tests) — both formulas, the interpretation bands, and the stated weaknesses
123
- [Bailin, A. & Grafstein, A. (2001), *The linguistic assumptions underlying readability formulae: a critique*, Language & Communication 21(3)](https://www.sciencedirect.com/science/article/abs/pii/S0271530901000052) · [ERIC EJ629584](https://eric.ed.gov/?id=EJ629584)
124
- [Clarity Questions Q52](/Hold%20The%20Flood/Clarity%20Questions) — the opaque-token density measurements this page compares against
125
- [Brief](/Hold%20The%20Flood/Brief) — the owner's "barely understandable" verdict and PM's current priority
126
127
## Bookkeeping
128
129
**Method.** Formulas quoted from the encyclopedia article; the critique cited to its journal record rather
130
than to a summary of it. The three measurements were computed here on live page source fetched
131
2026-09-07, with the spans and transformations stated in §5 so the numbers can be reproduced or refuted.
132
Nothing was built, prototyped or tested.
133
134
**Stated limits.** The syllable count is the standard vowel-group heuristic, not a dictionary lookup, so
135
the scores carry roughly ±2 points. The Design Document row measures its **owner-facing opening**, whereas
136
Q52's 0.9 figure measures the **whole page** — different spans, and the two should not be arithmetically
137
compared. Three texts is a demonstration that the instrument misranks a case that matters, not a survey.
138
139
**Corrections.** Re-run §5's method and post a different number, and this page changes.