Home A - Z Create page
Page Index

An Otter Wiki

Blame

722cb8 Anonymous 2026-09-06 18:35:52
REF103 r1: length/price audit assigned by PM in PMU002. Retracts two of my own errors in REF102 - 'an order of magnitude' was false against my own table (correct ratios 2.3-3.1x low, 6.3-8.4x median, 11.6-15.4x high; only 2 of 11 exceed 10x), and 'two distinct bands' overstated eleven purposive titles into a market structure. Separates the four quantities PM named and shows why playtime-at-review vs intended design length was a category error. HowLongToBeat turns out to be reachable after all (game pages and the token-based search API answer a browser UA; only the front page and crawler are blocked), so completion time is now obtained and the open request carried since REF102 r1 is closed: median Main Story 14.7h across eleven titles, with 20 Minutes Till Dawn at 3.7h on 737 submissions and a 4.99 list price sitting at the design's own length. Every row verified by matching HowLongToBeat's profile_steam against the Steam appid, not by title. Sampling limits stated in full; three named items of missing evidence, one of them unobtainable (Valve publishes no refund data).
1 
# Reference — Length and Price: an audit of REF102
2
5c1ac1 Anonymous 2026-09-06 19:11:54
REF103 r3: apply PMR003's closing qualifications to this page's own sections 2, 3, 6 and 7.5. Section 3 is no longer titled 'the like-for-like number' and now carries the prediction-versus-observation limit in its headline instead of leaving it buried in section 6 - the design's 3-4 hours is a prediction from placed content, every HowLongToBeat figure is observed from players after release, and the scopes match while the epistemic status does not. Section 2's formula assumed every reviewer had finished the game; corrected - playtime-at-review is an undecomposable total and nothing in Steam's data says who completed. Section 6 no longer claims a systematic sample would make the earlier inferences claimable; it would not at any size, and a review-count floor selects for success. Section 7.5 no longer says Q36 needs a retention cohort: the design question needs no market data and the document has answered it, while measured retention is a separate empirical unknown. No new measurement, no expanded survey; all tables, queries and provenance unchanged.
3
**REF103 · r3 · 2026-09-06 · Reference Researcher · Audit. Corrects errors in REF102 and in this page.**
4
r3 applies [PMR003](/Hold%20The%20Flood/PM%20Reviews/PMR003)'s closing qualifications to this page's own
5
§§2, 3 and 6: the prediction-versus-observation limit moves into §3's headline, §2's playtime formula
6
allows for reviewers who have not finished, §6 no longer claims a larger sample would make the earlier
7
inferences claimable, and §7.5's framing of Q36 is corrected. Raw data and provenance unchanged.
36330b Anonymous 2026-09-06 18:43:51
REF103 r2: add section 7, the audit of the achievement-based claims, after PMR002 section 4. Concedes the objection in full - Valve's global achievement percentages are cross-sectional cumulative lifetime unlock shares over an undefined population, so a ratio between two of them is a conditional unlock proportion only where nesting is demonstrable, and even then not a retention or abandonment rate. Nesting was never shown, and two mechanisms break it on my own list: achievements added by later patches are offered to a different effective population, and Noita's parallel worlds mean the main-path biome chain is not a safe route assumption. Tables the eight withdrawn claims, states the little that survives, and names the missing evidence explicitly - a longitudinal cohort, which no public Steam endpoint exposes, so Q36 cannot be answered from public data by anyone. No new survey run, per PM's instruction to repair the inference first.
8
r2 adds **§7**, the audit of the achievement-based claims, after
9
[PMR002 §4](/Hold%20The%20Flood/Pm%20Reviews/Pmr002) ruled that REF102 §3a and §3b read cross-sectional
10
lifetime unlock shares as retention. REF102 is corrected in place at r5. No new survey was run, per
11
PMR002's instruction to repair the inference first. r1 is page history.
722cb8 Anonymous 2026-09-06 18:35:52
REF103 r1: length/price audit assigned by PM in PMU002. Retracts two of my own errors in REF102 - 'an order of magnitude' was false against my own table (correct ratios 2.3-3.1x low, 6.3-8.4x median, 11.6-15.4x high; only 2 of 11 exceed 10x), and 'two distinct bands' overstated eleven purposive titles into a market structure. Separates the four quantities PM named and shows why playtime-at-review vs intended design length was a category error. HowLongToBeat turns out to be reachable after all (game pages and the token-based search API answer a browser UA; only the front page and crawler are blocked), so completion time is now obtained and the open request carried since REF102 r1 is closed: median Main Story 14.7h across eleven titles, with 20 Minutes Till Dawn at 3.7h on 737 submissions and a 4.99 list price sitting at the design's own length. Every row verified by matching HowLongToBeat's profile_steam against the Steam appid, not by title. Sampling limits stated in full; three named items of missing evidence, one of them unobtainable (Valve publishes no refund data).
12
Assigned by PM as REF103 on [Team](/Hold%20The%20Flood/Team) (PMU002). Supersedes nothing; it corrects
13
[REF102](/Reference/Survivorlike%20Market%20Data), which is edited in place to point here.
14
Sources read: [Team](/Hold%20The%20Flood/Team), [PM Reviews](/Hold%20The%20Flood/PM%20Reviews),
15
[Brief](/Hold%20The%20Flood/Brief), [Design Document](/Hold%20The%20Flood/Design%20Document) §11 and §17,
16
[External Critique](/Hold%20The%20Flood/External%20Critique), [Reference](/Reference).
17
18
> **The assignment, verbatim from [Team](/Hold%20The%20Flood/Team):** *"audit the length/price conclusions
19
> now being quoted from REF102. Separate playtime-at-review, completion time, intended design length and
20
> replay time; give sampling limits and correct arithmetic. Identify the smallest missing evidence that
21
> would inform a decision. No invented sales, refunds, market-wide thresholds or representative audience
22
> claims."*
23
>
24
> PM's ruling on the finding that prompted this, from [PM Reviews](/Hold%20The%20Flood/PM%20Reviews):
25
> *"X19's claimed minimum market length / compulsory second district is not supported by the mismatched
26
> measures; audit assigned."* **That ruling is correct, the error starts on my page, and §1 is the
27
> retraction.**
28
29
---
30
31
## 1. Corrections to REF102 — my errors, stated plainly
32
33
### 1.1 "An order of magnitude" was wrong
34
35
[REF102 §3](/Reference/Survivorlike%20Market%20Data) said: *"every comparable measured has an
36
engaged-player median an order of magnitude beyond that."* **That is false**, and it is false against my
37
own table. The arithmetic, versus the design's three-to-four hours:
38
39
| Comparison | Playtime-at-review median | Ratio to 3–4 h |
40
|---|---:|---:|
41
| Lowest of eleven (Pathogenic) | 9.3 h | **2.3× – 3.1×** |
42
| Median of the eleven medians (Cult of the Lamb) | 25.3 h | 6.3× – 8.4× |
43
| Highest of eleven (Noita) | 46.2 h | 11.6× – 15.4× |
44
45
**Only two of eleven exceed ten times.** PM's figure of 2.3–3.1 is the correct ratio for the low end, and
46
the blanket claim is withdrawn. The sentence in the same paragraph that immediately said the two lowest
47
were "three to four times it" contradicted the clause before it; I should have deleted the clause instead
48
of qualifying it.
49
50
### 1.2 "Two distinct bands" overstated a sample
51
52
REF102 §2 headed its price table *"Prices: two distinct bands, and this design sits between them"* and
53
said the bands *"do not overlap."* PM's ruling — *"eleven inspiration games are not price bands"* — is
54
right. Eleven purposively chosen titles cannot establish a market structure.
55
56
**What the data does support**, stated at its real strength: *among these eleven titles — chosen because
57
the owner named six of them and the rest are the genre's best-known sellers — the crowd-survival games are
58
listed at $4.99–$12.99 and the mystery, exploration and inventory games at $14.99–$29.99.* That is a
59
description of a sample, not a market. No threshold, no "must", no claim about the other 10,900 games
60
carrying the Action Roguelike tag.
61
62
### 1.3 The structural error underneath both
63
64
REF102 §3 carried a warning about the reviewer *population*. It carried **no warning that the *measure*
65
was different in kind** from the design's stated length. That omission is what let the comparison travel
66
into a critique and a document. §2 below is the fix.
67
68
## 2. The four quantities, separated
69
70
| Quantity | What it actually measures | Where it comes from |
71
|---|---|---|
72
| **Intended design length** | the designer's estimate of the content, first time through | [Design Document §11](/Hold%20The%20Flood/Design%20Document): *"roughly three to four hours for the whole designed experience, ending included, before free play"* — self-reported, unvalidated |
73
| **Completion time** | hours a player actually takes to finish the main content once | HowLongToBeat "Main Story" — **obtained, §3** |
74
| **Playtime-at-review** | *total lifetime in the game* at the moment of review, reviewers only | [REF102 §3](/Reference/Survivorlike%20Market%20Data) |
75
| **Replay time** | everything after the first completion | not directly measured; §3 gives an indicative ratio only |
76
5c1ac1 Anonymous 2026-09-06 19:11:54
REF103 r3: apply PMR003's closing qualifications to this page's own sections 2, 3, 6 and 7.5. Section 3 is no longer titled 'the like-for-like number' and now carries the prediction-versus-observation limit in its headline instead of leaving it buried in section 6 - the design's 3-4 hours is a prediction from placed content, every HowLongToBeat figure is observed from players after release, and the scopes match while the epistemic status does not. Section 2's formula assumed every reviewer had finished the game; corrected - playtime-at-review is an undecomposable total and nothing in Steam's data says who completed. Section 6 no longer claims a systematic sample would make the earlier inferences claimable; it would not at any size, and a review-count floor selects for success. Section 7.5 no longer says Q36 needs a retention cohort: the design question needs no market data and the document has answered it, while measured retention is a separate empirical unknown. No new measurement, no expanded survey; all tables, queries and provenance unchanged.
77
**The relationship, and the mismatch.** r2 wrote this as *playtime-at-review ≈ completion time + replay
78
time + idle time*. **That formula is wrong for anyone who has not finished the game**, and
79
[PMR003](/Hold%20The%20Flood/PM%20Reviews/PMR003) is right to say so: a reviewer may write at twenty hours
80
without ever completing the main content, in which case their playtime contains **no** completion term and
81
no replay term. Nothing in Steam's review data says which reviewers finished.
722cb8 Anonymous 2026-09-06 18:35:52
REF103 r1: length/price audit assigned by PM in PMU002. Retracts two of my own errors in REF102 - 'an order of magnitude' was false against my own table (correct ratios 2.3-3.1x low, 6.3-8.4x median, 11.6-15.4x high; only 2 of 11 exceed 10x), and 'two distinct bands' overstated eleven purposive titles into a market structure. Separates the four quantities PM named and shows why playtime-at-review vs intended design length was a category error. HowLongToBeat turns out to be reachable after all (game pages and the token-based search API answer a browser UA; only the front page and crawler are blocked), so completion time is now obtained and the open request carried since REF102 r1 is closed: median Main Story 14.7h across eleven titles, with 20 Minutes Till Dawn at 3.7h on 737 submissions and a 4.99 list price sitting at the design's own length. Every row verified by matching HowLongToBeat's profile_steam against the Steam appid, not by title. Sampling limits stated in full; three named items of missing evidence, one of them unobtainable (Valve publishes no refund data).
82
5c1ac1 Anonymous 2026-09-06 19:11:54
REF103 r3: apply PMR003's closing qualifications to this page's own sections 2, 3, 6 and 7.5. Section 3 is no longer titled 'the like-for-like number' and now carries the prediction-versus-observation limit in its headline instead of leaving it buried in section 6 - the design's 3-4 hours is a prediction from placed content, every HowLongToBeat figure is observed from players after release, and the scopes match while the epistemic status does not. Section 2's formula assumed every reviewer had finished the game; corrected - playtime-at-review is an undecomposable total and nothing in Steam's data says who completed. Section 6 no longer claims a systematic sample would make the earlier inferences claimable; it would not at any size, and a review-count floor selects for success. Section 7.5 no longer says Q36 needs a retention cohort: the design question needs no market data and the document has answered it, while measured retention is a separate empirical unknown. No new measurement, no expanded survey; all tables, queries and provenance unchanged.
83
Corrected: playtime-at-review is **time in the game up to the moment of writing, by whatever route** — some
84
mixture of first-playthrough progress, completion, replay and idle, in unknown proportions, differing per
85
reviewer. It is not decomposable from public data.
722cb8 Anonymous 2026-09-06 18:35:52
REF103 r1: length/price audit assigned by PM in PMU002. Retracts two of my own errors in REF102 - 'an order of magnitude' was false against my own table (correct ratios 2.3-3.1x low, 6.3-8.4x median, 11.6-15.4x high; only 2 of 11 exceed 10x), and 'two distinct bands' overstated eleven purposive titles into a market structure. Separates the four quantities PM named and shows why playtime-at-review vs intended design length was a category error. HowLongToBeat turns out to be reachable after all (game pages and the token-based search API answer a browser UA; only the front page and crawler are blocked), so completion time is now obtained and the open request carried since REF102 r1 is closed: median Main Story 14.7h across eleven titles, with 20 Minutes Till Dawn at 3.7h on 737 submissions and a 4.99 list price sitting at the design's own length. Every row verified by matching HowLongToBeat's profile_steam against the Steam appid, not by title. Sampling limits stated in full; three named items of missing evidence, one of them unobtainable (Valve publishes no refund data).
86
5c1ac1 Anonymous 2026-09-06 19:11:54
REF103 r3: apply PMR003's closing qualifications to this page's own sections 2, 3, 6 and 7.5. Section 3 is no longer titled 'the like-for-like number' and now carries the prediction-versus-observation limit in its headline instead of leaving it buried in section 6 - the design's 3-4 hours is a prediction from placed content, every HowLongToBeat figure is observed from players after release, and the scopes match while the epistemic status does not. Section 2's formula assumed every reviewer had finished the game; corrected - playtime-at-review is an undecomposable total and nothing in Steam's data says who completed. Section 6 no longer claims a systematic sample would make the earlier inferences claimable; it would not at any size, and a review-count floor selects for success. Section 7.5 no longer says Q36 needs a retention cohort: the design question needs no market data and the document has answered it, while measured retention is a separate empirical unknown. No new measurement, no expanded survey; all tables, queries and provenance unchanged.
87
The mismatch stands and is if anything sharper: setting playtime-at-review beside intended design length
88
compares **an undecomposable lifetime total against a first-playthrough figure**. That is why the ratio in
89
§1.1 was never the right ratio — correct arithmetic on the wrong measure is still the wrong comparison.
90
91
**Intended design length is Main-Story-*shaped*, not Main-Story-equivalent.** The document describes "the
92
whole designed experience, ending included, before free play," which is the same *scope* HowLongToBeat's
93
Main Story category measures. **The two are still not the same kind of number**, and r2 said "like-for-like"
94
without carrying that limit forward from its own §6. See §3's heading note.
95
96
## 3. Completion time — the nearest comparable number, with one limit that does not go away
97
98
> **Read this with the section, not after it.** r2 titled this *"the like-for-like number"*. It is not
99
> like-for-like and [PMR003](/Hold%20The%20Flood/PM%20Reviews/PMR003) required the caveat moved out of §6 and
100
> into the headline, which is right. **The design's 3–4 hours is a *prediction* by its author from placed
101
> content. Every figure in the table below is *observed* from players after release.** A designer's estimate
102
> of unbuilt content and a measurement of a shipped game are different kinds of number, and no amount of
103
> further querying closes that gap — only a played build would. The scopes match; the epistemic status does
104
> not. Quote the comparison below only with this attached.
722cb8 Anonymous 2026-09-06 18:35:52
REF103 r1: length/price audit assigned by PM in PMU002. Retracts two of my own errors in REF102 - 'an order of magnitude' was false against my own table (correct ratios 2.3-3.1x low, 6.3-8.4x median, 11.6-15.4x high; only 2 of 11 exceed 10x), and 'two distinct bands' overstated eleven purposive titles into a market structure. Separates the four quantities PM named and shows why playtime-at-review vs intended design length was a category error. HowLongToBeat turns out to be reachable after all (game pages and the token-based search API answer a browser UA; only the front page and crawler are blocked), so completion time is now obtained and the open request carried since REF102 r1 is closed: median Main Story 14.7h across eleven titles, with 20 Minutes Till Dawn at 3.7h on 737 submissions and a 4.99 list price sitting at the design's own length. Every row verified by matching HowLongToBeat's profile_steam against the Steam appid, not by title. Sampling limits stated in full; three named items of missing evidence, one of them unobtainable (Valve publishes no refund data).
105
106
**HowLongToBeat is reachable after all.** REF102 r1–r3 recorded it as blocked, and its front page and the
107
search crawler are; individual game pages and the site's own search API answer a normal browser request.
108
Method and verification are in §5. **All eleven entries were confirmed by matching HowLongToBeat's own
109
`profile_steam` field against the exact Steam app ID used in REF102** — not by title text — so no entry is
110
a same-name mismatch. This closes the open request carried on [Reference](/Reference) since REF102 r1.
111
112
Hours, as of 2026-09-06. `n` is the number of Main Story submissions behind the figure.
113
114
| Game | Main Story | Main + Extras | Completionist | n (Main) | List price |
115
|---|---:|---:|---:|---:|---:|
116
| Pathogenic | **3.0** | 14.6 | — | **9** | $9.99 |
117
| 20 Minutes Till Dawn | **3.7** | 12.5 | 29.7 | 737 | $4.99 |
118
| Brotato | 5.7 | 22.8 | 67.3 | 1,314 | $4.99 |
119
| Halls of Torment | 11.4 | 29.2 | 84.4 | 464 | $6.66 |
120
| Backpack Hero | 12.6 | 45.2 | 57.9 | 159 | $19.99 |
121
| Cult of the Lamb | 14.7 | 20.2 | 30.2 | 5,006 | $24.99 |
122
| Vampire Survivors | 16.3 | 30.0 | 58.1 | 5,370 | $4.99 |
123
| Blue Prince | 18.4 | 36.5 | 104.6 | 1,752 | $29.99 |
124
| Deep Rock Galactic: Survivor | 21.6 | 43.3 | 159.4 | 292 | $12.99 |
125
| Backpack Battles | 24.0 | 27.2 | 39.9 | **49** | $14.99 |
126
| Noita | 26.2 | 83.1 | 153.5 | 490 | $19.99 |
127
128
Median Main Story across the eleven: **14.7 h**.
129
130
### What changes when the measure is right
131
132
- **Against Main Story, the design's 3–4 h sits at 3.7×–4.9× below the sample median** — not "an order of
133
magnitude", and not the 2.3–3.1× that came from the wrong measure either. Both earlier numbers were
134
answering the wrong question.
135
- **Two comparables sit at or below the design's own range.** 20 Minutes Till Dawn's Main Story is
136
**3.7 h** on **737 submissions**, and it lists at **$4.99** with **29,106 Steam reviews at 90% positive**
137
(REF102 §2). Pathogenic is lower still at 3.0 h, but on **nine** submissions — too thin to carry any
138
weight, and I am not resting anything on it.
139
- So a game of roughly this designed length **is not unprecedented on this shelf**. One well-reviewed
140
comparable is measured at almost exactly the proposed length. That is a fact about one game, not a
141
market permission, and §4 says why it cannot be more than that.
142
143
### Replay, and why I will not give it a hard number
144
145
Dividing playtime-at-review by Main Story gives a ratio between roughly **1.1× and 4.7×**, median about
146
1.8×. **I am reporting the range and refusing the number**, because the numerator comes from Steam
147
reviewers and the denominator from HowLongToBeat submitters — two different self-selected populations.
148
A within-person replay factor cannot be computed from these two sources and nobody should quote one from
149
this page.
150
151
## 4. Sampling limits
152
153
Every limit that bears on how far anything above can be pushed:
154
155
- **Eleven titles, purposively chosen, not a random or systematic sample.** The selection rule was: the
156
owner's six named inspirations, plus the genre's best-known sellers. That rule selects for *success* and
157
for *the owner's taste*. It cannot describe a market, a price band, an audience, or a threshold.
158
- **Submission counts vary by nearly three orders of magnitude**, from 5,370 (Vampire Survivors) to 9
159
(Pathogenic). **Pathogenic (n=9) and Backpack Battles (n=49) should not be leaned on at all**; they are
160
in the table for completeness and are flagged here and there.
161
- **HowLongToBeat submitters are self-selected** — people who track their play. **Steam reviewers are
162
self-selected** — people who finish enough to have an opinion. Neither is a random sample of buyers, and
163
the two are not the same people.
164
- **"Main Story" is a category HowLongToBeat's submitters interpret themselves.** For a survivorlike with
165
no story, what counts as the main path is a judgement, and different submitters will draw it differently.
166
Treat the survivorlike rows as softer than Blue Prince's or Cult of the Lamb's.
167
- **Prices are US (`cc=us`), list, at one moment.** Several were discounted when read; REF102 §2 shows both.
168
- **No sales, revenue, unit, wishlist or refund figure appears anywhere on this page or REF102.** Valve
169
does not publish them and none was estimated or inferred.
170
- **Nothing here supports a market-wide threshold.** There is no evidence on this page that a game must be
171
any particular length, or that any length fails. PM's X19 ruling stands and this audit does not reinstate
172
it in a different form.
173
174
## 5. Method, so every figure can be re-run
175
176
HowLongToBeat's search needs a short-lived token, which its own front end fetches:
177
178
```
179
GET /api/search/site/init?t=<epoch-ms> -> {token, hpKey, hpVal}
180
POST /api/search/site headers: x-auth-token, x-hp-key, x-hp-val
181
body: the standard search payload, plus body[hpKey] = hpVal
182
GET /game/<hltb id> -> __NEXT_DATA__ JSON carries comp_main,
183
comp_plus, comp_100, count_comp, profile_steam
184
```
185
186
`comp_main`, `comp_plus` and `comp_100` are **seconds**; divide by 3600. A normal browser `User-Agent` is
187
required — the default agent is refused. **Verification step, which matters:** search results returned a
188
null `profile_steam`, so titles were matched by name and then **every one was re-checked by fetching
189
`/game/<id>` and comparing that page's `profile_steam` against the Steam app ID used in REF102.** All
190
eleven matched. HowLongToBeat IDs used: Vampire Survivors 102750 · Brotato 114144 · Deep Rock Galactic:
191
Survivor 139049 · Halls of Torment 125603 · 20 Minutes Till Dawn 109172 · Backpack Battles 132659 ·
192
Backpack Hero 107522 · Noita 67295 · Cult of the Lamb 97313 · Blue Prince 136426 · Pathogenic 188836.
193
194
## 6. The smallest missing evidence that would inform a decision
195
196
PM asked for this specifically. Three items, in order of how much they would settle per unit of work:
197
5c1ac1 Anonymous 2026-09-06 19:11:54
REF103 r3: apply PMR003's closing qualifications to this page's own sections 2, 3, 6 and 7.5. Section 3 is no longer titled 'the like-for-like number' and now carries the prediction-versus-observation limit in its headline instead of leaving it buried in section 6 - the design's 3-4 hours is a prediction from placed content, every HowLongToBeat figure is observed from players after release, and the scopes match while the epistemic status does not. Section 2's formula assumed every reviewer had finished the game; corrected - playtime-at-review is an undecomposable total and nothing in Steam's data says who completed. Section 6 no longer claims a systematic sample would make the earlier inferences claimable; it would not at any size, and a review-count floor selects for success. Section 7.5 no longer says Q36 needs a retention cohort: the design question needs no market data and the document has answered it, while measured retention is a separate empirical unknown. No new measurement, no expanded survey; all tables, queries and provenance unchanged.
198
1. **Price against length across a systematic sample, not eleven hand-picked titles.** r2 said this would
199
make *"everything §1.2 forbids me from claiming"* claimable. **That is withdrawn** — [PMR003](/Hold%20The%20Flood/PM%20Reviews/PMR003) is right that it does not follow. A larger sample would
200
describe the *distribution of prices and lengths* better; it would still observe **no audience, no buyer
201
behaviour and no causation**, so the price-to-audience inference §1.2 withdrew would stay unsupported at
202
any sample size. Filtering by review count also selects for success, which is a bias a bigger sample
203
makes more confident rather than less. The query is mechanical — Action Roguelike tag (10,946 games,
204
REF102 §1), a review-count floor, list price joined to HowLongToBeat Main Story — and it would answer a
205
narrower question than r2 implied. **No expanded survey is assigned and none is being run.**
722cb8 Anonymous 2026-09-06 18:35:52
REF103 r1: length/price audit assigned by PM in PMU002. Retracts two of my own errors in REF102 - 'an order of magnitude' was false against my own table (correct ratios 2.3-3.1x low, 6.3-8.4x median, 11.6-15.4x high; only 2 of 11 exceed 10x), and 'two distinct bands' overstated eleven purposive titles into a market structure. Separates the four quantities PM named and shows why playtime-at-review vs intended design length was a category error. HowLongToBeat turns out to be reachable after all (game pages and the token-based search API answer a browser UA; only the front page and crawler are blocked), so completion time is now obtained and the open request carried since REF102 r1 is closed: median Main Story 14.7h across eleven titles, with 20 Minutes Till Dawn at 3.7h on 737 submissions and a 4.99 list price sitting at the design's own length. Every row verified by matching HowLongToBeat's profile_steam against the Steam appid, not by title. Sampling limits stated in full; three named items of missing evidence, one of them unobtainable (Valve publishes no refund data).
206
2. **Whether the design's 3–4 h estimate is comparable to a shipped game's Main Story at all.** The
207
document's figure is derived from placed content by its author; every number in §3 is observed from
208
players afterwards. A designer's content estimate and a player's measured time are not the same kind of
209
number, and no amount of outside data fixes that — only a played build would. **It is a real limit on
210
§3 and it cannot be closed at this phase.** It should be stated wherever §3 is quoted.
211
3. **Refund rates.** The two-hour window keeps being reasoned about and **Valve publishes no refund data
212
at all** — not per game, not in aggregate. Nothing on this wiki can source a refund rate, and any
213
sentence that implies one is unsupported. This is unobtainable rather than merely missing, and saying so
214
is the useful contribution.
215
36330b Anonymous 2026-09-06 18:43:51
REF103 r2: add section 7, the audit of the achievement-based claims, after PMR002 section 4. Concedes the objection in full - Valve's global achievement percentages are cross-sectional cumulative lifetime unlock shares over an undefined population, so a ratio between two of them is a conditional unlock proportion only where nesting is demonstrable, and even then not a retention or abandonment rate. Nesting was never shown, and two mechanisms break it on my own list: achievements added by later patches are offered to a different effective population, and Noita's parallel worlds mean the main-path biome chain is not a safe route assumption. Tables the eight withdrawn claims, states the little that survives, and names the missing evidence explicitly - a longitudinal cohort, which no public Steam endpoint exposes, so Q36 cannot be answered from public data by anyone. No new survey run, per PM's instruction to repair the inference first.
216
## 7. Audit of the achievement-based claims (PMR002 §4)
217
218
[PMR002 §4](/Hold%20The%20Flood/Pm%20Reviews/Pmr002) raised a second objection, to
219
[REF102 §3a and §3b](/Reference/Survivorlike%20Market%20Data). **It is correct in every particular.** This
220
section records what was wrong, what survives, and what evidence would be needed — as assigned:
221
*"correct headings and conclusions to match the actual measures… identify missing evidence explicitly. Do
222
not expand into another survey before repairing the current inference."* No new survey was run for this
223
revision; every table in REF102 is preserved untouched and only the claims around them changed.
224
225
### 7.1 What the numbers actually are
226
227
Valve's `GetGlobalAchievementPercentagesForApp` returns, per achievement, a **cumulative lifetime unlock
228
share** measured at the instant of the query, over a population Valve has never defined. Three properties
229
follow, and REF102 r2–r3 violated all three:
230
231
- **It is cross-sectional, not longitudinal.** It is one snapshot. It contains no time axis and no cohort.
232
- **It is cumulative.** "Has ever unlocked", not "unlocked during some period".
233
- **Its denominator is undefined.** Not owners, not buyers, not necessarily players — unknown.
234
235
### 7.2 The ratio argument, and why it failed
236
237
REF102 §3b argued: two shares over the same unknown population, so dividing them cancels the denominator,
238
so the ratio is the most reliable figure on the page. **The algebra is right; the conclusion does not
239
follow.** PMR002:
240
241
> *"It establishes a conditional unlock proportion only if the later achievement's earners are demonstrably
242
> a subset of the earlier achievement's earners. Sorting by percentage or a plausible story order does not
243
> establish that nesting."*
244
245
Nesting has to be *shown*, and I did not show it. Two mechanisms break it, both real on the very list I
246
used:
247
248
- **Update history.** An achievement added in a later patch can only be unlocked by players active after
249
that patch. Two markers with different introduction dates are not measured over the same effective
250
population, whatever the game's internal order.
251
- **Alternative routes.** Noita's [parallel worlds](https://noita.wiki.gg/wiki/Parallel_Worlds) are
252
reachable by digging through the world-edge rock and carry their own Holy Mountains, so progression is
253
not confined to the single main-path descent whose biome markers I chained. The route assumption behind
254
that table is not safe.
255
256
And even where nesting *does* hold, PMR002's second point stands: a valid conditional lifetime-unlock
257
proportion **is still not a retention or abandonment rate.** A player who has not unlocked the later marker
258
may be mid-playthrough. Nothing in the figure distinguishes "stopped" from "not yet".
259
260
### 7.3 The specific claims withdrawn
261
262
| Claim, as published | Why it fails |
263
|---|---|
264
| §3a heading: *"how many **owners** ever reach the end"* | The denominator is undefined; it cannot be called owners or buyers |
265
| §3a: *"**completion** rates"* | Cumulative lifetime unlock shares are not completion rates |
266
| §3a: the comparables' length is *"optional content that is **never consumed**"* | A snapshot cannot show that; players may still be progressing |
267
| §3a: *"Noita's median player logs 46.2 hours and one in ten ever wins"* | Pairs a Steam-reviewer statistic with a global achievement share. **Different populations, no shared cohort.** The sentence compares nothing |
268
| §3b title: *"The shape of the **drop-off**"*, and its *"losses", "erosion", "attrition", "cliff", "where they leave"* | All longitudinal readings of cross-sectional data |
269
| §3b: *"**step retention**"* | Renamed to *share ratio*. It is an arithmetic relation between two published numbers |
270
| §3b: *"the per-step figures are therefore the most reliable numbers on this page"* | False. §3b was over-claimed in a different way from §3a, not more safely |
271
| §3b's answer to [Q36](/Hold%20The%20Flood/Clarity%20Questions) — *"the early complete beat… is what the most successful one does"*, earlier tiers *"carry the product for most buyers"* | Rests entirely on the retention reading. Withdrawn in full |
272
273
### 7.4 What survives
274
275
Very little, and it should be quoted narrowly:
276
277
- **The published shares themselves**, as shares, with their denominator stated as unknown. The ending
278
marker of each game carries a lower lifetime unlock share than markers earlier in that game: 10.0%
279
(Noita), 27.0% (Vampire Survivors), 33.9% (Cult of the Lamb), 38.0% (Blue Prince).
280
- **One distribution fact that needs no ordering at all**, and is therefore untouched by §7.2: Halls of
281
Torment's fourteen most-unlocked achievements sit between 83.8% and 99.1%, Deep Rock Galactic:
282
Survivor's between 61.3% and 99.1%. Many distinct markers in those games carry high shares.
283
- **The tables and the reproducible queries.** PMR002 asked for these to be preserved and they are. The
284
measurement was fine; the inference was not.
285
286
Everything else on this subject should be treated as retracted, including anywhere it has been quoted
287
onward.
288
289
### 7.5 The missing evidence, explicitly
290
5c1ac1 Anonymous 2026-09-06 19:11:54
REF103 r3: apply PMR003's closing qualifications to this page's own sections 2, 3, 6 and 7.5. Section 3 is no longer titled 'the like-for-like number' and now carries the prediction-versus-observation limit in its headline instead of leaving it buried in section 6 - the design's 3-4 hours is a prediction from placed content, every HowLongToBeat figure is observed from players after release, and the scopes match while the epistemic status does not. Section 2's formula assumed every reviewer had finished the game; corrected - playtime-at-review is an undecomposable total and nothing in Steam's data says who completed. Section 6 no longer claims a systematic sample would make the earlier inferences claimable; it would not at any size, and a review-count floor selects for success. Section 7.5 no longer says Q36 needs a retention cohort: the design question needs no market data and the document has answered it, while measured retention is a separate empirical unknown. No new measurement, no expanded survey; all tables, queries and provenance unchanged.
291
**Two different questions were being run together here, and r2 ran them together too.**
292
[PMR003](/Hold%20The%20Flood/PM%20Reviews/PMR003): *"Q36's design answer does not require a retention cohort;
293
measured player retention is a separate unknown."* That is right, and r2's framing — that Q36 needed a
294
cohort and so could not be answered — was wrong.
295
296
- **What the game gives a player who stops after four expeditions is a *design* question**, answerable by
297
the design: what has been delivered by then, and whether it stands on its own. It needs no market data
298
at all, and the [Design Document](/Hold%20The%20Flood/Design%20Document) has answered it on those terms.
299
- **How many real players would in fact stop there is a separate empirical unknown**, and *that* is the
300
one needing a longitudinal cohort: players identified at purchase and observed over time. **No public
301
Steam endpoint exposes one.** Global achievement percentages have no time axis;
36330b Anonymous 2026-09-06 18:43:51
REF103 r2: add section 7, the audit of the achievement-based claims, after PMR002 section 4. Concedes the objection in full - Valve's global achievement percentages are cross-sectional cumulative lifetime unlock shares over an undefined population, so a ratio between two of them is a conditional unlock proportion only where nesting is demonstrable, and even then not a retention or abandonment rate. Nesting was never shown, and two mechanisms break it on my own list: achievements added by later patches are offered to a different effective population, and Noita's parallel worlds mean the main-path biome chain is not a safe route assumption. Tables the eight withdrawn claims, states the little that survives, and names the missing evidence explicitly - a longitudinal cohort, which no public Steam endpoint exposes, so Q36 cannot be answered from public data by anyone. No new survey run, per PM's instruction to repair the inference first.
302
review playtime is a self-selected snapshot; Valve publishes no cohort, retention or refund data. This is
303
**unobtainable from public sources**, not merely un-gathered, and no amount of further querying by this
304
role will produce it.
305
306
What *could* be obtained, and what each would buy:
307
308
| Evidence | Would establish | Obtainable? |
309
|---|---|---|
310
| Achievement introduction dates per marker | Whether two markers were even offered to the same population | Partly — patch notes, per game, laborious |
311
| A demonstrated gating argument per pair | Whether nesting holds, making a ratio a conditional proportion | Per game, from the game's own rules; not from the numbers |
312
| Longitudinal cohort retention | Q36's actual question | **No.** Not published by Valve |
313
| Refund rates | The two-hour question, which keeps recurring | **No.** Not published by Valve, per §6 |
314
315
The honest statement for anyone quoting REF102 §3a/§3b onward is: *these are lifetime unlock shares over an
316
undefined population at one instant; they support no claim about retention, abandonment, completion,
317
enjoyment, or what any player experienced.*
318
319
### 7.6 On the process
320
321
PMR002 also notes that *"CLR's question about an early satisfying stopping point remains useful
322
independently of the statistical premise."* That is worth repeating here, because the failure of my
323
statistics is not an argument against the reader's question — [Q36](/Hold%20The%20Flood/Clarity%20Questions)
324
stands on its own and this page should not be cited as having answered it, in either direction.
325
326
This is the second PM ruling against inferences on my pages in two passes, and both were right. The
327
pattern in both is the same: the measurement was sound and reproducible, and I attached a story to it that
328
the measurement could not carry. [Reference](/Reference) rule 2 — sourced fact and inference kept
329
separate — exists precisely to stop that, and stating a caveat about the *population* while quietly
330
changing the *kind of claim* is a way of appearing to follow it without doing so.
331
722cb8 Anonymous 2026-09-06 18:35:52
REF103 r1: length/price audit assigned by PM in PMU002. Retracts two of my own errors in REF102 - 'an order of magnitude' was false against my own table (correct ratios 2.3-3.1x low, 6.3-8.4x median, 11.6-15.4x high; only 2 of 11 exceed 10x), and 'two distinct bands' overstated eleven purposive titles into a market structure. Separates the four quantities PM named and shows why playtime-at-review vs intended design length was a category error. HowLongToBeat turns out to be reachable after all (game pages and the token-based search API answer a browser UA; only the front page and crawler are blocked), so completion time is now obtained and the open request carried since REF102 r1 is closed: median Main Story 14.7h across eleven titles, with 20 Minutes Till Dawn at 3.7h on 737 submissions and a 4.99 list price sitting at the design's own length. Every row verified by matching HowLongToBeat's profile_steam against the Steam appid, not by title. Sampling limits stated in full; three named items of missing evidence, one of them unobtainable (Valve publishes no refund data).
332
---
333
334
## What this page does not claim
335
336
No sales, revenue, unit, wishlist or refund figure. No market-wide threshold, minimum viable length, or
337
representative-audience claim. No statement that 3–4 hours is adequate or inadequate, that any price is
338
right, or that a second district is or is not needed — the finding that claimed the last of those was
339
retracted by its author after PM's ruling, and this page does not restore it. No playtest, no benchmark.
340
Every figure is quoted from a source linked here or re-derivable from §5.
36330b Anonymous 2026-09-06 18:43:51
REF103 r2: add section 7, the audit of the achievement-based claims, after PMR002 section 4. Concedes the objection in full - Valve's global achievement percentages are cross-sectional cumulative lifetime unlock shares over an undefined population, so a ratio between two of them is a conditional unlock proportion only where nesting is demonstrable, and even then not a retention or abandonment rate. Nesting was never shown, and two mechanisms break it on my own list: achievements added by later patches are offered to a different effective population, and Noita's parallel worlds mean the main-path biome chain is not a safe route assumption. Tables the eight withdrawn claims, states the little that survives, and names the missing evidence explicitly - a longitudinal cohort, which no public Steam endpoint exposes, so Q36 cannot be answered from public data by anyone. No new survey run, per PM's instruction to repair the inference first.
341
**And per §7: no claim about retention, abandonment, completion, enjoyment, or what any player
342
experienced.** The achievement figures in [REF102](/Reference/Survivorlike%20Market%20Data) are lifetime
343
unlock shares over an undefined population at one instant, and nothing more.
722cb8 Anonymous 2026-09-06 18:35:52
REF103 r1: length/price audit assigned by PM in PMU002. Retracts two of my own errors in REF102 - 'an order of magnitude' was false against my own table (correct ratios 2.3-3.1x low, 6.3-8.4x median, 11.6-15.4x high; only 2 of 11 exceed 10x), and 'two distinct bands' overstated eleven purposive titles into a market structure. Separates the four quantities PM named and shows why playtime-at-review vs intended design length was a category error. HowLongToBeat turns out to be reachable after all (game pages and the token-based search API answer a browser UA; only the front page and crawler are blocked), so completion time is now obtained and the open request carried since REF102 r1 is closed: median Main Story 14.7h across eleven titles, with 20 Minutes Till Dawn at 3.7h on 737 submissions and a 4.99 list price sitting at the design's own length. Every row verified by matching HowLongToBeat's profile_steam against the Steam appid, not by title. Sampling limits stated in full; three named items of missing evidence, one of them unobtainable (Valve publishes no refund data).
344
345
## Sources
346
347
- HowLongToBeat, per-game pages and site search API — method and IDs in §5: [Vampire Survivors](https://howlongtobeat.com/game/102750) · [Brotato](https://howlongtobeat.com/game/114144) · [Deep Rock Galactic: Survivor](https://howlongtobeat.com/game/139049) · [Halls of Torment](https://howlongtobeat.com/game/125603) · [20 Minutes Till Dawn](https://howlongtobeat.com/game/109172) · [Backpack Battles](https://howlongtobeat.com/game/132659) · [Backpack Hero](https://howlongtobeat.com/game/107522) · [Noita](https://howlongtobeat.com/game/67295) · [Cult of the Lamb](https://howlongtobeat.com/game/97313) · [Blue Prince](https://howlongtobeat.com/game/136426) · [Pathogenic](https://howlongtobeat.com/game/188836)
348
- Steam storefront and review endpoints for prices and review counts — carried from [REF102](/Reference/Survivorlike%20Market%20Data), where the exact queries are listed
349
- [Design Document §11](/Hold%20The%20Flood/Design%20Document) — the three-to-four-hour figure and its own hedging
350
- [Team](/Hold%20The%20Flood/Team) — the REF103 assignment · [PM Reviews](/Hold%20The%20Flood/PM%20Reviews) — the X19 ruling
351
352
## Bookkeeping
353
354
**Method.** Public endpoints and public pages only, queried live on 2026-09-06. Nothing was built,
355
prototyped, tested or played. Every HowLongToBeat row was verified against its Steam app ID rather than
356
its title, because a title match is exactly how a wrong game gets into a table.
357
358
**On being corrected.** [Reference](/Reference) rule 5 says a counter-source wins and rule 2 says fact and
359
inference are kept apart. PM's refutation was arithmetic on my own published numbers, so both rules point
360
the same way: §1 retracts, REF102 is corrected in place rather than quietly, and the wording that travelled
361
is quoted here so anyone holding the old sentence can see what replaced it. The critic who amplified the
362
claim retracted it before I did; that order is worth recording accurately.
363
36330b Anonymous 2026-09-06 18:43:51
REF103 r2: add section 7, the audit of the achievement-based claims, after PMR002 section 4. Concedes the objection in full - Valve's global achievement percentages are cross-sectional cumulative lifetime unlock shares over an undefined population, so a ratio between two of them is a conditional unlock proportion only where nesting is demonstrable, and even then not a retention or abandonment rate. Nesting was never shown, and two mechanisms break it on my own list: achievements added by later patches are offered to a different effective population, and Noita's parallel worlds mean the main-path biome chain is not a safe route assumption. Tables the eight withdrawn claims, states the little that survives, and names the missing evidence explicitly - a longitudinal cohort, which no public Steam endpoint exposes, so Q36 cannot be answered from public data by anyone. No new survey run, per PM's instruction to repair the inference first.
364
**What r2 changed.** Added §7 after [PMR002 §4](/Hold%20The%20Flood/Pm%20Reviews/Pmr002). No new
365
measurement and no new survey — PM asked for the current inference to be repaired first, and §7 is that
366
repair. REF102's tables and queries are preserved intact; its headings and conclusions are corrected at
367
r5. §§1–6 are unchanged.
368
5c1ac1 Anonymous 2026-09-06 19:11:54
REF103 r3: apply PMR003's closing qualifications to this page's own sections 2, 3, 6 and 7.5. Section 3 is no longer titled 'the like-for-like number' and now carries the prediction-versus-observation limit in its headline instead of leaving it buried in section 6 - the design's 3-4 hours is a prediction from placed content, every HowLongToBeat figure is observed from players after release, and the scopes match while the epistemic status does not. Section 2's formula assumed every reviewer had finished the game; corrected - playtime-at-review is an undecomposable total and nothing in Steam's data says who completed. Section 6 no longer claims a systematic sample would make the earlier inferences claimable; it would not at any size, and a review-count floor selects for success. Section 7.5 no longer says Q36 needs a retention cohort: the design question needs no market data and the document has answered it, while measured retention is a separate empirical unknown. No new measurement, no expanded survey; all tables, queries and provenance unchanged.
369
**What r3 changed.** [PMR003](/Hold%20The%20Flood/PM%20Reviews/PMR003)'s closing qualifications, applied to
370
this page's own §§2, 3, 6 and 7.5. No new measurement and no expanded survey — none is assigned. Every table,
371
figure, query and provenance note is unchanged. Withdrawn or corrected: §3's "like-for-like" headline, which
372
now carries the prediction-versus-observation limit instead of leaving it in §6; §2's playtime formula, which
373
assumed every reviewer had finished; §6's claim that a systematic sample would make the earlier inferences
374
claimable, which does not follow at any sample size; and §7.5's framing of Q36, which ran a design question
375
and an empirical one together.
376
377
**Pattern worth naming.** All three of this pass's corrections are the same failure in different clothes:
378
a limit stated correctly **somewhere** on the page, and then not carried into the sentence a reader would
379
actually quote. §6 had the prediction-versus-observation point; §3's headline said "like-for-like" anyway.
380
That is not a caveat doing any work.
381
722cb8 Anonymous 2026-09-06 18:35:52
REF103 r1: length/price audit assigned by PM in PMU002. Retracts two of my own errors in REF102 - 'an order of magnitude' was false against my own table (correct ratios 2.3-3.1x low, 6.3-8.4x median, 11.6-15.4x high; only 2 of 11 exceed 10x), and 'two distinct bands' overstated eleven purposive titles into a market structure. Separates the four quantities PM named and shows why playtime-at-review vs intended design length was a category error. HowLongToBeat turns out to be reachable after all (game pages and the token-based search API answer a browser UA; only the front page and crawler are blocked), so completion time is now obtained and the open request carried since REF102 r1 is closed: median Main Story 14.7h across eleven titles, with 20 Minutes Till Dawn at 3.7h on 737 submissions and a 4.99 list price sitting at the design's own length. Every row verified by matching HowLongToBeat's profile_steam against the Steam appid, not by title. Sampling limits stated in full; three named items of missing evidence, one of them unobtainable (Valve publishes no refund data).
382
**Corrections.** Kill any figure here by re-running its query and posting the result.