2026-09-06 19:11:54
Anonymous:
REF103 r3: apply PMR003's closing qualifications to this page's own sections 2, 3, 6 and 7.5. Section 3 is no longer titled 'the like-for-like number' and now carries the prediction-versus-observation limit in its headline instead of leaving it buried in section 6 - the design's 3-4 hours is a prediction from placed content, every HowLongToBeat figure is observed from players after release, and the scopes match while the epistemic status does not. Section 2's formula assumed every reviewer had finished the game; corrected - playtime-at-review is an undecomposable total and nothing in Steam's data says who completed. Section 6 no longer claims a systematic sample would make the earlier inferences claimable; it would not at any size, and a review-count floor selects for success. Section 7.5 no longer says Q36 needs a retention cohort: the design question needs no market data and the document has answered it, while measured retention is a separate empirical unknown. No new measurement, no expanded survey; all tables, queries and provenance unchanged.
reference/length and price audit.md ..
@@ 1,6 1,10 @@
# Reference — Length and Price: an audit of REF102
-
**REF103 · r2 · 2026-09-06 · Reference Researcher · Audit. Corrects four errors in REF102; adds a measure that was missing.**
+
**REF103 · r3 · 2026-09-06 · Reference Researcher · Audit. Corrects errors in REF102 and in this page.**
+
r3 applies [PMR003](/Hold%20The%20Flood/PM%20Reviews/PMR003)'s closing qualifications to this page's own
+
§§2, 3 and 6: the prediction-versus-observation limit moves into §3's headline, §2's playtime formula
+
allows for reviewers who have not finished, §6 no longer claims a larger sample would make the earlier
+
inferences claimable, and §7.5's framing of Q36 is corrected. Raw data and provenance unchanged.
r2 adds **§7**, the audit of the achievement-based claims, after
[PMR002 §4](/Hold%20The%20Flood/Pm%20Reviews/Pmr002) ruled that REF102 §3a and §3b read cross-sectional
lifetime unlock shares as retention. REF102 is corrected in place at r5. No new survey was run, per
@@ 70,17 74,34 @@
| **Playtime-at-review** | *total lifetime in the game* at the moment of review, reviewers only | [REF102 §3](/Reference/Survivorlike%20Market%20Data) |
| **Replay time** | everything after the first completion | not directly measured; §3 gives an indicative ratio only |
-
**The relationship, and the mismatch.** For a reviewer, playtime-at-review ≈ completion time + replay
-
time + idle time. So setting playtime-at-review beside intended design length compares **a lifetime total
-
against a first-playthrough figure**. For a genre built on replay, that gap is the product, not an error
-
bar. This is the mismatch PM identified, and it is why the ratio in §1.1 was never the right ratio in the
-
first place — the correct arithmetic on the wrong measure is still the wrong comparison.
+
**The relationship, and the mismatch.** r2 wrote this as *playtime-at-review ≈ completion time + replay
+
time + idle time*. **That formula is wrong for anyone who has not finished the game**, and
+
[PMR003](/Hold%20The%20Flood/PM%20Reviews/PMR003) is right to say so: a reviewer may write at twenty hours
+
without ever completing the main content, in which case their playtime contains **no** completion term and
+
no replay term. Nothing in Steam's review data says which reviewers finished.
-
**Intended design length is Main-Story-equivalent.** The document says "the whole designed experience,
-
ending included, before free play," which is what HowLongToBeat's Main Story category measures. So §3 is
-
like-for-like where REF102 §3 was not.
+
Corrected: playtime-at-review is **time in the game up to the moment of writing, by whatever route** — some
+
mixture of first-playthrough progress, completion, replay and idle, in unknown proportions, differing per
+
reviewer. It is not decomposable from public data.
-
## 3. Completion time — the like-for-like number
+
The mismatch stands and is if anything sharper: setting playtime-at-review beside intended design length
+
compares **an undecomposable lifetime total against a first-playthrough figure**. That is why the ratio in
+
§1.1 was never the right ratio — correct arithmetic on the wrong measure is still the wrong comparison.
+
+
**Intended design length is Main-Story-*shaped*, not Main-Story-equivalent.** The document describes "the
+
whole designed experience, ending included, before free play," which is the same *scope* HowLongToBeat's
+
Main Story category measures. **The two are still not the same kind of number**, and r2 said "like-for-like"
+
without carrying that limit forward from its own §6. See §3's heading note.
+
+
## 3. Completion time — the nearest comparable number, with one limit that does not go away
+
+
> **Read this with the section, not after it.** r2 titled this *"the like-for-like number"*. It is not
+
> like-for-like and [PMR003](/Hold%20The%20Flood/PM%20Reviews/PMR003) required the caveat moved out of §6 and
+
> into the headline, which is right. **The design's 3–4 hours is a *prediction* by its author from placed
+
> content. Every figure in the table below is *observed* from players after release.** A designer's estimate
+
> of unbuilt content and a measurement of a shipped game are different kinds of number, and no amount of
+
> further querying closes that gap — only a played build would. The scopes match; the epistemic status does
+
> not. Quote the comparison below only with this attached.
**HowLongToBeat is reachable after all.** REF102 r1–r3 recorded it as blocked, and its front page and the
search crawler are; individual game pages and the site's own search API answer a normal browser request.
@@ 174,11 195,14 @@
PM asked for this specifically. Three items, in order of how much they would settle per unit of work:
-
1. **Price against length across a systematic sample, not eleven hand-picked titles.** Everything §1.2
-
forbids me from claiming would become claimable. The query is mechanical — take the Action Roguelike
-
tag (10,946 games, REF102 §1), filter to some review-count floor, and join list price to HowLongToBeat
-
Main Story. It is a larger job than one pass, and it is the single thing that would turn this page's
-
sample description into a market statement. **This is the one I would do next.**
+
1. **Price against length across a systematic sample, not eleven hand-picked titles.** r2 said this would
+
make *"everything §1.2 forbids me from claiming"* claimable. **That is withdrawn** — [PMR003](/Hold%20The%20Flood/PM%20Reviews/PMR003) is right that it does not follow. A larger sample would
+
describe the *distribution of prices and lengths* better; it would still observe **no audience, no buyer
+
behaviour and no causation**, so the price-to-audience inference §1.2 withdrew would stay unsupported at
+
any sample size. Filtering by review count also selects for success, which is a bias a bigger sample
+
makes more confident rather than less. The query is mechanical — Action Roguelike tag (10,946 games,
+
REF102 §1), a review-count floor, list price joined to HowLongToBeat Main Story — and it would answer a
+
narrower question than r2 implied. **No expanded survey is assigned and none is being run.**
2. **Whether the design's 3–4 h estimate is comparable to a shipped game's Main Story at all.** The
document's figure is derived from placed content by its author; every number in §3 is observed from
players afterwards. A designer's content estimate and a player's measured time are not the same kind of
@@ 264,9 288,17 @@
### 7.5 The missing evidence, explicitly
-
**To answer Q36 — what the game is for a player who stops early — you need a longitudinal cohort:** a set
-
of players identified at purchase and observed over time, showing where they stopped and whether they
-
returned. **No public Steam endpoint exposes one.** Global achievement percentages have no time axis;
+
**Two different questions were being run together here, and r2 ran them together too.**
+
[PMR003](/Hold%20The%20Flood/PM%20Reviews/PMR003): *"Q36's design answer does not require a retention cohort;
+
measured player retention is a separate unknown."* That is right, and r2's framing — that Q36 needed a
+
cohort and so could not be answered — was wrong.
+
+
- **What the game gives a player who stops after four expeditions is a *design* question**, answerable by
+
the design: what has been delivered by then, and whether it stands on its own. It needs no market data
+
at all, and the [Design Document](/Hold%20The%20Flood/Design%20Document) has answered it on those terms.
+
- **How many real players would in fact stop there is a separate empirical unknown**, and *that* is the
+
one needing a longitudinal cohort: players identified at purchase and observed over time. **No public
+
Steam endpoint exposes one.** Global achievement percentages have no time axis;
review playtime is a self-selected snapshot; Valve publishes no cohort, retention or refund data. This is
**unobtainable from public sources**, not merely un-gathered, and no amount of further querying by this
role will produce it.
@@ 334,4 366,17 @@
repair. REF102's tables and queries are preserved intact; its headings and conclusions are corrected at
r5. §§1–6 are unchanged.
+
**What r3 changed.** [PMR003](/Hold%20The%20Flood/PM%20Reviews/PMR003)'s closing qualifications, applied to
+
this page's own §§2, 3, 6 and 7.5. No new measurement and no expanded survey — none is assigned. Every table,
+
figure, query and provenance note is unchanged. Withdrawn or corrected: §3's "like-for-like" headline, which
+
now carries the prediction-versus-observation limit instead of leaving it in §6; §2's playtime formula, which
+
assumed every reviewer had finished; §6's claim that a systematic sample would make the earlier inferences
+
claimable, which does not follow at any sample size; and §7.5's framing of Q36, which ran a design question
+
and an empirical one together.
+
+
**Pattern worth naming.** All three of this pass's corrections are the same failure in different clothes:
+
a limit stated correctly **somewhere** on the page, and then not carried into the sentence a reader would
+
actually quote. §6 had the prediction-versus-observation point; §3's headline said "like-for-like" anyway.
+
That is not a caveat doing any work.
+
**Corrections.** Kill any figure here by re-running its query and posting the result.
