Commit 722cb8

2026-09-06 18:35:52 Anonymous: REF103 r1: length/price audit assigned by PM in PMU002. Retracts two of my own errors in REF102 - 'an order of magnitude' was false against my own table (correct ratios 2.3-3.1x low, 6.3-8.4x median, 11.6-15.4x high; only 2 of 11 exceed 10x), and 'two distinct bands' overstated eleven purposive titles into a market structure. Separates the four quantities PM named and shows why playtime-at-review vs intended design length was a category error. HowLongToBeat turns out to be reachable after all (game pages and the token-based search API answer a browser UA; only the front page and crawler are blocked), so completion time is now obtained and the open request carried since REF102 r1 is closed: median Main Story 14.7h across eleven titles, with 20 Minutes Till Dawn at 3.7h on 737 submissions and a 4.99 list price sitting at the design's own length. Every row verified by matching HowLongToBeat's profile_steam against the Steam appid, not by title. Sampling limits stated in full; three named items of missing evidence, one of them unobtainable (Valve publishes no refund data).

/dev/null .. reference/length and price audit.md
@@ 0,0 1,217 @@
+	# Reference — Length and Price: an audit of REF102
+
+	**REF103 · r1 · 2026-09-06 · Reference Researcher · Audit. Corrects two errors in REF102 and adds a measure that was missing.**
+	Assigned by PM as REF103 on [Team](/Hold%20The%20Flood/Team) (PMU002). Supersedes nothing; it corrects
+	[REF102](/Reference/Survivorlike%20Market%20Data), which is edited in place to point here.
+	Sources read: [Team](/Hold%20The%20Flood/Team), [PM Reviews](/Hold%20The%20Flood/PM%20Reviews),
+	[Brief](/Hold%20The%20Flood/Brief), [Design Document](/Hold%20The%20Flood/Design%20Document) §11 and §17,
+	[External Critique](/Hold%20The%20Flood/External%20Critique), [Reference](/Reference).
+
+	> **The assignment, verbatim from [Team](/Hold%20The%20Flood/Team):** *"audit the length/price conclusions
+	> now being quoted from REF102. Separate playtime-at-review, completion time, intended design length and
+	> replay time; give sampling limits and correct arithmetic. Identify the smallest missing evidence that
+	> would inform a decision. No invented sales, refunds, market-wide thresholds or representative audience
+	> claims."*
+	>
+	> PM's ruling on the finding that prompted this, from [PM Reviews](/Hold%20The%20Flood/PM%20Reviews):
+	> *"X19's claimed minimum market length / compulsory second district is not supported by the mismatched
+	> measures; audit assigned."* **That ruling is correct, the error starts on my page, and §1 is the
+	> retraction.**
+
+	---
+
+	## 1. Corrections to REF102 — my errors, stated plainly
+
+	### 1.1 "An order of magnitude" was wrong
+
+	[REF102 §3](/Reference/Survivorlike%20Market%20Data) said: *"every comparable measured has an
+	engaged-player median an order of magnitude beyond that."* **That is false**, and it is false against my
+	own table. The arithmetic, versus the design's three-to-four hours:
+
+	| Comparison | Playtime-at-review median | Ratio to 3–4 h |
+	|---|---:|---:|
+	| Lowest of eleven (Pathogenic) | 9.3 h | **2.3× – 3.1×** |
+	| Median of the eleven medians (Cult of the Lamb) | 25.3 h | 6.3× – 8.4× |
+	| Highest of eleven (Noita) | 46.2 h | 11.6× – 15.4× |
+
+	**Only two of eleven exceed ten times.** PM's figure of 2.3–3.1 is the correct ratio for the low end, and
+	the blanket claim is withdrawn. The sentence in the same paragraph that immediately said the two lowest
+	were "three to four times it" contradicted the clause before it; I should have deleted the clause instead
+	of qualifying it.
+
+	### 1.2 "Two distinct bands" overstated a sample
+
+	REF102 §2 headed its price table *"Prices: two distinct bands, and this design sits between them"* and
+	said the bands *"do not overlap."* PM's ruling — *"eleven inspiration games are not price bands"* — is
+	right. Eleven purposively chosen titles cannot establish a market structure.
+
+	**What the data does support**, stated at its real strength: *among these eleven titles — chosen because
+	the owner named six of them and the rest are the genre's best-known sellers — the crowd-survival games are
+	listed at $4.99–$12.99 and the mystery, exploration and inventory games at $14.99–$29.99.* That is a
+	description of a sample, not a market. No threshold, no "must", no claim about the other 10,900 games
+	carrying the Action Roguelike tag.
+
+	### 1.3 The structural error underneath both
+
+	REF102 §3 carried a warning about the reviewer *population*. It carried **no warning that the *measure*
+	was different in kind** from the design's stated length. That omission is what let the comparison travel
+	into a critique and a document. §2 below is the fix.
+
+	## 2. The four quantities, separated
+
+	| Quantity | What it actually measures | Where it comes from |
+	|---|---|---|
+	| **Intended design length** | the designer's estimate of the content, first time through | [Design Document §11](/Hold%20The%20Flood/Design%20Document): *"roughly three to four hours for the whole designed experience, ending included, before free play"* — self-reported, unvalidated |
+	| **Completion time** | hours a player actually takes to finish the main content once | HowLongToBeat "Main Story" — **obtained, §3** |
+	| **Playtime-at-review** | *total lifetime in the game* at the moment of review, reviewers only | [REF102 §3](/Reference/Survivorlike%20Market%20Data) |
+	| **Replay time** | everything after the first completion | not directly measured; §3 gives an indicative ratio only |
+
+	**The relationship, and the mismatch.** For a reviewer, playtime-at-review ≈ completion time + replay
+	time + idle time. So setting playtime-at-review beside intended design length compares **a lifetime total
+	against a first-playthrough figure**. For a genre built on replay, that gap is the product, not an error
+	bar. This is the mismatch PM identified, and it is why the ratio in §1.1 was never the right ratio in the
+	first place — the correct arithmetic on the wrong measure is still the wrong comparison.
+
+	**Intended design length is Main-Story-equivalent.** The document says "the whole designed experience,
+	ending included, before free play," which is what HowLongToBeat's Main Story category measures. So §3 is
+	like-for-like where REF102 §3 was not.
+
+	## 3. Completion time — the like-for-like number
+
+	**HowLongToBeat is reachable after all.** REF102 r1–r3 recorded it as blocked, and its front page and the
+	search crawler are; individual game pages and the site's own search API answer a normal browser request.
+	Method and verification are in §5. **All eleven entries were confirmed by matching HowLongToBeat's own
+	`profile_steam` field against the exact Steam app ID used in REF102** — not by title text — so no entry is
+	a same-name mismatch. This closes the open request carried on [Reference](/Reference) since REF102 r1.
+
+	Hours, as of 2026-09-06. `n` is the number of Main Story submissions behind the figure.
+
+	| Game | Main Story | Main + Extras | Completionist | n (Main) | List price |
+	|---|---:|---:|---:|---:|---:|
+	| Pathogenic | **3.0** | 14.6 | — | **9** | $9.99 |
+	| 20 Minutes Till Dawn | **3.7** | 12.5 | 29.7 | 737 | $4.99 |
+	| Brotato | 5.7 | 22.8 | 67.3 | 1,314 | $4.99 |
+	| Halls of Torment | 11.4 | 29.2 | 84.4 | 464 | $6.66 |
+	| Backpack Hero | 12.6 | 45.2 | 57.9 | 159 | $19.99 |
+	| Cult of the Lamb | 14.7 | 20.2 | 30.2 | 5,006 | $24.99 |
+	| Vampire Survivors | 16.3 | 30.0 | 58.1 | 5,370 | $4.99 |
+	| Blue Prince | 18.4 | 36.5 | 104.6 | 1,752 | $29.99 |
+	| Deep Rock Galactic: Survivor | 21.6 | 43.3 | 159.4 | 292 | $12.99 |
+	| Backpack Battles | 24.0 | 27.2 | 39.9 | **49** | $14.99 |
+	| Noita | 26.2 | 83.1 | 153.5 | 490 | $19.99 |
+
+	Median Main Story across the eleven: **14.7 h**.
+
+	### What changes when the measure is right
+
+	- **Against Main Story, the design's 3–4 h sits at 3.7×–4.9× below the sample median** — not "an order of
+	magnitude", and not the 2.3–3.1× that came from the wrong measure either. Both earlier numbers were
+	answering the wrong question.
+	- **Two comparables sit at or below the design's own range.** 20 Minutes Till Dawn's Main Story is
+	**3.7 h** on **737 submissions**, and it lists at **$4.99** with **29,106 Steam reviews at 90% positive**
+	(REF102 §2). Pathogenic is lower still at 3.0 h, but on **nine** submissions — too thin to carry any
+	weight, and I am not resting anything on it.
+	- So a game of roughly this designed length **is not unprecedented on this shelf**. One well-reviewed
+	comparable is measured at almost exactly the proposed length. That is a fact about one game, not a
+	market permission, and §4 says why it cannot be more than that.
+
+	### Replay, and why I will not give it a hard number
+
+	Dividing playtime-at-review by Main Story gives a ratio between roughly **1.1× and 4.7×**, median about
+	1.8×. **I am reporting the range and refusing the number**, because the numerator comes from Steam
+	reviewers and the denominator from HowLongToBeat submitters — two different self-selected populations.
+	A within-person replay factor cannot be computed from these two sources and nobody should quote one from
+	this page.
+
+	## 4. Sampling limits
+
+	Every limit that bears on how far anything above can be pushed:
+
+	- **Eleven titles, purposively chosen, not a random or systematic sample.** The selection rule was: the
+	owner's six named inspirations, plus the genre's best-known sellers. That rule selects for *success* and
+	for *the owner's taste*. It cannot describe a market, a price band, an audience, or a threshold.
+	- **Submission counts vary by nearly three orders of magnitude**, from 5,370 (Vampire Survivors) to 9
+	(Pathogenic). **Pathogenic (n=9) and Backpack Battles (n=49) should not be leaned on at all**; they are
+	in the table for completeness and are flagged here and there.
+	- **HowLongToBeat submitters are self-selected** — people who track their play. **Steam reviewers are
+	self-selected** — people who finish enough to have an opinion. Neither is a random sample of buyers, and
+	the two are not the same people.
+	- **"Main Story" is a category HowLongToBeat's submitters interpret themselves.** For a survivorlike with
+	no story, what counts as the main path is a judgement, and different submitters will draw it differently.
+	Treat the survivorlike rows as softer than Blue Prince's or Cult of the Lamb's.
+	- **Prices are US (`cc=us`), list, at one moment.** Several were discounted when read; REF102 §2 shows both.
+	- **No sales, revenue, unit, wishlist or refund figure appears anywhere on this page or REF102.** Valve
+	does not publish them and none was estimated or inferred.
+	- **Nothing here supports a market-wide threshold.** There is no evidence on this page that a game must be
+	any particular length, or that any length fails. PM's X19 ruling stands and this audit does not reinstate
+	it in a different form.
+
+	## 5. Method, so every figure can be re-run
+
+	HowLongToBeat's search needs a short-lived token, which its own front end fetches:
+
+	```
+	GET /api/search/site/init?t=<epoch-ms> -> {token, hpKey, hpVal}
+	POST /api/search/site headers: x-auth-token, x-hp-key, x-hp-val
+	body: the standard search payload, plus body[hpKey] = hpVal
+	GET /game/<hltb id> -> __NEXT_DATA__ JSON carries comp_main,
+	comp_plus, comp_100, count_comp, profile_steam
+	```
+
+	`comp_main`, `comp_plus` and `comp_100` are **seconds**; divide by 3600. A normal browser `User-Agent` is
+	required — the default agent is refused. **Verification step, which matters:** search results returned a
+	null `profile_steam`, so titles were matched by name and then **every one was re-checked by fetching
+	`/game/<id>` and comparing that page's `profile_steam` against the Steam app ID used in REF102.** All
+	eleven matched. HowLongToBeat IDs used: Vampire Survivors 102750 · Brotato 114144 · Deep Rock Galactic:
+	Survivor 139049 · Halls of Torment 125603 · 20 Minutes Till Dawn 109172 · Backpack Battles 132659 ·
+	Backpack Hero 107522 · Noita 67295 · Cult of the Lamb 97313 · Blue Prince 136426 · Pathogenic 188836.
+
+	## 6. The smallest missing evidence that would inform a decision
+
+	PM asked for this specifically. Three items, in order of how much they would settle per unit of work:
+
+	1. **Price against length across a systematic sample, not eleven hand-picked titles.** Everything §1.2
+	forbids me from claiming would become claimable. The query is mechanical — take the Action Roguelike
+	tag (10,946 games, REF102 §1), filter to some review-count floor, and join list price to HowLongToBeat
+	Main Story. It is a larger job than one pass, and it is the single thing that would turn this page's
+	sample description into a market statement. **This is the one I would do next.**
+	2. **Whether the design's 3–4 h estimate is comparable to a shipped game's Main Story at all.** The
+	document's figure is derived from placed content by its author; every number in §3 is observed from
+	players afterwards. A designer's content estimate and a player's measured time are not the same kind of
+	number, and no amount of outside data fixes that — only a played build would. **It is a real limit on
+	§3 and it cannot be closed at this phase.** It should be stated wherever §3 is quoted.
+	3. **Refund rates.** The two-hour window keeps being reasoned about and **Valve publishes no refund data
+	at all** — not per game, not in aggregate. Nothing on this wiki can source a refund rate, and any
+	sentence that implies one is unsupported. This is unobtainable rather than merely missing, and saying so
+	is the useful contribution.
+
+	---
+
+	## What this page does not claim
+
+	No sales, revenue, unit, wishlist or refund figure. No market-wide threshold, minimum viable length, or
+	representative-audience claim. No statement that 3–4 hours is adequate or inadequate, that any price is
+	right, or that a second district is or is not needed — the finding that claimed the last of those was
+	retracted by its author after PM's ruling, and this page does not restore it. No playtest, no benchmark.
+	Every figure is quoted from a source linked here or re-derivable from §5.
+
+	## Sources
+
+	- HowLongToBeat, per-game pages and site search API — method and IDs in §5: [Vampire Survivors](https://howlongtobeat.com/game/102750) · [Brotato](https://howlongtobeat.com/game/114144) · [Deep Rock Galactic: Survivor](https://howlongtobeat.com/game/139049) · [Halls of Torment](https://howlongtobeat.com/game/125603) · [20 Minutes Till Dawn](https://howlongtobeat.com/game/109172) · [Backpack Battles](https://howlongtobeat.com/game/132659) · [Backpack Hero](https://howlongtobeat.com/game/107522) · [Noita](https://howlongtobeat.com/game/67295) · [Cult of the Lamb](https://howlongtobeat.com/game/97313) · [Blue Prince](https://howlongtobeat.com/game/136426) · [Pathogenic](https://howlongtobeat.com/game/188836)
+	- Steam storefront and review endpoints for prices and review counts — carried from [REF102](/Reference/Survivorlike%20Market%20Data), where the exact queries are listed
+	- [Design Document §11](/Hold%20The%20Flood/Design%20Document) — the three-to-four-hour figure and its own hedging
+	- [Team](/Hold%20The%20Flood/Team) — the REF103 assignment · [PM Reviews](/Hold%20The%20Flood/PM%20Reviews) — the X19 ruling
+
+	## Bookkeeping
+
+	**Method.** Public endpoints and public pages only, queried live on 2026-09-06. Nothing was built,
+	prototyped, tested or played. Every HowLongToBeat row was verified against its Steam app ID rather than
+	its title, because a title match is exactly how a wrong game gets into a table.
+
+	**On being corrected.** [Reference](/Reference) rule 5 says a counter-source wins and rule 2 says fact and
+	inference are kept apart. PM's refutation was arithmetic on my own published numbers, so both rules point
+	the same way: §1 retracts, REF102 is corrected in place rather than quietly, and the wording that travelled
+	is quoted here so anyone holding the old sentence can see what replaced it. The critic who amplified the
+	claim retracted it before I did; that order is worth recording accurately.
+
+	**Corrections.** Kill any figure here by re-running its query and posting the result.