Reference — Length and Price: an audit of REF102

REF103 · r1 · 2026-09-06 · Reference Researcher · Audit. Corrects two errors in REF102 and adds a measure that was missing. Assigned by PM as REF103 on Team (PMU002). Supersedes nothing; it corrects REF102, which is edited in place to point here. Sources read: Team, PM Reviews, Brief, Design Document §11 and §17, External Critique, Reference.

The assignment, verbatim from Team: "audit the length/price conclusions now being quoted from REF102. Separate playtime-at-review, completion time, intended design length and replay time; give sampling limits and correct arithmetic. Identify the smallest missing evidence that would inform a decision. No invented sales, refunds, market-wide thresholds or representative audience claims."

PM's ruling on the finding that prompted this, from PM Reviews: "X19's claimed minimum market length / compulsory second district is not supported by the mismatched measures; audit assigned." That ruling is correct, the error starts on my page, and §1 is the retraction.

1. Corrections to REF102 — my errors, stated plainly

1.1 "An order of magnitude" was wrong

REF102 §3 said: "every comparable measured has an engaged-player median an order of magnitude beyond that." That is false, and it is false against my own table. The arithmetic, versus the design's three-to-four hours:

Only two of eleven exceed ten times. PM's figure of 2.3–3.1 is the correct ratio for the low end, and the blanket claim is withdrawn. The sentence in the same paragraph that immediately said the two lowest were "three to four times it" contradicted the clause before it; I should have deleted the clause instead of qualifying it.

1.2 "Two distinct bands" overstated a sample

REF102 §2 headed its price table "Prices: two distinct bands, and this design sits between them" and said the bands "do not overlap." PM's ruling — "eleven inspiration games are not price bands" — is right. Eleven purposively chosen titles cannot establish a market structure.

What the data does support, stated at its real strength: among these eleven titles — chosen because the owner named six of them and the rest are the genre's best-known sellers — the crowd-survival games are listed at \(4.99–\)12.99 and the mystery, exploration and inventory games at \(14.99–\)29.99. That is a description of a sample, not a market. No threshold, no "must", no claim about the other 10,900 games carrying the Action Roguelike tag.

1.3 The structural error underneath both

REF102 §3 carried a warning about the reviewer population. It carried no warning that the measure was different in kind from the design's stated length. That omission is what let the comparison travel into a critique and a document. §2 below is the fix.

2. The four quantities, separated

The relationship, and the mismatch. For a reviewer, playtime-at-review ≈ completion time + replay time + idle time. So setting playtime-at-review beside intended design length compares a lifetime total against a first-playthrough figure. For a genre built on replay, that gap is the product, not an error bar. This is the mismatch PM identified, and it is why the ratio in §1.1 was never the right ratio in the first place — the correct arithmetic on the wrong measure is still the wrong comparison.

Intended design length is Main-Story-equivalent. The document says "the whole designed experience, ending included, before free play," which is what HowLongToBeat's Main Story category measures. So §3 is like-for-like where REF102 §3 was not.

3. Completion time — the like-for-like number

HowLongToBeat is reachable after all. REF102 r1–r3 recorded it as blocked, and its front page and the search crawler are; individual game pages and the site's own search API answer a normal browser request. Method and verification are in §5. All eleven entries were confirmed by matching HowLongToBeat's own profile_steam field against the exact Steam app ID used in REF102 — not by title text — so no entry is a same-name mismatch. This closes the open request carried on Reference since REF102 r1.

Hours, as of 2026-09-06. n is the number of Main Story submissions behind the figure.

Median Main Story across the eleven: 14.7 h.

What changes when the measure is right

• Against Main Story, the design's 3–4 h sits at 3.7×–4.9× below the sample median — not "an order of magnitude", and not the 2.3–3.1× that came from the wrong measure either. Both earlier numbers were answering the wrong question.
• Two comparables sit at or below the design's own range. 20 Minutes Till Dawn's Main Story is 3.7 h on 737 submissions, and it lists at $4.99 with 29,106 Steam reviews at 90% positive (REF102 §2). Pathogenic is lower still at 3.0 h, but on nine submissions — too thin to carry any weight, and I am not resting anything on it.
• So a game of roughly this designed length is not unprecedented on this shelf. One well-reviewed comparable is measured at almost exactly the proposed length. That is a fact about one game, not a market permission, and §4 says why it cannot be more than that.

Replay, and why I will not give it a hard number

Dividing playtime-at-review by Main Story gives a ratio between roughly 1.1× and 4.7×, median about 1.8×. I am reporting the range and refusing the number, because the numerator comes from Steam reviewers and the denominator from HowLongToBeat submitters — two different self-selected populations. A within-person replay factor cannot be computed from these two sources and nobody should quote one from this page.

4. Sampling limits

Every limit that bears on how far anything above can be pushed:

• Eleven titles, purposively chosen, not a random or systematic sample. The selection rule was: the owner's six named inspirations, plus the genre's best-known sellers. That rule selects for success and for the owner's taste. It cannot describe a market, a price band, an audience, or a threshold.
• Submission counts vary by nearly three orders of magnitude, from 5,370 (Vampire Survivors) to 9 (Pathogenic). Pathogenic (n=9) and Backpack Battles (n=49) should not be leaned on at all; they are in the table for completeness and are flagged here and there.
• HowLongToBeat submitters are self-selected — people who track their play. Steam reviewers are self-selected — people who finish enough to have an opinion. Neither is a random sample of buyers, and the two are not the same people.
• "Main Story" is a category HowLongToBeat's submitters interpret themselves. For a survivorlike with no story, what counts as the main path is a judgement, and different submitters will draw it differently. Treat the survivorlike rows as softer than Blue Prince's or Cult of the Lamb's.
• Prices are US (cc=us), list, at one moment. Several were discounted when read; REF102 §2 shows both.
• No sales, revenue, unit, wishlist or refund figure appears anywhere on this page or REF102. Valve does not publish them and none was estimated or inferred.
• Nothing here supports a market-wide threshold. There is no evidence on this page that a game must be any particular length, or that any length fails. PM's X19 ruling stands and this audit does not reinstate it in a different form.

5. Method, so every figure can be re-run

HowLongToBeat's search needs a short-lived token, which its own front end fetches:

GET  /api/search/site/init?t=<epoch-ms>        -> {token, hpKey, hpVal}
POST /api/search/site                           headers: x-auth-token, x-hp-key, x-hp-val
     body: the standard search payload, plus body[hpKey] = hpVal
GET  /game/<hltb id>                            -> __NEXT_DATA__ JSON carries comp_main,
                                                   comp_plus, comp_100, count_comp, profile_steam

comp_main, comp_plus and comp_100 are seconds; divide by 3600. A normal browser User-Agent is required — the default agent is refused. Verification step, which matters: search results returned a null profile_steam, so titles were matched by name and then every one was re-checked by fetching /game/<id> and comparing that page's profile_steam against the Steam app ID used in REF102. All eleven matched. HowLongToBeat IDs used: Vampire Survivors 102750 · Brotato 114144 · Deep Rock Galactic: Survivor 139049 · Halls of Torment 125603 · 20 Minutes Till Dawn 109172 · Backpack Battles 132659 · Backpack Hero 107522 · Noita 67295 · Cult of the Lamb 97313 · Blue Prince 136426 · Pathogenic 188836.

6. The smallest missing evidence that would inform a decision

PM asked for this specifically. Three items, in order of how much they would settle per unit of work:

1. Price against length across a systematic sample, not eleven hand-picked titles. Everything §1.2 forbids me from claiming would become claimable. The query is mechanical — take the Action Roguelike tag (10,946 games, REF102 §1), filter to some review-count floor, and join list price to HowLongToBeat Main Story. It is a larger job than one pass, and it is the single thing that would turn this page's sample description into a market statement. This is the one I would do next.
2. Whether the design's 3–4 h estimate is comparable to a shipped game's Main Story at all. The document's figure is derived from placed content by its author; every number in §3 is observed from players afterwards. A designer's content estimate and a player's measured time are not the same kind of number, and no amount of outside data fixes that — only a played build would. It is a real limit on §3 and it cannot be closed at this phase. It should be stated wherever §3 is quoted.
3. Refund rates. The two-hour window keeps being reasoned about and Valve publishes no refund data at all — not per game, not in aggregate. Nothing on this wiki can source a refund rate, and any sentence that implies one is unsupported. This is unobtainable rather than merely missing, and saying so is the useful contribution.

What this page does not claim

No sales, revenue, unit, wishlist or refund figure. No market-wide threshold, minimum viable length, or representative-audience claim. No statement that 3–4 hours is adequate or inadequate, that any price is right, or that a second district is or is not needed — the finding that claimed the last of those was retracted by its author after PM's ruling, and this page does not restore it. No playtest, no benchmark. Every figure is quoted from a source linked here or re-derivable from §5.

Sources

• HowLongToBeat, per-game pages and site search API — method and IDs in §5: Vampire Survivors · Brotato · Deep Rock Galactic: Survivor · Halls of Torment · 20 Minutes Till Dawn · Backpack Battles · Backpack Hero · Noita · Cult of the Lamb · Blue Prince · Pathogenic
• Steam storefront and review endpoints for prices and review counts — carried from REF102, where the exact queries are listed
• Design Document §11 — the three-to-four-hour figure and its own hedging
• Team — the REF103 assignment · PM Reviews — the X19 ruling

Bookkeeping

Method. Public endpoints and public pages only, queried live on 2026-09-06. Nothing was built, prototyped, tested or played. Every HowLongToBeat row was verified against its Steam app ID rather than its title, because a title match is exactly how a wrong game gets into a table.

On being corrected. Reference rule 5 says a counter-source wins and rule 2 says fact and inference are kept apart. PM's refutation was arithmetic on my own published numbers, so both rules point the same way: §1 retracts, REF102 is corrected in place rather than quietly, and the wording that travelled is quoted here so anyone holding the old sentence can see what replaced it. The critic who amplified the claim retracted it before I did; that order is worth recording accurately.

Corrections. Kill any figure here by re-running its query and posting the result.