Methods, Data and Statistical Appendix

Executive summary 🔗

This appendix documents the complete pipeline behind "Do the World's Best Players Play the Openings That Suit Them?" — data sources, eligibility rules, exact definitions, engine configuration, statistical methods, full result tables, a transparent account of one data-processing error found and corrected mid-study, and the limitations of every claim made in the main article. It is not a second narrative and introduces no new analysis: every figure below is the same final, frozen output the article draws from.

In brief: we compared 100 recent classical games each for Magnus Carlsen and Fabiano Caruana, drawn from different but overlapping time windows. We measured repertoire breadth (opening-family concentration) and how often each player reaches a substantially cleared central pawn structure by move 20, then tested six primary objective-quality outcomes — Stockfish evaluation, expected score, player-caused loss, evaluation volatility, and two Central Control Score (CCS) measures — using an event-aware bootstrap, effect sizes, colour and opponent-strength splits, and an equal-size common-period resampling test. All six outcomes returned null results (95% CIs including zero, negligible effect sizes) in every view tested. Repertoire breadth and central-clearance frequency, by contrast, showed large, robust differences that survived every check applied. None of this is proof that the two players' repertoires are "the same" — only that no difference was detectable on the specific measures and views tested here.

1. Research questions and claims not being tested 🔗

The study's frozen question list (engine side) asked: does Carlsen's broader repertoire produce worse opening evaluations or greater volatility than Caruana's; does Caruana's concentration produce lower player-caused loss; do any such differences survive colour, opponent-strength, and common-period controls; and is CCS redundant with engine evaluation. The CCS side asked the analogous questions using Central Control Score instead of Stockfish evaluation.

Explicitly out of scope, by standing instruction, at every stage of this project: an "opening-fit score" of any kind; any psychological, MBTI, or personality inference; any causal claim connecting repertoire structure to match results; and any claim that one player's repertoire is objectively superior to the other's. The main article's "How would they answer the RBOTChess questionnaire?" section is explicitly an inference exercise, not a measurement, and is treated that way throughout — see §18.

2. Data sources and eligible-game criteria 🔗

Source data: four raw PGN exports (Carlsen-White, Carlsen-Black, Caruana-White, Caruana-Black), 3,146 raw Carlsen games and 2,777 raw Caruana games in total, spanning each player's available classical, rapid, and blitz output up to the cutoff date 2026-06-30. No source game carries a [TimeControl] PGN tag, so time-control classification is inferred entirely from event metadata (§5) — this is inference, not fact, and is documented as such throughout the pipeline.

A game is eligible for the study if: it has no colour-tag mismatch between the PGN header and the target player's actual colour; the player's name is found in the tags; it is not an exact duplicate of another game in the export; it is a genuinely played game (see below); and it falls within the confirmed- or high-confidence-classical time-control buckets (§5).

Played-game check. A record is excluded as non_played_or_malformed_record if fewer than two legal plies can be replayed from the PGN movetext (i.e. it does not represent at least one full move pair by both sides). This check removed 2 Carlsen records and 4 Caruana records from the raw pools before any sampling began, including the one described in §6.

3. Frozen 100+100 sample 🔗

Table 1 — sample construction.

Both final samples are exactly 100 games. Caruana's larger admissible pool (200 vs. 100) means his 100-game sample is the 100 most recent eligible games out of a larger pool; Carlsen's sample is effectively his entire admissible pool for the period, which is why his sample reaches further back in time (§4, §7).

4. Exact sample dates and colour distribution 🔗

Table 2 — sample windows and colour split.

The two samples are not time-matched — this is a real, acknowledged limitation, addressed methodologically in §7 rather than hidden.

5. Event-format verification and exclusion rules 🔗

Time-control classification follows a strict order of authority: (1) a PGN TimeControl tag if present (none were, for any source game); (2) a verified event registry (lib/event-registry.mjs) recording each tournament's actual format from external knowledge; (3) name-based heuristics (event names containing "blitz," "rapid," "bullet," "tiebreak," "playoff," "armageddon," etc. are excluded outright); (4) a conservative fallback to uncertain when nothing else applies. Heuristics never override verified registry entries. Knockout-format events (FIDE World Cup, American Cup) mix classical games with rapid/blitz tiebreaks under one event name and cannot be safely separated by name alone — every game from such an event is therefore left in the uncertain bucket and excluded from the sample, at the cost of some true classical games being discarded.

6. The excluded malformed record and its mechanical replacement 🔗

One record initially occupying a primary-sample place — Caruana vs. Narayanan, S.L., 2025-12-06, "Bundesliga 2025-26" — was found on review to contain only one legal ply, i.e. no completed move pair by both sides, and was reclassified as non_played_or_malformed_record and removed. Per the frozen replacement rule, the next-most-recent analysis-eligible Caruana game was promoted into its place mechanically, by the existing date-descending sort — not selected to adjust colour balance, results, or any other outcome. The promoted game (Caruana vs. Yoo, Christopher Woojin, 2024-10-16, a 96-ply/48-move decisive game, Round 5.5 of the 2024 U.S. Championship) carries an anomalous raw PGN Event tag ("ch-USA Cancelled 2024") that was preserved as-is for traceability and separately mapped to the correct canonical event via a registry alias — the tag anomaly is a PGN metadata artifact, not evidence the game was cancelled. This single substitution shifted Caruana's sample by exactly one game (199/200 combined games unchanged across both players) and moved the common-period start date from 2024-10-17 to 2024-10-16. Full accounting: sample_diff_audit.md.

7. Different historical windows and common-period robustness analysis 🔗

Because Carlsen's admissible pool for this period is smaller than Caruana's, his 100-game sample spans roughly 3.5 years while Caruana's spans about 1.5 years. Two independent strategies address this:

Common-period sample. All of Carlsen's primary-sample games (23 games) that fall within the date range shared by both players' full samples were compared directly against every Caruana primary-sample game in the same window (up to 100). This "common-period" comparison, on its own, is not the primary evidence for any claim in the article — it is one of several views checked.

Equal-size resampling. To avoid any spurious effect of Caruana's common-period pool being larger than Carlsen's fixed 23-game set, every common-period comparison was also run as a resampling test: 2,000 event-block resamples (whole tournaments drawn together, not individual games) of exactly 23 games each were drawn from Caruana's common-period pool, and Carlsen's actual value was located as a percentile within that distribution of 2,000 resampled statistics. A percentile near 0 or 100 means Carlsen's figure lies at or beyond the extreme edge of what any comparable-size Caruana subsample produces; a percentile in the middle of the range means it does not. This is the test used for every "survives equal-size resampling" claim in the article, and it is the same event-aware resampling machine used for the primary-outcome bootstrap in §12.

Leave-one-event-out check. For the two engine outcomes (E1, E4) that were affected by the CSV-parsing error described in §16, an additional sensitivity check recomputed the common-period percentile five times, once per Carlsen common-period event, with that event's games removed each time. Both percentiles moved substantially under this check (E1: range 35.0–81.8; E4: range 7.5–51.0) and never approached an extreme value under any single-event removal — confirming the corrected null result is not an artifact of one event's presence in the sample. Full table: common_period_e1_e4_audit.md.

8. Opening-family classification and repertoire-concentration measure 🔗

Games are grouped by ECO opening family as recorded in the source PGNs. Repertoire concentration is measured with a Herfindahl index: for a player with games distributed across opening families with shares s₁, s₂, …, s_n (each s_i = fraction of that player's games in family i), the index is:

H ranges from close to 0 (games spread evenly across many families) to 1 (all games in one family). Lower H means a broader, less concentrated repertoire.

Table 3 — repertoire concentration, common-period equal-size test.

Carlsen's common-period Herfindahl index sits at or below every one of the 2,000 equal-size Caruana resamples — the most extreme possible reading under this test, meaning his repertoire breadth is not explained by Caruana simply having more games available in the shared window.

9. Exact definitions of the structural measures 🔗

centralPawnClearance(fen) — a lenient measure. Counts total pawns on the d- and e-files for both colours combined; returns open if at most two such pawns remain on the board. This is a "mostly cleared" reading, not a requirement that the centre be entirely empty, entirely open in the file sense, or that central tension has been resolved by exchange rather than by pawns simply not having advanced there.

strictOpenCentre(fen) — a much stricter measure, defined independently, from a different source module. Returns true only if the count of central d/e-pawns for both colours combined is exactly zero — no d- or e-pawn remains for either side. These two measures answer different questions and are never treated as two versions of the same variable.

Central Control Score (CCS) — the current production implementation (src/lib/ccs.ts, v3), imported directly at runtime with no transpilation or hand-porting, so this study measures exactly what the live RBOTChess /ccs route would compute for the same position. CCS is a pure function of board position — no rating, Elo, or skill-tier input exists anywhere in its computation. Per colour, four components are summed into a total: occupation (a piece physically on a central square, weighted by piece type and core-vs-extended-centre location, with a −2 penalty if the piece is pinned to its own king); control (a central square attacked by only one colour, weighted 1.5× if the defending king hasn't castled); snipers (bishops, rooks, or knights attacking central squares from outside the centre, via open/half-open files or diagonals); and v2 additions (a fianchetto bonus, an unplayed pawn-break bonus, and a small development-lead term). ccsNetTarget = ccsTargetPlayer − ccsOpponent, positive always favouring whichever player's repertoire is being measured, mirroring the engine's perspective convention. Full component weights: ccs_implementation_freeze.md.

10. Engine configuration 🔗

Stockfish 18 Lite WASM, single-threaded (Threads: 1), NNUE file nn-9067e33176e8.nnue, Hash: 64, MultiPV: 1, node budget 300,000 nodes per position (not depth- or time-limited), Skill Level: 20 (full strength), UCI_ShowWDL: true. Positions were evaluated at plies corresponding to moves 10, 15, 20, and 25 for each player's own games (window: ply 14 to ply 50). Every raw engine result (cp, mate, WDL) was converted to a target-player perspective value (cpTarget, mateTarget) using the same sign-conversion logic as CCS, unit-tested independently for both colours before any real game was analysed. Results are cached by a composite key of FEN + engine version + full options object + node budget, so a position is only re-evaluated if any of those change — this is what makes the pipeline resumable and what allowed a later sample correction (§6) to reuse ~99.5% of prior engine work (7,131 of 7,168 final cache entries were untouched by the correction; 37 new positions were computed for the one promoted game).

11. Definitions of E1–E4, C1, C2 🔗

12. Statistical method 🔗

Game as the sole inferential unit. No position or individual move is ever treated as an independent observation for hypothesis testing — every primary-outcome statistic is one value per game, aggregated at the player level.

Event-aware bootstrap. All confidence intervals use a resampling procedure that draws whole tournament/match events (not individual games) with replacement, 2,000 iterations, so that games from the same event — which share an opponent pool, playing conditions, and preparation cycle — are never treated as independent draws.

Effect sizes. Cliff's delta (non-parametric; thresholds |δ| < 0.147 negligible, < 0.33 small, < 0.474 medium, else large, following Romano et al. 2006) as the primary statistic, reported alongside Cohen's d (pooled-SD) for a familiar parametric reading.

Views tested per primary outcome: raw (all 100 vs. 100 games), White-only, Black-only, opponent-Elo-difference tercile split (pooled thresholds), and common-period equal-size resampling (§7). E1 and E4 additionally received the leave-one-event-out sensitivity check described in §7.

13. Complete result tables, primary outcomes 🔗

Table 4 — Engine outcomes (E1–E4).

Table 5 — CCS outcomes (C1–C2).

Every raw 95% CI includes zero; every common-period percentile falls well inside the resampled range (none below 19 or above 66, against a 0–100 scale where an extreme finding would sit at or beyond the resampled range's edge, as in Table 3). Colour-split and Elo-tercile views (not tabulated here for space; full figures in engine_hypothesis_report.md) show no consistent direction for any outcome — where White and Black splits move in opposite directions for the same outcome (e.g. E4), this is itself evidence against a stable underlying difference, not for one.

14. Repertoire-breadth and central-clearance results 🔗

Table 6 — central pawn clearance @ move 20, primary sample.

n is smaller than 100 for both players because 2–3 games per player are shorter than move 20 and cannot reach the checkpoint — this is the correct denominator; any earlier document using the full 100-game count for this specific ratio used the wrong one. Both measures point the same direction and both survive the equal-size resampling test at the extreme edge of the distribution (percentile 100.0 for both, Table 3's companion result), meaning the gap is not explained by Caruana's larger available pool in the shared time window.

15. CCS–engine correlation and clean-divergence definition 🔗

Pooled across both players (n=195 positions with both measures available at move 20): Pearson r = 0.026, Spearman ρ = −0.044. Per player: Carlsen r = 0.108, Caruana r = −0.070. All four are negligible by any conventional threshold. This is reported as non-redundancy — CCS is not a proxy for, or a validation of, engine evaluation; the two measures were designed to describe different things (central space vs. objective advantage) and the near-zero correlation is the expected, honest confirmation of that, not a discovery that either measure is flawed.

Clean divergence is defined as:

Under this definition, 21 of 195 move-20 positions (10.8%) diverge — 10 from Carlsen's games, 11 from Caruana's. This rate is descriptive, not a hypothesis-tested claim with a confidence interval, and is consistent with (not independent evidence beyond) the near-zero correlation above.

16. Data-integrity audit and CSV-parsing correction 🔗

During preparation of the publication evidence pack, a bug was found in one analysis script's CSV reader: it split each row on literal commas (line.split(',')) without accounting for quoted fields, and 189 of engine_game_summary.csv's 200 rows contain a quoted comma in the opponent column (e.g. "Caruana, Fabiano"). This silently misaligned every column after the comma for those rows, corrupting the date+event+opponent join used to build the common-period comparison for two outcomes (E1, E4). The corrupted join had produced an apparent 100th-percentile and 0th-percentile "extreme" result for those two outcomes; both were retracted once the bug was found, not silently corrected — the retraction is recorded in engine_hypothesis_report.md's correction notice (commit 4cf0077).

A subsequent, dedicated audit (commit 70c220e) replaced every ad-hoc CSV parser across the pipeline with one shared, quote-aware implementation (lib/csv.mjs), added regression tests (quoted commas, escaped quotes, empty fields, numeric zero, embedded-newline rejection), switched several readers to the pipeline's canonical NDJSON/JSON outputs in place of re-parsing CSV, and added a standing consistency check (csv-json-consistency-check.mjs) that verifies every CSV file and its JSON/NDJSON sibling have identical row counts and identical row identifiers — all 10 checked pairs pass. Every publication-facing computation was rerun after the fix; the corrected E1/E4 figures in Table 4 are unchanged from the values first computed after the original fix, confirming the second audit's parser consolidation introduced no further change. One outcome (E2, expectedScore20) shows a 66.0 vs. an earlier-recorded 66.1 — traced to a floating-point display artifact ((66.05).toFixed(1) resolves to "66.0" because 66.05 has no exact binary representation), confirmed unrelated to the parser fix via a byte-for-byte comparison of the CSV and NDJSON inputs, and immaterial to the outcome's null classification either way.

17. Why unstable structural clusters were excluded 🔗

An earlier stage of this project clustered games by pawn-structure features (five clusters, k=5, chosen by a geometric elbow method). That clustering did not survive a robustness check that re-weighted the input features to remove an encoding artifact (the c/d/e/f pawn-structure block had 4× the distance-influence of the symmetry/space block purely from having more columns, not from being more informative) — the re-weighted clustering agreed with the original on only 49.7% of rows after optimal label matching (Adjusted Rand Index 0.33), well below the threshold for treating it as a stable, encoding-independent finding. Because of this, no claim in the article or this appendix rests on the specific five-cluster identities or their frequencies; the two structural findings that are used (repertoire-family concentration, central-clearance frequency) do not depend on that clustering at all.

18. Evidence map for every inferred competence-building claim 🔗

Table 7 — evidence tier for the "How would they answer the questionnaire?" section.

No row in the "Inference" tier is presented in the main article as measured, and no claim anywhere in this project asserts causality between repertoire architecture and match results, or infers a personality type.

19. Limitations 🔗

• Unequal, non-matched sample windows. Carlsen's 100-game sample spans about 3.5 years, Caruana's about 1.5 years, because Caruana's admissible classical-game pool for the recent period was larger. This is addressed via the common-period and equal-size-resampling views (§7), not eliminated.
• Time-control classification is inferential, resting on event names and a manually verified registry, because no source PGN carries a TimeControl tag. Ambiguous-format (knockout) events are excluded entirely rather than guessed at, which slightly shrinks both admissible pools.
• Small common-period Carlsen sample (n=23) makes the common-period view directional rather than fully reliable on its own; this is why every common-period finding is cross-checked against the raw 100-vs-100 view and, for E1/E4, a leave-one-event-out check.
• CCS and Stockfish evaluation are each single-snapshot, per-checkpoint measures (moves 10/15/20/25) — neither captures the full continuous course of a game, and neither should be read as a complete account of "opening quality."
• The centralPawnClearance/strictOpenCentre measures describe pawn count only, not tactical sharpness, king safety, or piece activity — a cleared centre can be simple or extremely sharp; this study cannot and does not distinguish the two.
• Public statements about either player's preparation approach are secondary reporting, not verified first-hand accounts of private study habits, and are treated in the article and this appendix as evidence for an inference, not as direct measurement.
• No causal claims are supported by this design. Every relationship reported (repertoire architecture, central clearance, CCS/engine divergence) is descriptive/associative; the study was not built to test whether repertoire choice causes any downstream result.

20. Reproducibility 🔗

All scripts referenced above live under scripts/elite-fit/ in the RBOTChess web application repository; raw and derived data live outside that repository and are available on request (not published alongside this article). Key files: qc.mjs (sample selection), lib/openings.mjs / lib/structuralFeatures.mjs (structural definitions), lib/ccsAdapter.mjs + src/lib/ccs.ts (CCS, frozen at commit 88dc82d), engine-run.mjs (Stockfish evaluation), hypothesis-analysis.mjs (primary-outcome statistics), engine-equal-size-robustness.mjs / common-period-audit.mjs (common-period tests), lib/csv.mjs + csv-json-consistency-check.mjs (data-integrity audit, §16), test-csv.mjs, test-ccs-adapter.mjs, test-perspective.mjs, test-stats.mjs (regression tests referenced throughout). Relevant commits, in order: d836a4f, ba1c167, d06783d, d8cb26c, 39b50cd, 0f4064f, 6348c33, 020913f, 4cf0077 (CSV-parsing correction), 70c220e (final data-integrity audit).

21. Sources 🔗

1. World Championship Game 2: Preparation counterpunch ↗ — ChessBase, 2018.
2. Magnus Carlsen — "I don't quite fit into the usual schemes" ↗ — ChessBase.
3. FIDE WCCM: Tie-break ↗ — FIDE, 2018 World Championship tiebreak result.