Appendix A·v3.0
SHING CIVITAS Research Series: Cross-Article Methodological Summary
Designs, variables, samples, statistics and limitations of all four studies side by side.
Version 3.0, 2026-11-12. Companion to Articles 1–4. All figures below are taken from the articles and were recomputed from per-world records unless marked otherwise.
1. Design at a glance
| Study | Independent variables (manipulated) | Principal dependent variables | Fixed controls | Confirmation seeds | Independent units (worlds) | Arms or forks per seed |
|---|---|---|---|---|---|---|
| P10 | Controller (old, mind9) × rule (base, noshare, nopunish, strong) | Child starvation; care transfers; thefts | 12 residents, 480 ticks, labour cost 1, carry cap 20, same seed per arm | 1201–1240 | 40 | 8 (320 worlds) |
| P11 | Cue; unknown-value rule; positive-control bonus; automatic sharing on/off; old controller | Consideration rate; selection rate; care transfers; child starvation | As P10; automatic sharing off except base_share/both_share | 1501–1540 | 40 | 8 (320 worlds) |
| P12 | Information contract (inherited, legal) × cue × unknown value | Consideration rate; first effective carers; positive transfers; child starvation | As P10 | 1701–1740 | 40 | 8 (320 worlds) |
| P12 care vs rest | Forced care or rest at one opportunity | Child alive at 48 ticks; alive-ticks; food transferred; fatigue | Same snapshot; 48 ticks | 1801–1840 | 38 eligible (1805, 1821 excluded) | 2 branches |
| P17 | World (C, F1_pop192, F2a_cap2, F3u_cap2, F3s_cap2) × controller (old, mind9) × fork (A, T, H, R, M, X) | Constraint index CIE; alive person-ticks (240 ticks) | Tick-60 snapshot; hash-checked twins | 18201–18240 | 40 | 5 worlds × 2 controllers × 6 forks |
| P18 | World (C, F2a_cap2) × controller × fork (A, T, K, KV, P, P1) | Alive person-ticks; focal food, wood, crafts | As P17 | 19201–19240 | 40 | 2 × 2 × 6 |
| P19 | World (F2a_cap2, C) × controller arm (m9, old, llm_mem, llm_nomem) | W (mean focal energy, death = 0); deaths; time allocation; plan-change rate | Same snapshot and focal adults; 180 ticks | 19911–19940 | 30 per world (60 for probe) | 4 (240 runs; 120 paid) |
| P14 | Predictor (persistence, endpoint, finite history, belief) × horizon (+1, +4, +12) | World-mean CRPS (also MAE, Brier) | P12 legal_both world; same labels for all predictors | 3101–3140 (controls 3301–3340) | 36–37 eligible per horizon | 4 predictors on identical labels |
Development and pilot seeds (not used for inference): P10 9401–9402, 1101–1110; P12 9601–9604, 1651–1652; P17 18001–18004 (18001 seen before protocol), 18101–18102; P18 19001–19008 (dev forks) and 19001–19016 (Stage C); P19 19901/19902; P14 9801–9807, 3001–3002.
2. Units of inference and what is not independent
In every study the unit is the world (seed). Residents, actions, ticks, focal instances (P19: 360 paid, 720 total), prediction labels (P14: 19,207 per horizon) and pooled stage counts (P11, P12) are dependent repeated measures. Predictors in P14 are paired on identical labels (not 160 worlds). Arms in each study are paired by seed (not 320 independent worlds).
3. Statistical methods by study
| Study | Effect measure | Test | Sidedness | Interval | Multiplicity | "Supported" rule |
|---|---|---|---|---|---|---|
| P10, P11, P12 | Mean paired difference per world | Exact sign test, zeros dropped | Two-sided | Seed-paired bootstrap, 10,000 resamples | Holm over 4 per study | Holm p < 0.05 and CI wholly on predicted side |
| P12 care vs rest | Difference in proportion alive; alive-ticks; fatigue | Descriptive | — | Bootstrap (frozen; 5,000 for some quantities) | None | No confirmatory family |
| P17 | Mean paired difference in CIE or alive person-ticks | Exact sign test | One-sided (greater) | Bootstrap, 10,000 | Holm over 16 | Holm p < 0.05 and CI lower bound > 0 |
| P18 | Mean paired difference in alive person-ticks | Exact sign test | One-sided (greater) | Bootstrap, 10,000 | Holm over 4 | As P17 |
| P19 | Mean paired difference in W | Exact sign test | Two-sided (W); one-sided (probe) | Bootstrap, 10,000 | Holm over 7 | Holm p < 0.05 and CI excludes 0 |
| P14 | Mean paired difference in world-mean CRPS | Exact sign test, zeros excluded | Two-sided | Bootstrap, 10,000 | Holm over 3 | Holm p < 0.05, CI upper < 0, ≥ 30 eligible worlds, ≥ 5% mean world-relative reduction; gate needs all three horizons |
4. Where the "both criteria" rule changed a label
| Case | Interval | Sign test | Frozen label |
|---|---|---|---|
| P18 mind9 P − A (F2a_cap2) | [8.975, 104.000] excludes 0 | Holm p = 0.182 | Not supported |
| P19 H1, abundant world | [−3.965, −0.717] excludes 0 | Holm p = 1 | Not supported |
| P14 +12 ticks | [−1.990, −0.099] excludes 0 | Holm p = 0.868; relative reduction 0.63% < 5% | Not supported (gate failed) |
| P10 H3 | [−0.525, 14.200] includes 0 | Holm p = 0.608 | Not supported |
5. Summary of primary-test outcomes
| Study | Primary tests | Supported | Not supported |
|---|---|---|---|
| P10 | 4 | H1 (+1.425 starvation), H2 (+6.125 care) | H3, H4 (theft) |
| P11 | 4 | All four | — |
| P12 | 4 | All four | — |
| P17 | 16 | 4 H1; 4 tool; 4 reserve (12 of 16 in total) | 4 house |
| P18 | 4 | old K − A | mind9 K − A, mind9 P − A, old P − A |
| P19 | 7 | W: llm_mem < m9 and < old in constrained world; memory probe | Three abundant-world tests; memory welfare effect |
| P14 | 3 | +1, +4 | +12 (gate failed) |
Totals across the series: 28 of 42 primary tests were supported (P10 2/4; P11 4/4; P12 4/4; P17 12/16; P18 1/4; P19 3/7; P14 2/3). This fraction is not a measure of success: the hypotheses were not equally risky, and several tests were controls or manipulation checks.
6. Comparing effect sizes across studies (cautions)
The outcomes use different units (deaths per world, proportions, person-ticks, energy units, CRPS). Magnitudes are not comparable across studies; only direction, consistency across worlds (positive/zero/negative counts) and the supported/not-supported status are comparable. No meta-analysis is attempted.
7. Cross-cutting threats to validity
Model dependence; short horizons; research-stage controllers with inherited information shortcuts outside the care pathway; internal preregistration only; development seeds that shaped some designs (P17 18001; P18 dev forks; P19 prompt repairs); no independent replication; AI-assisted drafting under owner direction; undistributed engine and analysis dependencies.
P19 model identification (v3): The language-model arms used the API name deepseek-flash, corresponding on the experiment date (2026-11-06) to DeepSeek-V4.1-Flash-1017 according to the provider's 2026-10-17 update. A pinned model snapshot was not archived.