Protocol Review: research-inquiry.md

Status update (2026-08-03): The recommendations in sections A–C and E were applied to the protocol in the commit following this review; section D's fixes were applied during the review itself. In the process, the dual workstream/section numbering was collapsed (workstream labels dropped; hypothesis IDs re-keyed to section numbers) per section D's residual nit — so hypothesis references in this document (H3.2, H8.1, H11.3, …) describe the reviewed version, one lower than the current IDs in the protocol.

Reviewed: 2026-08-03, against the version including Workstreams 1–15 Reviewer caveat: This review was produced by the same author as the protocol. Self-review catches structural problems but not shared blind spots; before execution, the protocol deserves one outside read from (a) a childcare-policy academic, (b) a budget-process practitioner, and (c) a current or former state subsidy administrator. Their objections will not overlap with this document's.

Verdict in one paragraph: The protocol's skeleton is sound — falsifiable hypotheses, named sources, kill conditions, a known-unknowns register, an explicit anti-advocacy posture, and a synthesis point (the architecture scorecard) that forces every workstream to cash out in a comparable currency. Its weaknesses are systematic rather than local: the precedent set is selection-biased toward progressive jurisdictions and misses the single most relevant live experiment; the hypothesis set tilts uniformly skeptical, which is a bias with better manners than advocacy but the same epistemology; the scorecard has no scoring method; the source lists are convenience samples from the author's memory rather than the product of a search procedure; and the protocol never states what the program is for, which makes "recommended architecture" formally undefined. All are fixable before execution. Ranked findings follow.

A. Critical gaps, ranked by decision-relevance

A1. Case-selection bias in the precedent set (highest-value fix in this review)

The protocol's US state cases — New Mexico, Vermont, DC, Multnomah County — are all recent, all progressive jurisdictions, and all primarily financing wins. This selection silently equates "universal childcare precedent" with "progressive financing innovation" and discards three categories of stronger evidence:

• Red-state universal pre-K, running for two decades. Georgia (lottery-funded universal 4s, mid-1990s), Oklahoma (universal pre-K, late 1990s — the Tulsa studies are among the most-cited outcome evaluations in the field), and Florida (universal VPK by constitutional amendment, mid-2000s). These matter for exactly the questions the protocol claims to care about: they are proof that universality can be enacted and politically durable in conservative states; they are the best available evidence on what universality without high per-child spending produces (Florida especially — high coverage, low standards, low cost); and Georgia's lottery earmark is a revenue instrument missing from §3's list. Their omission makes the political-feasibility analysis look harder than the record says it is, and the quality analysis easier.
• Canada's live nationwide rollout. The 2021 federal-provincial $10/day agreements (CWELCC) are the natural experiment for this exact inquiry: a federal system implementing a nationwide childcare framework through bilateral deals with subnational governments, in real time, with observable results — reported fee reductions, expansion shortfalls, workforce shortages, and an active for-profit-participation fight (Ontario) that is the §12.5 rate-participation risk actually happening. Quebec-only coverage of Canada studies the 1997 program and misses the 2021 one. (Details here are unverified recollection — same currency caveat as the protocol's own §14.1 — but the program's existence and relevance are not in doubt.)
• Failure and pathology cases. The UK's free-hours entitlement, where a funded rate persistently below delivery cost produced provider exits and cross-subsidy from unsubsidized hours — the cleanest national-scale realization of rate-participation risk. The Netherlands' childcare benefits scandal, where algorithmic improper-payment enforcement falsely accused thousands of families of fraud and ultimately brought down a government — the definitive cautionary case for demand-side architectures (§14.2 architecture 5) and for §A7 below. And NYC's pre-K scale-up, a domestic rapid-ramp case (tens of thousands of seats in about two years) directly relevant to §16's ramp-rate question.

Fix: Add Georgia, Oklahoma, Florida, Canada/CWELCC, the UK, the Netherlands scandal, and NYC to §15 with the same transferability template. Re-examine §10's framing once the red-state cases are in: the claim "universality is procedurally near-impossible" must be reconciled with the fact that three conservative states did it for 4-year-olds decades ago.

A2. Missing architectures: the caregiver-choice allowance, and a cash comparator

The §14.2 architecture list has no design in which public money can pay parents or kin to provide care — a home-care allowance of the kind several countries operate (Finland's is the canonical case; Norway ran one), and the version of this policy that the family-values right can actually support. CCDF already permits relative-caregiver payment at the margins, so it is not exotic. The evidence base is real and cuts both ways: home-care allowances measurably depress maternal employment (the Nordic literature) while polling well with exactly the constituencies every other architecture alienates.

Worth noting plainly: §12.4 instructs the researcher to check whether funder concentration causes "certain design options (for example, home-based care, or cash-to-families alternatives)" to receive "systematically less analytical attention." The protocol then omitted precisely that design from its own architecture list. The bias-check predicted the protocol's own blind spot. That is evidence the check works, and evidence it must be run against this document, not just against the literature.

Separately, the scorecard evaluates ten ways to build childcare but never benchmarks against not building childcare: an equivalently funded child tax credit / cash transfer. Opponents will make that comparison in every hearing; a report that has not made it first is unarmed. The cash benchmark is not a recommendation — it is the identification strategy for what care-specific provision adds per dollar.

Fix: Add architecture 11 (caregiver-choice allowance, payable to parents/kin, Nordic evidence attached) and a cash-equivalent comparator row scored on the same dimensions.

A3. No objective function — so "recommended architecture" is undefined

The protocol never states what the program optimizes: maternal labor supply, child development, family financial relief, or equity of access. These rank the architectures differently — Quebec optimized labor supply and is dinged on child outcomes; a child-development objective favors quality-conditioned designs that a labor-supply objective would reject as slow and expensive. §1.2 gestures at this (child vs. parent entitlement) but the scorecard has no objective attached, which means the "recommendation per age band" that §14 promises cannot be derived from the scorecard — it will be smuggled in.

Fix: Either (a) declare the objective as a §1 definitional decision and defend it, or — better — (b) present the scorecard's rankings under three explicit objective weightings (labor-supply-first, child-development-first, equity-first) and report where the ranking is stable across all three versus where it flips. Ranking stability across objectives is itself the most useful finding available.

A4. The hypothesis set tilts uniformly skeptical — a monoculture with no adjudication criteria

Read the ~30 hypotheses in sequence: workforce binds (H3.2), money becomes price inflation (H8.1), the rules select for a worse program (§11 preamble), no market path exists (H11.1), employer benefits defuse the coalition (H11.3), Tri-Share doesn't scale (H11.4), nothing passes intact (H10.1, H13.1), scaled programs underperform (H6.2). Every single one predicts difficulty. The protocol armors itself extensively against advocacy optimism and not at all against contrarian pessimism — yet a research protocol whose every prior points the same direction has a thesis, and "we tried to falsify it" is asserted without a mechanism.

Two fixes, both concrete:

1. Symmetric steelmanning. §18 requires a steelmanned opposition case. Require the mirror: a steelmanned feasibility case, built from the strongest evidence that this is more tractable than the protocol assumes (the red-state pre-K record, Canada's rollout speed, the DoD system's existence, the 1971 bill passing both chambers). Equal search effort, same evidentiary standard.
2. Pre-specified adjudication criteria per hypothesis. No hypothesis currently states what evidence would confirm or refute it, which leaves adjudication to the executor's judgment after seeing the data — the classic garden of forking paths. Before execution, each H gets two or three lines: refuted if…, supported if…, indeterminate if…. Example for H3.2 (pipeline caps the ramp): refuted if any US jurisdiction has demonstrably expanded its credentialed workforce faster than its pipeline throughput via wage increases alone (recruitment from adjacent sectors and returners, not new credentials); supported if DC's Pay Equity Fund raised wages substantially without a commensurate headcount increase inside three years; indeterminate if no jurisdiction has raised wages enough to test it.

A5. No search methodology — and the academic economics literature is nearly absent

Every source list in the protocol is a convenience sample of what its author could recall. That is availability bias operating at the layer where it does the most damage, because it is invisible downstream: the synthesis will look well-sourced while resting on whatever happened to be memorable. Two consequences already visible:

• The named sources skew heavily toward government agencies and policy shops. The empirical economics literature — the strongest causal evidence in the field — is almost entirely missing: the childcare-market microeconomics canon (Blau; Herbst on subsidies; the Handbook chapters), the quasi-experimental universal-program literature beyond Quebec (Havnes–Mogstad on Norway, Cornelissen et al. on Germany, Fort–Ichino–Zanella's negative findings for affluent children in Bologna — directly relevant to H6.2 and to the universal-vs-targeted fault line), and the Tulsa/Gormley pre-K studies.
• There is no way to distinguish "the literature says X" from "the sources I happened to list say X."

Fix: Add a search protocol per workstream: databases (EconLit, NBER WP series, Google Scholar, ERIC for the education side), search strings, date ranges, and inclusion/exclusion criteria, PRISMA-lite. The named sources become seed sources, not the corpus. This is the single largest robustness upgrade available for its cost.

A6. The scorecard has no scoring method

§14.2 is the analytical core: 19 dimensions × 10 architectures ≈ 190 judgment calls. As written, nothing constrains them — no scale, no anchors, no weights, no tie to evidence. Two competent analysts would produce materially different scorecards and both would look rigorous. And the implicit equal weighting of dimensions is itself a hidden judgment (is "Byrd survival" worth the same as "coverage of 0–3"?).

Fix, four parts:

1. Ordinal scales with anchored descriptions per dimension (what a 1, 3, 5 concretely means), written before scoring begins.
2. Every cell cites the workstream finding it rests on; an uncited cell is a flag, not a score.
3. No single weighted ranking. Present rankings under the three §A3 objective weightings plus an equal-weight baseline, and report rank stability.
4. Score the matrix twice, independently — by two people, or failing that by the same person from a shuffled architecture order weeks apart — and reconcile disagreements in writing. The disagreement log is a deliverable.

Add one missing dimension while in there: distributional incidence — who gains under each architecture by income, race/ethnicity, geography, and work schedule. Equity considerations currently appear as scattered bullets in five workstreams and then vanish at exactly the point where architectures are compared, which is where distribution is decided.

A7. Program integrity is missing entirely — and its risk is two-sided

No workstream addresses improper payments, fraud, or enforcement design. This is a guaranteed, program-shaping issue, and its two failure modes point in opposite directions:

• Fraud occurs and becomes the story. Improper-payment findings in CCDF are a recurring GAO/OIG theme, and the recent political history of child-serving programs includes a headline nine-figure pandemic-era fraud case in the child-nutrition space whose political damage radiated far beyond the program at fault. A young universal program does not survive many of those.
• Enforcement occurs and becomes the atrocity. The Netherlands ran aggressive algorithmic fraud detection on childcare benefits, falsely accused thousands of families — disproportionately those with immigrant backgrounds — clawed back benefits ruinously, and the scandal collapsed a government. Demand-side architectures (5, and partially 10) inherit this risk class wholesale.

Integrity design — payment verification, attendance-vs-enrollment tension (§6 already notes attendance-based payment destabilizes providers; integrity pressure pushes toward attendance-basis; that collision is unexamined), audit burden vs. the §11.6 small-provider participation problem, and due-process design for clawbacks — belongs in §6, and "integrity/enforcement risk profile" belongs on the scorecard.

A8. Operational blind spots (each small; together they decide feasibility in the field)

• Liability insurance. Availability and pricing of provider liability coverage is a documented center-closer and a hard constraint on FCC entry; a scaled public program may need a reinsurance or captive answer. One line in §4 (liability as a workforce deterrent) is the only trace. Belongs in §5.
• Transportation. School-age care logistics hinge on moving children between school and program; rural care generally hinges on transport. Zero mentions. Belongs in §5 and §8.
• The school-age workforce is a different labor pool. OST and summer staffing runs on part-time, seasonal, and young workers — different pipeline, different wage dynamics, different turnover. §4 models one workforce; it should model two.
• Background-check throughput. Post-2014 CCDBG comprehensive-check requirements produced months-long processing backlogs in some states (verify). At ramp speed, check-processing capacity is a binding input alongside credentialing. One bullet in §4's pipeline section.
• Tribal nations and territories. The scope line says "50 states + DC + territories," and CCDF has tribal set-asides with distinct rules — then neither tribes nor territories appear again. Either analyze or descope explicitly; silent omission is the one wrong option.
• Behavioral responses the cost model ignores: fertility effects (Nordic and Quebec literatures; second-order but compounding over a decade of cost projection) and interstate migration under uneven state adoption (real in the Medicaid gap; changes the political economy of holdout states). One sub-question each in §8/§9.
• Recession stress test. Childcare need is countercyclical; payroll-tax and state revenues are procyclical; state balanced-budget rules force cuts at maximum need. No architecture is currently stress-tested against a recession year. Add to §14's scorecard or §16's failure modes.

A9. No design-for-learning requirement

The protocol treats research as something that happens before rollout. The stronger position: the rollout is the research. Staged implementation is an evaluation design if built as one — lottery allocation where oversubscribed, staggered geography enabling difference-in-differences, pre-registered outcome dashboards tied to §16's leading indicators and to explicit decision points where evidence redirects the ramp. Each §14 architecture should specify its embedded evaluation design; an architecture that cannot be evaluated while operating is a worse architecture, and that belongs on the scorecard.

A10. Construct validity: the desert metric contradicts the protocol's own premise

§5 relies on "childcare desert" analyses whose standard metric (licensed slots per child in an area) counts only licensed supply — while §1.3 and §17 insist that FFN and license-exempt care are a large, poorly measured share of the real market. The protocol thus questions the completeness of licensed-supply data in one section and consumes a metric built entirely on it in another. Fix: treat "desert" as a claim to validate, not a fact to import — does the metric predict actual unmet demand (waitlists, parental search failure, labor-force effects) once informal care is accounted for?

A11. No unmediated parent or provider voice

Every parent and provider input arrives through an intermediary — surveys, associations, advocacy synthesis. For a feasibility study, tacit operational knowledge (why providers really decline subsidy participation; what makes parents distrust a program) systematically fails to survive aggregation. Full fieldwork is out of scope for desk research, but the gap narrows cheaply: public-comment dockets on CCDF rulemakings (providers describe participation barriers in their own words, on the record), state legislative testimony from operating providers, and OIG/GAO interview-based reports. Name these as required sources in §8 and §12 and log the limitation in §17.

B. Bias-shielding mechanisms to adopt

Ordered by leverage. B1–B4 should be non-negotiable.

1. Freeze and version the protocol; keep a deviations log. The pre-registration analog: once execution starts, the protocol text is the commitment device, and every mid-course deviation (dropped sub-question, added source type, reinterpreted hypothesis) is logged with a reason. Without versioning, the protocol will silently drift toward whatever the data made easy. (Requires the repo — see the mechanics note at the end.)
2. Per-hypothesis adjudication criteria before execution (§A4). This is the single strongest guard against motivated synthesis.
3. Symmetric steelmanning (§A4): the feasibility case gets the same effort as the opposition case.
4. Two-source rule with independence tracing. Every load-bearing empirical claim needs two sources of different types (e.g., administrative data + peer-reviewed study) that do not share a root citation. Much of this field is citogenesis — a single estimate laundered through a dozen reports. Trace to the root before counting sources as independent.
5. A formal source register. One table, maintained across all workstreams: claim → source → vintage → type (admin data / peer-reviewed / advocacy / industry / press) → funder / COI → root source → verified-by. §13's data-vs-position tagging becomes a column in this register rather than an aspiration.
6. Anchor quarantine. The protocol embeds roughly fifteen unverified numbers (BBB ~400B, ARPA 39B, labor-share fractions, and until this review a misleading "~180 non-school days"). Disclaimers do not defeat anchoring — the executor will still gravitate toward confirming the stated figure. Move every numeric prior into a single appendix table with columns stated value / verified value / delta / source, which Workstream 1 must complete line by line. An anchor that was wrong is a finding about the protocol's priors; the table makes those visible instead of silently corrected. (This review caught one such anchor: the "180 days" framing counted weekends; the employment-relevant figure is roughly 75 weekdays. Now fixed in the protocol — and exhibit A for why the quarantine table should exist.)
7. Red-team pass on the draft report. Before finalization, one reviewer's sole brief is to attack the draft's conclusions using the §13 opposition sources and the strongest contrary academic findings, in writing; the final report must answer or absorb each attack, also in writing.
8. Adversarial presentation for polarized literatures. Where the field genuinely disagrees (Quebec child outcomes, Tennessee VPK, Head Start fade-out), present each side's preferred specification and what evidence would resolve the dispute — never a silent pick of one side's estimate.
9. Structure before results. Fix the report's section structure, decision criteria, and scorecard scales (§A6) before synthesis begins, so the frame cannot be fitted to the findings.
10. Run the §12.4 funder-bias check against this inquiry itself. It already caught one blind spot (§A2). At the end of execution, ask it again: which options got the least analytical attention, and does that pattern track the funding structure of the sources used?

C. Grounding conclusions in data

1. Microdata-first baseline. Workstream 1 should be built from public microdata (ACS, SIPP, CPS supplements, NSECE) with a reproducible pipeline — code and data vintages committed alongside the report — not assembled from secondary reports' toplines. This converts the baseline from "citable" to "checkable," and it is the difference between a report and an artifact others can build on.
2. Named quantitative deliverables per workstream. Each workstream's "done when" should specify its output tables, with columns, now. Examples: §4 must produce required-FTEs by year by segment under three wage scenarios; §3 must produce the cost-estimate normalization table with a row per published model and a column per normalized assumption. If the table can't be specified in advance, the workstream isn't specified.
3. Uncertainty as bands, not adjectives. Key quantities reported as P10/P50/P90 (or low/central/high with stated drivers), propagated through to the scorecard — a cost cell should show its spread, not its midpoint.
4. Confidence grading on every major claim. A three-tier GRADE-style scheme (strong / moderate / weak evidence) attached to each finding in the deliverable, with the tier justified in the source register. The §18 requirement to distinguish "evidence that X" from "no evidence either way" becomes enforceable instead of rhetorical.
5. Two-method rule for load-bearing parameters. Labor share, turnover, take-up elasticity, maternal labor-supply elasticity: each estimated by at least two independent routes (e.g., survey-based and administrative; US quasi-experiment and international analog with stated transfer adjustment) with the divergence reported.
6. Operationalize "binding." The deliverable's central promise — constraints ranked, binding vs. difficult — currently rests on an undefined term. Proposed definition: a constraint is binding for a design if, with all other constraints at observed values, relaxing it alone would materially raise achievable coverage or ramp rate, and relaxing others would not. Practical test: if the budget doubled, what would stop output from doubling? That is the binding constraint. Apply per design and per age band, since the answer plausibly differs across both.

D. Mechanical defects found (fixed in the protocol during this review)

• §0 pointed to the Known-Unknowns Register as §13 (stale; now §17)
• §11's method pointed the feasibility matrix at "§12" for candidate designs (stale; now §14)
• §13.1 claimed "§16 requires a steelmanned opposition case" (stale; the requirement lives in §18)
• §14's question referenced "Workstreams 1–10" (stale; now 1–12)
• §1.4's "~180 non-school days" counted weekends; replaced with the employment-relevant ~75 weekdays framing

Residual structural nit, deliberately not churned: workstream N lives in section N+1 throughout, which is confusing and was the proximate cause of every stale reference above. If the protocol gets one more major revision, renumber so workstream and section coincide; otherwise leave it and stay vigilant.

E. Process recommendations

1. Phase-gate the execution. Fifteen workstreams executed in full is months of effort. Run a Phase 0 of a few days first: verify the anchor table (§B6), build the Workstream 1 skeleton, and scan the §A1 additions (red-state pre-K, Canada). Phase 0's job is to fire the kill conditions early or re-rank the workstreams before the effort is committed — several of this review's findings (especially A1) could materially reshape the protocol, and it is cheaper to find that out in days than months.
2. External review before execution (see reviewer caveat at top).
3. The protocol now lives in version control — required for §B1's freeze-and-deviations discipline.

If only five things get done

1. Add the missing precedents — red-state universal pre-K and Canada's live rollout (A1)
2. Write per-hypothesis adjudication criteria and the steelmanned feasibility case (A4/B2/B3)
3. Give the scorecard scales, evidence-tied cells, multi-objective weightings, and a distributional dimension (A3/A6)
4. Adopt the source register with the two-source independence rule (B4/B5)
5. Build the anchor-quarantine table and make Workstream 1 complete it (B6)