Skip to content

Study View: Compare by quartiles or median over several studies (STS-STM-12) - #494

Open
alisman wants to merge 2 commits into
masterfrom
studyview-quartiles-across-studies
Open

alisman wants to merge 2 commits into
masterfrom
studyview-quartiles-across-studies

Conversation

@alisman

@alisman alisman commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Summary

A numeric clinical chart's ☰ › Compare groups (or Analyze groups) › Quartiles / Median now works over several studies (STS-STM-12). Before, it was disabled there ("cut points are computed within one study"), and the server refused the split ("needs a cohort in a single study").

  • The cut: every selected study's values of the attribute, each with how many cases hold it, are pooled and cut by the one-study rule (pooledCuts: ClickHouse quantilesExact, the value at position ⌊p·n⌋), so the groups are cut at the selection's own quartiles.
    • The counts are the ones the proposal already reads per study: attributeStats with histograms (clinicalHistogramSql), now asked for when a quantile split spans studies.
    • A study with more than HISTOGRAM_CAP (20,000) distinct values is refused, naming it, rather than cut from part of its values.
  • Unchanged:
    • within one study, the SQL cut as before;
    • generic-assay charts still cut within one study, and keep their gate;
    • ties stay in one group, where the portal's splitData cuts by position and can split a tie across two groups. This is the existing one-study difference, not new here.
  • A statistics change, so its own PR.

Data boundary (for the Reviewer)

  • No new SQL: the per-study counts come from clinicalHistogramSql, through attributeStats (a peer answers it through its federation API's attributeHistograms). That builder is already in test/boundary.test.ts's BUILDERS: values and counts, no identifier, capped at HISTOGRAM_CAP and refused past it.
  • The same exposure as before: these are the value counts a numeric chart and the existing exact median / rank-sum paths already read.

Spec IDs

  • Covered: STS-STM-12 (Quartiles / Median over several studies).

Verification

  • The pooled rule against the SQL, one study each: coadread AGE 58 / 68 / 76 (n 592), luad MUTATION_COUNT 80 / 190 / 370 (n 566), msk_chord_2024 TMB 2.461 / 4.102 / 6.918 (n 25,040). The pooled cut equals attributeCutsSql's in every case.
  • Two studies: coadread + luad AGE quartiles cut at 58 / 67 / 74 ("AGE ≤ 58", "58–67", "67–74", "> 74"). The same rule over the portal's own 1,087 AGE values of the two studies (/api/studies/*/clinical-data) gives 58 / 67 / 74.
  • Browser, dev server, coadread + luad: Diagnosis Age › Compare groups › Quartiles is enabled and opens the comparison with those four groups. No page errors.
  • Boundary suite: 220 pass, 0 fail (no new builder).
  • Unit tests:
    • test/pooled-cuts.test.ts: quantilesExact's position, two studies' counts pooled, a value shared by both, no value;
    • test/analysis-partition.test.ts: quartiles over two studies end to end (both studies read, neither alone gives the cuts), a study past the cap refused, and the single-study median's refusal naming quantiles: 2.
      npm test 7,157 pass, 0 fail; tsc --noEmit and eslint clean.
  • Coverage: passes.
  • Planted bugs: 4 in scripts/mutants/study-view/quartiles-across-studies.ts (guard unit), two on the multi-study path (the first study only; a capped study used).

Tracker

Under "Study View → summary charts", this line (docs/PORTAL-PARITY.md:425):

  • Compare from a chart: Quartiles / Median across studies (ours one study only) (STS-STM-12)

is ticked:

Scorecard 30-study-summary.md, row STS-STM-12 (was | STS-STM-12 | 🟡 | C | Quartiles/Median cut by the server over the selection's values (compareQuantiles); Current bins one group per bar ✅; quartiles single study only |):

| STS-STM-12 | ✅ | C | Quartiles/Median cut by the server over the selection's values, over several studies pooled (pooledCuts, the one-study quantilesExact rule); Current bins one group per bar; ties kept in one group (the portal's splitData cuts by position) |

Preview

/lab?tab=explore&study=coadread_tcga_pan_can_atlas_2018,luad_tcga_pan_can_atlas_2018: Diagnosis Age › Compare groups › Quartiles.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Q3ezTaCo6wHFVsML6Qd4ht

…STM-12)

A clinical attribute's quantile split over a cohort that spans studies was refused ("needs a
cohort in a single study"). It now pools every study's values with their counts
(attributeValueCountsSql: values and counts, no case) and cuts them by the one-study rule
(pooledCuts: quantilesExact's ⌊p·n⌋), so the menu's Quartiles and Median work over several
studies. Within one study the cut is unchanged; generic-assay charts still cut within one study.
Checked: for coadread AGE, luad MUTATION_COUNT and msk_chord TMB the pooled cut equals the SQL's;
coadread + luad AGE quartiles cut at 58 / 67 / 74, the same rule over the portal's 1,087 values.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q3ezTaCo6wHFVsML6Qd4ht
@vercel

vercel Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
cbiolab-genie Ready Ready Preview Nov 19, 2026 10:09pm UTC
cbiolab-new Ready Ready Preview Nov 19, 2026 10:09pm UTC
cohort-lab Ready Ready Preview Nov 19, 2026 10:09pm UTC

Request Review

Comment thread lib/analysis.ts Outdated
// resolves the cutoff live from this cohort's own values, at the level the study records them.
// across studies, quantiles cut over every study's values pooled (STS-STM-12); the live median
// split below resolves its cutoff within one study
if (studies.length > 1 && req.quantiles != null) {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Untested branch. The new multi-study path in proposePartition (per-study viewAggregate, the hasViewAggregates guard, row pooling, the hand-off to quantilePartition) has no test. test/pooled-cuts.test.ts only exercises the pure pooledCuts, and both planted mutants live in pooledCuts too. test/analysis-partition.test.ts already has the mocked-SQL harness (on(/quantilesExact/...), the two-study cohort "Both"). Without a case like proposePartition({ cohort: "Both", attribute: "AGE", quantiles: 4 }) asserting the arms, a broken branch would pass every listed check: wrong SQL routed, rows not flattened, the branch condition inverted, or Number(row.v) reading the wrong column.

Comment thread lib/analysis.ts Outdated
const counts = (await Promise.all(studies.map(async (s) => {
const r = await resolverOf(s);
if (!hasViewAggregates(r)) throw new Error(s + " cannot answer a quantile split here");
return r.viewAggregate("partition:quantile values", def, (c, st, d) => attributeValueCountsSql(c, attribute, st, d));

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reuse / extra round-trip. mergedDistribution (line 272) has just run attributeStats on every study. With { histograms: true }, attributeStats already returns exactly these per-value counts (valueCounts, from clinicalHistogramSql, which is already in the boundary suite). This branch then runs a second query per study (and calls resolverOf/studyProfiles again) to fetch the same values through a new builder. Passing histograms: true and pooling valueCounts would drop attributeValueCountsSql, its new boundary entry and one ClickHouse/peer round-trip per study per click. It would also bring in histogramTruncated's refuse-rather-than-truncate handling.

Comment thread lib/federation.ts Outdated
"WHERE cancer_study_identifier = " + sq(study) + " AND attribute_name = " + sq(attribute) + "\n" +
" AND attribute_value != ''\n" +
" AND " + inCohortClinical(study) + "\n" +
"GROUP BY v HAVING v IS NOT NULL"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unbounded result. attributeValueCountsSql has no LIMIT, unlike clinicalHistogramSql (LIMIT HISTOGRAM_CAP + 1 BY a, per, whose caller then refuses). For a high-cardinality attribute, every distinct value of every study crosses to the coordinator on each Quartiles/Median click. The PR's own msk_chord TMB example is about 25k rows, and a large multi-study selection can be several times that. Each value held by a single case also leaves an aggregate-only peer as an exact value with n = 1. Consider the same cap-and-refuse as the histogram, or reusing it (see the comment on analysis.ts:284).

Comment thread lib/analysis.ts Outdated
}))).flat().map((row) => ({ v: Number(row.v), n: Number(row.n) }));
return quantilePartition(cohort, nameFor(attribute), attribute, req.quantiles, pooledCuts(counts, ps), (range) => ({ kind: "clinical", attribute, range }), { attribute });
}
if (studies.length > 1) throw new Error("'" + attribute + "' is numeric, and a median split needs a cohort in a single study (“" + cohort + "” spans " + studies.length + ")");

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Misleading message after this change. Median now works across studies through quantiles: 2 (the chart's ☰ › Median). But a multi-study request without quantiles still fails with "a median split needs a cohort in a single study". That tells an author a cross-study median is impossible when one is a parameter away. The comment at line 275 ("A median split … through a percentile leaf") also now sits above the multi-study quantile branch, which it doesn't describe. Suggest naming quantiles: 2 in the error (or routing the multi-study median through the pooled cut) and moving that comment down to the single-study code.

Comment thread lib/analysis.ts Outdated
// resolves the cutoff live from this cohort's own values, at the level the study records them.
// across studies, quantiles cut over every study's values pooled (STS-STM-12); the live median
// split below resolves its cutoff within one study
if (studies.length > 1 && req.quantiles != null) {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit (altitude): the same statistic now has two implementations. One is ClickHouse quantilesExact for one study (attributeCutsSql); the other is the JS pooledCuts for several. They can drift: the PR checks equality by hand on three attributes, and no test pins it. pooledCuts over one study's counts is the same cut, so a single path for 1..n studies (value counts → pooledCuts) would remove the branch and the risk of divergence. Alternatively, a test could assert that both give the same cuts on one fixture.

Comment thread lib/partition-arms.ts Outdated
const sorted = [...merged].sort((a, b) => a[0] - b[0]);
const n = sorted.reduce((a, [, k]) => a + k, 0);
if (!n) return { qs: [], hi: NaN, n: 0 };
const at = (i: number) => { let seen = 0; for (const [v, k] of sorted) { seen += k; if (seen > i) return v; } return sorted[sorted.length - 1][0]; };

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: the fallback return sorted[sorted.length - 1][0] in at can't be reached. Every caller passes Math.min(n - 1, …), so seen > i always fires by the last entry. Either the clamp or the fallback can go (keeping only the clamp is clearer), and the doc's "the last when that runs past the end" then describes one mechanism instead of two.

@alisman

alisman commented Nov 19, 2026

Copy link
Copy Markdown
Contributor Author

Review: changes requested (one blocking item, a missing test)

The method is right. The cut points are exact quantiles over every study's value counts pooled together, not an average of per-study quantiles. pooledCuts takes the value at position ⌊p·n⌋, the same rule ClickHouse's quantilesExact uses for one study, and it weights each value by how many cases hold it. This also matches legacy, whose splitData cuts the pooled values of the selected cases.

What's fine:

  • Units: each study counts at the level it stores the attribute, as the one-study path does. A study that stores it at another level is still refused, through mergedDistribution's level check.
  • NA: NA and non-numeric values are dropped by finiteSql.
  • Ties: ties go to the lower group (≤ cut), as in the one-study path. This is an existing difference from legacy, not a new one.
  • Remote peers: a participant with no viewAggregate makes the split fail with a named error. It does not fall back to the local studies alone (invariant 1).
  • No row limit: having no LIMIT is correct here, because a truncated list would give approximate cuts. The value/count exposure is the same as the numeric histogram's.

Blocking

  • The new multi-study branch has no test (lib/analysis.ts:279-287). test/pooled-cuts.test.ts and both planted bugs cover only the pure pooledCuts function. Nothing checks that every study in the cohort is queried, that the counts are pooled, or that the arms end up with the pooled cuts. For example, querying only studies[0] would pass npm test and the planted-bug check. Please add a "Both" case with quantiles to test/analysis-partition.test.ts, using its two-study stand-in warehouse. Give each study its own value counts, assert that both statements were sent and that the arm ranges equal the pooled cuts, and add a planted bug on that branch with a unit guard.

Not blocking (inline comments): when no quantiles are asked for, the multi-study error still says a median "needs a cohort in a single study", although quantiles: 2 now does it. The comment above the new branch describes the old code. The value counts could come from the attributeStats histogram that is already fetched. at()'s fallback can never run.

…lready reads; tested end to end

Review of #494:
- The multi-study cut now pools each study's value counts from attributeStats' histograms
  (mergedDistribution with histograms, clinicalHistogramSql, already in the boundary suite): no
  new builder, no second query per study, and a study past HISTOGRAM_CAP is refused, naming it,
  rather than cut from part of its values. attributeValueCountsSql is gone.
- A two-study quartiles test in test/analysis-partition.test.ts (both studies read; neither alone
  gives the cuts), the cap's refusal, and planted bugs on that path.
- The single-study median refusal names the way across studies (quantiles: 2); the comments sit
  over the code they describe; pooledCuts' unreachable fallback is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q3ezTaCo6wHFVsML6Qd4ht
@alisman

alisman commented Nov 19, 2026

Copy link
Copy Markdown
Contributor Author

Fixed in d565032.

  • Blocking (an untested branch): test/analysis-partition.test.ts now runs quartiles over the two-study cohort "Both" end to end. st1 holds 20–50 and st2 60–90; the arms are ≤ 40, (40, 60], (60, 80], > 80, which neither study alone gives. The test also asserts both studies' histograms were read. Two planted bugs sit on that path: the first study's counts only, and a capped study used.
  • Reuse: the cut now pools the value counts mergedDistribution already reads (attributeStats with histograms, clinicalHistogramSql). attributeValueCountsSql and its boundary entry are gone, there's no second query per study, and a study past HISTOGRAM_CAP is refused, naming the cap (tested).
  • Message: the single-study median refusal now says "across studies, ask for quantiles: 2". The percentile-leaf comment sits over the code it describes.
  • Fallback: pooledCuts' unreachable fallback is gone; the clamp is the one mechanism.
  • One path or two (nit): I kept ClickHouse's cut for one study, so this PR doesn't change that statistic. The two rules agreed exactly when I checked on coadread AGE, luad MUTATION_COUNT and msk_chord TMB, and test/pooled-cuts.test.ts pins quantilesExact's positions. Folding one study into the pooled path can be its own small PR if you'd like it.

Live, coadread + luad AGE still cuts at 58 / 67 / 74. npm test 7,157 pass, 0 fail; boundary 220 / 0; tsc --noEmit and eslint clean.

Comment thread lib/analysis.ts
if (kind === "num") return { kind, per };
// every study's (value, count) pairs, pooled: exact for a quantile cut across studies; none when a study's were cut
if (kind === "num") return opts?.histograms
? (perStudyStats.some((st) => st?.histogramTruncated || !st?.valueCounts)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A study with no values of the attribute in this cohort is refused here as if it were past the cap. attributeStats builds a numeric stat only from clinicalMomentsSql's rows (for (const r of nums.data)). When none of the cohort's cases in a study hold a finite value, that study gets no stat at all, so st is undefined here. !st?.valueCounts is then true, and proposePartition throws "a study holds more than 20,000 distinct values of it". The real count is zero. One example: a two-study cohort where AGE is declared numeric in both but blank for st2's selected patients. The categorical path just below treats a missing stat as no values (s?.values || []), and so did the previous head's per-study value-count query. The same false cap message is also the reason given for a peer that answered without valueCounts (one on older code that ignores attributeHistograms).

Suggested fix: let !st contribute []. Refuse with the cap message only on st.histogramTruncated. For a study that answered without valueCounts, give a separate refusal that names the study. Add a test with a study that returns no rows.

Comment thread lib/analysis.ts
// (pooledCuts: attributeCutsSql's quantilesExact) (STS-STM-12)
if (acrossStudies) {
if (dist.histogramTruncated || !dist.valueCounts)
throw new Error("cannot cut '" + attribute + "' into quantiles across “" + cohort + "”'s studies: a study holds more than " + HISTOGRAM_CAP.toLocaleString("en-US") +

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: the commit message and docs say the refusal names the study, but this message says only "a study". Carrying the study id (or ids) from mergedDistribution would make the refusal actionable.

@alisman

alisman commented Nov 19, 2026

Copy link
Copy Markdown
Contributor Author

Re-review of d565032: changes

The earlier blocking finding is fixed. test/analysis-partition.test.ts now runs a two-study quartiles case where st1 holds 20–50 and st2 holds 60–90. Neither study alone gives the expected cuts (40/60/80), and the test asserts that both studies were queried. The planted bug pooled-cut-first-study-only makes that test fail. The method has not changed:

  • clinicalHistogramSql counts the same clinical rows, under the same cohort and finite-value filters, as attributeCutsSql's quantilesExact. Each study's counts are at its own level, and mixed levels are still refused.
  • pooledCuts still takes the value at ⌊p·n⌋ with multiplicities.
  • A truncated histogram (over HISTOGRAM_CAP) is refused and never partly used.
  • The single-study path is unchanged.
  • No CI check has failed so far; gate-e2e and gate-warehouse are still pending.

Blocking:

  • lib/analysis.ts:191: a study with no values of the attribute in the cohort has no stat, so it is refused as "more than 20,000 distinct values". It should count as zero values, as the categorical path and the previous head did. A peer that answered without valueCounts gets the same false reason. See the inline comment.

Non-blocking:

  • nit: the cap refusal says "a study" without naming which.
  • nit: peers' value histograms are now pooled across sites. The PR body says so, and the federation API already serves attributeHistograms. But lib/stats.ts:1186-1191 and lib/views/clinical/run.ts:57-60 still say histograms are never combined across sites ("a boundary that must not be crossed"). Those comments now contradict this path, so update them.

This branch was successfully deployed

1 active deployment
Preview – cohort-lab — d565032d Deployed Nov 19, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant