Proportional Study Allocation — status and remaining plan¶
Assigned-reviewer validity requirement — 22 September 2026¶
Chris requires every valid allocation plan to verify each assigned reviewer's active membership and effective permission/access to the stage, its studies and the allocated activity, on save and activation, with revalidation and affected-plan repair after membership, permission or scope changes. See acceptance. Not met. Today only active membership is checked at save. Effective-permission checks are blocked on an authoritative out-of-request permission resolver (#3251), which does not exist: application groups are read only from the requester's own session claims. No approximation is used. Saved work, runtime authorization, approval gates and usage limits are unchanged.
Proposed evolution — 16 September 2026¶
The regime evolution plan records the owner's proposed direction for immutable allocation history, deterministic correction/reallocation, unaffected-reviewer continuity and optional verified assignment stamping. It is Draft, not implemented behavior or activation evidence. Its phased MVP preserves current assignments before introducing reallocation or query optimization. Historical delivery statements below require reconciliation with subsequent merged PRs; do not interpret this planning link as completing that audit.
Is the feature fully implemented? No. The original MVP is merged and dormant. The first
follow-up, direct annotation endpoint enforcement, merged in #3211 (fdc79576c). Reviewer UI
and administrator allocation progress merged in #3213 (761b3c70d5). No
environment flags are enabled by this change.
This document records what is done, what is deliberately not done, and the ordered plan to reach a feature reviewers actually benefit from. The design itself is in README.md; this file is the delivery status. The acceptance contract defines required behavior, failure paths, integration, performance and rollout evidence.
GitHub delivery tracking is consolidated in #3264. Reuse #3095 for editor consistency, #2042 for the approved membership-first deployment-migration/lifecycle sequence, #3251 for shared authorization, and #3252 for performance qualification. The umbrella retains integrated rollout acceptance and ordered enhancements; no partial PR closes it.
Allocation publication isolation is a required correctness follow-up under #3264. The implementation prepares a fresh private persisted project, never the shared cached instance, before backfill and optimistic-concurrency publication. Delayed/failing publication must not leak allocation into a sibling writer's project. This remains an unmerged slice until its exact-head review and CI complete; frontend save ordering alone is not cross-request isolation.
Where it stands¶
| Area | Status | Evidence |
|---|---|---|
| Domain model and allocation plan | Done | StageWorkloadShareConfiguration, StageWorkloadSharePlan, Study.WorkloadShareBucket |
| Assignment filtering for new annotation work | Done | StageReviewService filters only the NewForAnnotation candidate query |
| Fail-closed behaviour on stale or invalid configuration | Done | Empty result, never an exception; bucket-exhausted reviewers report complete |
| Configuration API | Done | StageWorkloadSharesController, optimistic concurrency, ProblemDetails conflicts |
| Feature flag | Done, off | proportionalStudyAllocation, default false, set in no environment |
| Admin configuration UI | Minimal | Enable toggle, per-reviewer percentage, distribute-evenly, total validation, save |
| Admin visibility into the resulting allocation | Merged in #3213 (761b3c70d5) |
Stage-design-authorized per-reviewer counts/progress and stale-membership guidance |
| Reviewer-facing UI | Merged in #3213 (761b3c70d5) |
Read API and stage-overview panel; see delivery continuation |
| Enforcement on direct study endpoints | Merged in #3211 (fdc79576c) |
Shared StageWorkloadShareEligibility gates ordinary direct GET and session PUT, and each freshly loaded submission retry; denied eligibility returns 404 before side effects |
| Allocation regime identity and review provenance (regime-evolution Phase 1) | Implemented in PR #3603; pending review/merge | Immutable StageAllocationRegime + transition log, stage pointer, one shared evaluator, reservation-to-session provenance, compatibility floor v1; see README |
| Deployed anywhere | No | Flag absent from staging and production values |
Merged in #2991 (merge commit 277802248),
superseding the abandoned #2450.
What the MVP does today¶
An administrator with the flag on can, on a disabled annotation-only stage with no review activity, give each active project member a percentage of the stage's review slots. Those percentages become a deterministic 10,000-bucket plan; each study carries a stable bucket derived from its id. When a reviewer asks for their next study, the candidate query is restricted to their buckets. Resuming a saved session and reconciliation remain unfiltered. The first follow-up also restricts direct annotation access and new annotation-session submissions to those same buckets. Saved ordinary work is exempt only for the same reviewer and stage, and submission must match the saved session ID. A reservation alone does not exempt a direct request. Disabled allocation retains legacy eligibility; a disabled runtime flag retains legacy behaviour, including submission identifier handling.
While an allocation is enabled the stage's review mode, study-selection mode and reviews-per-study are locked, because the plan is only valid against the count it was built for. A configuration naming a member who is no longer active, or one that no longer matches reviews-per-study, yields no new work rather than an error.
What it does not do¶
These are the gaps that make "merged" different from "usable", in the order they block value.
- Direct enforcement is merged in #3211 (
fdc79576c). It closes the ordinary direct GET and session PUT bypass with the same bucket policy as next-study selection. Focused tests cover both tracking modes, stale configuration, saved work, and denial without mutations or reservations. This is the prerequisite for the reviewer UI, not completion of that UI. - Reviewer visibility merged in #3213. It adds the caller's assigned studies, saved-work links, remaining/startable counts and enabled/paused state to stage overview.
- Administrator visibility merged in #3213. The same overview adds a stage-design authorized per-reviewer allocation/progress table, separate from configured percentages.
- Stage-review authorization is not validated. A member who cannot open the stage can still be given a positive share, and those slots then go unfilled. The check needs the reviewer's own application-group claims. Identity stores groups, but no canonical out-of-request, cross-provider authorization resolver is exposed to allocation configuration.
- Membership changes after activation are unhandled. Deactivating a configured member pauses that stage's new annotation work by design, but there is no reviewer lifecycle story beyond that. #3048 adds the deactivation admin surface and is unmerged. Pause visibility is implemented here; the user has authorized membership-first compatibility/migration implementation after the current allocation PRs ship, instead of extending runtime schema branches. Live migration execution and activation remain separate gates.
- No rebalancing. Shares can only be set before any review activity exists in the stage. A stage that is already underway cannot be re-divided.
Plan¶
Ordered by dependency, not by size. Items 1 and 2 are what turn this from a scheduling tweak into the feature people asked for.
| # | Work | Why it is here | Status |
|---|---|---|---|
| 1 | Enforce eligibility on the direct specific-study review and submission endpoints | Prerequisite for any browsing UI; without it shares are advisory | Merged in #3211 (fdc79576c). See implementation plan |
| 2 | Reviewer UI: my assigned studies, remaining share, allocation-active indicator | The reviewer-facing half of the feature; depends on 1 | Merged in #3213 (761b3c70d5); combined editor integration remains pending #3212 |
| 3 | Admin allocation view: per-reviewer assigned/completed counts and progress | Lets an administrator verify the split is working | Merged in #3213 alongside 2 (761b3c70d5) |
| 4 | Reviewer lifecycle handling for membership changes after activation | Deactivation currently pauses a stage silently | Pause visibility implemented; membership-first migration/lifecycle implementation authorized after current PRs ship; live execution gated |
| 5 | Validate stage-review authorization when configuring shares | Prevents configuring a member who cannot open the stage | Blocked on out-of-request authorization |
| 6 | Refresh the editor's reviews-per-study constraint on a Review Settings save | Client-side pre-validation only; server already fails closed | Implemented in #3212; shipping |
| 7 | Coordinate combined navigation-dialog SAVE ordering | Allocation publishes before the stage PATCH the domain then rejects | Implemented in #3212 with safe combined-save refusal; shipping |
| 8 | Memoise the derived plan by stage, configuration version and reviews-per-study | 10,000 buckets are walked per request; measurable CPU for a large team | Not started |
| 9 | Measure bucket-query performance at production scale | Decides whether materialized plans are ever needed | Budget (admin progress warm p95 ≤ 2,000 ms at 100k/50) not met: #3329 removed the per-reviewer fan-out; the covered-bucket design was measured and rejected in #3604 (misses the budget, regresses fully reviewed projects). Materialized per-reviewer counters are the evidenced next step. Protocol and evidence; production-like qualification remains outstanding |
| 10 | History-aware rebalancing for stages with completed work | Currently shares are fixed once review starts | Not started |
| 11 | Reserve/onboarding policy | Only after its review-slot semantics are specified | Deferred |
| 12 | Co-allocation optimisation and progress dashboard | Only if real usage demonstrates value | Deferred |
Current implementation slice¶
The direct endpoint plan maps the affected paths to acceptance criteria.
GET .../studies/{studyId}/review and ordinary PUT .../session/{sessionId} now return 404
for out-of-share new work before recording usage, reserving slots, mapping submissions, or
calling the save orchestrator. Flagged ordinary submission route/body stage and session IDs
must match (400 on mismatch), so one stage or saved session cannot authorize another.
Reconciliation uses the existing body mode and remains unchanged; the unused query parameter
cannot bypass an ordinary submission check. Session deletion and screening remain unchanged.
The ledger originated in PR #3202 and is carried forward here without changing that worktree. Do not treat this branch as deployed evidence. The delivery continuation tracks integrated reviewer/admin visibility, independent editor fixes and the remaining membership, third-party authorization and activation gates.
Local validation on 2026-09-05: full core suite 2,116 passed / one skipped; full API suite 1,227 passed / none skipped. API and Project Management endpoint builds succeed. Commands and the 66 newly added regression cases are recorded in the implementation plan.
Regime-evolution Phase 1 (PR #3603)¶
Scope: Phase 1 of the regime-evolution plan (merged in #3529). Allocation gets an identity, and each review records which regime admitted it. Who receives which work is unchanged. Correction, reallocation, stamping and reviewer-lifecycle changes are out of scope.
- Implemented: immutable regime records and an append-only transition log in their own
collections; the stage's current-regime pointer, published in one transaction with the
project; deterministic legacy-compatible regimes for pre-existing configurations; the shared
StageAllocationRegimeEvaluatorbehind every consumer; provenance on the typed annotation claim (review eligibility on) or the legacy reservation (off), carried onto the session; publication joined to the shared project-revision fence used by claim admission and the review-settings save (resynced onto #3579); the regime shown on the administrator progress read and panel; compatibility floor at regime schema 1 (per-request fail-closed, startup refusal, unknown BSON elements preserved). - Flag decision: no new flag. Assignment behaviour does not change, so there is nothing
for a flag to roll back. Regime records are only written by the existing configuration
endpoint, whose enable operation is already gated by
proportionalStudyAllocation. Provenance is additive metadata on writes that happen anyway. The fail-closed schema guard must be unconditional: plan section 8 notes that a flag cannot make an older binary understand newer data. The plan's separately guarded regime/reallocation behaviour belongs to the phase that first changes assignment (revision, correction or reallocation). - Deferred, not done:
- Bulk recording of legacy regimes for already-enabled configurations. This is plan section 8, step 3, a deployment-applied migration, and the repository has no migration runner yet (see #2042 / delivery plan). Until it runs, a legacy regime is identified deterministically, is fully reproducible from the unchanged stage configuration, and is recorded when first superseded.
- A free-text reason. Transitions record a reason code (
AllocationConfigured/AllocationDisabled); no UI collects free text yet. - Regime history and per-regime admission counts in the UI. The API and panel show the current regime; per-session provenance is persisted for later reporting.
- With
reviewEligibilityPolicyoff, direct-GET reservations (GET .../studies/{id}/review) are not stamped at claim time. The save-time admission check records the same regime when the session is created. In Phase 1 they cannot differ, because configuration changes are refused once any reservation or session exists. With the flag on, every claim goes through typed admission and is stamped. - The flag-off legacy claim writes only the study, not the project token. The fenced re-check catches such a claim if it committed before the publication's snapshot. One committing during an open publication is the residual #3321 window, which is unchanged.
- Validation: see the PR for exact per-project test counts. No environment, flag, live database or migration was changed.
Rollout¶
The flag is default-off and appears in no environment values file, so nothing changes for any user until someone deliberately enables it. Suggested sequence:
- Enable in a preview environment and configure a two-reviewer split on a test stage.
- Confirm each reviewer's "next study" stream stays inside their buckets and that a disabled member's stage yields no new work rather than an error.
- Ship items 1 and 2 before enabling anywhere reviewers will notice, because until then the allocation cannot be seen or relied upon.
- Enable in staging, then production, with the runtime flag as the rollback boundary.
Review-driven hardening already applied¶
Four review cycles on #2991 produced fixes worth recording, since they describe real failure modes rather than style: cache eviction after any failed allocation publish; navigation-guard save sequencing so the allocation and stage-settings writes cannot race; a per-request save token so a stale response cannot disable the wrong stage's Save button; review-settings form preservation when the allocation lock changes; reuse of an in-flight save instead of issuing a duplicate; share inputs locked while saving; allocation state cleared when a stage load fails; completion reported when a reviewer's buckets are exhausted; and an inherited reviews-per-study materialised on the stage so a later project agreement-mode change cannot invalidate an enabled allocation.