Proportional Study Allocation¶

Outcome¶

Project administrators can divide the new annotation work in a stage between active project members. A configured percentage is a share of review slots, not a share of studies. For example, in a two-review stage, a 50% reviewer is eligible for one review slot on every study.

This implementation replaces the abandoned approach in PR #2450. It deliberately delivers the smallest usable workflow and does not retain that branch's separate mapping collection, reserve pool, reviewer statuses, rebalance operations, dashboard, or OR-Tools dependency.

Administrator workflow¶

Create an annotation-only stage and set the number of reviews required per study.
Keep the stage disabled while configuring it.
Open Stage administration → Proportional workload allocation.
Enable allocation and assign percentages to the active project members. Use 0% for members who should receive no new work; the remaining total must be exactly 100%.
Save the allocation, then enable the stage for review.

Allocation can be changed or disabled only before any reviewer has started work in the stage. "Started" means a saved review session or an outstanding slot reservation: with active reviewer tracking a reviewer annotates against a reservation that graduates into a session only on their first save, so counting sessions alone would let an administrator republish the plan underneath live work. If the team needs different proportions after work starts, create a new annotation stage. This protects completed and in-progress work from being silently reassigned.

Assignment semantics¶

The feature is off by default behind proportionalStudyAllocation.
An unconfigured stage follows the existing assignment path byte-for-byte.
Configured stages use a fixed 10,000-bucket domain derived deterministically from the study ID.
Every bucket contains exactly Stage.SessionCountTarget distinct reviewers.
Reviewer bucket counts match their basis-point share of all review slots to within one basis point. A reviewer's exact target is basisPoints * reviewsPerStudy bucket slots, and those targets always sum to 10,000 * reviewsPerStudy. A reviewer holds at most one slot per bucket, so a target above 10,000 is unreachable; that happens only when the review count does not divide 10,000 (an equal three-way split on a three-review stage is 3,334/3,333/3,333, whose first target is 10,002). The plan clamps such targets to the bucket count and redistributes the few surplus slots to the reviewers with the most headroom, deterministically. The per-reviewer configuration cap is therefore ceil(10,000 / reviewsPerStudy), not the truncated value: the truncated cap made an equal split of exactly reviewsPerStudy reviewers impossible to save.
Saved-session resumption is never filtered; allocation applies only to new annotation work.
New annotation work is assigned through the atomic capacity-claim path introduced by PR #2467: the eligible buckets are applied to the random candidate query inside AtomicAssignRandomStudyForAnnotationAsync, and the study is then claimed with a SlotReservation under the stage's EnforceAnnotationTarget policy. An empty eligible set makes that query return no candidate, so the claim loop takes its ordinary "no study available" exit without retrying. An existing reservation is a resumption, not new work, and is returned before any bucket is computed.
The bucket plan assumes each study is filled by exactly Stage.SessionCountTarget distinct reviewers. A stage with EnforceAnnotationTarget off claims slots without the capacity guard, so concurrent claims can push a study past that target; eligibility stays correct (the candidate query still requires an insufficiently-allocated study) but the realised split can drift from the configured percentages. Stage.UpdateStage locks reviews-per-study while shares are enabled; it does not lock target enforcement.
The stage's study selection mode is locked to Annotation alongside its review mode. StudyAssignmentPolicy keys the fetch strategy off the selection mode alone, so an annotation-only stage left on ScreeningAndAnnotation would take the NewForScreeningOrAnnotation path, which receives no buckets.
When a reviewer's own buckets are exhausted but the unfiltered pool still reports new work, the assignment is re-resolved with the new-annotation pool treated as empty, so a reviewer holding unfinished saved sessions resumes them instead of being told the stage is complete.
Reconciliation pools are deliberately not filtered by workload shares. Shares steer who annotates new work, not who reconciles it.
A reviewer omitted from the configuration receives no new studies.
Missing, invalid, or stale reviewer membership fails closed for new work: when the configuration names a member who is no longer active, or no longer validates against the stage's current reviews-per-study, reviewers receive no new studies rather than an error. Resuming a saved session and reconciliation are unaffected in that state.

The domain type is StageWorkloadShareConfiguration, held by Stage.WorkloadShares; the derived plan is StageWorkloadSharePlan and each study carries a WorkloadShareBucket. Administration is served by api/projects/{projectId}/stages/{stageId}/workload-shares. "Allocation" remains the product-facing name of the feature and of its proportionalStudyAllocation flag.

Because the bucket plan is derived from the versioned Stage configuration, there is no second mapping record that can become stale. The immutable regime record that identifies each configuration (below) is published in the same transaction as the configuration itself.

Existing studies and indexes¶

New studies receive their stable allocation bucket when they are constructed, and Study.EndInit recomputes it on load whenever a document carries no bucket or an out-of-range one. The bucket is therefore a pure function of the study id for every in-memory instance, so a whole-document writer that loaded a legacy study before the backfill ran cannot write the sentinel back over it. Before enabling a configuration, the API additionally backfills missing or invalid buckets for every existing study in the project, so the bucket query does not depend on documents being read first. The configuration is saved only after that preparation succeeds.

MongoDB maintains the compound index ProjectId_1_WorkloadShareBucket_1. Rollback is safe: turning the feature flag off restores the existing selection path; bucket values and the index are inert and may remain in place. Rollback is also reversible: the flag gates only the enable operation on PUT .../workload-shares, so reading and disabling an already-configured stage keep working with the flag off. Gating the whole controller would strand such a stage, because its persisted configuration keeps Stage.UpdateStage refusing review-mode, selection-mode and reviews-per-study edits while the endpoint that lifts that lock has disappeared.

Direct annotation endpoint enforcement¶

With the runtime flag and stage allocation enabled, GET .../studies/{studyId}/review and ordinary annotation session submission use the same StageWorkloadShareEligibility bucket policy as next-study selection. Out-of-share or unconfigured reviewers receive 404 before usage writes, reservations, or submission. Stale/invalid configurations deny new work without throwing. Existing authorization and capacity checks still apply.

Saved ordinary sessions remain accessible for the same reviewer and stage, including completed work that needs correction. Submission must identify that exact saved session to receive this exemption. A different reviewer's session, another stage, a reconciliation session, or a bare slot reservation does not grant an exemption. Flagged ordinary submissions must have matching route/body stage and session IDs; mismatches return 400 before mapping or saving.

Reconciliation endpoints and submissions remain unfiltered. Submission mode follows the body used by the existing save path, not the unused query parameter. With the runtime flag off, or without enabled stage allocation, legacy eligibility behaviour is preserved.

Allocation regimes and review provenance¶

Phase 1 of the regime-evolution plan (PR #3603) gives every allocation an identity and records which one admitted each new review. Who receives which work is unchanged.

Regime record. Every publish through PUT .../workload-shares creates an immutable StageAllocationRegime in pmStageAllocationRegime. It holds everything the bucket plan reads: algorithm name and version, the stage id (the plan's tie-break seed), bucket domain, reviews per study and the reviewer shares in configured order. It also records predecessor, actor, time, reason and a SHA-256 input digest. StageAllocationRegimeEvaluator.Evaluate(regime) regenerates the plan from the record alone.
Current pointer. Stage.AllocationRegime is {RegimeId, SchemaVersion, ConfigurationVersion}. Only ConfigureWorkloadShares/DisableWorkloadShares move it, and each move produces an append-only StageAllocationRegimeTransition (pmStageAllocationRegimeTransition). StageAllocationRegimePublisher commits the regime, transition and project in one Mongo snapshot transaction. That transaction joins the shared project-revision fence used by typed claim admission (TryAdmitActivityReviewAsync) and the review-settings save. Its first write is the project admission token for the revision the change was prepared from, so the second of two overlapping fenced writers conflicts and aborts. It then re-checks stage review activity inside the snapshot, because a flag-off legacy claim never reads the project in a transaction and can commit after the controller's sequential pre-check (#3326, kept). A stale revision or a write conflict is reported as ProjectChanged and returns 409 "changed by someone else". Activity seen in the snapshot returns 409 "only before review starts". Nothing is written unless the whole publication commits. Disabling clears the pointer. Allocation-off mid-review is still not supported.
Legacy configurations. A stage with an enabled configuration and no binding pointer is governed by a legacy-compatible regime. Its id is derived deterministically from (stage id, configuration version), and it evaluates the same plan as before. It is recorded in pmStageAllocationRegime the first time it is superseded. Bulk materialisation of legacy records is a deployment-migration step and has not been done (see STATUS).
One evaluator. Next-study admission, direct access, submission and both read APIs all go through StageAllocationRegimeEvaluator via StageWorkloadShareEligibility. It calls the unchanged StageWorkloadSharePlan. Golden digests and a 30-configuration equivalence matrix (1–3 reviews per study, 3–40 reviewers, equal and unequal shares, members given no share) pin the bucket lists.
Provenance. Where the admitting regime is recorded depends on reviewEligibilityPolicy:
Flag on (typed claims). It is recorded on the annotation claim of the typed page by ActivityReservationAdmission.Reserve, from the allocation decision it evaluates against the project read inside the fenced transaction. Screening claims never carry a regime. Releasing the annotation claim clears it.
Flag off (legacy untyped reservation). The next-study claim records it on the SlotReservation in the atomic pipeline update.

In both paths, graduation carries it onto the AnnotationSession.AllocationRegimeId, and deleting and restoring the session carries it back onto the reservation or annotation claim. If no reservation carried one, the save-time server-side admission check records it on a session the save creates. Provenance is never taken from the client and never rewritten afterwards. Sessions from before this release, reconciliation sessions and unallocated work keep null, which means unknown. They are never backfilled with a guess. - Administrator inspection. GET .../workload-shares/progress returns regime (regimeId, origin = Configuration/LegacyConfiguration/LegacyAfterPointerMismatch, schemaVersion, supported), and the administrator progress panel shows it. LegacyAfterPointerMismatch means a release below the floor changed the configuration after a regime was published. That fallback is flagged rather than silent, and the fallback's legacy record names the displaced regime as its predecessor.

Compatibility floor¶

The regime schema version is StageAllocationRegimeSchema.CurrentVersion = 1. The first release containing PR #3603 is the minimum binary that understands regime fields.

A reader that meets a newer or unknown pointer schema fails closed. The stage serves no allocated new work, even if its configuration reads as disabled; saved work stays resumable. Allocation writes (PUT .../workload-shares, and review-mode/reviews-per-study changes) are refused with 409. The read APIs report stale and supported: false.
At startup, API and project-management run StageAllocationRegimeCompatibilityCheck (IRunAtStartUp). It refuses to start if either regime collection holds a schema this build does not understand, or if the store cannot be read.
The pointer and regime classes keep unknown BSON elements, so a whole-project replace by this release preserves fields a newer release adds.
Binaries below the floor cannot be made to refuse. They keep the pointer (Stage retains unknown elements) but would change a configuration without an audited transition. The evaluator then treats that stale pointer as the new configuration's legacy regime, reported as LegacyAfterPointerMismatch. Do not roll API or project-management back below the floor once a regime has been published.
These guards are unconditional. A feature flag cannot make an older binary understand newer data. With proportionalStudyAllocation off, allocation is not enforced at all, as before. That flag remains the rollback boundary for enforcement.

Delivery status¶

The original MVP is merged but dormant. Direct enforcement is in PR #3211. This follow-up adds the stage-overview reviewer allocation page and administrator counts/progress, backed by authorized read-only APIs. These changes remain pending review/merge and do not enable any environment flags. See STATUS.md for the authoritative remaining-work order and the delivery continuation for the acceptance criteria and unresolved gates.

The reviewer page uses GET .../workload-shares/my-studies?offset=0&limit=25; it shows distinct assigned studies, own saved status, remaining and currently available work, with bounded direct-study links. The administrator view uses GET .../workload-shares/progress and the existing stage-design policy. Both APIs return 404 with the runtime flag off. Counts are calculated in MongoDB without loading complete project studies into the API or browser. No extra collection or live event stream is introduced.

Assigned studies include preserved own ordinary saved work; remaining is assigned minus completed. Startable new work also respects the existing capacity query and is reported separately. Stale configurations show a paused state with no new assigned buckets, while keeping saved work visible. Administrators see inactive configured reviewers and recovery guidance, but membership mutations and automatic redistribution are not introduced.

MVP limits and follow-up order¶

Allocation and tracked-capacity acceptance¶

e2e/tests/allocation-capacity-integration.spec.ts exercises the combined contract in the isolated E2ETest environment using its existing authenticated test users. It enables allocation through the audited runtime-flag API, checks the tracking configuration, and restores the previous allocation override afterward. It configures an annotation-only stage with target two and enforcement enabled through the domain-backed setup endpoint (whose screening selection default is unchanged).

The journey checks assigned-study lists, concurrent next-study claims from two eligible reviewers, real SignalR presence, refresh deduplication, out-of-share GET/save denial, concurrent completion, and re-saving one's existing session at capacity. MongoDB assertions verify durable reservations and saved-session counts independently of UI indications. This does not change warning-only semantics or enable any preview, staging or production flag.

The browser journey requires a successful revision-specific E2E run before it constitutes acceptance; TypeScript compilation and setup-controller unit tests alone are insufficient. Existing lifecycle and save-guard suites remain necessary for disconnect, expiry, release and slot-loss behavior. Signed-in preview/staging acceptance is distinct from this isolated test environment, and the authorization assessment and publication/review-start fence remain open in the feature ledger.

The MVP supports annotation-only stages and active project members.

Known limitation: stage-review authorization is not validated. Eligible reviewers are the project's active members. A stage with a restricted StageReviewPolicy can therefore be given a positive share for a member who cannot open that stage, and those bucket slots are never filled. The check is not made here because it cannot yet be made faithfully: StagePermission.IsStageAuthorized needs the reviewer's own application-group claims. Identity stores groups, but there is no canonical out-of-request resolver that preserves the current cross-provider token and stored-role semantics. It also resolves its defaults through ResourceSecurity.Instance, a write-once static assigned only by the API composition root and absent from the project-management host and the domain unit tests. An approximate check evaluated with no claims would wrongly reject a member authorized through an application role. This remains an explicit dependency in STATUS.md.

The following are progressive enhancements, in order:

Review and merge direct endpoint enforcement, then deliver reviewer assigned-studies UI, remaining-share counts and an allocation-active indicator, followed by administrator assigned/completed counts. See STATUS.md for the complete delivery sequence.
Add explicit reviewer lifecycle handling for membership changes after activation.
Validate stage-review authorization when configuring shares and when listing eligible reviewers, once reviewer authorization can be evaluated outside a request principal.
Memoise the derived plan by stage id, configuration version and reviews-per-study. StageWorkloadSharePlan.Create walks all 10,000 buckets on every "next study" request and there is no reviewer-count limit, so a large team adds measurable CPU per review request.
Measure bucket-query performance at production scale before considering materialized plans.
Add history-aware rebalancing for stages that already contain completed work.
Add reserve/onboarding policy only after its review-slot semantics are specified.
Add co-allocation optimisation and a progress dashboard if real usage demonstrates value.
Refresh the allocation editor's reviews-per-study constraint when the stage entity changes without a stage switch (a Review Settings save that changes the target). The server still validates and fails closed with a 400, so this only affects client-side pre-validation.
Coordinate a combined navigation-dialog SAVE that both enables allocation and changes a locked stage setting (review mode, study-selection mode, reviews-per-study). The allocation publishes first and the domain then rejects the stage PATCH; surface that rejection or validate the stage settings first. Tracked with item 9 in the review follow-up issue.