Screening and annotation eligibility¶
Status: proposal for review, with D1/D2/D4/D6/D7 decided and D3a clarified by Chris on 22 September 2026. Approval records configuration applicability/defaults, the structural design direction, reservation applicability, completion meaning and the narrow D4 compatibility decision; it does not authorize implementation, database changes, runtime activation, merge or deployment. Implementation and operational rollout tasks remain.
Implementation workstream¶
Chris authorized implementation after #3552. The implementation goal is the complete approved MVP, not completion of an individual pull request. #3565 contains the initial unflagged reconciliation-save authorization correction. #3579 begins the configuration implementation; the remainder of the acceptance criteria below remain required.
| Slice | State and evidence required |
|---|---|
| Reconciliation-save grant and route identity | Implemented in #3565; focused endpoint tests and current-head CI pass. Broader ownership checks remain below. |
| Session-save row ownership (unflagged) | Implemented in #3671, flag-independent: SessionWriteOwnership is the one rule for which annotation and outcome rows an annotation-session save may create or replace. Incoming rows must be the caller's (this project and study, the save's reconciled flag, the caller as annotator/investigator for an ordinary save; outcome rows for this stage). A stored row reused by ID, or referenced as a parent/child, must be the caller's: same reconciled flag and, for an ordinary save, same annotator/investigator; reconciled rows are shared by reconcilers. A reused annotation must keep its question, and the question must be in the stage. Stage is not ownership: an annotation's or ordinary outcome row's StageId is a provenance stamp (the latest stage that saved that answer), so the caller's own row from another stage is replaced and re-stamped, as before; reconciled outcome rows keep the same-stage rule. Stored rows are not checked on StudyId or ProjectId (embedded-row stamps; legacy rows can carry a stale ProjectId, which the replacement re-stamps), and duplicate stored IDs are refused only when some copy is not the caller's. ExtractionInfo enforces it before mutating the study, and the flagged ActivityReviewWritePolicy calls the same rule. The server stamps investigator, project and reconciled on submitted outcome rows. Refusals are the typed AnnotationOwnershipMismatch 403. Not flagged: it closes a cross-reviewer overwrite, and legitimate web payloads only reuse the caller's own IDs. |
| Configuration boundary and creation | Implemented behind the default-off flag in #3579: closed activity/selection variants, versioned pure legacy mapper, new Combined Stop persistence, and creation UI. BSON, API, UI and flag-off checks pass locally; CI validation continues. |
| Settings and review admission | In progress in #3579: guarded settings GET/PUT, separate conversion/claim-release consent, revision checks, and current stage-review counts. Browser editing uses the loaded revision, preserves drafts on refusal, and requires separate explicit conversion choices and Apply anyway consent. Ordinary screening, annotation saves, and saved-session deletion recheck current stage permission, activity and ownership under a project/study transaction fence; first saved reviews record positive history. Typed refusals roll back optimistic screening, keep annotation drafts, and refresh stage settings. Outcome ID ownership and cross-stage outcome preservation have focused tests. #3695 (S2-C, flag-gated) fences the administrative screening writers (bulk study update and search import with screening columns) and uses statistics evidence for the settings save; see S2-C encoding. Pending: historical coverage and complete server/UI action decisions. |
| Reservations and protected settings save | In progress in #3579: typed independent claims, shared page lifecycle, per-activity graduation, project/study screening completion release, pure affected-claim planning, and transactional admission using the project revision fence. Replica-set races cover screening capacity across stages. Settings conflict/Apply anyway is connected to the browser editor. A connected reviewer's saved-session deletion restores its typed annotation claim without resetting the shared page timer or losing a screening claim. S3 (#3719, flag-gated) tests the idle, suspension and liveness paths over typed pages and the restore paths, performs the D8 decision 2 claim release in fenced admission with a typed warning, and refetches browser state after a release, a reservation conflict or Apply anyway; see S3 encoding. #3720 (flag-gated) adds the targeted SignalR claim-revoked event with durable intent and the ClaimReleased denial reason; see Claim-revoked event encoding. |
| Pure eligibility policy and truth table (S1a) | Implemented in #3646: pure Core ReviewEligibilityPolicy over explicit ReviewEligibilityFacts, deciding every action and selection pool separately with stable reason codes, and the shared truth table (C# ReviewEligibilityTruthTable in SyRF.Testing.Common, checked against its markdown export). Not wired into services, controllers, repositories or UI; encodings and recorded gaps are listed under S1a encoding and recorded gaps. |
| Mongo pool predicates against the truth table (S1b) | Server pools in progress in #3659 (flag-gated): ReviewEligibilityFactsBuilder and IReviewEligibilityService assemble facts from the stored aggregates; ReviewEligibilityPoolFilters holds one canonical Mongo predicate per pool (new screening, new annotation, resume saved with D8 hidden-excluded resumption, D8 reconciliation pool) whose stage gate is the policy's own PoolStageGate. With the flag on, Next status counts and every candidate sample come from these pools and new claims use typed admission; flag off runs the legacy queries unchanged. Every truth-table row is seeded in Mongo and the predicates, builder and policy agree row by row. #3663 (stacked, flag-gated) adds the additive eligibility section to NextStudyResponseDto on every review/reconcile study response: policy version, project/study revisions, one allowed/reason/warning decision per action and selection, and the reported leftover-claim release; null with the flag off. Pending: the UI route guard that blocks opening a disabled stage (D8). The leftover-claim release is performed by S3 (#3719). See S1b encoding. |
| Flagged D4/D8 refusal completion (S2) | Implemented in #3691. Deletion follows D8: the flagged ActivityReviewWritePolicy.EvaluateDeletion requires only an active member with the stage Review grant, plus ownership, matching the pure RemoveOwnAnnotation and the truth table (no StageDisabled/ActivityUnavailable refusal). With the flag on, RemoveSession does not restore an annotation claim on an inactive stage or one without annotation. With the flag on, every refusal on the three screening endpoints, SubmitSession and DeleteSession is typed (code) and leaves the data unchanged: controller-level NotFound/Forbid/BadRequest become typed refusals; a second session becomes DuplicateSession; an allocation denial becomes Forbidden; and a stage-policy 403 from the authorization middleware becomes typed through ReviewWriteAuthorizationResultHandler. A replica-set test matrix checks each refusal against the truth table where a row applies. The browser's session deletion handles typed refusals like the save paths. Flag-off responses are unchanged, except for the unflagged #3635 fix: the reconciliation payload check runs after the Reconcile-grant check and answers a typed 400, and SubmitSession's unused reconciliation query parameter is removed. |
| Legacy migration and release evidence | In progress. S4-A (#3732), the grouped configuration reader and writer floor, adds no behaviour change for unmigrated stages. GroupedReviewConfiguration is the D7 target shape: a closed activity variant; AnnotationStageSettings (target override, enforcement, incomplete-session limit, and an ExcludedWorkSetting that accepts only the editor's valid combinations) for AnnotationOnly and Combined; inert DormantAnnotation for ScreeningOnly; and workload-share allocation owned only by AnnotationOnly. Annotation form content and workload shares stay in their existing fields and are composed, not copied. A v2 ReviewConfiguration Stage element is read when present; otherwise the legacy fields are decoded with the production getters. Values that cannot construct a valid configuration are reported for a decision, never guessed. Only the migration writer (ApplyMigratedReviewConfiguration) writes v2, together with a legacy mirror and a digest of it. A later legacy edit by a writer below the floor therefore reads back as DivergedFromLegacyMirror. Settings edits on a migrated stage (the guarded PUT, the legacy JSON Patch route and stage creation/update) are refused with the typed ReviewConfigurationMigrated until activation. The mixed-version writer floor (ServiceVersionFloor) is satisfied only when every registered instance of each configured service records a version at or above its minimum. It fails closed on missing registry, service, instance or version. Instances record their version at startup, and version-less instances (binaries that predate this) are no longer pruned by age. Pending: S4-B migration tooling (dry-run plan, CAS apply, rollback, audit, fixtures) behind its configuration gate and the floor, S4-C settings API/editor, and staging acceptance. No production writes or activation are authorized. |
Flag decision: the new reviewEligibilityPolicy flag defaults off. This is a
cross-service behavior and persistence change whose action/reservation/migration paths
are still being implemented. Off retains existing creation and review behavior and
rejects explicit new-policy input; it must not erase a persisted explicit policy.
Do not enable this flag while only configuration creation is present. Enabling requires
the full server/browser contract and the migration/compatible-writer gates below.
The unflagged D4 reconciliation security correction is independent of this switch.
Activity-claim implementation boundary: typed page reservations carry independent screening and annotation acquisition times. Existing untyped BSON remains unchanged until the audited migration. Typed admission rejects unmigrated reservations and uses a real project-token write plus a study version check in the same Mongo transaction, including when materialized-statistics writes are off. Screening capacity is shared across stages; annotation tallies count annotation claims only. A successful admission can return no page (for example optional screening in Combined Allow), and a saved annotation can coexist with a remaining screening claim. SignalR consumes the committed admission result rather than treating an old reservation as permission to bypass a refusal. This partial path remains default-off: settings reconciliation, session-delete restoration, complete action refusals and rollout/migration gates must be finished before activation.
Settings-admission work in progress: the consent policy distinguishes unavailable
current statistics, reviewed/unknown-history conversion confirmation, and temporary-claim
release consent. Apply anyway bypasses only the last of these. The authoritative
StageSavedReviewCounts calculator counts stage-attributed screening and saved ordinary
and reconciliation sessions; reservations and reviews on another stage/project do not
contribute. This is current evidence, never proof of historical non-use. The existing
ProjectScreeningClassifier emits project-scoped moves even though the read catalogue
accepts stage selectors, so a project tally cannot substitute for these stage counts.
The guarded GET/PUT review-settings endpoint now uses an uncached project, a project-token
write and authoritative stage counts in one transaction. It returns separate conversion
and reservation conflicts, preserves inherited targets, and applies approved claim releases
with the settings write. The legacy JSON Patch route refuses edits while the flag is enabled.
A nullable monotonic historical marker is mapped without turning missing legacy evidence into
false. The browser form reads the raw inherited target and its effective value from the same
revision, keeps edits on refusals, and offers explicit conversion choices separately from
claim-release consent. Stage activation uses that same guarded save while the flag is on.
Changing a proposal invalidates its previous Apply anyway permission; a stale-version response
never silently rebases the draft. Ordinary screening and annotation submissions now join
the shared project fence, rechecking current stage access, activity and ownership before
the source write. The first successful save on a stage records positive review history in
the same transaction. A current transaction test covers an overlapping first screening
and conversion attempt; a rejected related write leaves neither review nor history.
This is still a partial writer cutover: other review owners and imports, historical
NeverReviewed proof, statistics bundle/fallback integration and mixed-writer deployment
gates remain. The new reviewer-save refusals have distinct server codes and the browser
keeps annotation drafts after those refusals, but complete route/refetch/timer behavior
and end-to-end browser acceptance are not yet established.
Storage boundary during implementation: legacy mode/selection fields remain readable
for compatibility; an additive nullable AdditionalScreeningPolicy BSON element records
explicit Combined choices. Missing remains distinguishable from explicit Allow/Stop.
The validated domain value derives single-purpose selection and cannot contain an Allow
override in a single-purpose variant. This bridge is not the completed D7 migration or
the complete annotation-settings grouping: preserve raw before-images and historical
annotation setup when those remaining mappings are implemented. Do not activate mixed
writers that can discard the added policy field on replacement saves.
Reconciliation prerequisite¶
Chris authorized implementation after the policy PR #3552. PR #3565 implements the first D4 security prerequisite: annotation-session submissions require the stage's Reconcile grant when either the payload or the stored session is reconciliation. Route/body identifiers must agree with allocation flags off as well as on, and a stored session cannot be submitted through another stage's route. This correction is unflagged because retaining the permission bypass is not a supported rollback behavior. It does not complete the broader ownership, activity, reservation or eligibility work. The endpoint regression selection passes 137 tests, covering ordinary and authorized reconciliation submission paths as well as refusals before the save service is called.
Policy and recorded decisions¶
The stage policy distinguishes whether another reviewer may record a screening decision after the project's screening-completion rule is met. D1/D2 below record the approved defaults.
Production uses annotation form one and permits this optional screening when a study is open and the other checks allow it. The incident that prompted #3551 occurred in annotation form two on staging. That PR introduced a wider restriction: a reviewer without an existing decision cannot submit one after screening is complete, regardless of the stage's selection mode. It did not add a requirement to screen before annotating.
The proposal makes Allow/Stop an explicit setting only for combined screening-and-annotation stages. Chris's latest clarification governs D1/D2/D6:
- Screening-only: always implicitly strict, for existing/historic and new stages. There is no configurable Allow override.
- Combined: existing stages with no saved setting resolve to Allow; newly created stages explicitly default to Stop, including new stages in existing projects. Administrators may choose Allow or Stop, and known explicit combined-stage selections are preserved.
- Annotation-only: no applicable screening policy.
Strict screening refuses a new reviewer decision after the project's screening-completion rule is met. Allow permits it on opened work when other checks allow it. Neither changes which studies Next searches for or permits an unavailable activity.
With tracking enabled, screening-only uses screening reservations; combined Stop uses separate screening and annotation reservations with a shared page timer; combined Allow uses annotation reservations only. Screening reservations require strict screening and tracking enabled. The lifecycle implementation remains proposed. These decisions do not authorize activation.
For a combined stage, the stated objective is that each study meets the required screening and annotation thresholds through reviewers' collective contributions. Individual reviewers can contribute independently. Selecting work that still needs either activity is consistent with that objective, but current selection and allocation checks do not by themselves certify that every study has completed both.
For example, two reviewers have completed a study's screening and a third reviewer opens it for annotation. With Allow, the third reviewer may also record their own screening decision if the other checks pass. With Stop, that new screening decision is refused; annotation follows its own rules. Changing a decision the third reviewer already recorded is a correction and is assessed separately.
The UI and API must apply the same agreed rules. The API must check again when saving, because another reviewer or an administrator may have changed the study or settings since the page opened.
Preserving production's legitimate workflow does not require preserving every historically accepted API request. Existing differences in authorization and mode enforcement must be resolved explicitly before implementation.
Terms used here¶
| Term | Meaning |
|---|---|
| Review Mode | Which activities the stage presents: screening, annotation, or both. The proposal would enforce those capabilities consistently in the UI and API; current API gaps are described separately. |
| Study Selection Mode | Which new studies Next searches for: those needing screening, those available for annotation, or either group. It does not decide every action permitted on an open study. |
| New screening decision | The reviewer has not previously recorded a screening decision for this study in this project. Other reviewers may already have screened it. |
| Correction | The reviewer changes their existing decision. Each reviewer has one decision per study per project, shared across stages. Submitting through another stage updates that decision and its stage attribution; it does not add an independent vote. |
| Reconciliation | A separately authorized process for resolving disagreements. It is not a way to bypass a refusal to accept a new reviewer decision. |
| Screening complete | The study meets the project's configured screening-completion rule. That rule can involve count and agreement. Completion does not always mean inclusion or agreement: manual dual-screening disagreement can be complete. |
| Annotation session | A reviewer's saved annotation work in a stage. It may be incomplete or completed and is distinct from a screening decision. |
| Capacity | Whether another temporary reservation may be added. Proposed screening capacity uses submitted project decisions and outstanding screening reservations; annotation capacity uses that stage's annotation sessions/reservations and annotation target. Reservations do not contribute to agreement. |
| Temporary reservation | A place held while a reviewer works, distinct from a saved annotation session or submitted decision. Current storage has one untyped reservation; separate screening/annotation reservations are proposed schema changes. |
| Strict / permissive | Shorthand for the proposed Stop / Allow new-screening policy on an opened study. A suggested default is not an enforced administrator setting. |
| Eligible candidate set | Studies eligible for fresh automatic selection for the requested activity; “set” refers to studies, not a group of people. |
| Collective stage completion | All eligible studies in the stage have been sufficiently reviewed according to configured settings (Chris’s D3a clarification). Contributions are collective; one reviewer having no available work does not establish this. No stage-closure concept is introduced. |
| Individual reviewer obligation | Whether an individual may contribute screening or annotation independently, or must finish both on an assigned study. A collective both-activities objective does not itself impose this obligation. |
What production supports and what changed on staging¶
We compare three baselines because production and staging did not run the same workflow. Chris confirmed that production has only used form one and that the incident prompting
3551 concerned form two on staging.¶
The production source shows that optional screening on annotation-selected work already existed in form one. Form two did not introduce that permission. The later ability to record a decision and stay on the study belongs to the redesigned review page; it is also separate from the choice of annotation form.
| Baseline | What it establishes | Limit |
|---|---|---|
| Production, form one | Optional screening on opened annotation work is supported; Include/Exclude submits and advances when local conditions permit. | Source and current deployment observations do not prove every historical user workflow. |
| Staging before #3551, form two | Optional screening was also accepted, subject to newer access/capacity behavior. The redesigned page could record and stay. | This is a pinned source comparison, not proof of the exact deployment or flags during the incident. |
| Staging after #3551 | A new reviewer screening decision is refused after the project's completion rule is met. Existing-decision corrections and reconciliation are exempt from this new restriction. | Other capacity, access and authorization checks still apply. |
Production is the compatibility baseline. Restoring the earlier staging code alone would not establish compatibility: staging already had newer capacity, reservation and navigation behavior. The D1 Allow fallback preserves optional screening in existing combined stages; screening-only stages are always implicitly strict. Explicit combined-stage administrator values take precedence. Further compatibility changes need comparison and approval.
The detailed implementation analysis below describes staging before and after #3551 unless a production source is explicitly identified. Do not assume those newer rules existed in production.
Production controls · Production API · Pre-change review host · #3551 diff
What administrators can configure today¶
Creating a stage and editing an existing stage behave differently. Creation chooses matching study selection for a screening-only or annotation-only stage. A combined stage offers all three selections.
Editing disables the selection control for single-purpose stages but retains its existing value. Saving includes that disabled value. A combined stage changed to annotation-only can therefore keep Screening selection: Next searches for work needing screening, while the page presents annotation. This can arise through the normal edit interface, not only through an API request.
The API generally stores both supplied values without requiring a matching pair. Persisted explicit selection values also take priority over defaults, including in older stages. Only a missing value defaults from Review Mode.
Enabled proportional allocation is a narrower exception: it requires Annotation review with Annotation selection and prevents changes to those modes or the allocation target until allocation is disabled.
These are current storage/UI behaviors, not the intended configuration contract. Chris has clarified that only combined stages own a selectable StudySelectionMode and the Allow/Stop setting. Single-purpose stages derive effective selection from ReviewMode. Migrate audited legacy records into the corrected domain model using the mappings below, preserving an audit of their original values. Migration design is authorized; executing production writes is not. The proposed domain model below makes valid configurations structural.
| Review Mode | Create-stage UI | Edit-stage UI | API/domain and legacy storage |
|---|---|---|---|
| Screening only | Computes Screening selection, regardless of hidden selection control | Selection is disabled; an existing non-null value is retained | No general requirement that selection=Screening |
| Annotation only | Computes Annotation selection | Selection is disabled; an existing non-null value is retained | No general requirement that selection=Annotation |
| Combined | Requires a choice among all three selection modes | Enables all three choices | All three are stored and used |
Production creation · Production editing · Production stage API · Production stored defaults · Current creation · Current editing · Save mapping · Current stage API · Stage domain · Legacy/default getter
Production inventory: verified aggregate findings, 22 September 2026¶
An explicitly authorized read-only production audit at 10:14:24–10:14:25 UTC inspected
2,315 projects and 2,292 stages. Of those projects, 1,435 contained stages; 880 had no stages.
The deployed production getters and BSON mappings at 104eca9b05dceb8a3504eec6ee319f3a6a428283
were checked before classifying stored values. This is observed production evidence, not an inference
from staging/form-two behavior or from proposed rules.
There were 36 effective mode mismatches across 30 projects: 26 active stages in 23 projects and 10 inactive stages in 9 projects. Two projects occur in both groups, so the project counts must not be added. “Mismatch” means inconsistent with this document's intended single-purpose pairing rule; it does not mean the production API rejects the combination. Combined ReviewMode permits all three valid StudySelectionMode values.
| Effective ReviewMode | Explicit StudySelectionMode | Active stages | Inactive stages | Total |
|---|---|---|---|---|
| Screening | Annotation | 1 | 3 | 4 |
| Screening | Combined | 5 | 4 | 9 |
| Annotation | Screening | 7 | 3 | 10 |
| Annotation | Combined | 13 | 0 | 13 |
| Total | 26 | 10 | 36 |
Separately, 52 legacy stages across 44 projects store both Screening=false and Annotation=false. They are suspicious raw configurations but effective-valid, not additional mismatches: the deployed schema-0 getter returns Screening whenever Annotation=false, and their selection defaults to Screening. Their absent/null activity field defaults to active. Across the entire inventory, 581 stages had defaulted selection/activity values (371 missing, 210 null); these were not falsely counted as mismatches. All 2,292 stages used effective schema 0, with no persisted ReviewMode as expected. No unknown effective enum, schema or activity values were found. Explicit non-null selection values take precedence over legacy/default derivation, even when they conflict with the intended single-purpose pairing.
A further read-only activity inspection at 10:23:02–10:23:51 UTC covered the 88 flagged stages in 74 projects, including both categories. 23 stages had surviving attributed review records; 65 had none in the inspected current records. This does not establish that the latter were never used. Recorded creation timestamps cannot identify the latest edit/review action: screening corrections can change StageId without updating creation time, and annotation-session status updates retain their creation time. No generic project modification, access or presence timestamps were substituted. Screening reconciliation is not separately marked in its stored decision record; annotation reconciliation was checked separately and no attributed records were found for these stages.
The exact bounded read-only queries, timestamps, affected project/stage identities, names, creation dates and per-stage activity details remain in the private administrator report outside Git. Only aggregate findings are included here. Queries returned narrow projections; there were no production writes, repairs or settings changes. The sequential and paginated reads were not a transactionally frozen snapshot, so concurrent changes and later audits can produce different results.
These findings supply D7's concrete migration inputs. The migration design below maps legacy records into valid configurations; this audit does not authorize executing it, deleting work, or treating an effective-valid both-false record as a mismatch. Include inactive stages in the migration plan and preserve their saved work.
How Next finds work¶
Next can select new work or resume a reviewer's saved annotation work. Its result depends on the stage settings and the reviewer's existing work.
Studies available for new screening: the study's current cached screening status says more screening is needed, and this reviewer has no existing screening decision for it.
Studies available for new annotation: the study is not sufficiently excluded, this reviewer has no annotation session outside reconciliation in this stage, and the study passes the annotation allocation query. An existing incomplete or completed session prevents it from being selected as new annotation work. Screening first is not required.
Studies with saved annotation work: this reviewer owns an incomplete annotation session in this stage and review context. The separate resumption rules determine when Next returns that work.
Screening selection searches the first group. Annotation selection searches the second. Combined selection searches either of those groups; a study does not have to belong to both. None of these selections requires the reviewer to complete both activities before leaving.
For positive allocation targets, the annotation query checks that fewer sessions are allocated than the target. It counts saved candidate sessions and, when tracking applies, reservations. There is a historical boundary exception: with no stage tally, the query admits the study even when the saved target is zero. A tracked claim with annotation capacity enforcement can then refuse it. Record that behavior separately in compatibility tests; do not silently change it by replacing the historical query with a simplified count comparison.
| Selection mode | New studies searched by Next | Must this reviewer do both before leaving? |
|---|---|---|
| Screening | Available for new screening | No |
| Annotation | Available for new annotation | No |
| Combined | Either group | No |
Selection filters · Count filters · Pool queries
Collective goal and selection copy¶
The goal is for each study to meet the required screening and annotation thresholds through the reviewers' combined contributions. Next offers studies eligible for either activity; each reviewer may contribute independently. The current selection rules allocate work and do not by themselves confirm that every study has completed both activities.
Suggested selection label: Offer work for screening or annotation. Keep the collective objective in the explanation; this option does not configure completed annotation thresholds or a stage-closing gate. “Available for annotation” describes allocation eligibility, which is not always the same as unfinished collective work. Existing labels
Worked examples¶
Screening complete, annotation not started. Two reviewers meet the project's screening rule by including a study. Maya has done neither activity and there is annotation capacity. Annotation or Combined selection can offer her the study; Screening selection does not offer it as new screening work. Production and earlier staging can still permit Maya's optional screening decision on the opened study. Later staging refuses it. Under the proposal, the stage's Allow/Stop setting would decide that question.
The reviewer already screened it. Maya previously screened the study through another stage. If she opens it here, changing her decision is a correction to the same project decision. It does not create a second vote. The current stage's other checks can still affect whether the correction saves.
Saved annotation becomes excluded. Maya saves incomplete annotation work and the study later becomes sufficiently excluded. It is no longer new annotation work. Whether Next resumes her saved work depends on the hide-excluded and resumption settings; direct opening and saving are separate checks. The proposal must not discard her saved or unsaved work merely because exclusion changed.
How reviewers open or return to a study¶
There are two choices in the interface: ask the application to choose a study, or choose a particular study yourself. Asking the application does not always produce new work: it can return a temporary reservation or resume saved annotation. A reservation holds capacity temporarily; a saved annotation session contains persisted work.
The journeys below are verified from the pinned source, not from a new browser test. The Incomplete Reviews list exists in production form one. The My studies navigator and record-and-stay behavior belong to the redesigned staging page; they are not consequences of enabling annotation form two alone.
Ask the application to choose: enter review or use Next¶
Entering the stage's review route without a particular study, or requesting Next, asks the server to choose work. Selection mode determines which new studies it searches for: screening, annotation, or either. Before searching, however, current tracking can return a study already reserved for this reviewer in the stage. Saved-work rules can also return unfinished annotation instead of new work.
For example, Maya returns to the review route while her temporary reservation still exists. The tracked path may return that same study. If she has already recorded a screening decision in this stage, the next-study request instead tries to release that reservation before choosing again. It does not delete her decision or saved annotation. Contention can prevent the release after bounded retries, so Next does not guarantee a different study on every request. Selecting a new candidate also does not guarantee a place: its atomic capacity claim can lose to another reviewer.
Ask for unfinished work: turn on the incomplete-reviews restriction¶
In production's Incomplete Reviews list, the toggle is Restrict random selection to studies with incomplete reviews. The redesigned staging My studies navigator has the corresponding Restrict random selection (Next study) to studies with incomplete reviews toggle. This changes what the next selection request asks for; it is different from clicking a study in the list.
With that restriction, the server tries to resume the reviewer's saved unfinished annotation. It can also resume saved work when the enabled incomplete-review limit is reached or the relevant new-work candidates are exhausted. If no saved work remains, a saved-work request can fall back to new selection; the toggle is not a permanent ban on new work. Excluded saved work is offered only when the existing hide-excluded setting allows it. Proportional allocation filters new Annotation-selection candidates, not saved resumption in the same way.
Choose a particular study: click its saved entry or open its URL¶
To return to Maya's own saved annotation, she can click its entry in Incomplete Reviews, or My studies on the redesigned page. Both navigate to that study's specific review URL. Opening a bookmarked review URL takes the same specific-study path. The server does not randomly substitute another study.
Navigation still follows the applicable unsaved-work guard; it does not silently swap out a dirty form. The server checks access and project/stage context and can apply in-progress, allocation and capacity restrictions. Where tracking requires it, the server attempts a reservation or recognizes the reviewer's existing reservation/saved session; a reread can also recognize their screening in that stage. A full target does not by itself remove an own valid saved session. A URL is therefore an alternative way to reach a study, not permission to bypass the checks or submit every activity.
Record a decision and stay: no new-study request¶
On the redesigned staging page, recording a screening decision can leave Maya on the same study to annotate. The record-only request does not ask the server to choose another study, and the current reservation is deliberately retained while she stays. The decision is already submitted; asking for Next later is a separate action.
The advancing screening action submits the decision and requests the next study in one request. Production form one's Include/Exclude follows this advancing pattern, with its existing local dirty-form restrictions. Do not describe staying, advancing and resuming saved annotation as the same operation.
Implementation trace: the no-study route dispatches getNextStudyForStage; choosing
a listed study uses its ID in route navigation. setRestrictedToSavedSessions carries
the saved-work request in the route parameter. _recordDecision(..., advance: false)
uses submitScreeningWithoutAdvancing; the advancing action uses submit-and-next.
TryReleaseScreenedReservationAsync is reached by a next-study request, not merely
because a reviewer records and stays.
Current SlotReservation storage has no screening/annotation activity type.
ShouldClaimAsScreening selects a capacity-check path but persists that same structure.
The independent activity reservations proposed below require a schema/lifecycle change;
they are not what the current UI journeys already implement. Random selection refuses a
disabled stage; the disabled-stage/mode-transition differences in direct access and
saves are the concrete compatibility case addressed by approved D4: refuse the newly
unavailable activity while preserving drafts and saved work. This is the intended
contract, not a claim that the current endpoints already enforce it.
Production incomplete-review controls · Redesigned navigator controls · Specific-study navigation and saved restriction · Route trigger · Next and screening effects · Navigation actions · Record/stay and advance host · Status/resume rules · Current selection policy · Reservation service · Direct access · Allocation contract
Which studies Next searches for, and which activities the page shows¶
The table describes current-source new-work routing and displayed activities, including persisted combinations excluded by the intended model below. A successful save still depends on authorization, current state and capacity. Four combinations can exist through editing or the API even though the creation wizard does not offer them.
| Review mode | Selection | New studies searched by Next | Activities shown on the page | Configuration status |
|---|---|---|---|---|
| Screening | Screening | Available for new screening | Screening | Offered at creation |
| Screening | Annotation | Available for new annotation | Screening, no annotation form | Can exist through editing or API; not offered at creation; can offer annotation-selected studies with no annotation surface |
| Screening | Combined | Either group | Screening, no annotation form | Can exist through editing or API; not offered at creation |
| Annotation | Screening | Available for new screening | Annotation, no screening decisions | Can exist through editing or API; not offered at creation; can select work whose missing activity is hidden |
| Annotation | Annotation | Available for new annotation | Annotation | Offered at creation |
| Annotation | Combined | Either group | Annotation, no screening decisions | Can exist through editing or API; not offered at creation; incompatible with enabled workload shares |
| Combined | Screening | Available for new screening | Screening and annotation | Offered at creation |
| Combined | Annotation | Available for new annotation | Screening and annotation | Offered at creation |
| Combined | Combined | Either group | Screening and annotation | Offered at creation |
Stage selectors · Review template
Intended configuration: represent only valid combinations¶
Group the two combined-only settings together in the domain model. A stage has one mode-specific configuration, rather than independently mutable review, selection and screening-policy flags. The following is a conceptual C# shape, not a final persistence schema or an implemented API:
abstract record ReviewConfiguration;
sealed record ScreeningOnly : ReviewConfiguration;
sealed record AnnotationOnly(AnnotationSettings Annotation,
AnnotationAllocation Allocation) : ReviewConfiguration;
sealed record Combined(CombinedSettings Settings,
AnnotationSettings Annotation) : ReviewConfiguration;
// Supporting annotation/allocation groups are described below.
// Construct through a validated factory; only the three supported selection
// values and the two supported additional-screening policies are accepted.
sealed record CombinedSettings(
StudySelectionMode Selection,
AdditionalScreeningPolicy AdditionalScreening);
In implementation, constructors/factories must enforce required settings groups, closed supported variants and supported enum values. The example does not claim that public C# enum parameters alone reject arbitrary integers. Keep construction controlled so every successfully constructed domain value satisfies these invariants.
| Domain configuration | Effective study selection | Additional screening | Administrator controls |
|---|---|---|---|
| ScreeningOnly | Derived Screening | Implicitly strict | Neither combined-only setting |
| AnnotationOnly | Derived Annotation | Inapplicable | Neither combined-only setting |
| Combined | Its validated Screening, Annotation or Combined choice | Its validated Allow or Stop choice | Both settings grouped together |
Single-purpose variants have nowhere to store an independent selection or Allow policy. UI and API use the same applicability: choosing a single-purpose mode derives selection and cannot enable an opposite pool; combined configuration supplies both choices. This is an intended contract, not a claim that current stored mismatches already behave this way. A mode transition constructs a new valid configuration atomically; it must not leave a partially updated pair of flags.
Defaults at the boundary: map an existing combined stage with missing additional- screening policy to Allow; preserve a known explicit combined selection and policy. Creating a new combined stage supplies and persists Stop by default, including in an existing project. Screening-only strictness derives from its variant, not a missing property. Annotation-only has no policy value to default. Selection defaults retain existing combined behavior where applicable; do not invent a new combined selection choice in this proposal.
Keep legacy persistence separate: decode the current raw DTO/storage fields through a versioned migration mapper before constructing valid domain configurations. Matching single-purpose records map directly; combined records map through validated settings and the approved existing missing-policy fallback. The migration table below handles every audited mismatch and raw both-false representation. This is a path into the new model, not a permanent rejection layer for legacy projects. Keep original values in the migration audit, not as independently authoritative flags in the corrected domain. Apply approved D7: keep each stage's existing ReviewMode and derive single-purpose selection from it. Document and review changed Next pools before activation; do not describe those changes as harmless. New API input validation is separate from migration of accepted legacy data. No production migration is executed by this documentation PR.
Checks this structure removes: downstream domain operations need not repeatedly ask whether a ScreeningOnly configuration contains Annotation selection or whether it has an Allow override: valid instances cannot express those combinations. Dispatching by variant determines permitted activities and effective selection.
Checks that remain: validate untrusted DTOs, enum values and legacy imports at the boundary; authorize administrators and reviewer access; verify route/session ownership; apply the study's current agreement/exclusion rules; and atomically enforce reservation capacity against current membership, configuration revisions and study state. A valid configuration does not prove permission or prevent concurrent claims. D4 still requires refusing newly unavailable activity while preserving drafts and saved work.
D7 approves keeping existing ReviewMode during migration into valid configurations. The existing-stage-to-combined row records the approved distinction between unused stages and stages with prior reviews. Neither requires keeping invalid combinations in the new domain type.
Other settings with evidenced applicability¶
The inspected form already disables its annotation-settings group for Screening mode,
but Stage stores those fields independently. Group their active domain meaning by
activity; do not infer applicability solely from where today's form happens to place a
control. These proposals follow the inspected consumers and existing constraints:
| Settings | Proposed owner / invariant | Evidence and migration effect |
|---|---|---|
SessionCountTarget, EnforceAnnotationTarget, MaxInProgress |
AnnotationSettings, present in AnnotationOnly and Combined. The target describes annotation; incomplete-session limits are not screening quotas |
Stage counts incomplete annotation sessions, including a separate reconciliation count; the form disables annotation settings for Screening. Preserve explicit target or inherited-target semantics, optional positive maximum, and enforcement value. Do not silently materialize a changing project fallback, except where existing enabled allocation already owns a fixed target. Screening capacity uses the project's screening rule instead. |
HideExcludedStudiesFromReviewers, ExcludedSessionStatsGrouping |
One annotation excluded-work setting that expresses supported combinations, instead of independently mutable contradictory values | Existing UpdateStage permits visible work grouped together/separately, or hidden work separate/unavailable; it rejects the other combinations. Preserve existing display/resumption meaning and saved records. Map valid pairs directly; report any invalid legacy pair for a specific migration decision without guessing whether to hide work. This does not add an exclusion-based save prohibition. |
Annotation questions, Extraction, category guidance |
Annotation form configuration inside AnnotationSettings; Extraction determines inclusion of system questions, not whether screening is allowed |
AllStageAnnotationQuestions combines configured questions with system questions when Extraction is enabled; guidance is consumed by annotation forms. Preserve question IDs, system-question behavior and inherited/overridden guidance. Historical annotation setup/work in a ScreeningOnly stage remains in migration/audit storage for recovery or a deliberate mode change, not as active annotation capability. |
WorkloadShares |
AnnotationAllocation owned only by AnnotationOnly: ordinary allocation or validated proportional allocation |
The domain explicitly requires Annotation review/selection, inactive configuration changes, active members and a fixed target while enabled. Structural ownership removes the repeated mode-pair check; membership, version, active-state and allocation-total checks remain. Preserve enabled configurations and their owned target; retain disabled configuration history. Do not extend proportional allocation to Combined as part of this work. |
IdleSessionTimeoutMinutes |
Shared review-page reservation timing, outside annotation-only settings | The proposed timer governs both temporary activity claims in strict Combined and screening claims in ScreeningOnly. Its present placement inside the form's annotation group is not the intended applicability. Preserve the stored duration/null-disabled meaning; expose it where tracked reservations apply. Moving it does not authorize enabling tracking. |
| Name, description, active state, stage security and study-scope filters | Common stage data | No evidence here justifies making these combined-only or annotation-only. Preserve them and their existing checks. AllowSelfReconciliation needs its actual workflow consumers assessed before a structural move; the field name alone is insufficient evidence. |
Stage settings and consumers · Annotation controls · Target fallback · Allocation contract
Use a validated value object for the excluded-work combinations and an optional positive value for the incomplete-session limit. UI-only enable/disable toggles need not become independent persisted flags. This eliminates contradictory active configurations without changing the existing exclusion policy. Other numeric defects, including legacy zero targets, retain their documented migration question; the structural change does not silently choose rejection or a replacement target.
Extend the same migration: preserve these raw settings in each before-image, then map applicable groups to AnnotationOnly/Combined and common timing to the stage. Retain non-applicable historical annotation setup for later recovery without allowing it to enable annotation on ScreeningOnly. The existing audit counted mode/default mismatches, not every value in these groups: dry-run their values and constraints before claiming additional automatic mappings are safe. Valid values map unchanged. Unknown/contradictory values need a recorded destination, not deletion or permanent legacy rejection. A mode change that enables annotation restores a known valid prior setup or explicitly obtains the required setup before constructing the destination configuration. Tests must cover these mappings and preservation of saved work; this is documentation/design, not a runtime refactor in this PR.
What a reviewer may do on an open study¶
Examples of #3551’s effect¶
| Situation in a combined stage | Before #3551 on staging | After #3551 on staging |
|---|---|---|
| Screening rule not met; reviewer has no decision | New decision can be accepted if other checks pass. | Same. |
| Screening rule met; reviewer has no decision; study is open | New decision can be accepted if other checks pass. | New decision is refused, including on saved/direct/reserved paths. |
| Reviewer already has a decision | A correction can be accepted if other checks pass. | Exempt from the new refusal; other checks still apply. |
| Reviewer annotates before screening or after screening is complete | Annotation follows its existing rules. | Same; #3551 adds no screening prerequisite. |
| Reviewer uses authorized reconciliation | Separate reconciliation rules apply. | Exempt from the new refusal of a reviewer’s first decision. |
“Screening complete” means the project's rule has been met. It does not necessarily mean inclusion or agreement. In manual dual screening, a disagreement can meet the completion rule. With an agreement-ratio rule, reaching the required count alone may be insufficient.
These examples assume all other applicable checks pass. Capacity, ownership, permissions, stage activity, allocation and local form state can still prevent an action. A study's absence from Next is not enough to deny an action on an already-open study; passing a Next filter is not enough to authorize a save.
A project might require two agreeing screening decisions while a stage permits five allocated reviewers. Before #3551, another reviewer could sometimes submit after the project rule was met because the stage still had capacity. After #3551, the new decision is refused even with room remaining. A correction is exempt from that new refusal, but current cross-stage capacity checks can still reject it.
Before screening controller · Current controller · UI threshold check · Capacity implementation
Candidate membership is not permission to save¶
These staging examples assume positive targets, current cached inclusion data and an authorized reviewer in a combined stage. “Candidate” means passing that new-work filter, not that Next must return this study or that a write must succeed. Saved work, reservations, capacity and allocation remain separate. See the zero-target exception in the evidence appendix.
| Study/reviewer state | Screening candidate group | Annotation candidate group | Combined candidate group |
|---|---|---|---|
| Below completion threshold, no own decision/session, room for annotation | Candidate | Candidate | Candidate |
| Below completion threshold, no own decision, annotation target full | Candidate | Not a candidate | Candidate |
| Screening complete and included, no own decision/session, room for annotation | Not a candidate | Candidate | Candidate |
| Screening complete and included, annotation target full, no own work | Not a candidate | Not a candidate | Not a candidate |
| Screening complete and excluded, no own work | Not a candidate | Not a candidate | Not a candidate |
| Manual-dual disagreement meeting the count, room for annotation | Not a candidate (cached Disagree is sufficient) | Candidate | Candidate |
| Count reached but agreement below/equal required ratio | Candidate | Candidate if annotation conditions hold | Candidate |
| Reviewer already screened; screening complete; no own annotation, annotation conditions hold | Not a candidate | Candidate | Candidate |
| Reviewer owns incomplete annotation, no screening; screening complete | Not a candidate | Not new annotation work; saved work may resume | Not a new-work candidate; saved work may resume |
| Reviewer owns completed annotation, no screening; screening complete | Not a candidate | Not a candidate | Not a candidate |
| Reviewer owns annotation but study still needs screening | Candidate | Not new annotation work | Candidate |
What #3551 changes, and what remains uncertain¶
3551 adds one restriction in this policy area: outside reconciliation, a reviewer without an existing project decision cannot submit after the project's screening-completion rule is met. It applies to advancing and staying on the study, in every stage and selection mode. It is independent of statistics and presence flags.¶
The change removes some previously accepted actions: optional screening on annotation-selected work, a first decision when returning to saved work or opening a direct link, and submission from a tab that was opened before another reviewer completed screening. Each example assumes the other checks would have allowed the action.
It does not change the new-work filters, require screening before annotation, remove existing decisions, or make a correction into another independent vote. Existing capacity and access rules still apply.
The stage's collective goal and the amount of work required from one reviewer are separate. The goal can require both activities across reviewers while allowing one reviewer to screen and another to annotate. #3551 neither establishes a global stage-completion check nor requires each reviewer to complete both activities.
Training or verification may be legitimate reasons to permit more reviewers to decide. We have not established which projects rely on those workflows. Likewise, the reported unwanted decision supports offering Stop, but does not establish that every existing project should be changed to Stop.
Proposed combined-stage setting¶
Only combined screening-and-annotation stages expose this setting. Screening-only stages are implicitly always strict with no Allow override; annotation-only stages have no screening policy. Derive applicability from ReviewMode, not study selection mode.
Once a study meets the project's screening-completion rule, may a reviewer who has not screened it add a decision in this stage?
Allow: accept a new reviewer decision on an opened study when the stage, authorization and capacity checks permit it.
Stop: refuse a new reviewer decision after the rule is met. Corrections to that reviewer's existing decision and authorized reconciliation are assessed separately.
This choice does not change which studies Next selects, make screening a prerequisite for annotation, or require both activities before leaving a study. It does not remove existing work. Other stages can have a different choice; because decisions are shared across the project, Stop in one stage is not a ban across the whole project on decisions accepted through another stage.
Existing combined stages without a saved value resolve to Allow, including those that ran under #3551. Newly created combined stages explicitly default to Stop, even within an existing project. Preserve known explicit combined selections. Only authorized project/stage administrators may change this combined-stage setting; changes are auditable.
Verified capacity coupling and proposed activity reservations¶
Current implementation, not proposed behavior: BuildScreeningCapacityFilter(stageId,
sessionCountTarget) counts submitted screenings attributed to that stage plus stage reservations,
excluding reservations whose reviewer already screened there, and requires their sum to be below
Stage.SessionCountTarget. Both screening reservation and guarded screening-save paths use it.
The administrator UI labels that value Required Annotators per Study and disables annotation
settings for screening-only review. When the stored value is absent, the domain getter falls back to
Project.AgreementThreshold.NumberScreened.
This is a domain coupling defect. An annotation target must never govern screening capacity. For example, minimum two screenings with two submitted decisions that still fail the configured agreement rule needs another reviewer; comparing two completed screenings with target two closes capacity instead. Conversely, a larger annotation target can open more screening places than intended. This newer reservation machinery must not be attributed to production's annotation-form-one renderer; see the separate production/staging comparison. Capacity filter · Screening reservation caller · Target fallback · Administrator label · Disabled annotation controls
Proposed policy when tracking is enabled:
| Stage and effective policy | Temporary reservations | Eligibility and continuation |
|---|---|---|
| Screening only, implicitly always strict | Screening reservation only | Reserve only if this reviewer/study can currently add a screening decision; use project screening capacity, never annotation target |
| Annotation only | Annotation reservation only | Reserve only when this reviewer/study can currently start annotation; existing own saved work retains its place |
| Combined, strict / Stop | Separate screening and annotation reservations | Acquire each only if that activity is currently eligible. One or both may exist; enabling both activities in ReviewMode is not enough |
| Combined, permissive / Allow | Annotation reservation only; no screening reservation | Optional screening on the opened study follows permissive action eligibility. Neither annotation occupancy nor a hidden screening-reservation check may reintroduce Stop |
Presence can remain independent of capacity reservations. With tracking off, do not claim temporary reservations are enforced; UI/API action eligibility still needs the agreed independent policy. A reviewer continuing their own reservation or saved annotation session retains that place even when total capacity is full. This is an idempotent continuation path, not permission to acquire another place or bypass changed access/activity eligibility. Annotation-target enforcement and its tracking flag remain explicit configuration dimensions rather than a new universal requirement.
Effective screening policy derives from ReviewMode first: screening-only is always strict, annotation-only has no screening policy, and only combined stages read the Allow/Stop setting. Existing combined stages missing it resolve to Allow; D2 requires newly created combined stages, including those in existing projects, to visibly default to and explicitly persist Stop. Preserve known explicit combined selections. There is no screening-only Allow override. Selection mode, form renderer and statistics flags do not determine strictness. Combined Allow has no screening reservation.
User-provided deployment context: Chris states that reviewer tracking has never been enabled. This records his deployment context, not an independently verified historical flag inventory. Verify environment/time scope before making a factual deployment claim or planning migration. Source traces of tracking-enabled paths describe conditional behavior, not evidence of legacy use; policy applicability derives from ReviewMode.
Proposed reservation lifecycle and one page-level idle timer¶
The following is a future contract, not a claim about today's single untyped reservation:
| Event | Proposed atomic effect / timer behavior |
|---|---|
| Open eligible study | Reserve only currently eligible activities. In a strict combined stage, independent screening and annotation reservations share one page-level idle timer |
| Before annotation edits | Idle timer runs for the page's temporary reservations |
| Annotation becomes dirty and unsaved | Suspend expiry of both temporary reservations under that shared timer; do not create a missing/ineligible reservation |
| Annotation becomes clean again | Restart the shared timer. Specify authoritative server transitions and reject stale timer deliveries with generation/version checks |
| Idle expiry or leaving | Release remaining temporary reservations. Leaving never deletes saved sessions or submitted screening decisions |
| First annotation save | Atomically replace only the annotation reservation with a saved session. A still-eligible screening reservation remains; subsequent session saves do not acquire another annotation place |
| Submit screening | Atomically persist the project-wide decision, remove its screening reservation and recalculate agreement. Annotation reservation/session is unaffected |
| Correct own decision | Update the existing project-wide decision and recalculate agreement; do not add another vote or treat correction as a new reservation acquisition |
| Clear own decision (proposal) | Atomically remove the decision and recalculate; if the reviewer remains present and is still eligible, attempt to restore a screening reservation consistently with capacity. Clearing is not leaving |
| Other reviewer or committed administrator change affects eligibility | The MVP settings flow below reports a reservation conflict first; an authorized administrator may confirm Apply anyway, which rechecks current impact and commits settings with affected-claim revocation under shared concurrency protection. For genuine eligibility loss under committed state, revoke only a now-ineligible temporary activity claim, persist state and reliably invalidate authorized clients. Disable only that activity and enforce it on the API before event receipt. Full allocation alone retains an own valid place; dirty drafts do not veto revocation |
The proposed clearing/restoration implementation needs a consistency contract: fresh presence/ownership, project and study versions, effective strictness, activity eligibility and capacity must agree at commit. An existing own reservation must not be double-counted. Concurrent clear, save, expiry and departure must not overbook capacity or restore a reservation after the reviewer left. Clearing remains a proposed action, not a verified ordinary endpoint in the inspected controller. If it is implemented, recommend reporting an authorized successful clear separately from a failed capacity restoration; there is no right to an extra slot. This recommendation is not a new user decision required to understand existing completion. No unconditional restoration or capacity bypass is implied. Saved-session removal/reopening has its separate existing lifecycle and ownership checks.
The shared timer is scoped to the page's temporary reservations; multiple tabs, disconnection/grace, dirty-to-clean races and stale scheduled deliveries need a defined server authority. Eligibility updates must not be delayed until that timer expires or until the annotation form becomes clean.
Proposed independent revocation and SignalR enforcement¶
Full capacity is not loss of eligibility. If the target is fully allocated because a reviewer already owns a valid reservation or saved annotation session, that reviewer retains their place. Correct atomic normal claim/submission writers should prevent over-allocation; do not evict an owner merely because a total reaches the target. Actual completion, permission loss, corrections, target or policy changes, and exceptional writes can instead make an activity genuinely ineligible or its temporary reservation unnecessary. The policy must distinguish those facts rather than use “at capacity” for both.
Under the applicable strict policy, revoke a now-ineligible temporary activity reservation independently:
- If screening becomes complete, remove the unnecessary screening reservation and disable new screening. Retain still-needed annotation reservation/session and its usable controls.
- If annotation eligibility is lost, remove the unnecessary annotation reservation and disable that annotation action. Retain screening if it remains needed and permitted. Merely filling allocation capacity, or another reviewer finishing while this reviewer's own place remains valid, is not by itself a reason to revoke it.
- Preserve unsaved input for authorized recovery instead of silently clearing the form. Dirty annotation suspends idle expiry only; it cannot override eligibility revocation. A preserved draft is not permission to save, and revocation must not wait for the form to become clean. Permission loss still applies the approved data-access rules; draft recovery must not disclose newly unauthorized data.
- Never delete a submitted decision or saved session as a side effect of temporary-reservation revocation. Whether a saved session may be edited/reopened after a configuration change remains a separately authorized action. Effective-Allow stages have no screening reservation to revoke; their screening actions follow Allow eligibility rather than inheriting strict completion refusal.
Target-three-to-two example: the initial settings save reports a conflict and leaves state unchanged if the reduction would invalidate current reservations. In the MVP, an authorized administrator may choose Apply anyway to release affected reservations. The server rechecks current impact before committing the reduction and revocations. Assume tracking and annotation-target enforcement apply; distinguish completed work from temporary places:
- One completed annotation plus two temporary reservations: three places were occupied, but annotation is not complete under the new target of two. Only one further completion is needed. One temporary reviewer can retain a place; the other must lose the annotation reservation under the reduced-capacity rule. The unanswered choice is which reviewer keeps it, not whether to keep everybody indefinitely.
- Two completed annotations plus one temporary reservation: the new target is already complete. The remaining temporary annotation reservation is no longer eligible and must be revoked. No choice of survivor is needed.
- A saved incomplete session: this is persisted work, not a disposable reservation. Keep its content and actual incomplete status; assess its continuation under the existing saved-session rules. Never delete it or count it as completed to make a reduced target fit.
D6 decided by Chris: in the MVP explicit-confirmation invalidation flow, retain the earliest still-valid annotation reservations by server acquisition order with a stable tie-break; revoke later excess reservations. In the first example, the earliest valid owner keeps the one remaining place. This is the agreed policy, not an unanswered choice between reviewers. Preserve saved work and unsaved input, and retain any independently eligible activity. The initial save reports the conflict; Apply anyway performs the protected change and invalidation. Materialized summaries remain follow-on work. This documentation PR implements neither the settings flow nor the summaries.
In every actual revocation, update the affected annotation state, refuse stale writes, and notify the reviewer through SignalR so only that activity becomes unavailable. Keep unsaved input for authorized recovery. A still-eligible screening reservation is independent and remains valid. Dirty input does not prevent revocation, and keeping a place does not exempt its owner from a later genuine loss of eligibility. The earlier full-capacity continuation rule applies while an owned place remains valid under the same configuration; it is not a promise to keep every owner after a target reduction.
The server is authoritative, even before a notification arrives:
- Recheck the complete activity eligibility and capacity conditions on atomic reservation claims and submissions, with current study/project/configuration admission. Normal writers must not rely on a prior candidate query or stale browser state to prevent excess claims or decisions.
- Commit the resulting reservation/revocation state and relevant revision before publishing its notification. Where a change affects multiple studies, fence affected actions immediately and use bounded durable cleanup; asynchronous notification/cleanup must not create a period of accepted invalid writes. Persist durable invalidation intent with the state change (or an equivalently proven reliable mechanism) so a process crash after commit cannot permanently suppress the update.
- Deliver authorized activity-specific invalidations across API pods through SignalR fan-out. The event identifies the affected context/activity and revision, not hidden reviewer decisions or private peer data. Recheck subscription/recipient authorization at delivery. This is a proposed eligibility/reservation contract; statistics invalidation alone is not proof of this behavior. Eligibility delivery must not depend on enabling a materialized-statistics consumer flag.
- Clients refetch authoritative eligibility and reservation state, then update signals/computed action state. Disable only the affected activity; preserve unrelated eligible work and unsaved input. Deduplicate revisions, ignore older snapshots and handle out-of-order responses without resurrecting revoked claims. Coalesce bursts while retaining a trailing refresh for changes arriving during a read.
- On reconnect/resubscription, obtain current state rather than assuming all events were replayed. Define bounded missed-event recovery and recover from a failed refetch with an explicit unavailable state. A delayed or lost SignalR event never permits an invalid API submission: return a typed reason, refresh authoritative state and offer appropriate recovery without a blind resubmit loop or draft loss.
Implemented for claim releases (#3720): points 1, 2, 3 and 5 for D8 decision 2 releases and Apply anyway revocations; see Claim-revoked event encoding. Other revocation causes remain proposals.
What is verified today: the annotation-save source converts an existing untyped reservation into a saved session, and existing code has reservation removal/dirty/idle primitives. The statistics notification path has a durable dispatcher, per-API-pod fan-out and delivery-time project authorization; its browser service treats messages as invalidation hints. Those are useful precedents, not evidence that independent activity claims, exceptional annotation revocation, activity-specific UI disabling or cross-pod eligibility recovery already exist or have passed staging acceptance. Automatic annotation revocation and the related exceptional-invalidation/UI behavior remain unverified. This document does not assert they work.
Current session conversion · Current annotation submission · Statistics dispatch · Statistics authorized fan-out · Client statistics invalidation
Proposed complete screening reservation predicate¶
Screening decisions have project/study/reviewer identity, even when submitted through different stages. Count submitted decisions and outstanding screening reservations in that same project/study scope. StageId can record origin/permissions, but must not partition the screening capacity counter. Count a reviewer once; a completed decision and its unremoved reservation must not count twice, and one reviewer must not reserve screening twice through different stages. Annotation reservations and saved annotation sessions do not contribute to screening capacity or agreement.
For a new screening reservation under the strict policy, let submitted be the number of distinct
reviewers with submitted project decisions, minimum be project.AgreementThreshold.NumberScreened,
and outstanding be distinct screening reservations without a submitted decision:
still-needs-screening-from-submitted-decisions
AND current-reviewer/project/stage-eligibility
AND reviewer-has-no-project-decision-and-no-existing-screening-reservation
AND (
(submitted < minimum AND submitted + outstanding < minimum)
OR
(submitted >= minimum AND outstanding == 0)
)
The final branch is safe only inside the outer still-needs-screening AND. A study with enough agreeing decisions must not qualify simply because it has no reservation. Initially reserve up to the minimum; after that count, admit one additional reviewer at a time only while the configured completion rule remains unmet. Reservations never count as agreement. Under manual completion rules, disagreement can already be complete: the authoritative completion predicate, not “disagree” alone, governs that case.
| Minimum 2, reviewer otherwise eligible | New screening reservation |
|---|---|
| 1 submitted, 0 outstanding | Allow |
| 1 submitted, 1 outstanding | Wait |
| 2 submitted, disagree and still incomplete, 0 outstanding | Allow one third reviewer |
| 2 submitted, disagree and still incomplete, 1 outstanding | Wait |
| 2 submitted, sufficient agreement / complete, 0 outstanding | Exclude through the outer completion predicate |
The following illustrative C# MongoDB query is a complete composition, not an implemented helper.
ScreeningReservations is proposed storage containing outstanding screening reservations only; the
actual schema, expiry and migration design remain open. Current storage is SlotReservations and must
not be treated as this typed array. reviewerProjectEligibility represents the approved reviewer/stage
capability and access policy under a project/configuration version admission check; it cannot be omitted
or replaced with an empty filter. The existing NewStudyFilters.InsufficientlyScreenedStudies supplies
the authoritative submitted-decision completion predicate, including agreement and manual semantics.
// PROPOSAL ONLY. Called after authenticated project/stage authorization, under
// the SAME project/configuration version admission as the atomic study claim.
FilterDefinition<Study> NewScreeningReservationFilter(
Project project, Guid studyId, Guid reviewerId,
FilterDefinition<Study> reviewerProjectEligibility)
{
var f = Builders<Study>.Filter;
var projectId = new BsonBinaryData(project.Id, GuidRepresentation.CSharpLegacy);
var reviewer = new BsonBinaryData(reviewerId, GuidRepresentation.CSharpLegacy);
var minimum = project.AgreementThreshold.NumberScreened;
if (minimum <= 0) throw new InvalidOperationException("Invalid screening minimum");
var submittedIds = new BsonDocument("$setUnion", new BsonArray
{
new BsonDocument("$map", new BsonDocument
{
{ "input", new BsonDocument("$filter", new BsonDocument
{
{ "input", new BsonDocument("$ifNull", new BsonArray
{ "$ScreeningInfo.Screenings", new BsonArray() }) },
{ "as", "decision" },
{ "cond", new BsonDocument("$eq", new BsonArray
{ "$$decision.ProjectId", projectId }) }
}) },
{ "as", "decision" }, { "in", "$$decision.ScreenerId" }
}),
new BsonArray()
});
// Proposed project/study-scoped typed array; StageId is not a count filter.
// Entries remain outstanding until atomically released/graduated.
var reservationIds = new BsonDocument("$setUnion", new BsonArray
{
new BsonDocument("$map", new BsonDocument
{
{ "input", new BsonDocument("$ifNull", new BsonArray
{ "$ScreeningReservations", new BsonArray() }) },
{ "as", "reservation" }, { "in", "$$reservation.InvestigatorId" }
}),
new BsonArray()
});
var submitted = new BsonDocument("$size", submittedIds);
var outstanding = new BsonDocument("$size", new BsonDocument("$setDifference",
new BsonArray { reservationIds, submittedIds }));
var capacity = new BsonDocument("$or", new BsonArray
{
new BsonDocument("$and", new BsonArray
{
new BsonDocument("$lt", new BsonArray { submitted, minimum }),
new BsonDocument("$lt", new BsonArray
{
new BsonDocument("$add", new BsonArray { submitted, outstanding }), minimum
})
}),
new BsonDocument("$and", new BsonArray
{
new BsonDocument("$gte", new BsonArray { submitted, minimum }),
new BsonDocument("$eq", new BsonArray { outstanding, 0 })
})
});
var noOwnDecisionOrReservation = new BsonDocument("$not", new BsonArray
{
new BsonDocument("$in", new BsonArray
{
reviewer, new BsonDocument("$setUnion", new BsonArray
{ submittedIds, reservationIds })
})
});
return f.Eq(study => study.Id, studyId)
& f.Eq(study => study.ProjectId, project.Id)
& NewStudyFilters.InsufficientlyScreenedStudies(project.AgreementThreshold)
& reviewerProjectEligibility
& new BsonDocument("$expr", new BsonDocument("$and", new BsonArray
{ noOwnDecisionOrReservation, capacity }));
}
// Candidate selection AND atomic FindOneAndUpdate claim use this full predicate;
// the claim update adds exactly one screening reservation on the matched study.
// Project/configuration admission must remain valid at that commit, not merely
// at the earlier authorization read. Recompute on concurrency retries.
Current completion filter · Existing screening candidate composition · Current atomic capacity composition · Current untyped schema
The schematic method targets one candidate study; a candidate-search variant may omit only the exact study-ID equality, not the outer eligibility/completion conditions. The same complete predicate must guard the atomic claim, so two readers cannot both take the post-minimum extra place. Configuration admission must protect threshold, membership, stage policy and tracking changes; an ordinary prior read is insufficient. Idempotent continuation of an own reservation is a separate non-allocating path that rechecks eligibility. Correction and explicit clearing likewise have separate contracts. Do not apply this strict new-reservation predicate to any effective-Allow screening submissions: that policy has no screening reservations and intentionally permits eligible optional new decisions.
Collective completion and individual contribution¶
Owner clarification, 22 September 2026 (D3a): stage completion means every eligible study in the stage has been sufficiently reviewed according to the configured stage settings. Preserve existing scope and eligibility. Do not introduce stage closure, a new closure gate, or a new population choice. A reviewer's no-work state is not collective stage completion.
The following answers distinguish verified code behavior from inference from that behavior and the owner's clarification. An inference documents the contract to preserve; it does not claim that a new global completion calculation is already implemented.
| Concern | Answer and evidence |
|---|---|
| Existing scope and activities | Verified: ReviewMode enables screening, annotation or both; StudySelectionMode routes Next. The inspected ordinary queries use project identity and the existing screening/annotation predicates; annotation sessions are stage-specific. Workload shares additionally restrict new annotation assignments and can exhaust one reviewer's work while others retain work. Inference: retain these existing boundaries and configured activities; do not offer an all-imported versus new-stage-population choice or treat an individual allocation bucket as the team's completion scope. Stage settings · Pool queries · Reviewer allocation exhaustion |
| Screening sufficiency | Verified: use the project's existing agreement rule, including minimum submitted screenings, strict agreement-ratio boundary and manual-mode behavior. Screening identity is project/study/reviewer, not a new vote in each stage. Inference: the stage uses that configured sufficiency result; Allow permits additional decisions but does not make them necessary for completion. Manual disagreement retains its existing separate inclusion/reconciliation meaning. Agreement rule · Shared identity |
| Completed annotation | Verified in production and staging: the stage tally counts ordinary sessions with Status == Completed in NumberOfCompletedCandidateSessions; saved incomplete sessions count only in NumberOfCandidateSessions. Reconciliation has separate started/completed fields. HasMinCompletedStageSessions compares completed ordinary sessions against its supplied minimum. The stage's “Required Annotators per Study” setting is SessionCountTarget, with the existing project-threshold fallback when unset. Inference: sufficient ordinary annotation is that stage's completed ordinary-session count meeting its configured target, not merely occupancy reaching that target. No new completion measure is needed. Production tally · Current tally · Completed-session predicate · Target label · Fallback |
| Saved, started and reserved work | Verified: saved incomplete sessions, dirty reservations and clean reservations are distinct from completed sessions. Staging's TotalAllocatedSessionCount adds saved sessions and reservations; TotalEngagedSessionCount adds saved sessions and dirty reservations. Neither is a completed count. Inference: a target of two with two incomplete sessions, or one completed session and one reservation, is not sufficient annotation. Full allocation can leave collective work incomplete while another reviewer has nothing available. Tally fields · Session status updates |
| Excluded studies — new work | Verified: the new-annotation predicate always removes sufficiently excluded studies under the project agreement rule, independently of Hide Studies. Screening uses its own insufficiency predicate. Inference: sufficiently excluded studies do not acquire a fresh annotation obligation merely because both activities are enabled. Preserve the existing eligible set; do not redefine exclusion or turn a display setting into a new population policy. New-annotation predicate |
| Excluded studies — saved work and display | Verified: HideExcludedStudiesFromReviewers controls excluded in-progress work in lists/resumption and max-in-progress counts. ExcludedSessionStatsGrouping controls whether existing excluded work is grouped with ordinary progress, shown separately or unavailable. The UI explicitly describes studies excluded after annotation began. Inference: hidden excluded saved work does not keep the ordinary reviewer work queue open; visible excluded incomplete work remains resumable under existing rules. Retain its actual incomplete/completed status and saved content. A grouping choice never manufactures completion, deletes annotations or creates a blanket direct-access prohibition. Settings and labels · Reviewer status · Assignment policy |
| Reconciliation | Verified: ordinary completed sessions and reconciliation-completed state are separate; reconciliation selection requires enough completed ordinary sessions and no completed reconciliation. Inference: do not append a new mandatory reconciliation step to ordinary annotation completion. Preserve reconciliation's own existing workflow, permissions and counters when that activity is being assessed. Reconciliation predicate · Tally |
| Independent reviewers | Verified: screening and annotation have separate reviewer identities and submission paths; combined selection is their union. Inference (D3b answered): Alice may screen and Bob may annotate. No same-reviewer-both requirement, new checkbox or assignment obligation is needed for this scope. |
| Save, skip and corrections | Verified: saving incomplete work preserves it; recording screening and requesting Next are distinct paths; status updates/reopening/removal affect the underlying tally. Inference: skip, leaving, reservation release and save-progress are not completion. Re-evaluate current sufficiency after corrections, reopening or removal; preserve saved work and drafts. Do not invent a closed-stage/reopening state machine or administrative exception. Navigation · Status mutation · Removal |
Known implementation discrepancy, not a policy question: legacy StudyStats uses a
hardcoded minimum of two in some annotation summary counters, in both pinned production
and staging. AnnotationTallyStateProfile deliberately mirrors that fixed threshold for
materialized parity. This differs from a configured stage target of one or three. Raw
started/completed tally distributions and the parameterized completed-session predicate
remain distinct from those fixed summaries. Do not report fixed-two counters as proof of
configured stage completion, change their parity definition silently, or ask Chris to
choose a new annotation measure to accommodate this defect. A later code change that
claims configured completion must address and test this discrepancy explicitly.
Production summary · Current summary ·
Materialized classifier
Reviewer no-work is separately verified: StudyAssignmentPolicy returns
StudyReviewStatus.Complete from the requesting reviewer's available and in-progress
counts. Its comment says “Stage complete,” but the inputs are reviewer-specific and new
annotation availability measures allocation, not completed work. The workload-share
fallback can also report this state when only that reviewer's buckets are exhausted.
It is therefore not evidence that every eligible study is sufficiently reviewed.
This distinction is an answered interpretation of the code, not a new completion gate.
Assignment policy · Status queries ·
Allocation fallback
Recorded decisions¶
Audit rule: owner decisions, verified behavior and documented inferences are recorded as answers. Implementation work and known discrepancies are not turned into product questions. Only choices for which the existing configuration and evidence do not determine an answer remain open. None of these documentation changes authorizes runtime activation.
| Item | Answer or concrete unresolved choice |
|---|---|
| D1 — Existing stages | Latest clarification by Chris, 22 September 2026: Allow/Stop applies only to combined stages. Existing combined stages with no saved setting resolve to Allow; preserve known explicit combined administrator selections. Existing/historic screening-only stages are implicitly always strict, with no configurable Allow override. Annotation-only has no screening policy. #3551 deployment history does not alter this contract. Missing=Allow remains approved within the property's combined-only scope; screening-only strictness derives independently from ReviewMode. |
| D2 — New stages | Clarified by Chris, 22 September 2026: newly created combined stages visibly default to Stop and persist it explicitly, including new stages within existing projects. New screening-only stages are implicitly always strict with no Allow/Stop setting; annotation-only stages have no screening policy. |
| D3a — Collective completion | Answered by Chris's clarification: every eligible study is sufficiently reviewed according to configured settings. The evidence/inferences above supply exclusion treatment and completed-session counting. Preserve existing scope; no closure concept, gate or new population choice. The fixed-two summary discrepancy is implementation work, not an unresolved product definition. |
| D3b — Individual contribution | Answered by existing behavior and compatibility inference: preserve independent contributions. Combined mode/selection does not require the same person to perform both activities. Do not introduce an individual-both option in this scope. |
| D4 — Authorization and compatibility | Answered security requirement: enforce the appropriate reconciliation grant and route/project/study/stage/session/annotation ownership. Historical unauthorized acceptance is not a workflow to preserve. Preserve verified legitimate production actions, subject to the explicit D1/D2 changes and this narrow D4 decision. Approved by Chris, 22 September 2026: for direct/resumed saves after a stage is disabled or its ReviewMode changes, refuse the newly unavailable activity while preserving drafts and saved work. Current endpoints do not uniformly reject it, so this records an intended compatibility change, not existing enforcement. It authorizes neither deletion of work nor runtime activation. Selection mode alone must not become an action prohibition. |
| D5 — Excluded work | Answered by existing settings and compatibility inference: preserve new-annotation exclusion, configured saved-work visibility/resumption and progress grouping. Do not introduce a new direct-access ban. Excluded saved records keep their true status and remain stored. See the evidence table above. |
| D6 — Capacity | Decided by Chris; implementation remains future work. Screening capacity uses the project's screening rule, never the annotation target. With tracking enabled, screening-only uses screening reservations; combined Stop uses separate eligible screening/annotation reservations with a shared timer; combined Allow uses annotation reservations only. MVP: directly check affected current reservations and report a conflict on initial settings save. An authorized administrator may choose Apply anyway. Recheck current impact and atomically apply settings/revoke affected claims with protection against competing reservation/review writes. Earliest still-valid reservations survive by server acquisition order with a stable tie-break; later excess claims revoke. If completed work meets the target, revoke remaining temporary claims for that activity. Force overrides reservation conflict only, never authorization or invalid configuration. Preserve saved decisions, saved annotation, unsaved input and independently eligible activity; use activity-specific SignalR plus authoritative API/reconnect recovery. Materialized summaries remain follow-on implementation; no D6 policy choice remains open. |
| D7 — Mode applicability and migration | Approved by Chris: keep each existing stage's ReviewMode during migration. Screening-only stages become ScreeningOnly with derived Screening selection and strictness; annotation-only stages become AnnotationOnly with derived Annotation selection and no screening policy. Combined retains its applicable valid selection/configuration and approved missing-policy fallback. The four mismatch rows below identify how Next changes; review these concrete workflow impacts rather than assuming the correction is harmless. No per-stage choice between preserving ReviewMode and preserving the incompatible selection remains open. This approves migration policy, not execution of production database writes. |
| D8 — Undecided eligibility combinations found by S1a | Decided by Chris, 24 September 2026; encoded in the pure policy and truth table (#3646). (1) Removing own annotation on a disabled stage or after a mode change is allowed with the stage Review grant and ownership, matching production, until annotation versioning replaces deletion with new versions. The UI offers no affordance there: the work stays hidden and preserved, unreachable because of (3) and the mode change. #3579's flagged ActivityReviewDeletionPolicy must later be relaxed to match (S2). (2) A leftover annotation claim on a study outside the reviewer's allocation grants no exemption: new annotation, the new-annotation pool and direct access are refused with AllocationNotAssigned. If the reviewer is no longer allocated the study and their allocation plus existing work leaves no available slot, the claim is released with a warning; the policy reports that release for S1b/S3 to perform. Under reallocation (regime plan Phase 3) claims count as protected work, and any not accommodated are released with a warning. (3) A disabled stage cannot be opened: no reviewer may open its review page (UI route guard in S1b) and the server refuses direct access with StageDisabled. Saved work is preserved but not viewable until the stage is re-enabled; a read-only view is a possible later enhancement. (4) Hidden excluded saved work mirrors the existing HideExcludedStudiesFromReviewers exactly: when on, sufficiently excluded saved work is absent from resumption (Next and the lists, ExcludedSavedWorkHidden) while direct access and saves still work. (5) Reconciliation readiness is a separate reconciliationPool selection that respects readiness (ReconciliationNotReady). It does not gate the reconciliation action: an administrator who deliberately opens a not-yet-ready study may reconcile, and the policy returns a non-blocking ReconciliationNotReady warning for the UI. Interpretation, 2026-09-24: own completed resave is not early reconciliation. Re-saving the reviewer's own completed reconciliation (which production allows) carries no warning; the warning remains when the study lacks enough completed sessions or another reconciler has completed. |
| An existing stage changes from one activity to both | Decided by Chris: new/unused stages use normal defaults without a change-of-behavior warning. If reviewing has occurred on this stage, changing it to Combined must warn the administrator about actual/potential behavior changes and require informed, explicit choices for study selection and Allow/Stop before saving. Explain consequences and preserve saved work. Prior combined values may inform the display but cannot bypass informed confirmation or silently save changed behavior. Prior reviews on this stage—not its active flag or reviews elsewhere in the project—trigger the warning. Reservation conflict/Apply anyway is separate consent. This answers the transition policy; UI implementation details are engineering work. |
Alternatives¶
| Alternative | Benefit | Cost / reason not recommended as the silent default |
|---|---|---|
| Keep #3551 universal Stop | Small implementation, prevents the reported extra vote | Removes historical optional/direct/reserved workflows without an admin choice; conflicts with D1 for existing combined unset stages and explicit combined Allow choices |
| Revert only the completion-threshold guard | Immediately restores old workflow | Loses the requested prevention option and leaves UI/API policy fragmented; feasible approved interim step, not authorized here |
| Derive action permission from selection | Appears simple | Makes optional activity impossible; mishandles saved/direct paths; contradicts three existing combined modes |
| Add separate stage screening quotas | Supports stage-specific review rounds | A reviewer’s decision is shared across stages; changes identity, statistics and scientific semantics; not this MVP |
| One combined-stage admin choice about new decisions after completion, plus common contract | Preserves combined missing=Allow and permits explicit prevention with matching UI/API; screening-only stays implicitly strict | Requires additive contract, coordinated rollout and explicit decisions about existing gaps |
Implementation and acceptance criteria¶
How the UI and API must agree¶
For the current reviewer and study, the server returns which actions are allowed and a reason for each refusal. The UI presents the permitted actions once local requirements, such as a valid form and no pending request, are satisfied. It explains refusals rather than deriving a different scientific rule from progress counters or selection mode.
Every save checks the same policy again. A response received when the page opened cannot authorize a later save after the study, settings, permissions or capacity have changed. A refused action must preserve the draft and refresh the available actions; it must not silently discard work or automatically resubmit the decision.
Engineering contract¶
Recommend a pure Core ReviewEligibilityPolicy with explicit facts, plus an
orchestrator that obtains authorized current facts. Reuse the existing
StudyReviewAccessResult/presence machinery for reservation state; do not build a
second lease model. Extend the established
review access contract,
whose implementation history is separate from
this policy decision. Return an eligibility section with the study response and
on refresh/refusal, containing:
- study/source revision, project/stage configuration revision and policy version;
- separate decisions for
submitNewScreening,correctOwnScreening, proposedclearOwnScreening,startAnnotation,saveOwnAnnotation,completeOwnAnnotation,reopenOwnAnnotation,removeOwnAnnotation, and authorized reconciliation actions; allowed, stable reason codes and bounded explanatory details for each action;- selection eligibility/reason separately (
newScreeningPool,newAnnotationPool,resumeSaved,directAccess), never a singlecanReviewbit; - activity-specific reservation ownership and shared-timer state, and capacity/ownership facts needed for an explanation, without exposing other reviewers' hidden decisions or personally identifying presence data.
These are existing lifecycle actions, not new user capabilities: production form one
exposes session removal, while form two reopens completed work by saving its status
as Incomplete and also exposes removal. The current delete path can restore a
reservation for a connected reviewer. Reopening and removal can therefore change
completed counts, capacity and subsequent eligibility. The proposal must describe
and test them alongside start/save/complete.
The provisional reopenOwnAnnotation result makes reopening explicit. Implementation
may instead model it as a named transition under saveOwnAnnotation if the same
permissions and tests remain explicit. removeOwnAnnotation covers removal of a
saved session; do not inherit new-session capacity or active-stage restrictions
without an approved rule. Specify owned versus other sessions, completed versus
incomplete work, route/stage/session identity, changed membership/access, and the
effect of deletion on reservations and refreshed eligibility. These are proposed
contract names, not approved permissions. Production removal ·
Production delete endpoint · Form-two reopening ·
Form-two removal · Current deletion/reservation path
Candidate reasons: ScreeningDisabledInStage, AnnotationDisabledInStage,
ScreeningAlreadySufficient, ExistingDecisionUseCorrection, StageDisabled,
AnnotationAtCapacity, ScreeningAtCapacity, ReservationExpired,
ExcludedFromNewAnnotation, AllocationNotAssigned, ConfigurationChanged,
EligibilityUnavailable. Authorization failures retain appropriate 403/404 behavior;
state/policy changes use typed conflict responses. Do not tell a screening-only reviewer
“annotation remains available” as the current generic 409 does.
Mongo pool predicates and the pure policy may need different implementations, but must share a specified truth table and executable fixtures. Prefer a common policy specification over ad hoc TS/C# duplication. Angular consumes the response through signals/computed values. Local draft validity, busy state and loss-of-connection states can temporarily disable an otherwise eligible action; those must have their own visible reasons and clear when resolved. The converse requirement does not mean rendering an annotation action in a screening-only view or bypassing validation.
On every submission outside reconciliation and every optimistic retry, reload/revalidate relevant study and configuration facts. Bind acceptance to the same source/configuration versions at commit, using the existing transaction/fencing conventions where available; an application-level reread alone leaves a race. Preserve tracking mode admission, idempotency and authorization. A client token is evidence of what it saw, not authority to bypass current eligibility. Concurrent new screenings under Stop must not both pass from the same old snapshot. Under Allow, do not accidentally reintroduce the new completion-threshold veto in a capacity or assignment helper.
On relevant screening/session/configuration changes, reconnect or a typed 409, refresh eligibility and current study facts through authorized reads. Coalesce bursts and cancel on route/user changes; preserve dirty forms and keep a trailing refresh when an event arrives during a request. Reevaluate a transient refusal when capacity/configuration changes so the UI does not permanently hide an allowed action. Do not use the current progress counters or asynchronously refreshed statistics as write authority. Future materialized counters could serve that role only under the transactional and concurrency conditions in the MVP/follow-on boundary below. Existing stats invalidations can trigger refresh but do not by themselves supply all policy facts or prove delivery. Never automatically resubmit a refused user decision.
S1a encoding and recorded gaps¶
The pure policy (#3646) encodes the contract above. Expected outcomes live in one place, the truth table; later Mongo pool predicates are tested against its selection columns. Encodings that the contract leaves to implementation:
- Reason codes reuse
ReviewWriteRefusalnames where they overlap, with the activity implied by the action:ActivityUnavailableis the candidateScreeningDisabledInStage/AnnotationDisabledInStage,ScreeningCompleteisScreeningAlreadySufficient, andAtCapacityisScreeningAtCapacity/AnnotationAtCapacity.NotFoundmeans the reviewer has no own decision/session for an own-work action;DuplicateSessionmeans continue the existing session. New codes:ExistingDecisionUseCorrection,ExcludedFromNewAnnotation,AllocationNotAssigned, and (D8)ExcludedSavedWorkHiddenandReconciliationNotReady. - Warnings are a separate, non-blocking channel on an allowed decision
(
ReconciliationNotReady), and the result carries a separate leftover-claim release with its warning (AnnotationClaimReleased). Warnings never refuse an action. - Leftover-claim release (D8 decision 2, corrected in S3) needs four facts: an own annotation
claim,
AllocationAssignment.NotAssigned, no own ordinary session, andAllocationLeavesSlotForOwnClaim = false. When a slot remains, the claim is kept and the reviewer may continue the study they hold (lead, 25 September 2026, option 1):startAnnotationanddirectAccessare allowed. The new-annotation pool still refuses the study withAllocationNotAssigned, so a kept claim never makes it new work. The truth table has one row for each case. - Capacity facts state whether a new place can be acquired, already folding in tracking and target enforcement. Only strict screening consults screening capacity.
- Selection pools are candidate-group membership and do not read selection mode, which routes Next; a sufficient study never re-enters the new-screening pool, even under Allow.
- Reopen is the form-two save of a completed own session as Incomplete: save permissions, no capacity check.
- Direct access is allowed when any ordinary action on the study is allowed; otherwise it carries the new-annotation refusal where the stage annotates, else the new-screening one.
The five combinations S1a found undecided were decided by Chris on 24 September 2026 (D8 under
Recorded decisions) and are encoded in the policy and truth table.
Not modelled yet: orchestrator-level codes (ReservationExpired, ConfigurationChanged,
EligibilityUnavailable), which depend on revision and freshness facts gathered in S1b.
S1b encoding¶
The server pools (#3659) define each study-level fact exactly as the facts builder reads it from the stored document, so a pool query and a policy evaluation of the same study agree. Encodings chosen here:
- Screening sufficiency for selection reads the study's cached screening status (
InclusionInfofor the project threshold), as Next always has. Selection is refused while inclusion info is being recalculated. - Screening capacity (accepted by the lead, 24 September 2026) mirrors typed admission: outstanding screening claims on any stage, held by reviewers who
have not submitted, must leave fewer than
max(1, required − submitted)places, and the reviewer must not hold one of them. Only strict screening consults it, and with tracking off it never refuses. With the flag on this narrows the legacy screening query, which left capacity to the claim, so Next never offers a study that admission would refuse. - Annotation capacity (
AnnotationPlaceAvailable, actions and direct access) mirrors typed admission: tracking off or an unenforced target never refuses; otherwise allocated sessions (saved candidates plus annotation claims) must be below the target, and a study with no tally has none allocated. A reviewer's own annotation claim keeps the study in the pool at full capacity. - Annotation target for selection (
UnderAnnotationTarget, decided by Chris via the lead on 24 September 2026) is a separate fact read only by the new-annotation pool. It keeps the legacy rule exactly: fewer allocated sessions than the target with tracking on, fewer saved candidate sessions with tracking off, whether or not the target is enforced, and a study with no stage tally passes even at a zero target (the historical exception). A study that meets it is refused from the pool with the new selection-only reasonAnnotationTargetMet(notAtCapacity, which is an enforced limit); opening it can still start annotation when the target is unenforced. A study ready to reconcile has at least the target of completed sessions, so it is never new-annotation work. - Reconciliation readiness (
EnoughCompletedSessionsForReconciliation, accepted by the lead, 24 September 2026) is the existing selection rule's non-reconciliation half: a tally with at least the target of completed ordinary sessions, not sufficiently excluded, and, without self-reconciliation, no ordinary session by this reviewer. Readiness also requires no completed reconciliation by anyone. Next's new reconciliation work is the pool minus studies the reviewer has already started; resuming a reconciliation keeps the legacy in-progress rule behind the reconciliation gate. - Allocation uses the #3603 evaluator's buckets for the reviewer as part of the new-annotation pool itself, so status counts already reflect the reviewer's share.
- Selection mode routes Next from the validated configuration variant rather than the raw stored selection.
The truth table gained rows for an unenforced target that is already met, for the zero-target exception (with the target unenforced and enforced), and, for historical data only, the reviewer's own completed reconciliation next to another reviewer's. The aggregate now allows one reconciliation session per study and stage, so the Mongo test stores that last combination directly. Rows whose study is full or ready to reconcile now state that the target is met; this changes only the new-annotation pool column of the ready-to-reconcile rows.
S2-C encoding¶
Administrative screening writers follow the reviewer save rules (owner decision, option 1), behind
reviewEligibilityPolicy; flag off leaves both paths unchanged.
- Bulk study update. A screening row is refused and reported, not applied, when the job's stage is disabled or annotation-only, a named reviewer is not an active project member (the ordinary screening save's membership rule), or a named reviewer already has a decision on the study. Decisions have project-wide identity, so a decision recorded under any stage is never overwritten. A refused row is refused whole: its identifiers are not applied either. Accepted rows are written in project-fenced transactions of at most 50 rows: the project-token write plus study version filters, with every rule re-evaluated on the transaction's own project and studies, and the first accepted row records the stage's positive review history in the same transaction. The job result reports the refused-row count and a sample of at most 100 row numbers with reason codes only; the result summary's unmatched count excludes refused rows.
- Search import with screening columns. A disabled or annotation-only stage refuses the import before
any Study is written (
ScreeningStageUnavailable). The reveal transaction, already a versioned project write, rechecks the stage (a refusal fails and cleans up the import) and records the stage's positive history when the search carries decisions attributed to it. No further fence is needed: the reveal is what makes the decisions visible, and its project write conflicts with a concurrent settings save. - Settings-save evidence. The project-screening family materializes project scope only, so
stage-attributed screening evidence is always authoritative. Stage-annotation statistics are read
through the bundle reader and shared decoder; only positive evidence (saved candidate sessions) is taken
from them, because their pinned snapshot is not the settings transaction's and a fresh zero cannot prove
the stage unused. Everything else, including reservation- or reconciliation-only tallies and stale,
rebuilding, fenced or unavailable reads, uses the authoritative stage-attributed calculation, which is
always available inside the settings transaction, so
Unavailableis not produced. GET exposes the evidence, its source and the historical marker additively.
S3 encoding¶
S3 (#3719) runs the reservation lifecycle on typed claims behind reviewEligibilityPolicy. Flag off, and pages
not yet migrated to typed claims, keep today's behaviour.
- One shared page timer. The idle, suspension and liveness consumers and hub disconnect handling work on the page: an idle mark or suspension covers every claim on it, and expiry or leaving removes the page with all its remaining claims. Releasing one activity's claim (screening submitted, annotation graduated or released under D8) changes neither the other claim nor the timer fields, so an already scheduled timer still applies.
- Restore paths. Reconnecting after suspension (
Resume) and interaction after an idle mark (ClearIdle) keep every claim, and the stale expiry is then a no-op. Deleting a saved session restores the annotation claim onto an existing screening page without touching its timer when the reviewer is connected. When they are not connected, the session is deleted and the screening page stays as it was. A legacy (unmigrated) page refuses the restore before anything is deleted. There is no "clear own decision" restore. - Slot for a leftover claim (
AllocationClaimSlot, derived from #3603's plan and the stored tallies). Count two groups of other reviewers on the stage: - reviewers with existing work: an ordinary saved session or an annotation claim;
- reviewers the plan allocates to the study's bucket who have no work there yet (outstanding demand).
A slot is left when the union of the two groups is smaller than the stage's annotation target. The plan gives
every bucket exactly "target" reviewers, so a reviewer outside the allocation has no slot while the plan can be
evaluated. A slot remains only when the regime fails closed (for example, an allocated member left the project
or the target no longer matches the regime). There is then no countable demand, and the claim is kept.
Allocated reviewers count as demand exactly as the evaluator's plan lists them. Because the evaluator refuses a
plan that names an inactive member, an inactive allocation shows up here as that fail-closed case, never as
demand quietly dropped. The reviewer-validity requirement (#3611, blocked on #3251's resolver) is where inactive
allocations will be repaired; once it lands, a repaired plan counts its reviewers again and the rule is unchanged.
- Kept claim continues. While a slot remains, the allocation access decision used by typed admission, the
flagged annotation save, direct access and the settings-save revocation plan (keepSlottedClaim) allows the
reviewer to continue the claimed study. It names no admitting regime, because the claim keeps its own
provenance. With the flag off the legacy allocation rule is unchanged.
- Release in fenced admission. Next's existing-reservation check, the Next candidate claim, direct access and
the hub join all run typed admission. Inside the project-revision fence, admission releases the own annotation
claim when the allocation denies it, there is no own session and no slot is left. The release commits even
though the study cannot be opened, and the result is reported as not admitted. The Next response
(claimRelease) and the join/direct-access result (StudyReviewAccessResult.claimRelease) carry
AnnotationClaimReleased with the released study's id only. The browser shows a snackbar and refetches the
stage's reviewer statistics and in-progress work. The join and direct-access denial carries the
ClaimReleased reason (#3720), and the browser shows the release warning instead of the capacity dialog.
Other reviewers see the release through the existing presence snapshot.
- Settings editor. After a ReservationConflict the editor refetches stage reviewer state and keeps the
draft and its opened revision. After Apply anyway it also re-reads the saved settings.
- Protected work. Publishing an allocation regime is already refused while any session or reservation exists
on the stage, typed pages included (StageReviewActivityFilter). Reallocation (regime plan Phase 3) does not
exist yet.
Claim-revoked event encoding¶
3720 implements points 1, 2, 3 and 5 of [Proposed independent revocation and SignalR¶
enforcement](#proposed-independent-revocation-and-signalr-enforcement) for claim releases, behind
reviewEligibilityPolicy (lead, 25 September 2026). The D6 decision approved activity-specific SignalR plus
authoritative API and reconnect recovery; the section's detailed contract remains a proposal, and these points
are its MVP acceptance criteria.
- Durable intent (point 2). Each release writes an intent into
pmActivityClaimRevocationOutboxin the same transaction: - typed admission's D8 decision 2 release;
- the guarded settings save's Apply anyway revocation.
Only these flag-on writers add intents, so no writer-floor item is needed. An intent names the claim owner, their
own study and stage, the released activities, the reason (AllocationNotAssigned or SettingsChanged) and the
study revision the release committed at. It never carries another reviewer's data. An aborted transaction writes
none.
- Commit before publish, at least once (point 2). A leased dispatcher runs in every API instance. It follows the
statistics notification outbox and the identity recovery-email outbox:
- it publishes only committed intents, and marks one delivered only after the bus accepts it, so a crash between
commit and delivery leaves the intent due;
- an expired lease is taken over by another instance;
- a failed send backs off and is abandoned after 10 attempts;
- delivered and abandoned intents expire after 7 days.
- Cross-pod fan-out with delivery-time authorization (point 3). There is no SignalR backplane; Valkey stores
BFF sessions only. Each intent is published to a temporary endpoint per API instance, as for statistics
invalidations, and each instance sends ActivityClaimRevoked to the owner's own local connections. The recheck at
delivery fails closed, so any of the following means no send:
- the flag is off;
- the project, stage, investigator or study is unknown;
- the owner is no longer an active member, or their account is deactivated;
- the study read is older than the release;
- the owner holds the released claim again;
- an activity is unrecognised.
- Client and reconnect recovery (points 4 and 5). The event and the response warning share one browser action:
- the warning names the reason and activity;
- a release reported by both within 30 seconds is warned about once;
- the stage's reviewer statistics and in-progress work are refetched;
- a reviewer on that study re-runs the join, keeping unsaved input.
There is no event replay. On reconnect, the existing rejoin and Next re-run admission and return the
authoritative state, and every save rechecks and refuses with a typed reason. A lost event therefore never
permits an invalid write.
- Denial reason. A join or direct-access denial that released the claim now carries
AccessDenialReason.ClaimReleased instead of the deliberate AtCapacity S3 kept.
Proposed implementation names; not approved schema¶
The provisional property is AdditionalScreeningPolicy, applicable only to combined
stages: whether a reviewer may add a new screening decision after the project’s
screening-completion threshold. Screening-only strictness is implicit and does not
read an Allow override; annotation-only has no screening policy.
| Value | Meaning | Compatibility/default |
|---|---|---|
AllowWhenOpened |
Permit a new screening decision on an opened study after the project’s screening-completion threshold is met, subject to approved capability/auth/activity-specific capacity; does not put complete studies back into the new-screening candidate group | D1 fallback for existing combined unset stages; administrator option for combined stages only; full production compatibility still requires the checks below |
StopAfterProjectSufficiency |
A new screening decision requires the project’s screening-completion threshold not yet to be met; corrections to that reviewer’s existing decision and reconciliation remain separate actions | D2 default for newly created combined stages, persisted explicitly, including new stages in existing projects. Preserve explicit combined selections. Screening-only has implicit strictness rather than this configurable setting; annotation-only has no screening policy |
No screening-before-annotation prerequisite, individual require-both obligation, universal exclusion prohibition or ungated annotation-target enforcement is introduced by this proposal. Reuse existing settings where they express the approved rule; keep additions independent and audited.
Minimum usable release¶
The first usable release provides one combined-stage administrator setting and matching UI/API behavior for the agreed screening and annotation actions, including saved and direct-open work. It applies D1/D2 by ReviewMode, preserves explicit combined selections, and enforces implicit screening-only strictness and combined Stop, including concurrent saves. Correct screening capacity and the complete atomic predicate are intrinsic correctness requirements. If tracking-enabled combined behavior is in that release, independent reservations, one authoritative page timer and compatible storage are required; a single untyped slot is not a valid substitute. Additive inactive schema/contract work can ship first, but do not activate only half of the claimed reservation lifecycle.
Settings changes in the MVP: directly read current reservations affected by a proposed setting change and evaluate them against the proposed configuration and current review facts. Include activity/mode, strictness, targets and timing changes where they affect a claim. An unrelated harmless edit must not be blocked merely because somebody is present. The administrator flow is:
- Save settings. If there is no reservation conflict, commit under the protected check below. Otherwise leave settings and reservations unchanged and explain the impact, for example: “Lowering the annotation target to two would release a reserved annotation place. Saved reviews will be kept.”
- Cancel or Apply anyway. Only an authorized administrator may explicitly confirm Apply anyway for the proposed settings. This overrides the reservation conflict only; it never bypasses authorization, valid domain configuration, existing allocation constraints or any other non-reservation requirement.
- Recheck and apply. The server reloads current impact rather than trusting the earlier preview or client-supplied claim IDs. It validates the proposed settings, retains earliest still-valid reservations where capacity remains, and commits the settings and affected-claim revocations within the same concurrency boundary. An intervening settings edit requires conflict handling, not overwriting a newer configuration. If current claims changed, compute their current valid survivors; the confirmation applies to the proposed settings and its affected reservations, not a stale frozen claim list.
- Refresh affected reviewers. Persist reliable invalidation intent with the state change, then deliver authorized activity-specific SignalR invalidations. Clients refetch eligibility and disable only affected actions while preserving unsaved input. The API refuses an ineligible stale submission even before notification arrives; reconnect/refetch repairs missed or out-of-order events.
The check, settings commit and revocations share concurrency protection with reservation creation, renewal, release, expiry, conversion to saved work and review submissions that change the relevant eligibility facts. A preflight read followed by an ordinary settings update is unsafe: a new claim could appear between them. Use a proven shared serialization boundary, for example a transaction that validates affected claims and writes the configuration/revocations while every competing writer also writes/checks the same stage/project admission revision. Snapshot reads alone do not prevent new matching claims; a shared write conflict/fence is required. Include project scope when a project-level screening rule affects claims across stages. On conflict, reread and re-evaluate before either ordinary or confirmed commit. New claims must use the admitted configuration, not their earlier cached settings. No newly invalid claim may escape revocation because it appeared after the first preview.
Cancel, an unconfirmed conflict, failed authorization or invalid configuration leaves settings and reservations unchanged. Ordinary reviewer-driven eligibility loss still uses independent revocation. D4 governs unavailable actions after a committed mode or disablement change. Settings changes and reservation invalidation must never delete saved screening decisions or saved annotation work. Preserve unsaved input for permitted recovery; retained input does not authorize an ineligible submission.
Acceptance and shortest critical path: build the direct impact check, initial conflict response, authorized Apply anyway action and shared protected commit; connect it to the existing eligibility invalidation contract. Test both unconfirmed and confirmed settings changes racing claims and submissions. A competing write either commits first and is included in the re-evaluation, or commits later against the new settings. Two conflicting writes must not both succeed from stale state. Confirm partial-capacity survivor order, completed-target revocation, force authorization/validation boundaries, saved/unsaved-work preservation and unaffected activity continuity. No materialized reservation dashboard or aggregation is required for this result.
Follow-on: materialized reservation/presence statistics across stage and project, activity breakdowns and richer administrator visibility or wait/progress tooling. The explicit Apply anyway confirmation is already MVP scope and does not depend on those summaries.
Future materialized counters can be authoritative if every relevant reservation lifecycle and review write updates them transactionally, and settings mutations share concurrency protection with those writes. Aggregate totals alone may still not identify which claims a particular activity, policy or timing change affects. Specify and prove that coverage before replacing the MVP direct check; asynchronously delivered statistics or SignalR messages do not meet it.
Characterization fixtures and an additive, inactive response contract can be reviewed in earlier PRs. Activation requires the complete agreed action coverage and compatibility/security decisions. Do not describe a screening-only predicate as full workflow parity.
Project policy templates, bulk migration interfaces and allocation redesign are later enhancements. This scope adds neither a stage-closure mechanism nor a same-reviewer-both requirement. Configured collective completion has the answered meaning above; a reviewer having no available work or all places being allocated does not establish it.
Test plan¶
The following tests are proposed work. This planning PR does not claim to have run them.
Staging setup: test both Allow and Stop on suitable staging stages with screening and annotation enabled. An existing combined stage with no saved setting uses Allow; the strict test explicitly configures Stop. Also verify a newly created combined stage persists Stop by default, including when created in an existing project. These are tests of decided policy, not additional policy choices. Select suitable stages and coordinate any shared-stage configuration changes when setting up the tests; this document selects no particular staging stage and authorizes no configuration change now.
Each fixture must identify the baseline, both stage modes, relevant flags/settings, reviewer history, study state and entry path. It must state separately whether the study is a new-work candidate, whether opening/resuming succeeds, which action the UI offers and what the API accepts. Include the expected refusal reason and whether existing work remains intact.
| Layer | Required acceptance cases |
|---|---|
| Configuration | Characterize current mismatches separately; domain variants cannot express incompatible single-purpose selections or screening-only Allow; validate malformed/null combined settings and unsupported enum values at boundaries; UI groups both choices only under Combined; API and derived selection agree; schema0/current absent/explicit mapping; workload-share lock; legacy mismatches migrate to recorded valid destinations with original values audited; idempotent reruns, interrupted batches and source-revision conflicts; no unapproved workflow changes |
| Shared policy | Characterize all 9 current mode pairs separately from the 5 valid intended mode/selection combinations; below completion threshold/included/excluded/manual disagreement/ratio boundary; no/existing reviewer decision; no/incomplete/completed annotation; Allow/Stop; screening-only always strict with no Allow override; existing combined missing policy Allow; new combined stages explicitly persist Stop in new and existing projects; explicit combined values preserved; annotation-only has no screening policy; settings UI exposes Allow/Stop only for combined stages |
| Completion semantics | Different reviewers may collectively satisfy the two activities; allocated or reserved annotation sessions alone do not prove completion; configured targets ½/3 use completed ordinary sessions, not fixed-two summaries; exclusion settings and correction/reopening effects preserve existing scope; reviewer no-work and full allocations never certify collective completion; no closure gate |
| Pool integration | Screening candidates, annotation candidates and their union against actual Mongo fixtures; annotation candidates include unscreened work but excludes sufficient exclusion; candidate versus reservation counts; no-tally/zero-target boundary with expected production, untracked staging and tracked enforced-claim outcomes; saved fallback, max-in-progress, allocation buckets |
| Screening writes | Advance and stay; tracking and statistics on/off; own correction including another stage; reconciliation separately authorized; existing reservation after threshold crossed; project count2/stage target5; disagreement at target; approved D4: reject newly unavailable screening after a ReviewMode change or stage disablement, including direct/resumed paths, and preserve drafts and saved work |
| Annotation writes | No screening prerequisite; no duplicate annotation session outside reconciliation; existing session continuation at target; new versus resumed work; target enforcement on/off with tracking on/off; excluded direct/saved paths; completed-session reopening and owned-session removal, reservation restoration and eligibility refresh; route/stage/owner/session identity; membership revocation; approved D4: reject newly unavailable annotation after a ReviewMode change or stage disablement while preserving drafts and saved sessions, including direct/resumed paths; a selection-mode-only change does not prohibit an otherwise permitted activity; reconciliation authorization |
| Reservation policy | Screening-only implicitly strict with screening reservation only when tracking is enabled; annotation-only, combined strict with zero/one/two independently eligible reservations, combined permissive with annotation reservation only; tracking off; own reservation/saved session retained at full capacity; no duplicate project-wide screening reservation through another stage |
| Screening capacity | Minimum2: submitted1/reserved0 allow; 1/1 wait; 2 disagree and incomplete/0 allow; 2 disagree and incomplete/1 wait; 2 complete/0 refuse. Match candidate and atomic claim truth tables; annotation target changes never affect screening; reservations never alter agreement; manual-complete disagreement stays excluded |
| Timer and lifecycle | Clean page timer, dirty unsaved annotation suspends both temporary claims, clean restarts, first annotation save replaces only annotation claim, screening save atomically removes screening claim/recalculates, expiry/leave release only temporary claims; other-reviewer eligibility loss disables/refuses activity despite dirty drafts; stale timer/multiple-tab races |
| Clearing and correction | Existing decision correction versus proposed clear; atomic remove/recalculate/conditional restore; restore loses capacity; reviewer left; cross-stage shared identity; no double count; saved work survives leave; no unconditional restoration |
| Mode-change warning | New/unused stages use normal defaults without behavior warning; prior stage reviews trigger an admin warning for active/inactive stages; require informed explicit selection and Allow/Stop choices; prior values cannot bypass confirmation; stage-review statistics/fallback and historical-used evidence, stale/missing/rebuilding not zero, first-review/settings-save race protected; save one valid configuration; preserve reviews; distinguish stage-specific history from project-wide screening identity; saved reviews without reservations still warn; reservation-release consent remains separate |
| Settings changes | Direct affected-reservation query; harmless edits allowed; initial conflict and Cancel do not mutate; authorized Apply anyway rechecks current impact and atomically commits settings/revocations; force cannot bypass authorization/domain/allocation constraints; earliest valid survivors at partial capacity; claims/submissions racing preview and confirmed commit; project-wide admission when needed; saved decisions/sessions never deleted |
| Concurrency | Two final votes race under Stop; same race under Allow; config/threshold/membership change during save and retry; slot expiry/capacity increase; source deletion; version refusal causes reread rather than success |
| UI and API parity | Server action true renders usable action once local preconditions pass; false renders reason and cannot emit; direct API bypass has same result; every entry/layout/form version; no 409 retry loop leaving buttons permanently wrong |
| Revocation and delivery | Full target preserves own valid reservation/session; actual completion/eligibility loss revokes only the affected temporary activity. Test correction/target/policy changes, delayed cleanup fences, crash after commit before notification, current recipient authorization, duplicate/out-of-order revisions and stale refetch responses |
| Cross-pod recovery | Two viewers connected to different API pods; simultaneous claim/submit; one viewer dirty, idle or disconnected; completion of either activity preserves the other. Reconnect/refetch and deliberately dropped events converge without reviving revoked claims or losing drafts; API rejects a stale action before its notification arrives |
| Real browser | Initial settings conflict offers Cancel/Apply anyway to authorized admins; confirmed change updates affected reviewers through SignalR/refetch without losing drafts; Combined × all three selections, Allow/Stop, two reviewers; study meeting completion threshold still annotatable; optional screening preserved under Allow; Stop hides/refuses it; correction and saved drafts retained; change/reconnect/409 refresh updates eligibility without reload |
| Rollout | Missing policy on old projects; additive DTO compatibility; mixed versions refuse unsupported activation; audited opt-in and rollback; no statistics flag changes action semantics |
In the main acceptance example, a combined stage offers a sufficiently screened study for annotation. With Allow, a reviewer without a screening decision can submit one if all other checks pass. With Stop, the UI does not offer that new decision and the direct API refuses it. Both choices preserve permitted annotation work. If another reviewer or administrator changes eligibility after the page opens, saving rechecks the current state and reports the result without losing the draft.
Migration and activation¶
Map legacy records into the corrected domain¶
Migration design is authorized; execution against production is not. The source values below are the effective values from the pinned production getters, not guessed meanings of raw booleans. Preserve project/stage identity, active state, configured thresholds, annotation targets, allocation settings, decisions and sessions in every mapping. ScreeningOnly always gains the intended strict contract; this is distinct from claiming all historical save paths enforced it.
| Legacy representation / observed count | Old effective behavior | Approved valid destination and impact | Migration treatment |
|---|---|---|---|
| Matching Screening / Screening | Next searches screening; screening UI | ScreeningOnly; same pool/activity, implicit strict screening contract | Automatic shape mapping after rollout authorization; no extra admin policy choice |
| Matching Annotation / Annotation | Next searches annotation; annotation UI | AnnotationOnly; same pool/activity, no screening policy | Automatic shape mapping; preserve allocation and targets |
| Combined / any supported selection | Both activities; Next uses the saved/effective selection | Combined with that selection; existing absent policy maps Allow, known explicit policy is preserved | Automatic mapping using D1. Do not apply the new-stage Stop default to migrated existing stages |
| Screening / Annotation — 4 stages (1 active, 3 inactive) | Annotation pool, screening UI | ScreeningOnly: Next now searches screening, while the screening activity remains unchanged | Approved D7 mapping; document the change from annotation-selected to screening-selected work for review |
| Screening / Combined — 9 stages (5 active, 4 inactive) | Union pool, screening UI | ScreeningOnly: Next narrows from the union to screening; screening activity is unchanged | Approved D7 mapping; annotation-only candidates leave this stage's Next pool, which is a workflow change to review |
| Annotation / Screening — 10 stages (7 active, 3 inactive) | Screening pool, annotation UI | AnnotationOnly: Next now searches annotation, while the annotation activity remains unchanged | Approved D7 mapping; document the changed candidate pool and preserve saved annotation |
| Annotation / Combined — 13 stages (13 active) | Union pool, annotation UI | AnnotationOnly: Next narrows from the union to annotation; annotation activity is unchanged | Approved D7 mapping; screening-only candidates leave Next; preserve saved annotation and applicable workload-share constraints |
| Both raw activity flags false — 52 stages in 44 projects | Schema-0 getter produces Screening; selection defaults Screening; absent/null active defaults active | ScreeningOnly with that effective active state; preserve records even if they contain historical work | Automatic mapping from verified getter semantics; do not invent a disabled/no-activity mode |
| Missing/null selection or active values — 581 stages (371 missing, 210 null) | Production getters supply the effective selection/activity | Map the derived values into the applicable variant and preserve effective active state | Automatic mapping where decoded values are valid; this defaulting category can overlap other rows and is not an extra stage population |
Approved D7 migration rule: keep each of the 36 stages' current ReviewMode and replace its incompatible independent selection with the selection derived from that mode. No migration to Combined is needed to preserve an old mismatched pool. Existing combined stages keep their applicable valid configuration, including explicit policy values and the existing missing=Allow fallback.
The approved mapping changes Next behavior for these 36 stages. Include the before/after pool, active state and surviving saved-work context in the private migration plan for review; the table above gives the public aggregate impacts. Inactivity or absence of surviving records does not prove the change is harmless. This impact review implements the approved policy; it is not another request to choose the mapping for every stage.
Move all mapped records into valid domain configurations through the versioned migration. Unknown values or concurrent edits still require investigation/replanning. Retain the old compatible reader only for the controlled version transition, not as a permanent home for mismatches. Executing production migration and activating the new behavior remain separately gated.
Mode changes after migration¶
Decided transition behavior: new or unused stages use the normal defaults without a change-of-behavior warning. Newly created combined stages explicitly default Stop, including within existing projects; the approved missing=Allow fallback for stages already combined remains unchanged.
When reviewing has occurred on a stage and an administrator changes it to Combined, warn that administrator before saving. Show which activity becomes available, the actual or potential change to Next's selection, and what Allow versus Stop means for additional screening. Require informed, explicit choices for both study selection and Allow/Stop. Prior saved combined values may inform what is displayed, but cannot bypass confirmation or silently persist changed behavior. Save the administrator's confirmed settings as one valid configuration. Preserve all saved reviews.
Trigger this on prior reviews on this stage, regardless of whether it is active or inactive. Saved stage-attributed screening decisions and annotation sessions are relevant evidence; an unsaved draft or temporary reservation is not a saved review. Screening decisions have project-wide identity and their stage attribution can change on correction, as the audit notes. Do not broaden the warning to every stage merely because the project has reviews, or assume no surviving records proves a stage has never been reviewed. Use reliable stage-specific review history where available and preserve that history through migration. Resolving the source of that history is an implementation task, not a new warning-policy choice.
Use review statistics for this check. Chris specifies stage review tallies/materialized statistics as the source; this dependency is part of the warning work now. It is separate from the new materialized reservation/presence summaries deferred above. The pinned statistics catalogue and read contract provide these starting points:
| Evidence needed | Existing family / contract | Consumer requirement and limits |
|---|---|---|
| Screening reviews on the stage | project-screening accepts ProjectStage scope and exposes ScreeningTallyCounts; membership/reviewer screening families have different scope |
Request the actual stage-scoped tally and verify its source grain counts reviews attributable to this stage. A project-wide positive total or merely a project-stage identifier is not proof of stage usage. Do not substitute sufficiency or full-allocation counters for review existence. Integration must prove the stage-attribution mapping before using it for this warning. |
| Saved annotation on the stage | stage-annotation has ProjectStage scope and candidate-session count distributions; membership-stage-annotation has MembershipStage session/reconciliation counts |
Read positive saved-session evidence, incomplete or completed, across excluded and unexcluded work; include saved reconciliation where relevant. Do not use total study count or sessionedCount alone: a reservation-only tally can exist, and ordinary zero-session cells can also arise from reconciliation-only work. Use the supported saved-session/reconciliation facts or their authoritative calculation to distinguish these cases. |
| Whether the answer is usable | ProjectStatisticsBundleReadResult reports read source, fallback reason, availability and revision watermarks; the reader checks lifecycle/version/fences |
Use a valid fresh response or complete its required authoritative fallback. Missing, stale, rebuilding, incompatible, fenced, unavailable or capacity-failed data is not zero. If the authorized fallback cannot establish the answer, keep the transition unresolved and report/retry through the existing statistics contract rather than silently saving as unused. |
Statistics catalogue · Screening tally projection · Annotation count caveats · Read result · Freshness/fallback reader
Current zero is not “never used.” Deletion, correction and moved screening attribution
can remove current counts even after reviewing occurred. Retain stage-specific historical
review evidence, for example a monotonic HasEverBeenReviewed indicator set with the
first saved review and never cleared by removal or attribution changes. This is a proposed
implementation mechanism, not a claim that the current metric catalogue already provides
it. Seed it during migration from positive stage tallies and trustworthy stage-specific
history; a zero current tally cannot safely seed false without historical coverage.
Record unknown legacy history as unknown, not unused. If complete stage-specific
history cannot be recovered, show the same behavior-change warning and require the
same informed choices as for a reviewed stage; do not block that transition indefinitely
or falsely rewrite unknown history as proven reviewed/unused. The pinned current families do not
by themselves prove complete ever-used coverage; implementing that coverage is an explicit
technical dependency, not another product-policy question or a change to the separate
statistics task's authorized scope.
Recheck when saving: the browser's warning check is not sufficient. Revalidate the review-statistics/history evidence within the protected settings-save admission boundary. A first review committed after the form opened must either cause the save to return the reviewed-stage warning and require informed choices, or serialize after the new settings and follow them. Share the relevant source/configuration revision protection with review writers; freshness at the earlier read alone does not close this race. Recover unavailable current statistics through the existing contract before saving; for irrecoverable historical uncertainty, retain the unknown evidence state and use the reviewed-stage warning/explicit-choice path. Protect that path with the same save-time recheck. Never let Apply anyway turn missing review evidence into “unused.”
Separately, D6 checks affected temporary reservations and offers authorized Apply anyway where appropriate. Saved reviews without reservations still require the reviewed-stage warning; reservations without saved reviews still require their own conflict handling. Acknowledging the behavior warning is not consent to release reservations, and force confirmation does not replace the informed settings choices. Neither may delete saved work. The warning is for the administrator making the change, not for reviewers.
Changing Combined to a single-purpose mode constructs ScreeningOnly or AnnotationOnly atomically, deriving selection and screening strictness/applicability. Keep previous combined choices in history for a later return, not as active flags capable of overriding the single-purpose variant. On a later return to Combined after prior review, show that history but still require the informed choices above. Preserve work and refuse newly unavailable activity under D4. A combined-to-combined settings edit preserves the other explicitly selected setting and remains subject to existing allocation constraints.
Versioned execution, audit and activation¶
Implementation and rollout tasks, not another product decision: deploy compatible backend and frontend versions before enabling the new behavior. Every backend allowed to write must enforce the new settings and reservation rules; supported clients must understand refusals and refresh affected activity. Keep the behavior disabled until those pieces are ready. Plan rollback so it preserves saved work and explicit settings, including Stop. Engineering must name the rollout switch, version requirements and rollback steps; this document does not request an abstract additional policy approval. Actual migration execution and deployment still require their existing authorization.
- Dry run against current records. Decode schema-0 and any encountered versions with the production getter semantics; classify matching/defaulted mappings, approved D7 mismatch mappings with changed-pool impacts, and already-migrated records. Report aggregate counts and private per-stage before/after plans. Reconcile against the observed 2,292 stages, 36 mismatches and 52 both-false cases as a dated baseline, not a frozen expected total. Include active/inactive counts, completed/saved work and allocation constraints; never add overlapping categories as separate populations.
- Store a versioned plan. Use a migration identifier, source schema/configuration revision, chosen destination and an audit before-image. Preserve known explicit combined values; map absent policy on existing combined to Allow. For each D7 mismatch, record the unchanged ReviewMode, derived selection and resulting Next-pool change under Chris's approved migration policy. Unknown future schema/enum values are diagnostics requiring investigation, not guessed mappings.
- Apply idempotently after execution approval. Compare-and-swap the source revision into the new version/configuration and write the audit atomically, with one migration marker per stage/version. An already-applied destination is a no-op. A concurrent edit fails the revision match and must be reread/replanned, not overwritten. Resume partial batches from markers; preserve identities and all review work. Never use the new-stage creation default for migrated existing combined stages.
- Verify before activation. Reconcile planned/applied/skipped/conflicted counts; verify every migrated stage constructs a valid variant, effective selection matches its approved destination, and decisions/sessions/targets/active states remain intact. Test all four mismatch mappings, both-false/default getters, explicit combined policies, annotation-setting groups/common timing, preserved dormant setup, reruns, interruption and concurrent configuration edits. Run staging reviewer journeys for any changed pool or activity before broader activation.
- Coordinate readers and writers. Deploy readers that understand both versions and writers that cannot recreate independent incompatible settings. Gate activation by configuration version and a supported writer/client floor; old writers must be fenced from migrated records before cutover. Do not make mixed-version clients show Stop as enforced before every admitted writer enforces it. Keep not-yet-migrated records on the compatible old path only during the controlled migration; unknown values and revision conflicts require investigation, not a new decision about the approved D7 mappings.
- Writer floor for bulk-update refusal reports (#3695). Do not enable
reviewEligibilityPolicywhile any API or PM replica older than #3695 is running. Older replicas cannot deserializeStudyUpdateParseResultdocuments that carry refusals (their implicit class map does not ignore the added elements), so a Project with such a job would fail to load there. Flag-off results are unchanged and remain readable by every version. - Preserve rollback information and policy. Retain private before-images and destination revisions. Rollback must be version-aware and must not overwrite post-migration configuration edits or review work. Stop rollout and reconcile such conflicts. A compatible fallback writer must preserve explicit combined Stop and single-purpose derivation/strictness; restoring an older binary that silently bypasses them is not a valid rollback. Reinstating a historical incompatible workflow requires an explicit decision, not automatic copying of old flags.
- Coordinate reservation schema separately. Chris reports tracking has never been enabled; verify its environment/time scope and actual outstanding records before execution. If none exist, migrate configuration without fabricating reservation work. If untyped claims do exist, define a drain/transition before admitting typed writers; never guess activity ownership or double-count saved decisions/sessions. Combined Stop requires independent eligible claims and the shared timer; combined Allow has annotation claims only. Staging verification and production activation remain separately authorized operations.
| Deployment state | Behavior that must be specified before activation |
|---|---|
| Contract/policy gate off | Which approved existing behavior remains, including production versus current staging differences. |
| New server contract available; settings UI not yet active | Whether the new response is read-only/additive and how unsupported client versions behave. |
| Administrator can choose Stop | Which server writers and supported UI versions enforce it; what prevents premature activation. |
| Rollback after explicit Stop | Minimum supported writer behavior or other coordinated rollback rule that preserves the chosen restriction. |
Compatibility acceptance: apply D1/D2 within the combined-only setting scope and screening-only implicit strictness. Preserve other approved production workflows for equivalent state, subject to D7's intended selection derivation and approved D4: refuse newly unavailable activity after stage disablement or a ReviewMode change while preserving drafts and saved work. Record and obtain approval for any further intentional compatibility change. Preserve earlier/later staging fixtures without calling them production evidence.
The named release gate and its off behavior remain implementation work governed by the compatibility requirements above. This documentation-only PR adds no feature flag or runtime behavior. The future feature requires a dedicated rollout decision independent of statistics/presence flags. Before activation, specify the writer floor and rollback handling for an explicit Stop; silently falling back to Allow is not an acceptable rollback.
Evidence and limitations¶
Deployment and flag observations¶
User-provided context: Chris reports that production has only used annotation form one, and that the incident prompting #3551 occurred in annotation form two on staging. The screenshot is evidence of that reported staging workflow, not a production reproduction. The read-only checks below independently establish the currently deployed versions; they do not prove every past flag value or user action.
| Baseline | Source/deployment evidence | What it establishes |
|---|---|---|
| P — actual production, form one | Web 7.39.0, public informational version 7.39.0-6c9b72e, source 6c9b72e2f16ea45f91b0776c2cbf743dc7bc3a81; API image 9.44.1-restore.104eca9, source 104eca9b05dceb8a3504eec6ee319f3a6a428283 |
Production web and API have different revisions. Neither is #3551's parent. Read-only Kubernetes observation on 2026-09-22 at 06:14 UTC: 3 web replicas and 1 API replica ready. The image/source association for the API comes from its deployed tag and GitOps pin, not an independently attested image build. |
| S0 — staging before #3551, form two | Source comparison snapshot f0e257459e3ee98d75fa09ff9ee09fe48c18c7d0 (merge parent including #3547); Chris's form-two staging report |
This is the pre-change source baseline. The exact serving pod revisions and complete effective flag snapshot at the instant of the report were not captured here; do not promote the merge parent into proof of that request's deployment. |
| S1 — staging after #3551 | Source 9ddb8ec659bd63fa3b7b9fb65a05bad205471790; Web 7.147.4-sha.9ddb8ec, API 9.117.0-sha.9ddb8ec |
Independently rechecked on 2026-09-22 at 06:14 UTC: 2 web replicas and 1 API replica ready; public web informational version matches. |
Production version pins are in GitOps web configuration and
GitOps API configuration, pinned to the inspected GitOps commit.
Live observations used read-only kubectl get deployments -A -o json,
production public web configuration,
staging public web configuration
and staging runtime flags.
Those endpoints are mutable; the dated values here are the observation record.
No credentials, private study data or whole configuration payloads are included.
Production's public config has annotationSettingsConfigurable=false and
screeningSettingsConfigurable=false; it has no annotationFormV2 key, and its pinned
review template mounts form one directly. Production's runtime-flag endpoint returned
HTTP 404. This is not evidence that every modern flag exists there with value false. The
production stage settings below were traced in its own source.
Staging's effective runtime snapshot revision 69, refreshed
2026-09-22T06:16:41.2723643Z, has annotationFormV2=true,
stageReviewRedesign=true, stageReviewDockview=true,
annotationSettingsConfigurable=true, screeningSettingsConfigurable=false, and
proportionalStudyAllocation=true. Static web config instead has redesign and
Dockview false: static config alone does not identify the effective UI. API and web
deployment environment both show ActiveReviewerTrackingEnabled=false; that key
was absent from the browser runtime snapshot. These are current observations,
not a reconstructed flag history for S0. No authenticated per-user override or
project/stage settings inventory was performed. Form-two eligibility can still fall back
to form one for unsupported contexts; the form flag and layout flags are separate.
Renderer selection · Form-two admission
Detailed production/staging comparison¶
| Concern | P: production form one and its API | S0: pre-#3551 staging form two and its API | S1: change made by #3551 |
|---|---|---|---|
| New-study selection | Screening candidates, annotation candidates, or either group; annotation selection excludes sufficiently excluded studies and reviewers with an existing annotation session, without requiring prior screening | Same basic candidate-group/union semantics, with newer optional reservations/allocation/access machinery | No selection-filter change |
| Activities shown | ReviewMode controls screening and annotation surfaces. Form one can be used without first screening. Production settings explicitly describe annotation-selected screening as optional | Same mode distinction; form two replaces the annotation renderer, not the candidate groups or screening permission | No new annotation prerequisite |
| New screening decision after completion threshold | On a permitted opened study, Include/Exclude remain available when local data/dirty/recalculation conditions permit; no completion-threshold veto in the production screening submission path | Also permitted before #3551, subject to newer access/capacity conditions | New UI veto and API 409 for a new decision outside reconciliation; applies independently of which annotation renderer is mounted |
| Correction to existing decision | Shared reviewer/project/study decision is updated, including stage attribution; no second stage vote | Same shared identity; newer capacity path can distinguish existing same-stage decisions | Correction exempt from the new veto |
| Decision/navigation interaction | Include/Exclude submit and request next study; dirty annotation disables those controls until saved/discarded | Legacy layout retains that pattern. Redesigned layout can record and stay with dirty work or a stay preference, using a separate non-advancing write path | Adds threshold veto to both advancing/staying paths; does not introduce record-and-stay |
| Annotation save/complete | Form-one validation emits session/annotation/outcome payload to its host; session API has no prior-screening prerequisite or general completion-threshold refusal | Form two has its own validation/persistence and save/complete events. Newer API has capacity/access/allocation guards absent from P; renderer choice alone does not create these guards | No annotation-save change |
| Stage creation | Single-purpose mode computes matching selection; combined mode offers three choices | Same basic creation semantics | Unchanged |
| Stage editing | Single-purpose mode disables selection but retains its explicit value; save uses raw form value. Combined enables all three choices | Same behavior, plus newer enabled-workload-share restrictions | Unchanged |
| Stage API/storage | Both values passed through; no general matching-pair normalization. Explicit stored selection wins; absent value defaults from review mode | Same basic storage contract, plus newer allocation constraints | Unchanged |
Production evidence: review template, screening controls, review host, form-one submission, screening/session endpoints, domain submission, screening identity, pool filters, union query, creation, settings edit/save, settings labels, stage API, stage defaults. Staging evidence: pre-change review host, pre-change form two, pre-change navigation policy, pre-change API, and the section-specific references below. These traces establish code behavior, not a claim that every production workflow was exercised in a browser.
Attribution: optional screening of annotation-selected studies already exists in P's UI, labels and submission path; form two did not originate that permission. Form two changes annotation rendering/validation/persistence. Record-and-stay is a separate redesigned-host behavior available before #3551, not a consequence of the form-two flag alone. Reservations, capacity enforcement and proportional allocation are intervening backend features, not changes made by #3551 or necessarily active in the reported session. No complete historical inventory of their activation was performed. #3551's exact diff attributes the new completion-threshold veto and live Stage Overview work specifically to S0 → S1.
Compatibility boundary: P is the production baseline. Restoring S0 is not a sufficient compatibility test: its access/capacity rules and navigation already differ from P. The proposed Allow value preserves P's demonstrated optional-screening permission, but does not by itself establish full form-one/form-two parity. Before production rollout, characterize P's selection, settings, actions and saves against the proposed implementation, apply the recorded D1/D2/D4/D7 decisions and explicitly approve any further intentional compatibility change. Keep S0/S1 fixtures separately for staging regression and the reported prevention option.
The detailed implementation evidence describes S0/S1 unless a production reference is explicit. Do not apply their newer capacity/allocation rules retroactively to production. The separately authorized read-only production inventory is recorded above; no authenticated production write/reproduction was performed. UI labels establish communicated meaning, not proof that every API constraint matches it.
Current checks and UI/API differences¶
This table describes inspected staging paths. The explicitly marked D4 contract is an approved future change; it is not a claim of current enforcement.
| Dimension | What the page or Next does today | What the staging save path checks, except where marked |
|---|---|---|
| Review capability | ReviewMode hides screening or annotation surfaces | Screening-decision and annotation-session endpoints outside reconciliation use StageReview authorization; inspected save paths do not independently reject the opposite review mode or check selection mode |
| Route/study membership | Route/UI visibility is not the security boundary | Stage access, project/study relationship and active membership checks do not independently prove session/annotation ownership |
| Session/annotation identity | Existing work must belong to the authorized context | Route/body checks are conditional in the inspected save path; existing-session mutation matches by ID. D4 requires independent route, session, stage and annotation identity/ownership tests |
| Reconciliation authorization | Separate workflow/grant | A reconciliation-marked submitted session must require the proper grant; payload consistency alone is insufficient |
| Screening-completion threshold | New in #3551: hides new screening after screening completes when no matching existing decision by this reviewer, except reconciliation | New in #3551: same veto on a new screening decision outside reconciliation before each attempt; does not alter annotation save |
| Existing screening | Current reviewer's decision can be changed | One screening per reviewer/project/study; a correction updates it and its stage attribution, rather than adding an independent stage vote |
| Screening capacity (current defect, not proposed policy) | Presence may lock controls; eligibility UI does not calculate the complete write filter | With verified tracking on, save allows a persisted reservation, an existing screening in that stage, or stage screening/reservation count below SessionCountTarget. With tracking off, no such capacity predicate. #3551 applies before both branches. This incorrectly couples screening to the annotation target and partitions project-wide screening by stage; see the proposed replacement above |
| Annotation capacity | Presence/access can lock the form; Screening selection can still land on a study that has no annotation room | Guard enforced for saves outside reconciliation only when tracking is available and stage EnforceAnnotationTarget is true. Own persisted reservation/session can permit continuation at target. Flag-off behavior is different; don't present the setting as unconditional |
| Prior annotation | No new annotation candidate for own incomplete/completed session; saved/direct paths remain | Saving the existing session is supported; a different annotation session ID outside reconciliation for the same reviewer/stage is rejected. Completion does not add a screening prerequisite |
| Exclusion | New annotation selection always excludes sufficiently excluded studies; hide-excluded affects saved lists/resumption. Annotation form mounts by review mode/access, not an independent exclusion policy | Session submission has no general sufficient-exclusion refusal. A direct/open excluded study can therefore differ from its Next-pool eligibility |
| Disabled stage / changed ReviewMode | Current Next refuses a disabled stage; guarded direct access can deny. UI activities follow ReviewMode | Current: no uniform stage.Active check in the inspected submission paths outside reconciliation; StageReview authorization is not such a check. Approved D4 contract: direct/resumed submissions must refuse the newly unavailable activity after disablement or a ReviewMode change, preserving drafts and saved work. Selection mode alone is not an action prohibition; implementation remains future work |
| Reconciliation | Separate context; #3551 exempts its screening controls | Screening reconciliation uses StageReconcile authorization and bypasses the completion-threshold and capacity checks used outside reconciliation. Session endpoint uses StageReview and a reconciliation payload flag; its local ReconciliationCheck checks annotation flags, not a separate Reconcile grant. This needs an authorization audit, not a compatibility assumption |
| Data pending / dirty | Inclusion recalculation/presence can hide actions. Legacy layout blocks screening while annotation is dirty; redesigned layout records and stays, preserving draft | Valid persisted submission can still be accepted; form validity/in-flight navigation are transport/UI preconditions, not a second scientific eligibility policy |
| Freshness | Angular computed eligibility reacts to supplied store data; stats refresh does not itself prove current study agreement data refreshed | #3551 retries reload the Study, but retains the request's Project object. A concurrent threshold/stage-setting change is not automatically revalidated simply because study CAS is retried |
Required release gate for the eligibility contract: the reconciliation authorization gap below must be fixed and tested before the new eligibility contract ships. It is not deferred audit work.
Authorization boundary to resolve before implementation: the annotation save path checks stage-review access outside reconciliation but does not independently require the reconciliation grant when the submitted session is marked as reconciliation. Payload consistency is not authorization. D4 must specify and test reconciliation, route/session identity and ownership enforcement. Historical acceptance is not evidence that these requests should remain permitted.
Session endpoint · Payload consistency check · Submission service · Existing-session identity branch
Zero-target candidate and capacity exception¶
The Mongo predicate is “no matching stage tally has count at least the target”, not an unconditional mathematical count-below-target test. Production uses candidate sessions; staging can include reservations. A missing tally therefore passes even at target zero. The stage update DTO/domain do not establish general positivity; the UI’s minimum of one does not prove persisted/API values are positive. No stage inventory established whether deployed projects contain such values.
The following are expected characterization results, not tests run by this PR. Assume an authorized reviewer with no prior annotation session, an eligible study, no tally for the stage, and saved target zero:
| Baseline/path | Candidate-query result | Assignment/claim consequence |
|---|---|---|
| Production | Passes the missing-tally predicate | No modern tracked claim; can return the candidate if other selection conditions pass |
| Staging with tracking off | Passes the predicate | Can return the candidate without a tracked claim |
| Staging with tracking and annotation capacity enforcement | Passes the predicate | Atomic annotation claim rejects the missing tally at target zero; candidate membership does not guarantee assignment |
A future correction must explicitly choose configuration rejection or a changed selection rule and migration behavior; neither is authorized here. Production predicate · Staging predicate · Atomic claim · Untracked return path · Update DTO · PATCH mapping · Stage assignment
Threshold and storage details¶
For both mode enums, Screening=1, Annotation=2 and Combined=3. These are separate properties in current storage, not one permission bitmask. In the inspected legacy getters, explicit stored selection takes precedence for schema zero and later; absent selection defaults from review mode. The intended domain variants above replace that independent selection authority for single-purpose stages; activation remains separate. Enums · Legacy getter
3551 compares the project threshold against the study’s AgreementMeasure, allows¶
a missing threshold through and exempts the reviewer’s existing decision. The client duplicates that computation. The strict ratio comparison matches the server: threshold and study measure have different runtime types, so ValueObject equality does not make equal numeric fields complete. Test count, strict-ratio and manual disagreement boundaries separately. Agreement rule · Diff
Assessment of the existing Claude comments¶
| Existing recommendation | Assessment and treatment in this revision | Disposition |
|---|---|---|
| Reconciliation authorization / proposed D8 | Substantive concern verified. D4 now makes the security requirement prominent and mandatory; a duplicate D8 is unnecessary. The comment's query-string shorthand is incomplete: the submitted session's Reconciliation field drives the relevant branches; query=true alone is not the demonstrated mechanism. |
Addressed in the document: prominent D4 boundary and identity tests; no runtime fix claimed. Authorization is answered as mandatory; Chris has also approved the narrow disabled-stage/mode-transition contract: refuse the newly unavailable activity and preserve drafts and saved work. Runtime enforcement remains future work. |
| Misleading combined-selection label | Chris clarified collective completion intent; the revised explanation and selection copy preserve it. OR selection does not make that intent factually wrong. D3a now defines completion from existing eligibility and configured sufficiency; no closure or individual-both concept is introduced. | Original diagnosis superseded by owner clarification. The revised text records configured collective completion and distinguishes it from reviewer no-work and allocation counts. No application copy or enforcement changed. |
| Missing=Allow for all S1 projects | Chris's latest clarification makes Allow/Stop combined-only: existing combined unset stages use Allow, newly created combined stages explicitly default Stop even in existing projects, and known explicit combined selections are preserved. Screening-only is always implicitly strict; annotation-only has no screening policy. | D1/D2/D6 reconciled throughout. Missing=Allow remains approved for the combined-only property. With tracking, screening-only reserves screening; combined Stop reserves each eligible activity separately with a shared timer; combined Allow reserves annotation only. Staging tests cover both Allow and Stop as described in the test plan; no runtime activation is inferred. |
| Feature flag and off behavior | The migration section requires a dedicated rollout decision, mixed-version gate and preserved off/rollback behavior. The activation table now requires mapping server enforcement, response contract, settings UI and rollback to explicit states; the switch name, compatible versions and rollback steps remain implementation tasks. | Recorded as rollout engineering, not an open product decision. Specify compatible backend/frontend versions, disabled-state behavior and rollback before enabling the feature; this documentation does not authorize deployment. D1/D2 settle defaults, D3a clarifies completion, D4 settles the narrow compatibility case and D7 defines structural configuration applicability; none authorizes activation. |
Statistics refresh is separate from action eligibility¶
This planning PR adds no SignalR change or new browser proof.
3551 was merged and staging deployment verified on 2026-09-21 at 19:37 UTC:¶
API 9.117.0-sha.9ddb8ec, Web 7.147.4-sha.9ddb8ec (both web replicas ready),
Project Management 11.113.0-sha.9ddb8ec; public web config matched the merge SHA.
Neither reservation-to-session conversion nor the statistics transport demonstrates automatic annotation
revocation or exceptional-invalidation UI behavior; the proposed independent revocation contract above
requires its own implementation and acceptance evidence.
That is the prior deployment observation; the web/API revisions were independently
rechecked on 2026-09-22 as recorded above. No new SignalR browser proof was run.
Stage Overview subscribes to statistics invalidations and has missed-event recovery;
local browser acceptance proved authorized HTTP/store/UI recovery, with notification
logic separately unit-tested. Authenticated staging two-tab/cross-pod transport was
not proven by that local run. Progress refresh and eligibility refresh are different
contracts. This planning PR neither changes nor redeploys SignalR.
The release's identity timing flake passed a targeted retry; its PDF integration timeout remains tracked in #3539. Neither that timeout nor successful statistics parity settles eligibility intent.
Source references¶
All code links below are pinned; search symbols are named in the relevant sections.