Quality management

Manual home · Role guides · Coverage · Gaps

Manual conventions and recovery apply to every workflow. Evidence level: code inspected; related tests were inspected, not executed.

flowchart TD
  A[Eligible interaction and published scorecard] --> B[Claim or manager allocation]
  B --> C[Draft with revision guarded saves]
  C --> D{Complete and valid?}
  D -->|No| C
  D -->|Yes| E[Immutable submission and official score]
  E --> F[Manager correction]
  E --> G[Independent calibration]
  F --> H[Independent re-review]
  G --> I{Disagreements?}
  I -->|Yes| J[Immutable adjudication]
  I -->|No| K[Agreement recorded]

The main diagram covers audit-based calibration. Manual group calibration is a separate creator/reference/distribution process in QMS-10–12; publication controls when participants see comparisons.

QMS-01 — Queue, claim, assign and reassign manual reviews

Status: Implemented. Verification: code inspected; no runtime completion claim.

Purpose: queue, claim, assign and reassign manual reviews.

Actors and access: audits:review to claim/list own work; audits:manage for explicit reviewer/unassigned work and managed scope.

Prerequisites: Existing interaction and a published scorecard selected directly or by resolution.

Entry point: Interactions → audit action; Review queue; Administration.

1. Claim a review through the interaction action, or create manager-assigned/unassigned work.
2. When creating unassigned pool work, an enabled review-assignment policy may deterministically select an eligible reviewer for the configured percentage; check the returned reviewer rather than assuming the item remains unassigned.
3. Managers filter managed work and assign/reassign an active eligible reviewer; reviewers open their own queue.
4. Open the pinned interaction/scorecard context and start the workbench.

Result: An idempotently claimed audit is owned by its reviewer or awaits allocation.

Exceptions and recovery: A submitted audit cannot be reassigned. Interaction ownership is distinct from audit reviewer ownership. Foreign/non-owned reviews may return not found; use authorised managed views rather than guessing URLs.

Source evidence: services/api/src/audits/routes.ts; services/api/src/app.ts; apps/web/src/features/administration/assignment.tsx; apps/web/src/features/audits/audit-queue.tsx.

QMS-02 — Draft a review with evidence and knowledge selection

Status: Implemented. Verification: code inspected; no runtime completion claim.

Purpose: draft a review with evidence and knowledge selection.

Actors and access: audits:review and ownership of the unsubmitted audit.

Prerequisites: Available pinned scorecard and interaction artifacts.

Entry point: Review workbench.

1. Read the full interaction and scorecard criteria. For interaction-relevant knowledge sections, select valid suggested articles or explicitly confirm no relevant article applies.
2. Choose outcomes, add comments and attach available artifact/time-offset or pinned knowledge-section evidence.
3. Wait for autosave success; use the current draft revision for further saves.

Result: A revision-guarded draft stores reviewer answers and evidence.

Exceptions and recovery: On a conflict, preserve unsaved notes privately, reload the server version and reconcile deliberately. Invalid outcome IDs, foreign/missing artifacts and wrong knowledge revisions are rejected. An unassigned audit must be claimed before editing.

Source evidence: services/api/src/audits/routes.ts; services/api/src/audits/knowledge-selection.ts; apps/web/src/features/audits/audit-workbench.tsx.

QMS-03 — Review AI suggestions and generate a findings summary

Status: Implemented. Verification: code inspected; no runtime completion claim.

Purpose: review AI suggestions and generate a findings summary.

Actors and access: audits:review and audit ownership; audits:manage for the AI review queue/retry.

Prerequisites: Configured AI processing and redacted evidence.

Entry point: Review workbench → AI suggestions.

1. Inspect each proposed outcome and its grounded evidence; use, confirm or dismiss supported suggestions.
2. Use bulk adoption only after review; inspect what changed and any preserved answers.
3. Request the findings summary if useful and monitor its status; retry failed queued evaluation from the manager queue when appropriate.

Result: Reviewer decisions are recorded separately from AI proposals; a summary supports review.

Exceptions and recovery: Absence checks supply no quote and require a manual answer. AI failures or unsupported evidence do not justify inventing an outcome. Automated assessment is advisory; official submission is QMS-04.

Source evidence: services/api/src/ai-suggestions/routes.ts; services/api/src/ai-suggestions/queue.ts; apps/web/src/features/audits/audit-workbench.tsx.

QMS-04 — Submit a review and inspect its official result

Status: Implemented. Verification: code inspected; no runtime completion claim.

Purpose: submit a review and inspect its official result.

Actors and access: audits:review and ownership.

Prerequisites: Complete required answers, valid evidence and confirmed knowledge selection where required.

Entry point: Review workbench → Submit.

1. Check all required outcomes and add supported Molecule/overall feedback with its intended audience.
2. Ensure the latest draft is saved, then explicitly submit.
3. Inspect the server-calculated score, critical failures, breakdown and official submitted state; follow generated alerts or notifications.

Result: An immutable submitted audit records the authoritative score and original version context.

Exceptions and recovery: Incomplete/mismatched answers and stale revisions block submission. Critical failures can override an apparent passing score. Use a correction or independent re-review for later changes; submitted answers are not edited in place.

Source evidence: services/api/src/audits/routes.ts; services/api/src/audits/scoring.ts; apps/web/src/features/audits/audit-workbench.tsx.

QMS-05 — Record a manager correction

Status: Implemented. Verification: code inspected; no runtime completion claim.

Purpose: record a manager correction.

Actors and access: audits:manage.

Prerequisites: A submitted source audit and a valid Atom/outcome correction.

Entry point: Administration → corrections.

1. Review the source result and supporting evidence.
2. Select the Atom, corrected outcome and rationale; link calibration context where applicable.
3. Record the correction and inspect its ledger/activity entry; commission independent re-review if needed.

Result: A linked correction records the manager's decision while preserving the submitted audit.

Exceptions and recovery: Invalid source/outcome combinations fail server validation. A correction does not overwrite historical source answers; examine the correction ledger when interpreting later reporting.

Source evidence: services/api/src/corrections/routes.ts; apps/web/src/features/administration/assignment.tsx.

QMS-06 — Assign and complete independent re-review

Status: Implemented. Verification: code inspected; no runtime completion claim.

Purpose: assign and complete independent re-review.

Actors and access: audits:manage to create; re_reviews:review and ownership to draft/submit.

Prerequisites: Submitted source audit, supported correction linkage and a different active eligible reviewer.

Entry point: Administration → re-reviews; Re-review queue/workbench.

1. Create a re-review for the independent reviewer.
2. The assigned reviewer reads the evidence, saves answers using the current revision and completes required criteria.
3. Submit and inspect the independent result; original audit score is shown after submission.

Result: An immutable independent review is linked to the source audit.

Exceptions and recovery: The source reviewer cannot perform their own re-review. Resolve save conflicts and missing evidence before submission; do not copy source outcomes to defeat independence.

Source evidence: services/api/src/re-reviews/routes.ts; apps/web/src/features/re-reviews/re-review-workbench.tsx; apps/web/src/features/re-reviews/re-review-queue.tsx.

QMS-07 — Perform an independent calibration

Status: Implemented. Verification: code inspected; no runtime completion claim.

Purpose: perform an independent calibration.

Actors and access: calibrations:review and reviewer ownership.

Prerequisites: A submitted source audit reviewed by somebody else.

Entry point: My calibration queue/workbench; calibration API.

1. Claim the submitted audit for calibration and open the independent workbench.
2. Answer the pinned criteria and save with revision protection while baseline answers stay hidden.
3. Submit and inspect agreement rate, exact disagreements, score variance and critical-failure comparison.

Result: A submitted calibration compares independent decisions against the original audit.

Exceptions and recovery: Self-calibration is rejected. Complete valid answers/evidence before submission. Disagreement is routed to QMS-09 rather than editing either immutable review.

Source evidence: services/api/src/calibrations/routes.ts; services/api/src/calibrations/comparison.ts; apps/web/src/features/audits/audit-workbench.tsx.

QMS-08 — Configure calibration sampling

Status: Implemented. Verification: code inspected; no runtime completion claim.

Purpose: configure calibration sampling.

Actors and access: calibrations:manage.

Prerequisites: An agreed sampling rate and supported channel/team scope.

Entry point: Administration → sampling policies.

1. Create a named sampling policy with scope, rate and status.
2. Inspect or update the existing policy, including disabling it when needed.
3. Check the resulting audit/calibration signals for eligible submitted work.

Result: The saved policy participates in sampling selection for eligible work.

Exceptions and recovery: A sampling flag is not evidence that an independent calibration has been completed. Avoid presenting a configured rate as measured reviewer coverage; use actual calibration records.

Source evidence: services/api/src/calibration-sampling/routes.ts; services/api/src/calibration-sampling/selection.ts; services/api/src/audits/routes.ts; apps/web/src/features/administration/assignment.tsx.

QMS-09 — Adjudicate calibration disagreements

Status: Implemented. Verification: code inspected; no runtime completion claim.

Purpose: adjudicate calibration disagreements.

Actors and access: calibrations:adjudicate.

Prerequisites: A submitted calibration with disagreements and its immutable source scorecard.

Entry point: Adjudications.

1. Open the disagreement queue and compare original and calibration decisions.
2. Resolve every disputed Atom using a valid final outcome and rationale.
3. Record the adjudication and inspect the server-derived upheld, overturned or partially-overturned decision.

Result: One immutable adjudication records the resolution for the calibration.

Exceptions and recovery: Incomplete/invalid coverage is rejected. Identical replay is idempotent; conflicting second decisions are not a way to edit the ledger. Submitted source reviews remain intact.

Source evidence: services/api/src/adjudications/routes.ts; services/api/src/calibrations/routes.ts; apps/web/src/features/adjudications/adjudications.tsx.

QMS-10 — Create a manual calibration reference and distribute it

Status: Implemented. Verification: code inspected; no runtime completion claim.

Purpose: create a manual calibration reference and distribute it.

Actors and access: calibrations:manage; creator-scoped management.

Prerequisites: An eligible manual audit context with pinned interaction/scorecard.

Entry point: Manual audit → calibration; Calibrations I created.

1. Create the session and review/save reference outcomes, comments and feedback.
2. Choose active eligible auditors, optionally narrowing by company/division, and set the due date.
3. Distribute the session and monitor participant assignments; save later reference edits as new revisions before finalisation.

Result: Participants receive private calibration assignments linked to the reference session.

Exceptions and recovery: The creator cannot be a participant. Invalid/inactive auditors or unsupported reference decisions fail validation. Reference editing does not silently alter submitted participant decisions.

Source evidence: services/api/src/manual-calibrations/routes.ts; apps/web/src/features/audits/manual-calibration-workbench.tsx.

QMS-11 — Complete a manual calibration assignment

Status: Implemented. Verification: code inspected; no runtime completion claim.

Purpose: complete a manual calibration assignment.

Actors and access: calibrations:review and assignment ownership.

Prerequisites: A distributed assignment and available interaction evidence.

Entry point: Review queue → Manual calibration assignments.

1. Open your assignment and inspect the full interaction without seeing private reference decisions.
2. Save outcomes, evidence and feedback with the current revision.
3. Complete all required items and submit; return after publication to inspect reference comparisons.

Result: Your immutable submission remains private until the creator publishes results.

Exceptions and recovery: Stale revision and unavailable evidence block saves/submission. Publication controls visibility, not whether a submission can be rewritten. Reload conflicts and reconcile before submitting.

Source evidence: services/api/src/manual-calibrations/routes.ts; apps/web/src/features/audits/manual-calibration-workbench.tsx.

QMS-12 — Finalise and publish manual calibration results

Status: Implemented. Verification: code inspected; no runtime completion claim.

Purpose: finalise and publish manual calibration results.

Actors and access: calibrations:manage and session creator.

Prerequisites: Saved complete reference answers and completed participant submissions.

Entry point: Calibrations I created → session.

1. Inspect participant progress and final reference decisions.
2. Finalise completed submissions to calculate exact-answer alignment against the reference revision.
3. Explicitly publish results so participants can view the permitted comparisons.

Result: Finalised comparisons can become participant-visible through publication.

Exceptions and recovery: Incomplete assignments remain open when completed submissions are finalised; inspect each status. Reference completeness and revision/state checks still apply. Finalise and publish are separate actions, not an implied approval of unfinished work.

Source evidence: services/api/src/manual-calibrations/routes.ts; services/api/src/manual-calibrations/alignment.ts; apps/web/src/features/audits/manual-calibration-workbench.tsx.