This repository serves as a centralized log of CAPAs taken in response to significant outages, performance degradations, or operational issues.
To maintain focus, items are added to this repository only when the issue cannot be resolved within normal team activities, warrants postmortem documentation, and fits into one of two categories:
- Category 1: Customer-Facing Failures Incidents where users cannot access key functionality (e.g., contact form submissions failing, donate/payment flow errors, primary site downtime).
- Category 2: Dev & Productivity Failures Sustained disruptions to internal workflows (e.g., blocked development branches, CI/CD deployment halts, major infrastructure downtime).
Governance Rule: CAPA entries may only be created or approved by Executive Team members.
| Role | Key Responsibilities |
|---|---|
| Executive Team | Approves or creates the CAPA issue; determines if the incident meets the CAPA threshold. |
| SWAT Team | Claims initial assignment within 15–20 minutes. Validates the issue and coordinates with the affected teams. |
| Originating Team | Technical representatives assigned to assist with Root Cause Analysis (RCA) and implement permanent fixes. |
- An Executive Team member creates/approves the CAPA as a high-severity GitHub issue.
- A member of the SWAT team is assigned immediately.
- Identify the Originating Team (e.g., ML, Web, or API) in the comments and assign at least one technical representative as a collaborator.
- Implement short-term resolutions (temporary patches, rollbacks) to restore functionality immediately.
- Strict Logging: All updates, communications, and actions must be logged as comments on the issue with timestamps (e.g., logging exactly who was contacted and when).
- Investigate the underlying conditions that allowed the issue to occur using evidence.
- Identify failures in processes, testing, deployment, or system architecture.
- Corrective Actions: Define and implement steps to permanently fix the specific root cause.
- Preventive Actions: Identify where similar vulnerabilities exist elsewhere and implement safeguards.
- Verify the fix is effective (via monitoring, tests, or metrics).
- Update the original issue with a final summary of the corrective actions taken and preventive measures planned.
- Close the CAPA ticket.
Use this structure when creating a CAPA issue in GitHub to ensure consistent tracking.
Incident Summary
- Description:
- Issue Since (Date & Time):
- Current Impact (Systems/ Features impacted):
Timeline:
- Maintain a chronological log of events below (e.g., YYYY-MM-DD HH:MM - Issue detected).
Resolutions:
- Short-Term:
- Findings from Root Cause:
- Long-Term (Corrective):
- Preventive:
Action Items, Owners to be mentioned in the comments.