Before investigating
- Record the failing user operation, first-seen time, timezone, and impact.
- Compare from a second vantage point and check recent changes.
- Capture `date -Is`, `uptime`, `free -h`, `df -h`, and `df -i`.
- On systemd hosts, inspect failed units and the relevant service journal.
- Separate observed facts, hypotheses, and next tests.
- Preserve evidence before a restart or configuration change.
Before a release
- Record the commit, artifact checksum, target, and change owner.
- Verify the checksum against a trusted build record.
- Identify which meaningful tests ran, and exercise a failing test.
- Confirm the current healthy version and retain its artifact.
- Define the health check: status, expected content, and critical operation.
- Review data/schema compatibility and recovery boundaries.
- Confirm access, backup/restore plan, maintenance window, and rollback owner.
After a release
- Check from the reader/user network path, not just localhost.
- Compare errors and latency with the baseline over an agreed window.
- Run the critical user operation; a 200 response alone is insufficient.
- Roll back only when the documented trigger and data-safety conditions apply.
- Verify the original user operation again and record the final version.
- Record what failed, what worked, remaining risk, and next owner.
Handoff record
Impact:
Environment / version:
Time and timezone:
Observation:
Hypothesis:
Test and exact output:
Change / artifact / checksum:
Verification:
Rollback boundary:
Next action and owner:
Version: 30 September 2026 · Adapt to your environment and change process. Evidence policy.