[Kubernetes DR EP8] DR drill과 Chaos Engineering: 복원해 보지 않은 백업은 없는 백업이다

Kubernetes DR 스터디 커리큘럼의 여덟 번째 모듈, 운영 검증을 깊게 다룹니다. GitLab 2017년 사고와 Google SRE book의 복구 원칙에서 출발해 Principles of Chaos Engineering, Netflix의 Chaos Monkey와 Chaos Kong, Google DiRT와 game day, NIST SP 800-84의 tabletop exercise를 살펴봅니다. etcd 스냅샷 검증, 관측 지표, runbook 관리까지 Kubernetes 운영에 연결합니다.

참고 자료

• Principles of Chaos Engineering (Last Update: 2019 March)
• chaos-eng.github.io commit 7b630f5 (2017-04-04): adds 'Minimize Blast Radius' to Principles of Chaos Engineering
• Basiri et al., Chaos Engineering, IEEE Software 33(3), 2016 (arXiv 1702.05843)
• Google SRE Book, Chapter 26 - Data Integrity: What You Read Is What You Wrote
• Google SRE Book, Chapter 33 - Lessons Learned from Other Industries
• GitLab: Postmortem of database outage of January 31
• NIST SP 800-84 Guide to Test, Training, and Exercise Programs for IT Plans and Capabilities (September 2006)
• AWS Well-Architected Reliability Pillar REL13-BP03 Test disaster recovery implementation to validate the implementation
• AWS Well-Architected Reliability Pillar REL12-BP05 Conduct game days regularly
• AWS Well-Architected Operational Excellence Pillar OPS07-BP03 Use runbooks to perform procedures
• etcd v3.5 Metrics
• etcd v3.5 Disaster recovery
• Certificate Management with kubeadm
• Netflix/chaosmonkey (GitHub README)
• Netflix/SimianArmy (GitHub, archived 2021)

이 에피소드의 음성은 TTS로 합성되었습니다.