When I read about these things I always think about some of the writing of @pluralistic on graceful failure modes. A product (system) is not defined by its success but by how good or poorly it fails. I've been teaching students that not considering (poor) failure modes is a huge liability.