We Got Compromised!
We missed a internet facing box with Log4j. Even the CTO Advisor team with over 75-years of operations experience makes mistakes.
This morning, I read an article about the active exploitation of internet-facing appliance we have in production. I called one of our engineers to take a look to make sure we patched it. Unfortunately, we missed it, and someone installed a Crypto miner on it. Things could have been worse.
Note: It’s important to note this is a lab without customer data.
We are going through a root cause analysis (RCA) to improve our patch management processes. However, one of my engineers asked what advice would we give a customer in a similar situation.
This post is a CTO Short so, there’s not enough space to go into detail. We’ll save that for a more extensive post on https://thectoadvisor.com.
In short, this is a conversation you want to have before being compromised by Ransomware. At the minimum, start with a tabletop exercise on how you’d recover from a ransomware attack. Think immutable backups, system access, restricting the blast radius, identification, and remediation of the point(s) of intrusion.
In other words, have a documented plan. And then test that plan. Then, repeat and rinse as necessary based on the risk to your business. For example, we could have a complete ransomware loss and rebuild our infrastructure in a few days with minimum impact on our customers. So, while we need to have a plan, the overall risk to our business is low compared to others.
How have you approached recovery from Ransomware?