There Are Times I Just WannaCry About Patching
The ransomware excitement of last Friday afternoon demonstrated that the work that Delt has done on cyber security and incident response has significantly paid off. There is some significant credit to be shared that we remained not only ransomare free but that our preparation for such an event was such that I got to spend the evening writing press releases that explained we were all clean and why rather than trying to explain why we had been breached. At this moment in Delt’s life and with significant expansion on the cards a cyber security breach could have cost us much more than a few negative press stories.
Those of you who worked late into the night on Friday, over the weekend and into Monday to protect our customers, thank you.
We did well and our customers, both current and potential, have been vocal in their praise to our response. However we need to be self aware enough to know that we dodged a bullet.
- We had a high level of patching but it wasn’t 100% on either clients or servers
- When we looked, our different management tools reported very different things. Neither AD or CMDB were anything like up to date. It was not easy to establish AV and patch status
- It took until Monday afternoon for us to fully identify unpatched devices and take steps sort it
- On Monday evening, 72 hours after the attack we still had almost 200 machines that were not patched. Not all of these were active and would have patched on next boot but some were still vulnerable
- We thought that a protocol used by the attack was blocked across some of the estate when it wasn’t
Applying a patch that fails is no good unless we know and quickly correct it. Applying a patch that needs a reboot and then not forcing a timely reboot is fairly useless. Deciding not to apply a patch to a server because it requires downtime and the client won’t agree to needs robust challenge.
There are a number of lessons learned from the weekend, some of which are already in progress, like forced reboot and some of which you’ll see over coming weeks, like improving AD quality.
the future it wouldn’t be the end of the world to explain that we had a system down or an outage because of a patch current policy but I don’t ever want to have to explain why we were breached by something that should already have been patched. I know patching is time consuming, expensive and disruptive but we must patch current and be able to prove it.
Giles Letheren – Chief Executive Officer