I was wondering whether I would be able to publish this week’s post. Why? What happened? In this post I will tell you the incident it occurred at my cloud provider and how I managed to stay online. That is why companies should have an Incident Response and a BCP plan!
If you have not seen it so far, OVH was on fire this week. It is not an expression or such a thing, I actually mean they had a real fire that destroyed one of their buildings, where they had one datacenter. This is a major incident that affected many many clients, including companies. Of course, OVH has a plan to respond to the incident, but it is taking a lot of time and effort for them to manage it.
We have a major incident on SBG2. The fire declared in the building. Firefighters were immediately on the scene but could not control the fire in SBG2. The whole site has been isolated which impacts all services in SGB1-4. We recommend to activate your Disaster Recovery Plan.— Octave Klaba (@olesovhcom) March 10, 2021
Non, les données contenues dans les serveurs de SBG2 ne seront vraisemblablement pas récupérables.— xgarreau 🤝 (@xgarreau) March 10, 2021
Mais si vous utilisez le Backup FTP #OVH, vos sauvegardes sont dans un autre datacenter. (Les photos sont des pompiers) https://t.co/tRonks8qIw pic.twitter.com/EL5BcMjbc7
Incident response: the plan B
Incident Response is a concept widely known by cybersecurity experts. Together with the Business Continuity Plan (BCP), it is the safeguard for business to be able to have a plan when things go wrong. That is, they have thought in advance their critical business processes they have to restart, how they should communicate with their clients, who they should call to as soon as they have detected their incident, and a bunch of things they should have written.
My plan B: two weeks of free hosting
Where is my incident response documentation? Well, I am not a company, so I do not have such a thing! What I had to do is what people usually call having a plan B.
So, whith no access to my server, I had to figure out what to do to publish again my website. Of course, I had a copy of the source I am using for this. As I explained in a previous article, I am using the Hugo static site generator to render my website so, all I had to do is host somewhere the complete website and point my DNS there.
One of my options is to hire another web hosting service meanwhile. However, after a long time thinking and looking at my website (only
localhost, of course), I remembered Github Pages!
Github Pages is a free service from Github that gives you the opportunity to have a website for you and your projects. That is, they will automatically host and deploy a website from your source code at your repositories. So, as long as you have an account at Github, you can make your own website.
There are two types of websites at Github Pages:
- Your main website, usually powered by Jekyllrb: martinord.github.io.
- Your project sites: martinord.github.io/my-project
So, what I did is configure a new Github repository named
martinord.github.io, and enable Github Pages in the Settings section like this:
As you may have seen in the above configuration, I have specified a custom domain name. This setting is for you to be able to point your DNS servers to Github Pages and tell them where you have your website. Therefore, now you have to create a new
CNAME entry at your DNS provider like this:
Some incidents you would not expect to happen sometimes, well… they just happen. For this reason, if you are a company, you should have some action plan, like an Incident Response or a BCP. They are critical for a business, as offline time means lost money.
In my case, I had my website down some days, but finally managed to keep it online. The important thing here is to never stop learning so, when things go wrong, you will stay creative and find a solution.
Obri! I hope you have found this post interesting, see you next week!