30.07.2015

Sorry for the disruption

Many of you noticed that Vertikal.net was offline for a couple of hours this morning... It seems that the host building in Zurich suffered some major disruption, including issues with fibre optic cabling, which cut outgoing capacity.

As a result the system apparently cut service to a number of countries, including The Netherlands, UK and Germany. This is the first such major disruption since 2006, and hopefully the last for some time. We are planning to look into options such as dual location hosting as it also cut off our email systems… but… given the rarity of such events may well chose not to add further complexity.

We are now working to catch up with our emails and reporting and appreciate your patience.

UPDATE

It now seems that the fibre optic cables into the hosting building were severed by workman installing a new section in the building. The hosting company has several redundant sytems intended to avoid what happened yesterday, including duplicated fibre optic lines into the building. However the two feeds join up inside and that - of course - was where they were cut. It also has two other facilities that can take over the operations while one is down, but it turns out that those also link in to the junction to extract and continue to run the data.

So in spite of having at least three seemingly redundant systems - they all overlapped at one point and that was of course what got hit yesterday.

Needless to say a redesign is currently being drawn up. Hats off to the team at Nine which did manage to get the facility up and running on a temporary basis in under three hours and have the main system repaired tested an back on line a few hours later.