Ovh den açıklama gelmiş kesintiyle ilgili. gsw routerı üzerindeki konfigürasyonun yanlışlıkla silinmesi nedeniyle gsw pop noktasının düşmesi, akabinde yaşanan bgp sorunu nedeniyle trafiğin otomatik olarak diğer pop noktasına aktarılamaması nedeniyle kesinti yaşanmış.
Özürlerini iletiyorlar
Alıntı
Hello,
An incident has just occurred on the routing of the 2 routers in Paris: gsw-1-a9. The incident was caused by a human error. One of the engineers in the network team (that’s my team) accidentally deleted the OSPF configuration on the router, despite double checking the configuration. He confirmed that he had done it without realising (in pilot mode) .. and as a result the gsw-1-a9 router went down.
Everything should still have continued to function, however. But a bug on the 3rd reflector router rf-3-a1 stopped this router alerting the rest of the backbone that gsw-1-a9 was down. rf-2-a1 did it, but rf-2-a1 was down during the outage. The backbone therefore continued to behave as if the gsw-1-a9 router was up. We saw that there was a looping problem in the traceroutes.
We’ve rebooted all BGP sessions on rf-3-a1, but given that only rf-2-a1 was synchronising the BGP between all the routers in Europe because rf-1-a1 and gsw-1-a9 were down, the European network was intermittent: each router was pinging or not pinging for 60-120 seconds.
Everything then came back up; then we reconfigured the gsw-1-a9 router.
The backbone is up.
Please accept our sincere apologies for this outage. Human errors can happen and the backbone is meant to prevent this type of issue. We will investigate to find the bug on our RR (ASR1002). Then we will give the team a good hiding…
