Vint.ee outage 18.11.2016

Hello, This was the biggest outage in recent years, where access to vint was limited for 6 hours (21:00 - 03:00). We have both good and bad news: Good news: We found a way to restore normal life to vint.ee. This is remarkable, because initially even a server restart did not help fix the situation. Bad news: We are currently unable to come up with a logical connection between the fix we made and the cause of the error. In simple terms: We do not know exactly what happened, we fixed it, and it works :) From the technical details: php-fpm max_children limit was 5. It had been 5 for these months on the new server and it had been 5 all these years on the old server. We increased this number to 25, restarted fpm and everything started working! Of course, best practices recommend moving the connection between nginx and fpm from native socket to tcp-ip, which we will do tomorrow morning. I would like to thank Jaanus Jaakmaa, who came from a party at 02:30 and solved the problem. I also came from a party at the same time, but that is less important ;) Keeping Vint.ee running seemed easy before today. At the moment, that is no longer the case. Jaanus and I will keep an eye on the server's behavior. Especially during the upcoming Mind Sports Olympiad. Best regards, Marten