Forum Outage
- Geoff the Medio
- Programming, Design, Admin
- Posts: 13603
- Joined: Wed Oct 08, 2003 1:33 am
- Location: Munich
Forum Outage
For most of the last 16-24 hours, the forums were inaccessible.
This happened because the host disabled the MySQL database for the forums. They did that because the forums were being flooded with requests originating from Chinese IP addresses, which was causing problems for the shared server. They tried blocking individual IPs, but were unable to slow the traffic, so disabled the database to prevent it from affecting any other users.
We've since set up a free tier of CloudFlare distributed cache, and I've banned a range of IPs in the forum software itself that were apparently the source of most of the issues, which dropped the stable 5-min concurrent users from about 300 (of which 99+% were guest users) to 50, with the 50 mostly being CloudFlare-related IPs, based on the whois info the forum shows me.
Hopefully this combination of tactics will reduce the issues for the host and the server and bandwidth usage for freeorion.org.
This happened because the host disabled the MySQL database for the forums. They did that because the forums were being flooded with requests originating from Chinese IP addresses, which was causing problems for the shared server. They tried blocking individual IPs, but were unable to slow the traffic, so disabled the database to prevent it from affecting any other users.
We've since set up a free tier of CloudFlare distributed cache, and I've banned a range of IPs in the forum software itself that were apparently the source of most of the issues, which dropped the stable 5-min concurrent users from about 300 (of which 99+% were guest users) to 50, with the 50 mostly being CloudFlare-related IPs, based on the whois info the forum shows me.
Hopefully this combination of tactics will reduce the issues for the host and the server and bandwidth usage for freeorion.org.
Re: Forum Outage
I noticed.
Hopefully it all clears up. Wonder if this was really a target or they are just testing some stuff out.
Hopefully it all clears up. Wonder if this was really a target or they are just testing some stuff out.
Re: Forum Outage
What can be possibly be (financially) gained by launching a DDoS attack against our forums? This is an open source project, no money involved at all here, nothing to gain...
- Geoff the Medio
- Programming, Design, Admin
- Posts: 13603
- Joined: Wed Oct 08, 2003 1:33 am
- Location: Munich
Re: Forum Outage
I think the proposed motive is to get access and post spam, phishing links, or search engine optimization links.
Re: Forum Outage
Just hit the database problem again (max connections), though cleared up when I reloaded the page.
Code: Select all
SQL ERROR [ mysqli ]
User tzlaine_4 already has more than 'max_user_connections' active connections [1203]
Re: Forum Outage
Forum is almost not usable for me right now.
4 out of five tries i GET the 'max_user_connections' active connections issue
4 out of five tries i GET the 'max_user_connections' active connections issue
Any code or patches in anything posted here is released under the CC and GPL licences in use for the FO project.
Look, ma... four combat bouts!
Look, ma... four combat bouts!
Re: Forum Outage
In my case it's variable. Sometimes it needs ten retires per page, sometimes I only need to retry one every few pages.
I wonder if there is a way to ban the IPs that are constantly crawling the forum and are not useful bots.
- Geoff the Medio
- Programming, Design, Admin
- Posts: 13603
- Joined: Wed Oct 08, 2003 1:33 am
- Location: Munich
Re: Forum Outage
I already banned all the IPs in the range from the Hong Kong ISP that is causing the issues. The remaining connections are coming through the CDN now so can't be blocked in the forum software. Tyreth should be setting up a CAPTCHA for those IPs in the CDN though, qhich should deter bot connections before they reach the forum server itself.
Re: Forum Outage
Well, I'm not really an IT security expert, but how is a rather slow/lazy DDoS attack like this supposed to get you into the forum to do these things? To achieve this, I'd rather expect outright hacking attempts, and who would waste such efforts on our forum?Geoff the Medio wrote: ↑Sat Oct 26, 2019 9:47 pm I think the proposed motive is to get access and post spam, phishing links, or search engine optimization links.
Or ARE all these access attempts by bots actually automated hacking attempts...? But even if, that's a lot computer power wasted on such a small, insignificant forum...
Re: Forum Outage
Tyreth has just enabled some countermeasures (CAPTCHA for China and Hong Kong) on the cloudflare tier level, if I understand correctly. Immediately afterwards the issues were gone. So it looks like that worked. I can use the forum just fine now.
Re: Forum Outage
Thank you all for the arrangements. Now this is smooth again.