1

Backstory: Recently someone somehow compromised a Xenforo installation linked to my server and injected a endlessly looping php mail() script. It sent thousands of emails from within the domain to the outside world, ending up getting my domain blacklisted from all of the major email ISP's before I had even noticed it was happening. I found the file, which was somehow injected into a cache and skin_cache directory of the forum, and removed them and set up a permanent redirect (using a php header redirect) to an anti-spam harvesting site.

Current problem: I now am seeing a steady and never-ending flow of POST requests to aforementioned spam scripts. The IP's are different every time, and seem to never stop coming. This has been happening for over a two weeks. So much so that my apache is maxing out its MaxClients setting and running into memory problems, and starting to shut down other processes to compensate.

This is what's showing up in the log:

190.40.7.126 - - [28/Mar/2013:18:58:30 -0500] "POST /forum/cache/sslFDoB.php HTTP/1.1" 200 3889 "-" "Mozilla/5.0"
190.104.19.189 - - [28/Mar/2013:18:58:39 -0500] "POST /forum/cache/sslFDoB.php HTTP/1.1" 200 3889 "-" "Mozilla/5.0"
78.251.159.173 - - [28/Mar/2013:18:58:57 -0500] "POST /forum/cache/sslFDoB.php HTTP/1.1" 200 3889 "-" "Mozilla/5.0"
219.78.213.10 - - [28/Mar/2013:18:59:09 -0500] "POST /forum/cache/sslFDoB.php HTTP/1.1" 200 3889 "-" "Mozilla/5.0"
69.123.20.137 - - [28/Mar/2013:18:59:09 -0500] "POST /forum/cache/sslFDoB.php HTTP/1.1" 200 3889 "-" "Mozilla/5.0"
2.234.181.7 - - [28/Mar/2013:18:59:37 -0500] "POST /forum/cache/sslFDoB.php HTTP/1.1" 200 3889 "-" "Mozilla/5.0"
187.207.223.67 - - [28/Mar/2013:18:59:44 -0500] "POST /forum/cache/sslFDoB.php HTTP/1.1" 200 3889 "-" "Mozilla/5.0"
24.242.122.42 - - [28/Mar/2013:19:01:56 -0500] "POST /forum/cache/sslFDoB.php HTTP/1.1" 200 3889 "-" "Mozilla/5.0"
213.49.254.102 - - [28/Mar/2013:19:02:32 -0500] "POST /forum/cache/sslFDoB.php HTTP/1.1" 200 3889 "-" "Mozilla/5.0"
82.247.48.152 - - [28/Mar/2013:19:02:38 -0500] "POST /forum/cache/sslFDoB.php HTTP/1.1" 200 3889 "-" "Mozilla/5.0"
41.135.146.136 - - [28/Mar/2013:19:02:43 -0500] "POST /forum/cache/sslFDoB.php HTTP/1.1" 200 3889 "-" "Mozilla/5.0"
91.187.93.36 - - [28/Mar/2013:19:03:04 -0500] "POST /forum/cache/sslFDoB.php HTTP/1.1" 200 3889 "-" "Mozilla/5.0"
194.90.37.132 - - [28/Mar/2013:19:03:40 -0500] "POST /forum/cache/sslFDoB.php HTTP/1.1" 200 3889 "-" "Mozilla/5.0"

I need suggestions on the best way to go about handling this problem. Simply redirecting or 404'ing them isn't going to stop the resource usage on the server. There are also so many different IP's at such a constant rate that it's hard to see if there are any duplicate IP's at all. They are all seeming to be on random ranges and of different origins. I'm honestly not sure if these are fake requests, or genuinely compromised clients victim to a spamming xss or virus of some sort.

Would it be reasonable to come up with a way to ban any IP that requests that file dynamically so they can't retry? I need suggestions and help with this please.

Improve this question

1 Answer 1

Reset to default
3

It's likely that the URL to the malicious script was distributed to a botnet, which is trying to use your webserver as a beach head. The requests will eventually subside as it's not profitable for the botnet to continue requesting a script that doesn't function.

The quick and easy fix: Add a <Location> block to your Apache configuration or a rewrite rule which will ignore these requests.

The brazen, haphazard method: Use iptables based rate limiting. iptables -A INPUT -p tcp --dport 80 -m limit --limit 60/min -j ACCEPT with a following rejection rule might help, though it may also block legit traffic, including search engine indexers and legitimate bots.

The better solution: Implement a web application firewall like Apache's mod_security. You can add an additional rule to match the incoming requests and blacklist them, or any other suitable side-effect.

Regardless of which option you take, I highly doubt that 404 errors are harming the resource usage on your server. You need to consider other factors:

  • Your PHP scripts could be causing some significant performance/efficiency issues.
  • Your Apache configuration may not be optimized to handle larger amounts of traffic.
  • Your server is compromised and you now have a much bigger problems to deal with.

You may also want to consider putting Apache behind Nginx as a reverse-proxy backend. Depending on the cause, using an HTTP cache in front of your Apache server (like Varnish) could reduce the load on your server to a relatively constant factor instead of a varying one.

TL;DR: It depends. Your server has probably been mis-configured or compromised.

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

Thank you for the ideas. The server load issue may be a misconfiguration with my postfix setup. I just found some issues in the log for that as well and had been setting up dovecot and opendkim yesterday. As for ridding of the incoming connections your suggestions were all read and I will look into seeing which one will do the job the best. Thanks for your response. :]
I've now configure mod_security to block all incoming requests to that file. Thanks again for the logical ideas, lol.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.