Stop an ongoing DoS attack

Many server managers say that effectively stopping a DoS attack is impossible. This is only half true.

Under a DoS siege the majority of the server administrators will try to create a null route for each offensive IP, IP block or even whole subnets. This is done as an effort to keep the service online for legitimate clients, but deny service for the offenders. This is very time consuming operation as the IPs are faked so null routing almost always is impossible.

So what other methods are available?

I can tell you my experience only, and that is extremely simple. First, I start to look for the port the attackers chose. Usually and unfortunately this port is almost always port 80 aka HTTP. If the server became completely unusable and the number of connections is unbelievably high, then one simple step: shut down the ethernet card which handles the public network. No one will be able to use the server until you re-enable the ethernet card so this is a very cruel option in many’s mind. But why? If no one can access the server because the DOS attack, why is bad if I shut down the ethernet card?
And why do I shut down completely the traffic? This is again my own experience: the offender thinks that the mission was accomplished and simply gives up. I had only once situation when re-enabling the ethernet card after a few hours brought the offender back too, one occasion from a few dozen. So in my humble opinion it’s still better to have a few hours of silence than having a few hours of stress caused by the efforts of blocking the unwanted traffic.
On the other hand if the server didn’t deny the service yet, you may try to null route the offenders. It’s an extremely time consuming operation and if the IPs are faked… well. You can’t really null route. But if you desperately want to keep the server up and running, you may try it. Be prepared for great stress, have your coffee machine prepared too, and if you smoke, a few packs of cigs in the nearby is also required.

Another option I heard about is a bit more interesting. But it involves DNS administration so you should be familiar with it before start it.
Let’s take the following situation: you have www.example.com functioning on the 192.168.0.1 IP. When you observe that your IP is under attack and your website functioning under the example.com domain became slow, you move your domain name under another IP on another server, the offenders will still attack the old IP but they cause no trouble since that IP is out of service.
This is very beautiful in theory, but in real world may function or not. Usually doesn’t, since while the update in your DNS travels through the whole world, your website may be already on its knees.

What can be done to just not experience DoS attacks?

To be prepared for the attack. There will be always black hat hackers and hacker kids in suburban basements who has nothing better to do but try to offend websites or whole servers, so our only option is to be prepared. Better said, the datacenters.
DoS attacks can be recognized quite easily, there is a listening system installed on every major internet backbone too. What they can do is to early recognize that something is happening and start to block the malicious traffic while is born.
Choose your provider wisely. If you want a good hosting solution go with well known web hosts like GoDaddy or MediaTemple. If you need servers, choose well known data centers like RackSpace or Softlayer. They are all prepared for DoS attacks and can help you in no time.

If you go with a server, a must have is a good hardware (!) firewall, which can distinguish the good traffic from the bad one, then block the unwanted. This kind of firewalls, stateful firewalls are manufactured by Cisco for example. They do a hell’a good job.
Other than that, you can only beg for mercy. Nothing else. Every system is vulnerable to these attacks at any time, it’s not just you. The only difference is how much the servers can survive without being killed.

What is null routing and how to use it

There are times when a server admin has to re-route some of the traffic coming from specific IPs. This is called null routing.

OK, now simpler: say, 192.168.0.1 is trying to connect to your server. You know that this IP has bad intentions so you decide to block it. Creating an IPTables deny entry is one of the options, null routing is another (adding an entry in the routing tables), more effective one. Why? Because while IPTables is only a software, null routing happens on the server’s level, thus putting less strain on the hardware.
Null routing means basically that you block the IP, but you can also re-route it. For example you can easily re-route it to the loopback address, 127.0.0.1, thus when the client tries to connect to the server, the server instantly redirects the connection to the client’s address. This is often called blackhole routing.

Null routing is the preferred option when a server is under DoS siege.

How to create a null route.

The offender’s IP address in my examples is 192.168.0.1, the gateway mask is 255.255.255.0 and the new route is 127.0.0.1 . Obviously, you replace these values with the offender’s IP and the preferred route.

on Linux, you log in as root, and type:
route add 192.168.0.1 gw 127.0.0.1 lo

On Windows based systems it seems re-routing is not really possible. You can specify a gateway to use to route a specific connection through, but my experience is that it’s just not working in the way I would like to. Even though Microsoft Support tells me the contrary.
Anyway, the shell command you should use for rerouting on Windows based systems is the following:
route add 192.168.0.1 MASK 255.255.255.0 127.0.0.1

The problem is that this created for me a loop, but on the wrong side: on my side.

How do you recognize a DoS attack

Early recognition of a DoS attack is essential. This post will help you learn how to recognize a DoS attack so you can take necessary steps.

In my previous post, I talked about DoS attacks in general (see end of the post for the link) and if you’ve read it, you know how are they working. If you didn’t read it, do it now then return here.

When a website is under DoS siege, the most prominent effect is that the website becomes slower and slower as the time passes by and the number of incoming connection increases to the point the webserver either resets (in my humble opinion the best thing) or denies to serve.

In one of my previous posts I mentioned that a webserver, for example Apache can handle only a limited number of connections. This number solely depends on the architecture of the webserver’s core, the amount of available hardware resource and how the server manager configured the webserver. This only applies to standalone servers, not balanced ones or mainframes. In how much time will the server becomes unusable, solely depends on the above factors. For example a server with high amount of RAM could handle a very big number of connections, if the configuration of the software allows it.

So, the first eye-blowing effect of a DoS attack is the enormous number of unexpected connections.

The architecture of the webservers is designed so that when a connection is opened, it isn’t closed for quite a lot time, usually 250-300 seconds. So when a DoS client connects to a server, it’s not disconnected for the TTL time. Imagine thousands of connections waiting to close.

As per the above paragraph, the second prominent thing which happens with a webserver is that there are a lot of unclosed, waiting connections.

The web server allocates an amount of hardware resource for each incoming connection. Under a DoS siege the number of connections is enormously high, thus a lot of hardware resource is consumed. First the RAM will be not enough, when the RAMs are filled up the server will try to use the hard disk thus causing the server to “swap” but as there is not enough RAM the CPU neither will be able to handle a thing. The server is on its knees.

So, the last thing you should observe is that the server load is too high. I say this is the last you have to look for because if only this applies to your situation, it’s most likely you are not under a DoS siege but either a misconfiguration happened on the server’s level, or a bad-written, low quality script is using up the resources.

What is a DOS attack and how to initiate it

The best method to deflect an attack is to understand how the offender planed his attack, thus you know what will happen next or how to stop a previous action.

This post is about the basics of a distributed DOS attack. DoS is the abbreviation of Denial of Service. I won’t list everything you need, not the software you need nor anything which you can start a DOS attack with, but the architecture of such an attack to help you form a picture about how the DOS attacks works thus help you avoid them.

First of all, what is a DoS attack?

DoS stands for Denial of Service. Basically, a server is bombarded with fake traffic until it denies to serve normal traffic coming from legitimate users. DoS is a relatively new thing on the internet and they can be temporary or permanent. Temporary is when it can be let’s say fixed easily and permanent is when it can’t be fixed with a simple reboot or by shutting down the ethernet port. Permanent DoS is very sever as it rewrites a network component’s firmware, thus the owner of the hardware either replace it or re-rewrites the firmware (usually replacing is advised).

There are many types of DOS attacks, the most common are probably the ICMP floods. ICMP is a protocol on which the ping requests are handled and as almost every server will respond to ping requests, they are vulnerable to these type of attacks.

So, what happens in the background? A user with connection superior to the server’s which will be attacked, sends a high amount of ping requests to the server and keeps sending them until the server is brought down. Sounds simple? It’s not. The above example is oversimplified, there is involved one other step which i didn’t mention: the IP faking.

So, how ping is working?

1. user initiates a ping to a specified server.
2. the server receives the request
3. the request contains data about where the server has to reply, the initiator’s IP address. The server replies to this address
4. the initiator acknowledges the answer and the connection is closed

So how to initiate a DoS attack

The problem occurs when the pinger’s IP is faked as the server will try to answer for the request but as the address is faked it can’t. One issue with this: as the architecture of the web servers is designed so that if a connection can not be handled, the connection is kept open for a very long time. Too many connections open will bring the server on its knees in no time, mostly if there are not enough hardware resource like RAM.

Of course, modern web servers will not be brought down with only one offending client. An attack to a well designed webserver, network component or network is much harder and requires more offenders. The attack should be synchronized very well and distributed over many, a few hundred or even thousand and even millions of attacking PC, solely depending on how advanced is the attacked party.

So how DoS is possible then?

The easiest way the hackers achieve to sync and distribute the attack is that they, let’s say hire spammers to send spam containing either links to malware which is used as a DoS client or the message has the malware attached. When the user clicks the link or opens the attachment, the script installs itself as a service on a Windows based OS.

On a side note, it’s interesting that the most expensive OS is in the same time the most vulnerable to malware and virus attacks, too.

The whole of the PCs which has this malware installed is called botnet.

The installed malware will start to work on a specified time and date like in MyDoom’s (Link to Wikipedia article, read it, it’s very interesting) case when millions of computers started synchronized distributed attack against several websites. This is called DDoS, abbreviation for Distributed Denial of Service. It’s incredibly effective against lower end web servers and can put offline even high-end architectures like Google’s search back-end network.

Next article is about how to recognize DoS attacks.

If something is not clear, feel free to ask.

The end of the CAPTCHA era

Every webmaster should know what a CAPTCHA (abbreviation for Completely Automated Public Turing test to tell Computers and Humans Apart) is, but for who doesn’t know, the little images with obfuscated text on it under the registration forms or comment forms are CAPTHCAs. They are meant to stop robots, for example spam-bots from registering on a website or to post unwanted, spam comments. They are very effective. But the problem is, they are easier to brake than many thinks.

Braking, hacking CAPTCHAs

I am not a good desktop-software developer, but even for me, took less then 2 days to write a .NET software which can recognize with quite high reliability (over 80%) the characters in random CAPTCHAs i found on the internet. I was one step away from releasing a SPAM-Bot with CAPTCHA recognition as all I would have need is a network connector, a script which handles the data transfer on TCP between my PC and the remote website.

So, why are these CAPTHCAs so easy to break?

Basically these are just text, random letters and numbers on a random background then saved as an image file. When braking these images, the software has to follow these steps:

1. separate the background and the foreground
2. segment the characters from the image into separate blocks
3. finally, match the blocks with templates, so all the letters from the dictionary plus the numbers

I think it’s too simple.

If a spammer team hires humans to decipher CAPTCHAs, the situation is even easier. In 3rd world country anyone would do it for a price of $0.0001 per CAPTHCA. Someone who is good enough, can solve about 5000 CAPTCHA’s per day, go figure. How much the spammers earn with their spam-campaign i have no idea, but since they doing this i figure way more than i would think.

So,

How to stop spammers effectively?

It’s very hard. A very good initiative is the Akismet project. Before a comment appears on a website is submitted to Akismet which will try to identify if the text from within the comment could be spam or it’s a legitimate comment. The effectiveness of this service is incredibly high. Sincerely, I didn’t see a spam comment on any website which is protected by Akismet. But this service has a great vulnerability: the service is free and the threshold the comments are matched against, is practically a database. The spammers use random text for their comments, so the owners and developers of Akismet have to update their database day-by-day, the database can only grow and updating the database is also time consuming… for a free service is not a good thing. They will either have to go further as a payed service, get sponsors or to give up. Neither one is an easy decision.

Another good alternative would be to make a website registration dependent and the registration data to be obtained from a trusted third-party which verifies extensively the identity of the users. Such initiative is OpenID. If the third-party which verifies the identity of the users – these entities are called OpenID Servers, such server is for example Verisign, one of the most trusted entities on the internet- makes a good job, the spammers can’t pass through the net thus can not post unwanted commercial comments. If the OpenID Server doesn’t do a good job, then the whole thing is meaningless.

And the list of the possible options has been exhausted… I think. If you know more, let me know.

Now let’s see what upgrade options for the CAPTCHA system would be possible. The official CAPTCHA website lists some great ideas, but at the time of my visits neither worked. The first is that they take hand-written (or not) words from old books and while a user types the letters (and numbers), they also digitize the books. Well, maybe it’s just me but this is the old CAPTHCA refurbished.
The second interesting initiative is a… I have no idea as it’s not working. They neither list any detail but, that it’s their newest CAPTCHA. Good to know.
And the last: 4 random images which relates in a way or other with each other. The user has to choose then the thing which is related to each image.

The last initiative I think is the most reliable and the same time unbreakable initiative as I can’t think of a reliable(!) solution which could recognize random objects from an image, of course i might be totally wrong.

Do you know of another solution which might work on a grand scale? Share your thoughts, the comments are open.

Next Page »