Google server running cPanel

While setting up a web-server is not a big deal, cPanel makes the whole process so much easier. Seemingly, Google takes the advantages of this software, too. We already knew that Google uses the open-source and so-popular Apache web-server, most likely because it can be adapted to various situations and has the lowest footprint yet amazing capabilities. But that they use cPanel is a new thing, here’s the proof:

cPanel is good for beginner server administrators, but for professionals it’s like a WYSWYG editor for the web-developers. It’s good cos it’s simple to use, bad, because it adds one more daemon process to the persistent processes list, which is less than ideal for servers which are bombarded with so many connections like the Google servers.

Another thing which is interesting: as I stated above cPanel makes setting up web-servers extremely easy. But why do Google need this help when the employment requirements for a server administrator job are the following :

• BA/BS in Computer Science or related field, or equivalent experience.
• 3 - 5 years experience with UNIX systems administration (5-15 years for Senior position).
• Solid scripting skills in Shell, PHP, Perl or Python.
• Proven technical troubleshooting and performance tuning experience.
• Experience in a high-volume or critical production service environment.
• Ability to handle periodic on-call duty as well as out-of-band requests.
• Tack-sharp analytical abilities.
• A strong sense of ownership, urgency, and drive.
• Fluent written communication and unusual verbal agility are strong assets.
• SQL experience a plus, MySQL a plus.
• Experience leading short projects involving outside teams is a plus.

The picture was taken when we typed “google.com” in the address-bar of Firefox and we resolved to that page. This can be caused by they putting the server too fast in the cluster, even before they could mirror the homepage’s software.

Ban IP on server level after a number of unsuccessful logins

This is a widely used function amongst the server managers. Depending on your system configuration, the server will ban the enforcer’s IP, either putting it in the firewall’s deny list or, on Linux servers with IPTables installed, will put the IP in the drop list.

To achieve this feature, the easiest way is to install a software firewall. My recommendation is CSF, Configserver Security & Firewall developed and maintained by Way To The Web Limited. It’s an extremely efficient software, and integrated into WHM is very easy to manage it, even a beginner can handle almost everything.

So, if configured correctly, CSF has a, say, extension: LFD or Login Failure Daemon.

This stuff is what we search for at the moment. You can configure which ports to listen on, so if the enforcer tries on SSH, POP3 or FTP and even HTTP authentication, it will get banned after a few tries.

You can also put IPs to its ignore list. This is very useful feature if you don’t want to get yourself banned.

Country bans: Good or not?

During my daily routine, when I check all the sites I have to, Google Webmaster Group, and other forums I visit, I noticed a, well… Let’s say trend: more and more webmasters think that it’s a good idea to ban whole countries from their websites and servers. So is this wise or just a result of a momentary panic?

Let’s see first why would do it? The most convenient explanation would be that you get attacks, let it be SPAM or DOS, from a specific country and to stop it, you just ban the whole country. Let’s say you have a basic server, running Apache. To ban a country is quite easy, you supply a feed with the IPs you want to ban and you’re done. Even with IPTables would be easy enough to ban a country, say most half an hour with searching included.
Let’s take a small number of aggressive clients, say 1.000 clients concurs for connection to the servers/website, and you decide to ban a whole country. The most offensive country at the moment of this post is China. China has approximately 1/6 of the World’s population. Basically if you ban 1 billion people from accessing your site just because of those 1.000 who are attacking it, well… it’s pity. You can only lose, mostly visitors coming from search engines. You will have you webserver standing steadily, but you lost revenue, as visitors equals revenue. And as a general rule of the thumb, the aggressive clients WILL give up after a moment, switch off the server, shut down the ports they are using for an hour, something, anything, but ban a whole country?!

The second case, you ban countries just because you don’t offer anything for those countries. Or you think you don’t offer.
This was very painful for me to learn, but for some reason big corporations’ webmasters do it often, and it’s so frustrating. I test a lot of IT equipment, usually stuff which didn’t appear yet in my country, I try to visit the manufacturer’s website to download a driver and I can not, because the IP I have, and all the country has, is banned from the server I want to access! How foolish…

Recent case is one of my tests with an eMachines notebook. I knew the firm is owned by Acer, yet Acer has no drivers for eMachines equipments. I check the website, emachines.com and miracle: I can’t access it, it times out. This is a common case when your IP is banned on server level, since the server won’t serve you anything, no 403 message, no nothing, not even a single ICMP package. So I tried to access the website through a server which is located in a different country, still in Europe. No luck. After a few more tries with EU servers, I try with a US server, located in Dallas. And what a joy: I could access the above link. Later I tried with a Canadian server and I could access the website. In total, I tested it with 46 servers which are under my management, and ONLY the US and Canadian servers could fetch the site.
Yet, eMachines started to export notebooks in my country and since they are cheep, people buy it like sugar, but their only option is the provided driver CD/DVD… which is not good for XP
So if you need an XP driver for something, you either code it for yourself (i know), or you switch to Vista as the drivers provided on that media is good only for this OS.

So, what do you think, is it good to ban a whole country or not?

Restrict access to directory or domain by IP, using .htaccess

I don’t blah too much on this subject.
Basically, you can restrict or allow who can connect to your site or who can access specific directories using .htaccess .

Here’s the code to block one specific IP, I use 192.168.0.1 to block, you replace it with the IP you want to deny.


order allow,deny
deny from 192.168.0.1
allow from all


That is. Placed in the root of your site, the user with the IP 192.168.0.1 will not be able to access your site at all. If you place it in a specific subdirectory of your site, the user won’t access the specific subdirectory. If you want to put more IPs in your deny list, just add one more deny line for each IP.

To block by domain, replace the IP with a domain. For example:


order allow,deny
deny from .comcast.net
deny from .google.com
allow from all


If you look hard, you will observe that I put a dot in front of the domains. It has only one meaning: if you put a dot in front of the domain, all the sub-domains will be blocked. For example, in the second deny rule i said to deny everybody from google.com, including www.google.com, googlebot.google.com, finance.google.com, you get it.

And as always, we saved the world again.

Be Google’s friend: Make your URLs canonical with .htaccess

This subject is… is… well
Every second site on the net has at least one article about this subject. But to be honest, it’s good to have so many articles about this, in a way. At least people recognize they should use it. Or not.
So, what’s the fuss around the URL canonicalization? One thing only: the search engines and their hate of duplicated content. If your website is accessible both on www.example.com and the plain example.com the search engines will index both areas, they think you duplicated your content to get more positions in the search results, so they penalize your domain. Weird. They should know it’s the same website, or at least the coders should teach them that www is the same with non-www. Or at least on well-configured servers.
So, here pops Apache in and throws a resolution for the issue: the mod_rewrite engine, again. You will have to have mod_rewrite bundled into Apache and working correctly.
As always, here’s the code for those who just want to copy&paste and then the explanation for all the lines.


RewriteEngine On
RewriteCond %{HTTP_HOST} ^www\.example\.com [NC]
RewriteRule ^/(.*) http://example.com/$1 [R=301,L]


That is. Placed in a .htaccess file, it will redirect with code [301:Moved Permanently] all the queries sent to the www.domain.com to domain.com. Now let’s explain it line-by-line:

1. We switch On the mod_rewrite module, thus telling Apache we want to work with it.
2. If the hostname contains “www”, apply the rule, so this a condition
3. This last line is the rule which has to be done if the condition can be applied on the HTTP request. In our case do a 301 redirection to the non-www version of the site

That was all. Search engines are now happy, World saved again.
As always, if something is unclear, drop a comment and i answer as soon as possible.

Hotlink Protection using .htaccess made easy

This is one of the most used tricks by the webmasters who care about their allocated bandwidth. The code which controls what are domains where your images can show up is very short, 4 line that is.
As always, I provide the full code, then below it I explain everything.
To use this code, you have to have an Apache web-server with mod_rewrite correctly installed.
So, let’s see the code for those who don’t want the explanations:


RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http(s)?://(www\.)?example.com [NC]
RewriteRule \.(jpg|jpeg|png|gif)$ - [NC,F,L]


Now some explanations:
The first line,

RewriteEngine on

practically tells Apache we will do something with mod_rewrite so turn it on. This line is optional if you already turned it on before in the same .htaccess where you put the above code in.
The second line,

RewriteCond %{HTTP_REFERER} !^$

this is nastier. Basically, if there is no referrer, let the image to be displayed. I guess this needs a bit of explanation. When you navigate on the internet from one site to the other, the browser always sends a “referrer” header to the host you are accessing. So, for example if you are currently on http://www.Google.com and you navigate to http://yahoo.com, the browser will send yahoo the following : “Referrer: http://www.google.com”. This header is what we use in our .htaccess to prevent hotlinking, BUT! Some antiviruses, firewalls clears this header on the clients’ side so there is no referrer at all, thus we don’t know the user browses our site, or it’s hotlinking our image on another site. Thus we just let the image to be displayed if there is no referrer.

RewriteCond %{HTTP_REFERER} !^http(s)?://(www\.)?example.com [NC]

If the referrer domain is our own domain, display the image. We set: http(s)?://(www\.)?yourdomain.com, so our condition will work on HTTP, HTTPS and also on our www and non-www hostname/domain.

And the last step is to tell Apache which files to protect:

RewriteRule \.(jpg|jpeg|png|gif)$ - [NC,F,L]

In the above case the jpg, jpeg, png and gif images will be protected. If you want to protect your Flash-files as well, put swf in the list and your movies will not display embedded in remote sites.
On our domain the php files are also protected because the Imagick examples are parsed by php codes.

I hope the above example was somewhat useful, if you need help with it, just say your problem below and will answer as soon as possible.

Tightening PHP

OK, let’s see what’s my point of view on this subject. Please note that nothing I say is a must, probably others would do in another way what I did, many would do things better or less good.
First of all, I would like to say a “thank you” to all of those who I manage their servers and have cPanel/WHM installed. Since the folks from cPanel integrated EasyApache in their software, the job got much easier for us. Less work, more time to drink coffee.

So. My very first suggestion is to install PHP as a mod. For example, on Apache there is mod_php4 or mod_php5. Security holes are in both setup, whatever you do, but if you install PHP as a mod, at least you have less headaches. Maybe it’s just me, but when installed as CGI it was a nightmare. First of all, the developer will have to work in the cgi-bin directory (if the web-server is well configured), secondly, sometimes you have to place the php executable in the CGI directory which is not good… at all. Thirdly, why bring even more security holes in the server by involving a CGI wrapper if PHP already has enough? Probably the last is the best reason, and please, don’t tell me the wrappers don’t have security holes.

OK, let’s step forward. We still remain at Apache, two other mods: mod_suphp and mod_security. None of these is a must, it depends on your needs and on what will you do with your server. mod_security is “just” a software firewall, it can do many things, live-protection, monitoring, etc, there are a lot of features, better have a read here on mod_security official page. it’s pretty cool, but if you already have a firewall installed like CSF, probably is just a waste of resource. On the other hand, mod_suphp will execute php scripts as the script’s owner. OK, will try to explain: php is called by default “nobody”. This can vary on systems as it can be changed easily. So, by default, every PHP script is executed as “nobody”, say “nobody” will be the owner of all PHP scripts. Sounds weird. Let’s say, you run many virtual hosts on your server, example1.com is John’s, your best friend from school, Ran, your girlfriend’s ex has example2.com on your server. If suphp is not installed (or not working correctly), if Ran or John writes a script and runs on your server, it will be run as “nobody” as, remember, by default every PHP script is nobody’s. If you have suphp installed, then Ran’s PHP scripts will be run as Ran’s, John’s as John’s, and so on. Why is good to have it installed? Well, everyone will be able to execute its own script but not the others’. Again, an example: John will be able to execute his own PHP scripts but not Ran’s and vice-versa, and definitely John will not be able to execute nobody’s scripts. This tool is a must if the server will not be used only by you, else you can live without it, but you will have a security hole!

Install always the most up to date and STABLE release of PHP. Why everyone thinks that this is not good? The folks from PHP releases new versions cos the fixed or upgraded something, not cos they were bored. If you think your users’ PHP scripts won’t be compatible with the new PHP, first let them know, give ‘em a few days to fix their codes, then throw them in the deep water and upgrade PHP. Sincerely, I saw the other day a webhost which still used PHP 3. When I asked their live support why the php 3, they said, well, thy poor girl from the support desk that their users aren’t prepared for the upgrade yet. Yup, 2 or 3 years isn’t enough time. Too bad PHP 6 is knocking on the door.

Next step, let’s disable functions which might be dangerous on production servers! Let’s give a straight-forward example: you definitely don’t want your users who are all potential black-hat hackers to be able to run commands on your system like
format c:
Right? So let’s disable some inbuilt PHP functions, consider the following:

ini_set
system
show_source
shell_exec
passthru
exec
proc_open
popen

system, exec, passthru, shell_exec, proc_open, popen are related to command line, thus are good to have them disabled, remember, format c:. strong>ini_set, you already configured your PHP, why would you let others to reconfigure it? Finally show_source, would you like to show your PHP script’s source to the hackers?
You may also disable phpinfo, as you don’t really want to let others know what is your PHP configuration. If you don’t want to send e-mails from your script, you may want to consider disabling the mail function as well.

Ok, another one: fopen() . Did you know it can be used to open remote files? For example to open a YouTube page and view through your webpage. That’s the less severe case anyway. It could be used to place exploits on your server or for many other purposes. So
allow_url_fopen = Off

That’s for now, I will talk later about how and when to limit the server resources PHP can eat up.