Returning BAD BOTS to where they came from

Banning BAD BOTS to where they came from

By Strictly-Software

Recently in some articles I mentioned some .htaccess rules for returning "BAD BOTS" e,g crawlers you don't like such as IE 6 because no-one would be using it anymore and so on.

Now the rule I was using was suggested by a commenter in a previous article and it was to use the REMOTE_ADDRESS IP parameter to do this.

For example in a previous article (which I have now changed) about banning IE 5, 5.5 and IE 6, I originally suggested using this rule for banning all user-agents that were IE 5, 5.5 or IE 6.

RewriteRule %{HTTP_USER_AGENT} (MSIE\s6\.0|MSIE\s5\.0|MSIE\s5\.5) [NC]
RewriteRule .* http://%{REMOTE_ADDR} [L,R=301]

Now this rewrite rule uses the ISAPI parameter {REMOTE_ADDR} which holds the originating IP address from the HTTP request to send anyone with IE 6 or below back to it.

It is the IP address you would normally see in your servers access logs when someone visits.

Problems with this rule

Now when I changed the rules on one of my own sites to this rule and then started testing it at work for a work site by using a user-agent switcher add-on for Chrome I ran into the problem that every time I went to my own site I was sent back to my companies gateway router page.

I had turned the switcher off but for some reason either a bug in the plugin, a cookie or session variable must have caused my own site to believe I was still on IE 6 and not the latest Chrome version. So everytime I went to my site with this rule I was kicked back to my companies gateway routers page.

Therefore after a clean up and a think and talk with my server techie guy he told me I should be using localhost instead of the REMOTE_ADDR IP address .The reason was that a lot of traffic, hackers, HACKBOTS, Spammers and so on would be hitting the Gateway page for their ISP for potential hacking,

These ISP's might get a but pissed off with your website sending their gateway routers page swathes of traffic that could potentially harm them,

Therefore to prevent getting letters in the post that you are sending swathes of hackers to your homes or phones ISP gateway - as a lot of phones or tablets use proxies for their browsers anyway - is to send them back to their own localhost or 127.0.0.1.

Also instead of using a 301 permanent redirect rule you should use a 302 temporary redirect rule instead as that is the more appropriate code to use,

Use this rule instead

Therefore the rule I now recommend for anyone wanting to ban all IE 5, 5.5 and 6 traffic is below.

RewriteRule %{HTTP_USER_AGENT} (MSIE\s6\.0|MSIE\s5\.0|MSIE\s5\.5) [NC]
RewriteRule .* http://127.0.0.1 [L,R=302]

This Rewrite rule bans IE 5, 5.5 and IE 6.0 and sends the crawler back to the localhost on the users machine with a 302 rewrite rule. You can obviously add other rules in with BOTS and SQL/XSS injection hacks as well

This is a more valid rule as it's not a permanent redirect for the traffic such as if a page has changed it's name. Instead it's down to an invalid parameter or value in the HTTP Request that the user is being redirected to the new destination with a redirect.

If the user changed it's user-agent or parameters then it would get to the site and not be redirected with a 301 OR a 302 status code but instead get a 200 OKAY status code.

So remember, whilst an idea might seem good at first until you fully test it and ensure it doesn't cause problems it might not be all that it seems.

Running WAMP on Windows 8 alongside IIS - 403 Forbidden Index

Problems with WAMP on Windows 8 - Forbidden 403 - Running IIS and Apache side by side

By Strictly-Software

I have just got a new laptop which has come with Windows 8.1.

Thankfully this version of Windows gives me more of  a Windows 7 feel as I really hate those big tablet like buttons. I can't use them as a touch screen anyway and I code not spent all my time on Facebook so they give me no benefit.

I want to get where I am going without Windows telling me how things should look and feel and Windows 8 is numptyfing the user interface even more so that old people can buy their wool with a few swipes but a coder has to hack about for ages to get anything working.

Anyway as I develop in PHP and C#, ASP I need to run both IIS and WAMP side by side on the same PC.

If you have read my earlier article on getting round issues with IIS blocking port 80 you will know that I like to change the port Apache uses to 8888 (a common HTTP alternative) so that I can run both web servers side by side without having to switch IIS off before switching WAMP on and vice versa.

Quick overview is to edit the httpd.conf file and change the listen port from 80 to 8888 e.g

Listen 8888

Also changing the line that mentions the server to:

ServerName localhost: 8888

You can read this article here. Troubleshooting WAMP Server on Windows 7

However with Windows 8 I found this didn't fix the problem and when I tried to access localhost:8888 I would get a Forbidden 403 status error code back when accessing index.php or phpmyadmin.php.

Apparently there are multiple solutions depending on what you want to do with your server.

As Windows 8 is an IPv6 Operating system and WAMP is IPv4 the loopback address is NOT 127.0.0.1 but ::1.

A simple ping to localhost in your command prompt will prove this as you will get back ::1 and not 127.0.0.1.

To fix this problem there are two solutions depending on whether you want to run IIS alongside WAMP. If you don't mind toggling between IIS and WAMP then follow Method 1. If not try Method 2.

Method 1

To get round the differences between WAMP being IPv4 and Windows 8 being IPv6 you need to edit some files.

Instead of changing your httpd.conf file as my previous article does you change the following line.

Listen 80

to

Listen 0.0.0.0:80

Then you would need to edit your hosts file in c:\windows\system32\drivers\etc to comment out the line ::1 localhost e.g:

# localhost name resolution is handled within DNS itself.
127.0.0.1       localhost
# ::1                   localhost

However this doesn't stop the issue of running WAMP alongside IIS.

Therefore if you do want both to run side by side without toggling them on/off ignore what I just said and instead follow method 2.

Method 2

Follow all my steps in the earlier article: Troubleshooting WAMP Server on Windows 7 and then you need to also change the following lines of code in your phpmyadmin.conf file which will be in your c:\wamp\alias\ folder to the following:

<Directory "c:/wamp/apps/phpmyadmin3.5.1/">
    Options Indexes FollowSymLinks MultiViews
    AllowOverride all
        Order Deny,Allow
 Deny from all
 Allow from all
</Directory>

And then in your C:\wamp\bin\apache\apache2.2.22\conf\httpd.conf file (or whatever version you are using) you need to have these lines of code.

Remove the allow from 127.0.0.1 and replace with allow from all like we did in the previous file.

<Directory />
    Options FollowSymLinks
    AllowOverride None
    Order deny,allow
    Allow from all
</Directory>

Now you should be able to access phpmyadmin with either http://localhost:8888/phpmyadmin/ or http://127.0.0.1:8888/phpmyadmin/ and still get your .NET or ASP classic code running with IIS from a simple http://localhost or http://127.0.0.1.

These two methods should sort you out if you get stuck like I did and I am sure when the next version of Windows comes out we will all have some more problems to solve to get WAMP running alongside IIS!

Troubleshooting WAMP Server Installation on Windows 7 machines

Troubleshooting WAMP Server installation on Windows 7 computers

I like to code in both PHP and .NET or ASP so on my laptop or PC I need to be able to run
both PHP code and .NET code.

As both languages use different web servers to run their code this means on Windows PC's I have been installing WAMP Server to run and test any PHP code. However ever since moving to a new laptop (Windows 7 64 bit) I found that getting WAMP Server to work was a right pain in the arse.

I had managed it on my Work PC (also Windows 7 64 bit) without any problems at all so I don't know why on a fresh install I had issues. Anyhow this is how I troubleshooted WAMP Server installation on a Windows 7 laptop.

After installing wampserver turn it on and start all service then try and access it e.g http://localhost

You will probably be met with an IIS 7 home screen and not the WAMP Server home screen you might have expected.

This is because windows PC's come with their own IIS web server which sits on the same port 80 that WAMP Server does.

First off try turning off IIS by going into Administrator Tools, open IIS and disable it.

Or you could go into Administrator Tools, open Services and turn off World Wide Web Service.

Then restart all the WAMP Server services and try http://localhost or http://127.0.0.1 again

If it works then you will be met with the WAMP Server Home page but if like me on Windows 7 you might be met with a 404 page that just says

Not Found


HTTP Error 404. The requested resource is not found.

So WAMP is still not running even though the other web server running on that port is off. So something else is blocking or listening that port.

You can try to find out what this is by doing the following:

Running Command Prompt (under administrator rights)

Typing in: netstat -b -o

You need the -o so that you can see the Process ID column as long as you remembered to run under admin privileges and you need the -b to see which program is running or creating the connection consuming the listening port in this case port 80 on 127.0.0.1 (localhost).

You will get something back like this

C:\windows\system32>netstat -b -o

Active Connections

  Proto  Local Address          Foreign Address        State           PID
  TCP    127.0.0.1:80           my_pc_name:49719       TIME_WAIT       0
  TCP    127.0.0.1:80           my_pc_name:49721       TIME_WAIT       0
  TCP    127.0.0.1:80           my_pc_name:49723       TIME_WAIT       0
  TCP    127.0.0.1:80           my_pc_name:49725       TIME_WAIT       0
  TCP    127.0.0.1:80           my_pc_name:49727       TIME_WAIT       0
  TCP    127.0.0.1:80           my_pc_name:49729       TIME_WAIT       0
  TCP    127.0.0.1:80           my_pc_name:49731       TIME_WAIT       0
  TCP    127.0.0.1:80           my_pc_name:49733       TIME_WAIT       0
  TCP    127.0.0.1:1110         my_pc_name:49718       TIME_WAIT       0
  TCP    127.0.0.1:1110         my_pc_name:49720       TIME_WAIT       0
  TCP    127.0.0.1:1110         my_pc_name:49722       TIME_WAIT       0
  TCP    127.0.0.1:1110         my_pc_name:49724       TIME_WAIT       0
  TCP    127.0.0.1:1110         my_pc_name:49726       TIME_WAIT       0
  TCP    127.0.0.1:1110         my_pc_name:49728       TIME_WAIT       0
  TCP    127.0.0.1:1110         my_pc_name:49730       TIME_WAIT       0
  TCP    127.0.0.1:1110         my_pc_name:49732       TIME_WAIT       0
  TCP    127.0.0.1:5354         my_pc_name:49155       ESTABLISHED     1368
 [mDNSResponder.exe]
  TCP    127.0.0.1:27015        my_pc_name:49441       ESTABLISHED     1128
 [AppleMobileDeviceService.exe]
  TCP    127.0.0.1:49155        my_pc_name:5354        ESTABLISHED     1128
 [AppleMobileDeviceService.exe]
  TCP    127.0.0.1:49441        my_pc_name:27015       ESTABLISHED     4768
 [iTunesHelper.exe]
  TCP    192.168.1.7:49619      ww-in-f125:5222        ESTABLISHED     4884
 [googletalk.exe]

Not very helpful as all the Proccess ID's for port 80 on 127.0.0.1 are set to 0.

If it's not set to 0 you can find out what process is listening on it by going to your Task Manager, ticking the "Show processes from all users" box and then searching for the PID (process ID).

If the PID column is not shown then go to View > Select Columns and tick the PID (Process Identifier)
option.

So because we are no nearer to working out what is using this port we are stuck as the other web server is disabled and not listening on port 80 but it seems nothing else is. A virus scan with a good tool helps here just in case something is running that shouldn't be.

As I haven't been able to find out what is using the port I have to now resort to a hack to get WAMP Server running.

This hack is actually useful if you want to be able to run WAMP and IIS on the same machine at the same time.

If you have done a default install then go to the httpd.conf ini file which should be at:

c:\wamp\bin\apache\Apache2.2.21\conf\httpd.conf

Then find the Listen option which will be under some comments near the top starting with

# Listen: Allows you to bind Apache to specific IP addresses

and change it to a port you want to use instead of 80 and one you know you are not using. I chose port 8888 a common HTTP alternative port.

So add this line in under the comments.

Listen 8888

This should get your WAMP Server default page up after a WAMP restart by accessing:

http://localhost:8888

However you should also change another directive in the file which identifies the servers name.

It starts with these comments

# ServerName gives the name and port that the server uses to identify itself.

So replace

ServerName localhost

with

ServerName localhost:8888

or if you chose a different port then use that instead of 8888.

You will now find that you can access WAMP Server from your PC okay and if you still want IIS to run alongside then it's a good idea to change the PORT of WAMP Server anyway to prevent having to keep turning IIS off (or permanently disabling it).

A bit of a pain to get going but it works for my 64 bit Windows 7 PC and I know someone else who had the same problem with their new PC. On my older XP laptops and PC's I had no problems at all getting WAMP working and it's only since moving to Windows 7 I have had this issue.

If anyone know what causes this issue then please let me know but please don't reply SKYPE or some other application could be the problem as we ruled that out with the netstat -p -o command prompt scan earlier.

Anyway this hack gets WAMP Server running and it is good to know how to debug the problem.