The Mysterious Random 404

I’ve been wrangling with these mysterious “Error 404″ messages that I “randomly” get whenever I post an article on this blog. For instance, while editing the article [PHP4 and PHP5 Side-by-Side Installation on Breezy][php4php5], I was continuously plagued with this error. It turns out, after a lot of trial and error, that the problem is caused [by mod_security][mod_security]. After discovering this, I went out and [posted][wp_forum_post] on the WordPress forums. It turns out, a lot of WordPress users seem to be having this problem.

I posted on the forum about 3 months ago and eventually, I forgot about it. Until it bit me in the ass again today. I tried to update [PHP4 and PHP5 Side-by-Side Installation on Breezy][php4php5] today to point out that it also works under “Dapper”. There it was again — Error 404. The funny thing is, if I saved the unmodified article, I still get the Error 404. It was only about 3 months ago that I posted it, and it passed mod_security validation. So through the same trial and error process described below, I was able to update the article.

mod_security

mod_security works by intercepting requests to the webserver, be it a GET request or a POST request. When you post an article to WordPress by clicking “Save and Continue Editing”, “Save”, or “Publish”, a POST request is made to your server. This request is then intercepted by mod_security which does some analysis of the POST’ed data. Based on filtering rules set forth by your hosting provider, your post will either pass or fail to validate. If your POSTed data fails to validate, the default action would be to generate a 404 error.

Working Around mod_security

Temporarily Disabling mod_security

The best way to work around mod_security is to disable it while you are posting and enable it when you are done posting. If you are under virtual hosting like myself, you would be lucky if you are allowed to turn off mod_security via .htaccess. To check out if you are able to turn off mod_security this way, add^[1][note1] the following lines to your .htaccess file:

<ifmodule mod_security.c>
SecFilterEngine Off
SecFilterPost Off
</ifmodule>

Upload your update .htaccess file back to your server. Then check if your configuration worked by visiting your home page. If you see your home page, then the above configuration works. Now you should be able to post anything without it being intercepted by mod_security. If the above configuration does not work (i.e., you get an “Internal Server Error” instead), you might want to try the next one. Remove the above configuration from your .htaccess and replace it with the following:

<ifmodule mod_env.c>
SetEnv MODSEC_ENABLE Off
PassEnv MODSEC_ENABLE
</ifmodule>

Upload your updated .htaccess file. Then visit your home page and make sure that it does not generate an “Internal Server Error”. Most of the time, the SetEnv directive does not cause an “Internal Server Error”. It only requires FileInfo permissions to use it in your .htaccess and most hosting providers allow FileInfo by default. Try posting the problematic post (i.e., the one that causes the 404 error) and see if it works. If it does not work^[2][note2], then you are out of luck and must resort to the “infernal HTML entities work-around”.

Let me stress the point that if either one of the the above configuration settings work for you, you should not make them permanent. Your hosting provider installed mod_security to protect the sites that they are hosting. If you leave mod_security disabled and your site gets broken into it will be your fault!

Infernal HTML Entities

So you’ve tried disabling mod_security and it did not work, what do you do now? Well, you can try working around mod_security by “fooling” it. As of mod_security version 1.9.x, it seems that HTML entities will pass through just fine. However, before you think of encoding your entire post into HTML entities you should keep in mind that mod_security filters only certain word combinations. You can ask your hosting provider what rules they use for mod_security, but I doubt that they will give you that information. If they do give it to you, you will need to decipher them. See the mod_security documentation for that.

Unless you are able to acquire the mod_security rules from your hosting provider and you are able to decipher them, your only option would be to discover those rules by trial and error. First save your troublesome article text into a file. Then post the contents of the file paragraph by paragraph (i.e., post the first paragraph, then the second one, so on…). When you encounter the 404 error on a certain paragraph, stop right there and post the first sentence of the offending paragraph. Continue posting the sentences of the offending paragraph until you hit the sentence that causes a 404 error. From there, you should post the sentence word by word until you find the word that causes the 404 error.

When you find the word that causes the 404 error, modify the word by encoding any of the characters in the word using HTML entitiesm you can use [this table][htmlentities] for reference. For example, I’ve found that through trial and error, my post triggers the 404 error when the word “curl” appears in the text along with other words. Using the table [here][htmlentities], I encode ‘c’ into its HTML entity like so:

curl

If you look up the character for the decimal value 99 in the [table][htmlentities], you will find that it’s the character ‘c’. Now, try posting the encoded text. You should now be able to post it without problems and you can continue with your trial and error or you can try posting the remainder of your text file after the encoded word and see if it gets posted without errors.

That’s it. Good luck and happy hunting!

NOTES:

¹ If you are using “pretty URLs” for WordPress, make sure you download your current .htaccess file and add the mod_security directives to it.

² In my case, using SetEnv did not work. I think it has something to do with my PHP installation. On the server hosting this blog, PHP is installed as a CGI binary. Even PassEnv does not seem to work.

[php4php5]: http://abing.gotdns.com/2006/03/14/9/ “Permanent link to PHP4 and PHP5 Side-by-Side Installation on Breezy” [mod_security]: http://www.modsecurity.org/projects/modsecurity/apache/index.html “ModSecurity – ModSecurity for Apache” [wp_forum_post]: http://wordpress.org/support/topic/61431/page/3 “WordPress Support Forums: 404 after save or publish a post” [note1] #note1 “Note 1″ [note2] #note2 “Note 2″ [htmlentities]: http://www.utoronto.ca/webdocs/HTMLdocs/NewHTML/iso_table.html “ISO 8859-1 (Latin-1) Characters List”

a

Update July 6, 2006: This tutorial has been tested and is known to work under Ubuntu Linux 6.06 LTS (aka “Dapper Drake”).

Under Ubuntu (or Debian), there are three options:

1. PHP is installed as an Apache (or Apache2) module

2. PHP is installed as a CGI binary

3. PHP is installed as a CLI binary

The first two installation options are for Web-based applications. The third option is used if you want to use PHP as a scripting language for command line programs.

With a side-by-side installation, you can install one version of PHP as an Apache module and the other version as a CGI binary. If you like, you can also install them both as CGI binaries. It requires a bit more work (at the moment) to have them both installed as an Apache module since you would have to forgo your distribution’s package management system and install PHP4 and PHP5 from source.

Your choice if installation is entirely up to you. You can choose from one of the following setups:

A. PHP4 is installed as an Apache module; PHP5 is installed as a CGI binary

B. PHP5 is installed as an Apache module; PHP4 is installed as a CGI binary

C. both PHP4 and PHP5 are both installed installed as CGI binaries

The easiest route to take is to install these packages using apt-get. For my setup, I chose setup A: PHP4 as an Apache module and PHP5 as a CGI binary. This setup is ideal for virtual hosting environments where you want your customers to have PHP4 as the default PHP version and PHP5 as an available alternative.

To begin, install PHP4 as an Apache (or Apache2) module:

Apache 1.3.x:

$ sudo apt-get install libapache-mod-php4

Apache 2.0.x:

$ sudo apt-get install libapache2-mod-php4

That should install the core PHP4, which is next to useless without some extensions:

$ sudo apt-get install php4-curl php4-domxml php4-gd php4-mcrypt php4-mhash php4-mysql php4-pgsql php4-sqlite php4-xslt

Once you have installed all the modules you need, check that you have Apache or Apache2 up and running and verify that your configuration was updated:

Apache 1.3.x:

$ sudo kill -0 `cat /var/run/apache.pid` && echo "Apache is running."

Apache 2.0.x:

$ sudo kill -0 `cat /var/run/apache2.pid` && echo "Apache is running."

You should see the message “Apache is running”. If you see an error message instead, then you should try to start Apache first:

Apache 1.3.x:

$ sudo invoke-rc.d apache start

Apache 2.0.x:

$ sudo invoke-rc.d apache2 start

Once you have Apache up and running, check to see if you have successfully configured PHP4, you can use the HEAD Perl program from libwww-perl:

$ HEAD http://localhost/ | grep PHP
Server: Apache/2.0.54 (Ubuntu) mod_fastcgi/2.4.2 PHP/4.4.0-3ubuntu2 mod_ssl/2.0.54 OpenSSL/0.9.7g mod_perl/2.0.1 Perl/v5.8.7

Or, you can use the curl client from the curl package:

$ curl --head http://localhost/ | grep PHP
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
Server: Apache/2.0.54 (Ubuntu) mod_fastcgi/2.4.2 PHP/4.4.0-3ubuntu2 mod_ssl/2.0.54 OpenSSL/0.9.7g mod_perl/2.0.1 Perl/v5.8.7

If you see output similar to the ones above (contains the text PHP) then your setup is now working properly.

Next step is to install PHP5 as a CGI binary:

$ sudo apt-get install php5-cgi

This will install all the packages needed to have a core PHP5 installation. Again this would be next to useless without the extensions:

$ sudo apt-get install php5-curl php5-gd php5-mhash php5-mysql php5-pgsql php5-sqlite php5-xmlrpc php5-xsl

Once you are done installing all the extensions you need, it’s time to configure your Apache webserver:

Apache 1.3.x:

$ sudo gedit /etc/apache/sites-enabled/default

Apache 2.0.x:

$ sudo gedit /etc/apache2/sites-enabled/default

Add the following just before the line that says </VirtualHost>:

Action php5-script /cgi-bin/php5
AddHandler php5-script .php5

This will define a new content handler and default action for files with a .php5 extension. This directive will apply to your default virtual host.

Then restart Apache:

Apache 1.3.x:

$ sudo invoke-rc.d apache restart

Apache 2.0.x:

$ sudo invoke-rc.d apache2 restart

Try it out:

Apache 1.3.x:

sudo gedit /var/www/apache-default/phpinfo.php5

Apache 2.0.x:

sudo gedit /var/www/apache2-default/phpinfo.php5

Enter the following lines and save the resulting document:

<?php
phpinfo();
?>

Then with your browser, open the URL http://localhost/phpinfo.php5:

$ sensible-browser http://localhost/phpinfo.php5

If you see the auto-generated phpinfo page, then congratulations! You now have PHP4 and PHP5 installed on your server.

If you want to use setup B, just exchange php4 and php5 in the installation commands above and when you setup the Action and AddHandler directive, you should use .php4 instead of .php5.

If you want to use setup B, do not install PHP4 as an Apache module as instructed above. Instead install the CGI binary:

$ sudo apt-get install php4-cgi

And then proceed to install the PHP4 modules. Then setup the Action and AddHandler directives for PHP4:

$ sudo gedit /etc/apache2/sites-enabled/default

Add the following just before the line that says :

Action php4-script /cgi-bin/php5
AddHandler php4-script .php4

If you want the server to handle the .php extension as PHP4, add another line:

AddHandler php4-script .php

Restart Apache:

Apache 1.3.x:

$ sudo invoke-rc.d apache restart

Apache 2.0.x:

$ sudo invoke-rc.d apache2 restart

PHP CGI Binary Installation and suExec with Apache2

One of the great things when you have setup PHP to run as a CGI binary with Apache 2 is that you get the capability to run it under suExec. What this means is that if your script requires write privileges to a directory under the document root, you no longer have to make that directory world writeable. If you use Smarty with caching enabled, then you are probably familiar with this requirement.

suExec should already be installed when you installed Apache2, since it comes in the apache2-common package. All you need to do is enable it:

$ cd /etc/apache2/mods-enabled
$ sudo ln -s /etc/apache2/mods-available/suexec.load .

Then start and stop Apache2 (force reload may not be enough to enable it):

$ sudo invoke-rc.d apache2 stop
$ sudo invoke-rc.d apache2 start

Check that Apache2 has indeed loaded suExec:

$ cat /var/log/apache2/error.log|grep suexec
[Sat Dec 24 15:42:14 2005] [notice] suEXEC mechanism enabled (wrapper: /usr/lib/apache2/suexec2)

You should see something like the output above to indicate that suExec is now properly installed. If not, check the output of the following command:

$ ls -l /usr/lib/apache2/suexec2
-rwsr-x---  1 root www-data 10332 2005-12-06 00:36 /usr/lib/apache2/suexec2

Note that the setuid bit must be set as above. If it is not, then:

$ sudo chmod +s /usr/lib/apache2/suexec2

And stop and start Apache2 again. Check that you have suExec loaded and if it is, then proceed with the following:

$ cd /var/www
$ sudo cp /usr/lib/cgi-bin -r
$ sudo chown www-data:www-data cgi-bin -R
$ sudo chown www-data:www-data apache2-default -R
$ sudo gedit /etc/apache2/sites-enabled/default

Insert the following lines before the line that says </VirtualHost>:

<IfModule mod_suexec.c>
    SuexecUserGroup www-data www-data
</IfModule>

Then restart Apache2:

$ sudo invoke-rc.d apache2 restart

To test your new suExec setup, try the following:

$ sudo gedit /var/www/apache2-default/suexectest.php5

Enter the following and save the resulting file:

<?php
$fp = fopen('example.txt', 'a+');

$hello = "Hello World!\n";

fwrite($fp, $hello, strlen($hello));

fclose($fp);
readfile('example.txt');
?>

Then on the command line enter the following:

$ sudo chown www-data:www-data /var/www/apache2-default/suexectest.php5
$ sensible-browser http://localhost/suexectest.php5

You should see the phrase Hello World! in your browser window to confirm that suExec works.

Note that the following directives are used in a container2:

SuexecUserGroup
Action
AddHandler

1. The virtual host document root and cgi-bin must be under /var/www/

/var/www/customer/html and /var/www/customer/cgi-bin

1. The owner and group of the document root and cgi-bin directories must be the same as the owner and group you set with SuexecUserGroup for that virtual host.

   e.g.: If you have SuexecUserGroup ucustomer1 gcustomer1, then:

$ sudo chown ucustomer1:gcustomer1 /var/www/customer -R

NOTES:

¹ Article is already in progress. I will post it here as soon as I find the time (and energy) to finish it.

² You can also have the Action and AddHandler directives in your .htaccess file to provide per-directory configuration. However, your virtual host must be configured to AllowOverride at least FileInfo.

a