Installing Gearman is pretty easy as there are packages for it in Ubuntu:

sudo apt-get install gearman libgearman-dev

The development headers (libgearman-dev) are only required if you need to compile a library for your programming language such as a PHP extension. To install the PHP module you would run:

sudo pecl install channel://pecl.php.net/gearman-0.7.0

If you have trouble with the above step then it is probably because you are running an older version of Ubuntu. In this case take a look at my previous post Getting gearman to install on Ubuntu.

Moving onto mod_gearman_status, which is an Apache module to show the status of Jobs and their associated workers. It looks like the following:

Firstly lets ensure that the system has the build-essentials package installed:

sudo apt-get install build-essentials

We now need to install the Apache2 development headers, which assuming you installed the standard Ubuntu Apache2 package will be the prefork edition. If you don’t understand this then don’t worry you can just continue below.

sudo apt-get install apache2-prefork-dev

Now download modules C file and run the following command to build it and install it as an Apache module:

sudo apxs2 -c -i mod_gearman_status.c

Apache now needs to be told how to load the module and what configuration settings to use. In /etc/apache2/mods-available you need to create two files.

/etc/apache2/mods-available/gearman_status.load:

LoadModule gearman_status_module /usr/lib/apache2/modules/mod_gearman_status.so

/etc/apache2/mods-available/gearman_status.conf:

<IfModule mod_gearman_status.c>
    <Location /gearman-status>
        SetHandler gearman_status
    </Location>
</IfModule>

Enable the module with the following command:

sudo a2enmod gearman_status

Restart Apache to load the module:

sudo service apache2 restart

Now in your browser you can visit http://example.org/gearman-status where example.org is your servers address.

Introduction

When working in a team it is very useful to have a central web server with multiple environments and a configuration as close to the live server as possible. This can be a bit of a nightmare though if you need to setup a new VirtualHost container in Apache every time a new project is brought on or when a developer wants to work on a version of the site in their own environment.

The good news is that this can all be handled automatically and new sites can be setup by simply adding a new directory to the file system. There are at least two ways of getting this going; the first of which is the mod_vhost_alias module for Apache and the second is enabled via mod_rewrite. I prefer to use the second method as it is more flexible and it allows you tap into the ability of mod_rewrite to introduce environment variables and redirect requests (this is particularly useful for robots.txt - you’ll see).

The Apache2 Manual does have a very good page dedicated to overcoming this problem, but I will be sharing with you all the settings I am using which you will need to stop Google et. al. from crawling your sites served from the staging environment for example.

Read More

A project I am working on at the moment requires time delayed job queues and having found nothing yet that can manage it properly so I decided to wrap up `at` into a PHP class. This gives you simple methods to add, list and remove jobs from the `at` queue using object oriented code.

The code is very simple and I have documented it reasonably well so along with the examples you should get on your way quickly. The class can, of course, be used from the command line as well so if you want to write batch scripts in PHP to handle adding a collection of predefined `at` jobs for example - it can make it easier.

I feel the important features of `at` have been added into the code, but if you want to wrap any of the other functions then please do fork my code and make a patch or post a pull request. For more information on what `at` can do please either run `man at` in your console or visit the Edinburgh University’s man at page.

You can get the code from my repository on GitHub: PHP-at-Job-Queue-Wrapper.

If you do have any trouble getting the `at` daemon (atd) going then my previous post may help you debug it - please see If you are having problems getting Ubuntu atd running for more on that.

If you just cannot get atd to start running check the permissions on your /var/spool/cron/atjobs and /var/spool/cron/atspool directories. The should be `chmod 770` and then `chmod +t`. Also they should be owned by `daemon:daemon`.

I was getting this error when attempting to set new at jobs:

Can’t open /var/run/atd.pid to signal atd. No atd running?

If I tried to start the daemon through the service management I was getting:

sudo service atd start
start: Job is already running: atd

But running `ps -elf | grep ‘atd’` yielded no appropriate results.

When trying to start the atd daemon manually on the command line I was getting:

simon@forge:~$ sudo atd -d
Cannot change to /var/spool/cron/atjobs: Permission denied

Which finally gave me the clue!

Sun VirtualBox Logo

Often Linux just does it better!  Often I find myself developing a Windows machine without access to a Linux development server, but I still need to access to some of the Linux binaries and features such as cron jobs, the at command and binaries such as imagemagick, pdftotext, etc.  Some things can be emulated with ported binaries or through Cygwin, but I feel a lot more comfortable developing on a platform that is representative of the live server the web site will run on.

I have not removed XAMPP as it is very useful for developing small scripts or sites without the overhead of running a virtual machine.  Therein lies the problem with this solution – it is virtual machine based and that will mean your local development machine will suffer when the virtual machine steals computing time from the CPU or more memory.  So whilst I can use XAMPP on my netbook I am not so sure a VM will run smoothly.

Setting up a new development environment

1. Download VirtualBox 2.1 from http://www.virtualbox.org
2. Download the latest Ubuntu from http://www.ubuntu.com
   Your choice of server or desktop. I chose desktop to give me another web browser testing environment.
3. Install VirtualBox and open it up
   1. Click create new VM (Virtual Machine)
   2. Give your VM name (I called mine Development)
   3. From the drop downs choose
      1. Linux
      2. Ubuntu (only choose Ubuntu 64 bit if you are running 64 bit Windows)
   4. Choose how much of your machines physical memory the virtual machine is allowed to steal. I chose 512MB as I am running the desktop version of Ubuntu.
   5. You will need to create a new virtual hard disk – a new hard disk wizard will open
      1. Choose dynamically expanding storage
      2. Choose
         1. Give the virtual hard disk a name (mine is called Development just like my virtual machine)
         2. The location of the storage on your hard drive
4. Now your new virtual machine should be setup so right click on it and choose Start
   1. The Virtual Machine will pop up a message asking you to go through its first run wizard
   2. Choose your installation location
      1. CD/DVD ROM
      2. Image file – point this to the location of your Ubuntu installation ISO you downloaded and saved earlier
   3. This will then mount your CD image and boot from it
5. Install Ubuntu by going through its installation wizard which is very easy and does not require detail here (if you have chosen the desktop version this will be even easier because you can use the mouse!)
6. Once you have the virtual machine setup you will want the nice Linux drivers so you can have a larger screen size and for networking etc. For this you need to install the Virtual Guest Additions.
   1. Mount a new CD drive/ISO Image in your virtual machine whilst it is running from the top menu item called Devices
   2. Add the new image which you will find in your VirtualBox program folder
7. Back to your virtual machine and open up a terminal window
   1. In your terminal navigate to the CD Rom drive
   2. Execute the following command sudo sh ./VBoxLinuxAdditions-x86.run
   3. Once complete you need to reboot the machine so execute sudo reboot
8. Once the machine reboots get back into the terminal and run the following commands
   1. sudo apt-get update
   2. sudo apt-get upgrade
   3. sudo apt-get install apache2 mysql-server-5.0 php5 php5-xdebug
   4. Just to be on the safe side restart apache with sudo /etc/init.d/apache2 restart
   5. Try accessing your web server by opening a web browser in the virtual machine and typing in localhost if you chose the desktop and you should see a message like It Works!
9. We need to be able to see folders on our host machine inside the development machine quickly and easy so from the top menu bar choose Devices -> Shared Folders
   1. Add a new folder
   2. Tell it which folder to look in
   3. Give it a memorable name
   4. Tick make permanent
10. Jump back into the virtual machine and execute the following commands to add your shared folder to the virtual machine, in a terminal
    1. sudo mkdir /mnt/yourFolderNameHere (mine is /mnt/htdocs/)
    2. sudo mount -t vboxsf memorableFolderName /mnt/yourFolderNameHere
       1. Replace memorableFolderName with name you set in step 9.III
       2. Replace /mnt/yourFolderNameHere with the folder you made in step 10.I
    3. If you navigate to /mnt/yourFolderNameHere and execute ls you should see a list of the files on your host systems shared folder
    4. If that worked then we want to add our folder to the fstab file so that the mount point is loaded every time our virtual machine boots up
       1. sudo vim /etc/fstab
       2. Add the following line to the bottom of the file:
          memorableFolderName /mnt/yourFolderNameHere vboxsf defaults 0 0
          1. Replace memorableFolderName with name you set in step 9.III
          2. Replace /mnt/yourFolderNameHere with the folder you made in step 10.I
11. Now to make this development machine available over the network shut the virtual machine down by going to top menu bar Machine -> Close and choosing Power Off
    1. Right click on the virtual machine and choose settings
    2. From the left hand menu choose Network
       1. Tick Enable Adapter
       2. Adapter Type: Intel PRO/1000 T Server
       3. Attached to: Host Interface
       4. Tick Cable Connected
       5. Choose the interface on your host machine that gives you access to the outside world. You can find out which one this is by looking in your control panel.
    3. Save the settings and start your virtual machine back up again
    4. To find out the IP address of your virtual machine open a terminal execute ifconfig
       
12. You should now be able to visit your web server from your host machine by entering that IP address into your web browser.
    

Configuring our web server

1. We need to enable mod_rewrite which involves making a simple symbolic link
   1. Navigate to /etc/apache2/mods-enabled
   2. Execute sudo ln -s /etc/apache2/mods-available/rewrite.load rewrite.load
2. Reboot the server sudo /etc/init.d/apache2 restart and test it is function ok.

Setting up Mass Virtual hosts

The idea behind this step is to minimise the time it takes to setup a new host on the server. Basically all you have to do is create a new directory an away you go. (Don’t forget it will still need to appear in your hosts file).

1. We need to enable mod_vhost_alias
   1. Navigate to /etc/apache2/mods-enabled
   2. Execute sudo ln -s /etc/apache2/mods-available/vhost_alias.load vhost_alias.load
2. Now to setup the rules it will create virtual hosts by
   1. Execute sudo vim /etc/apache2/apache2.conf
   2. Go to the base of the file and add the following rules:

      http://gist.github.com/294736

      VirtualDocumentRoot sets the directory that the VHosts public directory is contained in. This will basically convert http://subdomain.localhost/ to /mnt/htdocs/subdomain and pull the relevant files through.

3. Now restart the apache process sudo /etc/init.d/apache2 restart

A side note when using this method that you should be aware of. This will affect your rewrite rules if they are placed into a .htaccess file. To avoid any problems always declare the RewriteBase rule in your .htaccess.

For example:

http://gist.github.com/294737

VirtualBox Shared Folders Permissions

Nathan brought a glaring omission from my post to the fore – thanks!  Currently you maybe having permissions issues with your shared folders because they might be being mounted as root:root.  To get them to load with a specified user and group you will need to edit your /etc/fstab file again and change ‘defaults‘ to be ‘uid=username,gid=groupname‘ – an example would be ‘uid=simon,gid=www-data‘.

A full line example would be from:
htdocs /mnt/htdocs vboxsf defaults 0 0
to
htdocs /mnt/htdocs vboxsf uid=simon,gid=www-data 0 0

Apache HTTP Server

I really like the way the Apache modules and virtualhosts are seperated out on Debian into folders containing those, which are available and those which are enabled. There is one small problem with this – it is more work than before! Luckily there are some helper scripts.

The Apache configuration files are layed out in the following way:

mods-available – the actual text files containing the modules configuration
sites-available – the vhosts text file for the site

mods-enabled – a symlink to the actual text file in mods-available
sites-enabled – a symlink to the actual text file in sites-available

Anything listed in the enabled directories will be loaded when Apache is therefore enabling the respective site or module. Manually symlinking these up can be a right pain so I use the following scripts to assist me:

“a2ensite sitename” – will create the symlink for you
“a2dissite sitename” – will remove the symlink for you
“a2enmod modulename” – will create the symlink for you
“a2dismod modulename” – will remove the symlink for you

When the symlink is place the module or site will be loaded and vice versa.

Do not forget that you still need to reload the configurations into Apache by running “/etc/init.d/apache2 reload”.

Certificates are a useful way of restricting access to your SSH server because a user must have three things to log onto the server:

1. Username
2. Password
3. Certificate

Normally they would only need to have a password and username, which can be guess at or (potentially) brute forced. Forcing the user to supply a certificate on log on means that they must also have a tangible source of identification (without the key file they cannot log in!).

Creating the key pair

Log onto the server and run ssh-keygen and you will get asked a few questions as follows (enter a passphrase):

user@host$ sudo ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/user/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/user/.ssh/id_rsa.
Your public key has been saved in /home/user/.ssh/id_rsa.pub.
The key fingerprint is:
95:60:c2:31:2e:94:cf:66:b6:fa:8b:b8:45:6c:dd:22 user@server
The key’s randomart image is:
+–[ RSA 2048]—-+
| .o+.o |
| ….+ . . |
| .o. o |
| . o*. . |
| E+o.S |
| o … |
| .. |
| o.. |
| o…o. |
+—————–+

This will generate two files:

1. id_rsa
2. id_rsa.pub

in your home directory (if you chose the defaults). They are your private and public keys respectively. The public key is the one that goes on your server and the private key is the one you use when logging into the server.

Add the public key

Now on the server run

user@server:~$ cat id_rsa.pub » ~/.ssh/authorized_keys
user@server:~$ rm id_rsa

which adds the public key to the list of authorized keys for this user.

Activate the key based authentication on the server

To edit the config run

user@server:~$ sudo vim /etc/ssh/sshd_config

Ensure that you have the following lines uncommented and set correctly in your configuration file:

RSAAuthentication yes
PubkeyAuthentication yes

If you wish to lock non-certified logins from the server then also ensure you activate the following settings:

ChallengeResponseAuthentication no
PasswordAuthentication no
UsePAM no

Now reload the SSH configuration to get the new settings going:

user@server:~$ sudo /etc/init.d/ssh reload

Conversion to PuTTY

PuTTY has its own private key format and the private key you created earlier now needs to be converted, which is a very simple process. Firstly you need to download the key from the server and save it to your computer.

1. Now open PuTTYgen, which comes when you install PuTTY (look in the PuTTY program files directory if there is no shortcut in the start menu).
2. Click the “Load” button and point it to the private key we downloaded earlier (depending on the file extension you gave the key file you may need to adjust the file extension filter on the PuTTY load key dialogue)
3. It will ask you for the passphrase you set when you were generating the certificates – enter it!
4. PuTTYgen should now tell you that you it has successfully imported the certificate – click ok.
5. Click the “Save private key” button and save it

Getting this to work with PuTTY

Now the new key we just saved is compatible with PuTTY we can start a new PuTTY session as usual, but don’t forget to tell PuTTY where the key file is located by looking in the Category tree menu and clicking on Connection -> SSH -> Auth. You can now click the “Browse” button and point PuTTY to the key file you just created.

Click open and a new session will load:

1. Enter your username as normal
2. When prompted; give the passphrase you gave when creating your key (do not make the mistake of using your linux user account password as it won’t work!)