Most of our development is done on Vista desktops (although I also run Ubuntu of course) and we wanted a web interface for easy browsing of code so I needed to use a pervasive and well supported VCS. After much hunting around and looking at Bazaar, Mercurial and git I decided to go with the latter due to its support in the community.  There are still some reservations I have against git, which work well for Linux kernal development but not our day to day web development – the major one being that subversion would allow you to commit and update on a file by file basis where as git versions the entire repository at once as far as I am aware.

Anyway now I have an Ubuntu server that I am using for storing the repositories and as a pretty frontend I have installed InDefero which also allows us to manage tickets.  InDefero is much like the code.google.com forge to manage and look at it so it is very simple.  I like Trac but I was not happy with the existing integrations with git.

It was not all plain sailing with the InDefero install as I had to make some very minor changes in the git plugin’s core code (the git plugin comes as part of InDefero and does not need to be installed seperately).  Basically the plugin was creating new repositories with very strict permissions which meant that the neither my backup user nor the apache process user could access them.  What this meant from the frontend was that the repository statistics (disk usage) were not displayed because it could not stat the directories and my back up process could not open them to make backups.

This problem is caused by the use of permissions setting in a PHP mkdir() command in the src/IDF/Plugin/SyncGit/Serve.php on lines 94 and 193.  You will want to increase the file permission octal just enough for your other users to gain access.  It is also worth mentioning that on line 111 there is a umask() command which you might also wish to tweak to improve your access.  Also don’t forget to update all $pluf_path values in the files in the scripts directory.  The rest of the way you should be fine just following the included documentation in doc/syncgit.mdtext

So far I am enjoying using InDefero though I think I will need to hack it so that when you add a new repository/project all the users names are placed in the project members as default to save vast amounts of copying and pasting.  Our development team is not large enough to warrant only giving certain people access to certain repositories/projects.

[UPDATE] I have made a small (and nasty) hack to the source of InDefero to cause the members text area when creating a new project to be pre-filled with my member’s names.  This involved changing /src/IDF/Form/Admin/ProjectCreate.php on line 107 by setting the array element initial to the value you want to appear in the text area.

For our desktop access to git we are using msysGit, PuTTY Plink and TortoiseGit.  This combination is working well for us so far and means that even non-technical users can begin versioning their documents and push them to a central repository for backup purposes.

I use most of these commands every day to simplify my terminal interactions with an Ubuntu development box.  This is more of a personal reference but thought I would share incase you find it useful.

Memcached and APC are two tools that you can install on your server and gain almost instant gratification!  APC basically caches executions that you send to a PHP process so that the next time you ask the parser to run your script it only has to look for some pre-chewed opcode in memory rather than parsing your PHP from the disk.  APC also has another feature up its sleeve, memory object caching, which allows you to store objects such as results from a database table in memory.  Memcached works in much the same way, but it can be used as a session handler as well as a persistent store for object data.  Another advantage memcached has over APC is that it can be distributed so you could have a number of servers all maintaining memcached daemons to distribute the load.

APC is faster as a straight PHP call on the memory cache, but you can only access the memory cache from the local PHP process.  I have not tested it but I am unsure if that means you can access it from CLI PHP as well as via PHP running on Apache.  Often I will have a PHP frontend with any server scripts (for example cronjobs) written in Python so it is very handy to be able to access the cache from a central location, which with memcached is possible but not with APC as it is PHP specific.  For more information on performance and advice on when various caching methods may be more useful Peter over at MySQLPerformanceBlog has written an interest article Cache Performance Comparison.

So for code that I know will only ever be used by the local PHP process I store the object in the APC memory object store, but if I need portability or the server is under high load and I want to separate out my object cache server from my script server then I use memcached.

I have written two articles detailing the steps involved in installing both the binaries on RedHat Enterprise Linux and Ubuntu or Debian.  Currently the live servers I use are either running CentOS or RedHat and the local development server is very nice setup based upon Ubuntu.

The MySQL documentation contains a very nice set of examples and documentation on using memcached to reduce hits on the database, which includes examples in a few languages including Python and PHP.  There is also some very handy hints to be gleened from An Introduction to memcached by Jeremy Ashcraft a.k.a MrSpooky over at Search-This.

Installing APC on Debian or Ubuntu is as simple as:

user@server:/directory/$ sudo apt-get install php-apc

Now let us reboot the Apache process to enable our new cache:

user@server:/directory/$ sudo /etc/init.d/apache2 restart

APC should now be ready to run on your server.  Try running the following command to verify it is setup; you should get something in response like mine:

user@server:/directory/$ php -r ‘phpinfo();’ | grep ‘apc’
apc
MMAP File Mask => /tmp/apc.s5jA6w
apc.cache_by_default => On => On
apc.coredump_unmap => Off => Off
apc.enable_cli => On => On
apc.enabled => On => On

<…SNIP…>

Now lets move onto installing Memcached, which again is very simple:

user@server:/directory/$ sudo apt-get install memcached
user@server:/directory/$ /etc/init.d/memcached start

The PHP Memcached module can be installed through Apt-Get as well:

user@server:/directory/$ sudo apt-get install php5-memcache

Now to configure PHP to use Memcached to store the session information we need to edit our /etc/php5/apache2/php.ini file and find the lines like the following:

session.save_handler = files
;session.save_path =

and change them so that they now look like this:

session.save_handler = memcache
session.save_path = “tcp://localhost:11211?persistent=1&weight=1&timeout=1&retry_interval=15″

That just leaves us to restart the Apache2 process:

user@server:/directory/$ /etc/init.d/apache2 restart

You are now up and running with Memcached PHP sessions and APC served PHP.

So far I have tried out Ubuntu, eeeBuntu, EasyPeasy Linux and CrunchBang Linux (all of which are Debian based) on the Asus Eee PC.  Ubuntu was a bit of an annoyance to get setup and it was troublesome trying to get all the buttons or the sound and microphone to work so I then tried EasyPeasy.  It was easy to install (it already contains the Array.org kernal) and I liked the NBR interface with its easy to use tabbed system not to mention that after using Ubuntu it was nice to be able to see most of the system dialogues on screen (Ubuntu’s dialogues were so long they fell off screen! – Hint: hold down alt and click to move dialogues without titlebars).  The major problem I had with it was that it took way too long to boot and came preloaded with a whole host of applications I will never need.

So after some research I saw a lot of good things being said about eeeBuntu, which again in the version I chose uses the NBR interface.  Also whilst it was very nice to use not everything worked, which was very annoying.  The microphone and some of the shortcut buttons did not work, which is useless if you need to use Skype.  The selection of preloaded applications was also not to my tastes (although better than EasyPeasy) and boot time was also quite painful although not as bad as EasyPeasy.  Speaking of boot times make sure you remove any SD cards from the onboard slot before booting the machine as this will add significantly to the load time of the OS.

After using eeeBuntu for some time I finally grew tired of the lag and decided it was time to do some more research into the available packages for easy installing on the Eee PC.  This is when I came across CrunchBang (#!) Linux, which is a very light weight version of Ubuntu running the Openbox shell.  This distro boots quickly, comes with a nice set of applications that I can easily build upon.  I am really pleased with the OS so far and I am even writing this post from within it.

Installation is very simple just use UNetbootin to transfer the Cruncheee image to a USB key and boot from the USB key to try out the ‘live disc’ version of the OS to see if it is for you.  If you like it then right click on the desktop and choose install from the resultant menu.

Not everything will work right of the box though.  The microphone does not record immediately and requires some tweeking to the Volume Control interface.  You can bring up the interface by pressing the shortcut Super + v (in my case the Super key is emblazoned with the Windows logo).  Now click the preferences button at the bottom of the dialogue and check the boxes beside Front Mic Boost, Capture and Input Source.  Now in the playback tab unmute Front Mic and raise the volume, on the recording tab unmute the microphone icon and raise the volume and finally on the options tab choose Front Mic from the drop down.

Unfortunately there is one further annoyance.  The volume buttons do not bring up the on screen display but Super + v will bring up the mixer which is fine for me at the moment.

Sun VirtualBox Logo

Often Linux just does it better!  Often I find myself developing a Windows machine without access to a Linux development server, but I still need to access to some of the Linux binaries and features such as cron jobs, the at command and binaries such as imagemagick, pdftotext, etc.  Some things can be emulated with ported binaries or through Cygwin, but I feel a lot more comfortable developing on a platform that is representative of the live server the web site will run on.

I have not removed XAMPP as it is very useful for developing small scripts or sites without the overhead of running a virtual machine.  Therein lies the problem with this solution – it is virtual machine based and that will mean your local development machine will suffer when the virtual machine steals computing time from the CPU or more memory.  So whilst I can use XAMPP on my netbook I am not so sure a VM will run smoothly.

Setting up a new development environment

1. Download VirtualBox 2.1 from http://www.virtualbox.org
2. Download the latest Ubuntu from http://www.ubuntu.com
   Your choice of server or desktop. I chose desktop to give me another web browser testing environment.
3. Install VirtualBox and open it up
   1. Click create new VM (Virtual Machine)
   2. Give your VM name (I called mine Development)
   3. From the drop downs choose
      1. Linux
      2. Ubuntu (only choose Ubuntu 64 bit if you are running 64 bit Windows)
   4. Choose how much of your machines physical memory the virtual machine is allowed to steal. I chose 512MB as I am running the desktop version of Ubuntu.
   5. You will need to create a new virtual hard disk – a new hard disk wizard will open
      1. Choose dynamically expanding storage
      2. Choose
         1. Give the virtual hard disk a name (mine is called Development just like my virtual machine)
         2. The location of the storage on your hard drive
4. Now your new virtual machine should be setup so right click on it and choose Start
   1. The Virtual Machine will pop up a message asking you to go through its first run wizard
   2. Choose your installation location
      1. CD/DVD ROM
      2. Image file – point this to the location of your Ubuntu installation ISO you downloaded and saved earlier
   3. This will then mount your CD image and boot from it
5. Install Ubuntu by going through its installation wizard which is very easy and does not require detail here (if you have chosen the desktop version this will be even easier because you can use the mouse!)
6. Once you have the virtual machine setup you will want the nice Linux drivers so you can have a larger screen size and for networking etc. For this you need to install the Virtual Guest Additions.
   1. Mount a new CD drive/ISO Image in your virtual machine whilst it is running from the top menu item called Devices
   2. Add the new image which you will find in your VirtualBox program folder
7. Back to your virtual machine and open up a terminal window
   1. In your terminal navigate to the CD Rom drive
   2. Execute the following command sudo sh ./VBoxLinuxAdditions-x86.run
   3. Once complete you need to reboot the machine so execute sudo reboot
8. Once the machine reboots get back into the terminal and run the following commands
   1. sudo apt-get update
   2. sudo apt-get upgrade
   3. sudo apt-get install apache2 mysql-server-5.0 php5 php5-xdebug
   4. Just to be on the safe side restart apache with sudo /etc/init.d/apache2 restart
   5. Try accessing your web server by opening a web browser in the virtual machine and typing in localhost if you chose the desktop and you should see a message like It Works!
9. We need to be able to see folders on our host machine inside the development machine quickly and easy so from the top menu bar choose Devices -> Shared Folders
   1. Add a new folder
   2. Tell it which folder to look in
   3. Give it a memorable name
   4. Tick make permanent
10. Jump back into the virtual machine and execute the following commands to add your shared folder to the virtual machine, in a terminal
    1. sudo mkdir /mnt/yourFolderNameHere (mine is /mnt/htdocs/)
    2. sudo mount -t vboxsf memorableFolderName /mnt/yourFolderNameHere
       1. Replace memorableFolderName with name you set in step 9.III
       2. Replace /mnt/yourFolderNameHere with the folder you made in step 10.I
    3. If you navigate to /mnt/yourFolderNameHere and execute ls you should see a list of the files on your host systems shared folder
    4. If that worked then we want to add our folder to the fstab file so that the mount point is loaded every time our virtual machine boots up
       1. sudo vim /etc/fstab
       2. Add the following line to the bottom of the file:
          memorableFolderName /mnt/yourFolderNameHere vboxsf defaults 0 0
          1. Replace memorableFolderName with name you set in step 9.III
          2. Replace /mnt/yourFolderNameHere with the folder you made in step 10.I
11. Now to make this development machine available over the network shut the virtual machine down by going to top menu bar Machine -> Close and choosing Power Off
    1. Right click on the virtual machine and choose settings
    2. From the left hand menu choose Network
       1. Tick Enable Adapter
       2. Adapter Type: Intel PRO/1000 T Server
       3. Attached to: Host Interface
       4. Tick Cable Connected
       5. Choose the interface on your host machine that gives you access to the outside world. You can find out which one this is by looking in your control panel.
    3. Save the settings and start your virtual machine back up again
    4. To find out the IP address of your virtual machine open a terminal execute ifconfig
       
12. You should now be able to visit your web server from your host machine by entering that IP address into your web browser.
    

Configuring our web server

1. We need to enable mod_rewrite which involves making a simple symbolic link
   1. Navigate to /etc/apache2/mods-enabled
   2. Execute sudo ln -s /etc/apache2/mods-available/rewrite.load rewrite.load
2. Reboot the server sudo /etc/init.d/apache2 restart and test it is function ok.

Setting up Mass Virtual hosts

The idea behind this step is to minimise the time it takes to setup a new host on the server. Basically all you have to do is create a new directory an away you go. (Don’t forget it will still need to appear in your hosts file).

1. We need to enable mod_vhost_alias
   1. Navigate to /etc/apache2/mods-enabled
   2. Execute sudo ln -s /etc/apache2/mods-available/vhost_alias.load vhost_alias.load
2. Now to setup the rules it will create virtual hosts by
   1. Execute sudo vim /etc/apache2/apache2.conf
   2. Go to the base of the file and add the following rules:

      http://gist.github.com/294736

      VirtualDocumentRoot sets the directory that the VHosts public directory is contained in. This will basically convert http://subdomain.localhost/ to /mnt/htdocs/subdomain and pull the relevant files through.

3. Now restart the apache process sudo /etc/init.d/apache2 restart

A side note when using this method that you should be aware of. This will affect your rewrite rules if they are placed into a .htaccess file. To avoid any problems always declare the RewriteBase rule in your .htaccess.

For example:

http://gist.github.com/294737

VirtualBox Shared Folders Permissions

Nathan brought a glaring omission from my post to the fore – thanks!  Currently you maybe having permissions issues with your shared folders because they might be being mounted as root:root.  To get them to load with a specified user and group you will need to edit your /etc/fstab file again and change ‘defaults‘ to be ‘uid=username,gid=groupname‘ – an example would be ‘uid=simon,gid=www-data‘.

A full line example would be from:
htdocs /mnt/htdocs vboxsf defaults 0 0
to
htdocs /mnt/htdocs vboxsf uid=simon,gid=www-data 0 0

Apache HTTP Server

I really like the way the Apache modules and virtualhosts are seperated out on Debian into folders containing those, which are available and those which are enabled. There is one small problem with this – it is more work than before! Luckily there are some helper scripts.

The Apache configuration files are layed out in the following way:

mods-available – the actual text files containing the modules configuration
sites-available – the vhosts text file for the site

mods-enabled – a symlink to the actual text file in mods-available
sites-enabled – a symlink to the actual text file in sites-available

Anything listed in the enabled directories will be loaded when Apache is therefore enabling the respective site or module. Manually symlinking these up can be a right pain so I use the following scripts to assist me:

“a2ensite sitename” – will create the symlink for you
“a2dissite sitename” – will remove the symlink for you
“a2enmod modulename” – will create the symlink for you
“a2dismod modulename” – will remove the symlink for you

When the symlink is place the module or site will be loaded and vice versa.

Do not forget that you still need to reload the configurations into Apache by running “/etc/init.d/apache2 reload”.

Certificates are a useful way of restricting access to your SSH server because a user must have three things to log onto the server:

1. Username
2. Password
3. Certificate

Normally they would only need to have a password and username, which can be guess at or (potentially) brute forced. Forcing the user to supply a certificate on log on means that they must also have a tangible source of identification (without the key file they cannot log in!).

Creating the key pair

Log onto the server and run ssh-keygen and you will get asked a few questions as follows (enter a passphrase):

user@host$ sudo ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/user/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/user/.ssh/id_rsa.
Your public key has been saved in /home/user/.ssh/id_rsa.pub.
The key fingerprint is:
95:60:c2:31:2e:94:cf:66:b6:fa:8b:b8:45:6c:dd:22 user@server
The key’s randomart image is:
+–[ RSA 2048]—-+
| .o+.o |
| ….+ . . |
| .o. o |
| . o*. . |
| E+o.S |
| o … |
| .. |
| o.. |
| o…o. |
+—————–+

This will generate two files:

1. id_rsa
2. id_rsa.pub

in your home directory (if you chose the defaults). They are your private and public keys respectively. The public key is the one that goes on your server and the private key is the one you use when logging into the server.

Add the public key

Now on the server run

user@server:~$ cat id_rsa.pub » ~/.ssh/authorized_keys
user@server:~$ rm id_rsa

which adds the public key to the list of authorized keys for this user.

Activate the key based authentication on the server

To edit the config run

user@server:~$ sudo vim /etc/ssh/sshd_config

Ensure that you have the following lines uncommented and set correctly in your configuration file:

RSAAuthentication yes
PubkeyAuthentication yes

If you wish to lock non-certified logins from the server then also ensure you activate the following settings:

ChallengeResponseAuthentication no
PasswordAuthentication no
UsePAM no

Now reload the SSH configuration to get the new settings going:

user@server:~$ sudo /etc/init.d/ssh reload

Conversion to PuTTY

PuTTY has its own private key format and the private key you created earlier now needs to be converted, which is a very simple process. Firstly you need to download the key from the server and save it to your computer.

1. Now open PuTTYgen, which comes when you install PuTTY (look in the PuTTY program files directory if there is no shortcut in the start menu).
2. Click the “Load” button and point it to the private key we downloaded earlier (depending on the file extension you gave the key file you may need to adjust the file extension filter on the PuTTY load key dialogue)
3. It will ask you for the passphrase you set when you were generating the certificates – enter it!
4. PuTTYgen should now tell you that you it has successfully imported the certificate – click ok.
5. Click the “Save private key” button and save it

Getting this to work with PuTTY

Now the new key we just saved is compatible with PuTTY we can start a new PuTTY session as usual, but don’t forget to tell PuTTY where the key file is located by looking in the Category tree menu and clicking on Connection -> SSH -> Auth. You can now click the “Browse” button and point PuTTY to the key file you just created.

Click open and a new session will load:

1. Enter your username as normal
2. When prompted; give the passphrase you gave when creating your key (do not make the mistake of using your linux user account password as it won’t work!)

A very nice article: Keeping SSH access secure

I use the following in /etc/ssh/sshd_config:
AllowUsers username
PermitRootLogin no

Which kills root login access to the server meaning you will need to login as the username provided in AllowUsers and then su to root (eg. su root) or sudo the commands if you have sudo setup (apt-get install sudo).

You may also wish to change the port through which SSH occurs by adding:
Port 2345

Where 2345 is the new port number. This will stop people from attacking through the standard port 22, which can help against script kiddies and those using pre-packaged scripts. It will however not stop someone from finding the new port via ICMP sweeps.

Kick over SSHd:
/etc/init.d/ssh restart

http://workaround.org/articles/ispmail-sarge

VPSs are nice but how do you host multiple domain email addresses? Well the above tutorial a fantastic guide on Debian Sarge.

An item worthy of note is creation of the Courier SSL certificates. By default there will be a couple automatically created by Courier at install time. This are useless as your email client WILL reject them because localhost is not the same domain as domain1.com etc…

You need to edit both /etc/courier/pop3d.cnf and /etc/courier/imapd.cnf and change the details to reflect your company or name.

CN – Fully qualified hostname (eg. mail.domain.com)
C – Country (2 letter code eg. AU)
ST – State or province
L – Location/City
O – Organisation
OU – Organisation Unit/Division

CN must be set to your mail servers hostname!

Now you need to remove the old certificates because the new ones cannot be generated with them in place:
cd /etc/courier
rm imapd.pem pop3d.pem

Now to make the new certificates, which will last for 365 days at which point you need to re-generate them. If you need them to last longer then you can adjust the /usr/sbin/mk*****cert shell script.
mkimapdcert
mkpop3dcert