Agavi Framework

Bitextender backed Agavi is a very secure and helpful open source (LGPL) MVC framework with the core development being headed by David Zülke (Wombert) and Felix Gilcher (certainly in the IRC channel!). It can take some time to get the hang of the framework so I have put together all the resources I use or have used to help you get started.

Documentation Resources:

• Official Agavi Tutorial - Incomplete at time of writing
• API Documentation
• Agavi Cookbook
• Official Agavi FAQ
• Unofficial Agavi FAQ – Very helpful
• Package Docs in SVN – You need to dig around these folders in the SVN source code viewer of Trac (some of it is also old)
• Veikko Mäkinen’s Blog: http://blog.veikko.fi/tagged/agavi
• David Zülke’s Blog: http://blog.wombert.de/tagged/agavi (take a look through the PHPLondon talk and learn of the Mike controversy mentioned in one of my previous posts! Plus lots of Agavi tips as well of course.)
• IBM Developer Works Article series:
  • Part 1
  • Part 2
  • Part 3
  • Part 4
  • Part 5
  • Part 6

Support Resources:

• Agavi Support page IRC is very helpful (be patient when waiting for a response though)

I worked on a project while ago that required the use of iFrames to create “AJAX” file uploads.  It took me a little while but I finally worked out how to get the contents of an iFrame using jQuery.  To get the contents of an iFrame we need to wait until the iFramed content has finished loading as well.

http://gist.github.com/294740

JQuery is a fantastic tool but sometimes its functionality can be obscure or doing it one way might not work in a certain browser (MSIE6 anybody!).  I have often found myself trying to remember the best way to work with HTML select lists so I am compiling this list of hints for future use and I hope that you find it useful.  All the examples below are written where this represents the select element of the select list.

First up getting the selected value from a select list is as simple as using the jQuery shortcut $(this).val();.  Simple isn’t it!  Setting the selected item is just as easy too…

jQuery has a nice shortcut for setting the selectedIndex/value of an HTML select list and its syntax is as so $(this).val('value');.

Finding an option in a select list by its value works out to be something like this $('option[value="value-to-search-for"]', this);.

Often I need to be able to reset the select list to its initial state when clearing/resetting a form for the next submission.  There is a jQuery shortcut for this which is $(this).val('');, but it does not work in IE for whatever reason so I devised $(this).val($('option:first', this).val());.  This basically sets the select lists value equal to the first option in the list.

The next item on the list is resetting the select list to a blank option, which is easy with the following syntax $(this).val(null);.

Now to add an option into a select list with jQuery $(this).append('<option value="Option Value">Option Name</option>'); and to remove an option from the list by value $('option[value="value-to-search-for"]', this).remove();.

The aforementioned tips and some extras are included in a syntax highlighted manner below for your perusal.

http://gist.github.com/294741

Most of our development is done on Vista desktops (although I also run Ubuntu of course) and we wanted a web interface for easy browsing of code so I needed to use a pervasive and well supported VCS. After much hunting around and looking at Bazaar, Mercurial and git I decided to go with the latter due to its support in the community.  There are still some reservations I have against git, which work well for Linux kernal development but not our day to day web development – the major one being that subversion would allow you to commit and update on a file by file basis where as git versions the entire repository at once as far as I am aware.

Anyway now I have an Ubuntu server that I am using for storing the repositories and as a pretty frontend I have installed InDefero which also allows us to manage tickets.  InDefero is much like the code.google.com forge to manage and look at it so it is very simple.  I like Trac but I was not happy with the existing integrations with git.

It was not all plain sailing with the InDefero install as I had to make some very minor changes in the git plugin’s core code (the git plugin comes as part of InDefero and does not need to be installed seperately).  Basically the plugin was creating new repositories with very strict permissions which meant that the neither my backup user nor the apache process user could access them.  What this meant from the frontend was that the repository statistics (disk usage) were not displayed because it could not stat the directories and my back up process could not open them to make backups.

This problem is caused by the use of permissions setting in a PHP mkdir() command in the src/IDF/Plugin/SyncGit/Serve.php on lines 94 and 193.  You will want to increase the file permission octal just enough for your other users to gain access.  It is also worth mentioning that on line 111 there is a umask() command which you might also wish to tweak to improve your access.  Also don’t forget to update all $pluf_path values in the files in the scripts directory.  The rest of the way you should be fine just following the included documentation in doc/syncgit.mdtext

So far I am enjoying using InDefero though I think I will need to hack it so that when you add a new repository/project all the users names are placed in the project members as default to save vast amounts of copying and pasting.  Our development team is not large enough to warrant only giving certain people access to certain repositories/projects.

[UPDATE] I have made a small (and nasty) hack to the source of InDefero to cause the members text area when creating a new project to be pre-filled with my member’s names.  This involved changing /src/IDF/Form/Admin/ProjectCreate.php on line 107 by setting the array element initial to the value you want to appear in the text area.

For our desktop access to git we are using msysGit, PuTTY Plink and TortoiseGit.  This combination is working well for us so far and means that even non-technical users can begin versioning their documents and push them to a central repository for backup purposes.

Memcached and APC are two tools that you can install on your server and gain almost instant gratification!  APC basically caches executions that you send to a PHP process so that the next time you ask the parser to run your script it only has to look for some pre-chewed opcode in memory rather than parsing your PHP from the disk.  APC also has another feature up its sleeve, memory object caching, which allows you to store objects such as results from a database table in memory.  Memcached works in much the same way, but it can be used as a session handler as well as a persistent store for object data.  Another advantage memcached has over APC is that it can be distributed so you could have a number of servers all maintaining memcached daemons to distribute the load.

APC is faster as a straight PHP call on the memory cache, but you can only access the memory cache from the local PHP process.  I have not tested it but I am unsure if that means you can access it from CLI PHP as well as via PHP running on Apache.  Often I will have a PHP frontend with any server scripts (for example cronjobs) written in Python so it is very handy to be able to access the cache from a central location, which with memcached is possible but not with APC as it is PHP specific.  For more information on performance and advice on when various caching methods may be more useful Peter over at MySQLPerformanceBlog has written an interest article Cache Performance Comparison.

So for code that I know will only ever be used by the local PHP process I store the object in the APC memory object store, but if I need portability or the server is under high load and I want to separate out my object cache server from my script server then I use memcached.

I have written two articles detailing the steps involved in installing both the binaries on RedHat Enterprise Linux and Ubuntu or Debian.  Currently the live servers I use are either running CentOS or RedHat and the local development server is very nice setup based upon Ubuntu.

The MySQL documentation contains a very nice set of examples and documentation on using memcached to reduce hits on the database, which includes examples in a few languages including Python and PHP.  There is also some very handy hints to be gleened from An Introduction to memcached by Jeremy Ashcraft a.k.a MrSpooky over at Search-This.

Installing APC on Debian or Ubuntu is as simple as:

user@server:/directory/$ sudo apt-get install php-apc

Now let us reboot the Apache process to enable our new cache:

user@server:/directory/$ sudo /etc/init.d/apache2 restart

APC should now be ready to run on your server.  Try running the following command to verify it is setup; you should get something in response like mine:

user@server:/directory/$ php -r ‘phpinfo();’ | grep ‘apc’
apc
MMAP File Mask => /tmp/apc.s5jA6w
apc.cache_by_default => On => On
apc.coredump_unmap => Off => Off
apc.enable_cli => On => On
apc.enabled => On => On

<…SNIP…>

Now lets move onto installing Memcached, which again is very simple:

user@server:/directory/$ sudo apt-get install memcached
user@server:/directory/$ /etc/init.d/memcached start

The PHP Memcached module can be installed through Apt-Get as well:

user@server:/directory/$ sudo apt-get install php5-memcache

Now to configure PHP to use Memcached to store the session information we need to edit our /etc/php5/apache2/php.ini file and find the lines like the following:

session.save_handler = files
;session.save_path =

and change them so that they now look like this:

session.save_handler = memcache
session.save_path = “tcp://localhost:11211?persistent=1&weight=1&timeout=1&retry_interval=15″

That just leaves us to restart the Apache2 process:

user@server:/directory/$ /etc/init.d/apache2 restart

You are now up and running with Memcached PHP sessions and APC served PHP.

Installing APC on Redhat is as simple as:

[user@server directory]# yum install php-pecl-apc

APC should now be ready to run on your server.  Try running the following command to verify it is setup; you should get something in response like mine:

[user@server directory]# php -r ‘phpinfo();’ | grep ‘apc’
apc
MMAP File Mask => /tmp/apc.s5jA6w
apc.cache_by_default => On => On
apc.coredump_unmap => Off => Off
apc.enable_cli => On => On
apc.enabled => On => On

<…SNIP…>

Now lets move onto installing Memcached, which again is very simple:

[user@server directory]# yum install memcached
[user@server directory]# /etc/init.d/memcached start

The PHP Memcached module can be installed through YUM as well:

[user@server directory]# yum install php-pecl-memcache

Now to configure PHP to use Memcached to store the session information we need to edit our /etc/php.d/memcache.ini file and jump to the bottom of the file where we uncomment the following lines by removing the preceeding semicolon (‘;’):

; Options to use the memcache session handler

; Use memcache as a session handler
;session.save_handler=memcache
 ; Defines a comma separated of server urls to use for session storage
;session.save_path=”tcp://localhost:11211?persistent=1&weight=1&timeout=1&retry_interval=15″

So that it now looks like this:

; Options to use the memcache session handler

; Use memcache as a session handler
session.save_handler=memcache
 ; Defines a comma separated of server urls to use for session storage
session.save_path=”tcp://localhost:11211?persistent=1&weight=1&timeout=1&retry_interval=15″

That just leaves us to restart the Apache/HTTPd process:

[user@server directory]# /etc/init.d/httpd restart

You are now up and running with Memcached PHP sessions and APC served PHP.

The latest beta 2 release includes web workers, which are essentially threads allowing you to farm off Javascript heavy lifting to background processes so that the interface can continue to load without being impacted upon.  The Mozilla developer center [sic] has an interesting article on implementing them; Using web workers, which includes a couple of worked examples based on the Fibonacci sequence.

Geolocation is an interesting one for services like Twitter, Jaiku and possibly Facebook as it would allow users an easy way of updating all the services with their current location simply by Firefox broadcasting the information.  But even more interestly Yahoo! and Google maps (or a custom implementation of them – I am tempted but do not have the time at the moment) could use it to real time track the location of a user.  So potentially you could use the maps like a satellite navigation application with the watchPosition() function all the user would need is a GPS mouse with a Firefox plugin to read the position from it.  Oh yes and of course they would need a laptop with Firefox installed on it and a mobile internet connection.  

Obviously for any of this to work you will need to be running the latest beta available from the Mozilla site.

Agavi Framework Logo

A bit slow off the mark with this one but Agavi 1RC1 has been released and it is of course looking very nice. The 1.0 feature set is complete and potential bug fixes are the only things standing between now and 1.0 in earnest.  Features that I am particularly interested in include the recent refactoring of the configuration/routing files, the new validation interface and the addition of unit testing to the framework.

Sun VirtualBox Logo

Often Linux just does it better!  Often I find myself developing a Windows machine without access to a Linux development server, but I still need to access to some of the Linux binaries and features such as cron jobs, the at command and binaries such as imagemagick, pdftotext, etc.  Some things can be emulated with ported binaries or through Cygwin, but I feel a lot more comfortable developing on a platform that is representative of the live server the web site will run on.

I have not removed XAMPP as it is very useful for developing small scripts or sites without the overhead of running a virtual machine.  Therein lies the problem with this solution – it is virtual machine based and that will mean your local development machine will suffer when the virtual machine steals computing time from the CPU or more memory.  So whilst I can use XAMPP on my netbook I am not so sure a VM will run smoothly.

Setting up a new development environment

1. Download VirtualBox 2.1 from http://www.virtualbox.org
2. Download the latest Ubuntu from http://www.ubuntu.com
   Your choice of server or desktop. I chose desktop to give me another web browser testing environment.
3. Install VirtualBox and open it up
   1. Click create new VM (Virtual Machine)
   2. Give your VM name (I called mine Development)
   3. From the drop downs choose
      1. Linux
      2. Ubuntu (only choose Ubuntu 64 bit if you are running 64 bit Windows)
   4. Choose how much of your machines physical memory the virtual machine is allowed to steal. I chose 512MB as I am running the desktop version of Ubuntu.
   5. You will need to create a new virtual hard disk – a new hard disk wizard will open
      1. Choose dynamically expanding storage
      2. Choose
         1. Give the virtual hard disk a name (mine is called Development just like my virtual machine)
         2. The location of the storage on your hard drive
4. Now your new virtual machine should be setup so right click on it and choose Start
   1. The Virtual Machine will pop up a message asking you to go through its first run wizard
   2. Choose your installation location
      1. CD/DVD ROM
      2. Image file – point this to the location of your Ubuntu installation ISO you downloaded and saved earlier
   3. This will then mount your CD image and boot from it
5. Install Ubuntu by going through its installation wizard which is very easy and does not require detail here (if you have chosen the desktop version this will be even easier because you can use the mouse!)
6. Once you have the virtual machine setup you will want the nice Linux drivers so you can have a larger screen size and for networking etc. For this you need to install the Virtual Guest Additions.
   1. Mount a new CD drive/ISO Image in your virtual machine whilst it is running from the top menu item called Devices
   2. Add the new image which you will find in your VirtualBox program folder
7. Back to your virtual machine and open up a terminal window
   1. In your terminal navigate to the CD Rom drive
   2. Execute the following command sudo sh ./VBoxLinuxAdditions-x86.run
   3. Once complete you need to reboot the machine so execute sudo reboot
8. Once the machine reboots get back into the terminal and run the following commands
   1. sudo apt-get update
   2. sudo apt-get upgrade
   3. sudo apt-get install apache2 mysql-server-5.0 php5 php5-xdebug
   4. Just to be on the safe side restart apache with sudo /etc/init.d/apache2 restart
   5. Try accessing your web server by opening a web browser in the virtual machine and typing in localhost if you chose the desktop and you should see a message like It Works!
9. We need to be able to see folders on our host machine inside the development machine quickly and easy so from the top menu bar choose Devices -> Shared Folders
   1. Add a new folder
   2. Tell it which folder to look in
   3. Give it a memorable name
   4. Tick make permanent
10. Jump back into the virtual machine and execute the following commands to add your shared folder to the virtual machine, in a terminal
    1. sudo mkdir /mnt/yourFolderNameHere (mine is /mnt/htdocs/)
    2. sudo mount -t vboxsf memorableFolderName /mnt/yourFolderNameHere
       1. Replace memorableFolderName with name you set in step 9.III
       2. Replace /mnt/yourFolderNameHere with the folder you made in step 10.I
    3. If you navigate to /mnt/yourFolderNameHere and execute ls you should see a list of the files on your host systems shared folder
    4. If that worked then we want to add our folder to the fstab file so that the mount point is loaded every time our virtual machine boots up
       1. sudo vim /etc/fstab
       2. Add the following line to the bottom of the file:
          memorableFolderName /mnt/yourFolderNameHere vboxsf defaults 0 0
          1. Replace memorableFolderName with name you set in step 9.III
          2. Replace /mnt/yourFolderNameHere with the folder you made in step 10.I
11. Now to make this development machine available over the network shut the virtual machine down by going to top menu bar Machine -> Close and choosing Power Off
    1. Right click on the virtual machine and choose settings
    2. From the left hand menu choose Network
       1. Tick Enable Adapter
       2. Adapter Type: Intel PRO/1000 T Server
       3. Attached to: Host Interface
       4. Tick Cable Connected
       5. Choose the interface on your host machine that gives you access to the outside world. You can find out which one this is by looking in your control panel.
    3. Save the settings and start your virtual machine back up again
    4. To find out the IP address of your virtual machine open a terminal execute ifconfig
       
12. You should now be able to visit your web server from your host machine by entering that IP address into your web browser.
    

Configuring our web server

1. We need to enable mod_rewrite which involves making a simple symbolic link
   1. Navigate to /etc/apache2/mods-enabled
   2. Execute sudo ln -s /etc/apache2/mods-available/rewrite.load rewrite.load
2. Reboot the server sudo /etc/init.d/apache2 restart and test it is function ok.

Setting up Mass Virtual hosts

The idea behind this step is to minimise the time it takes to setup a new host on the server. Basically all you have to do is create a new directory an away you go. (Don’t forget it will still need to appear in your hosts file).

1. We need to enable mod_vhost_alias
   1. Navigate to /etc/apache2/mods-enabled
   2. Execute sudo ln -s /etc/apache2/mods-available/vhost_alias.load vhost_alias.load
2. Now to setup the rules it will create virtual hosts by
   1. Execute sudo vim /etc/apache2/apache2.conf
   2. Go to the base of the file and add the following rules:

      http://gist.github.com/294736

      VirtualDocumentRoot sets the directory that the VHosts public directory is contained in. This will basically convert http://subdomain.localhost/ to /mnt/htdocs/subdomain and pull the relevant files through.

3. Now restart the apache process sudo /etc/init.d/apache2 restart

A side note when using this method that you should be aware of. This will affect your rewrite rules if they are placed into a .htaccess file. To avoid any problems always declare the RewriteBase rule in your .htaccess.

For example:

http://gist.github.com/294737

VirtualBox Shared Folders Permissions

Nathan brought a glaring omission from my post to the fore – thanks!  Currently you maybe having permissions issues with your shared folders because they might be being mounted as root:root.  To get them to load with a specified user and group you will need to edit your /etc/fstab file again and change ‘defaults‘ to be ‘uid=username,gid=groupname‘ – an example would be ‘uid=simon,gid=www-data‘.

A full line example would be from:
htdocs /mnt/htdocs vboxsf defaults 0 0
to
htdocs /mnt/htdocs vboxsf uid=simon,gid=www-data 0 0