Due to the nature of my current employment I spend alternating periods of time as primary oncall responder when issues arise in our infrastructure. I've long been a fan of Nagios, which we use at work as well. It lacks a native iPhone interface, but provides easy-enough hooks to be able to build one. This gave me a fantastic opportunity to hone my iOS SDK skills with a app that would certainly "scratch my own itch". I'm a strong believer that you learn best when addressing a problem you're actually experiencing - you can taste the pain going away. With OnCall for Nagios, the iPhone becomes a fantastic platform to be able to do basic triage and diagnosis as well as respond and communicate to other participants of your infrastructure team.

The great thing about this as my first real application was that it required a deeper dive into many of the core iPhone SDK APIs as well as the building of a more complicated (from a development perspective) UI.

Under the hood it's interaction with Nagios is accomplished via screen-scraping. This allows the app to work right out of the box, acting as just another client to the existing Nagios web interface without any server side changes required. For HTML parsing I built a light abstraction around libxml and used xpath queries to retrieve the relevant data. Certain views required multiple asynchronous HTTP requests, each with the same delegate, which required another light abstraction layer around NSURLConnection to be able to pass in identifiers for the connections so that incoming data would be appended to the correct NSMutableData property. Seemingly simple tasks such as HTTP authentication for async NSURLConnection requests and (optionally) ignoring self-signed SSL certificates took significant digging through documentation to gain a better understanding of connection's didReceiveAuthenticationChallenge method.

OnCall for Nagios uses a TabBarController with child NavigationBarControllers - this gives you the familiar tabbed buttons at the bottom with the ability to navigate forwards and backwards at the top. It also uses a combination of custom and NSUserDefaults to provide saved settings - allowing multiple Nagios instances to be setup and switched between within the app.

Interface development was slow and tedious for the initial setup. Determining the correct nesting order (TabBarController is the parent of multiple NavigationBarControllers) was huge. Once you get the hang of IBOutlets and where to "connect the dots" duplicating previous discoveries becomes easier. For certain interface details I often found myself hand-coding these elements instead of arguably spending more time figuring out how to do them correctly in Interface Builder. I'd be curious to hear of other's experiences on where to draw that line. I found that anything non-trivial required some level of code (take custom UITableViewCells) although I'm sure there are other ways to accomplish this.

One of the most important aspects to developing in Objective-C is a solid of understanding of it's memory management. With a solid background in C as well as Python - this wasn't terribly difficult. This document was extremely helpful. If you follow the golden rule "you only release or autorelease objects you own" you'll be fine. I found Xcode's static code analysis (clang) to be extremely helpful in situations that weren't immediately obvious, diagnosing and eliminating problems in my code. It's quite impressive to see arrows being overlaid onto your source code illustrating the code path that's causing the issue.

I also highly recommend Flurry, a free analytics SDK. It's extremely easy to integrate and provides a wide variety of metrics without much effort. It was also helpful in identifying when Apple was reviewing the application. The approval process took longer than I had hoped but wasn't as problematic as I had imagined it to be.

I also found it extremely beneficial to release a lite version. For this type of application the choices were obvious as to limiting functionality (display fewer problems, dont allow multiple instances, only show 2 hosts, etc.). In terms of implementation of the lite version - it exists in the same codebase. In Xcode I defined a new target with a different output binary and a specific C define which the code pivots on.

#ifdef LITE_VERSION
    if ([table count] < 2) {
        [table addObject:data];
    } else if (!didShowUpgrade) {
        didShowUpgrade = TRUE;
        UIAlertView *alertBox = [[UIAlertView alloc] initWithTitle:@"Lite Version" message:@"The lite version only displays 2 problems, consider upgrading!" delegate:self cancelButtonTitle:@"Ok" otherButtonTitles:nil];
        [alertBox show];
        [alertBox release];
    }
#else
    [table addObject:data];
#endif

The lite version sees much higher downloads and facilitates the need for experimentation and testing before purchasing.

I've certainly learned a lot throughout this process and I'm looking forward to continuing to develop features for this app as well as getting back into game development on this powerful platform. If you'd like more detail on anything I've glossed over feel free to ask questions in the comments or e-mail me directly.

Good luck!

Related posts:

1. Django 1.1 Released
2. PHP 5.3.0 Released and Firefox 3.5

I was investigating some intermittent issues in an application at work - transient DNS issues were causing the application to become unresponsive. This was confusing at first because it was written to perform HTTP requests asynchronously. The important thing here is that it was specifically DNS resolution that was failing. As I dug deeper I realized that simple health checks, ones that did not perform any HTTP requests, were also hanging... something had to be blocking.

Taking a look at the Tornado source, unsurprisingly, it was leveraging the multi functionality of libcurl (via pycurl). What was surprising to me was that the DNS resolution portion of the multi interface, by default, blocks on non-windows installations. Only when compiled with c-ares support does it perform these async.

You learn something new every day!

Related posts:

1. Python’s Tornado has swept me off my feet
2. Tornado 1.0 Released
3. Deployment Using Capistrano / Webistrano via Rails / Phusion Passenger

Essentially what it does is list the contents of your releases_path for the first host in the list of hosts and assumes that all those individual release directories will be present on all other hosts in the current deploy. This is a poor assumption. What if one host isn't deployed to as frequently (perhaps a QA environment?). These edge cases cause trouble with the default code.

What it should do is, for each host you're deploying to, check the releases_path for that host with keep_releases and delete only those old directories on just that host. I re-wrote deploy:cleanup to do just that using some of the features of run() to execute commands only on specific hosts...

  task :cleanup, :except => { :no_release => true } do
    count = fetch(:keep_releases, 5).to_i
    run "hostname" do |c, s, hostname|
      local_releases = capture("ls -xt #{releases_path}", :hosts => [hostname]).split.reverse
      if count >= local_releases.length
        logger.important "no old releases to clean up on #{hostname}"
      else
        logger.info "keeping #{count} of #{local_releases.length} deployed releases on #{hostname}"

        (local_releases - local_releases.last(count)).each { |release|
          run "#{sudo} rm -rf #{File.join(releases_path, release)}", :hosts => [hostname]
        }
      end
    end
  end

Related posts:

1. Deployment Using Capistrano / Webistrano via Rails / Phusion Passenger
2. Be Language Agnostic – Solve the Problem!
3. 8 Books To Get A Developer For The Holidays

Tornado is a non-blocking web server written in Python. It's structure is similar to web.py so users of that popular Python web framework will feel right at home. This is a structure that lends itself really well to developing RESTful APIs as the methods you write to handle incoming requests are named after the HTTP methods used:

class PlaceHandler(tornado.web.RequestHandler):
    def get(self, id):
        # respond to a GET
        self.write('GETting something')

    def post(self):
        # respond to a POST
        self.write('POSTing something')

You match URI paths to "handlers" (the controller for those MVC folk) via a list of regex, handler tuples that instantiate an "application".

application = tornado.web.Application([
    (r"/place", PlaceHandler),
    (r"/place/([0-9]+)", PlaceHandler)
])

if __name__ == "__main__":
    http_server = tornado.httpserver.HTTPServer(application)
    http_server.listen(8888)
    tornado.ioloop.IOLoop.instance().start()

As usual any values that are captured from the regex are passed, in order, to the method that receives the request in the handler.

Because of it's non-blocking nature Tornado bundles an asynchronous HTTP client for use internally. Additional modules include a command line and config file convenience library, escaping, 3rd party authentication (Facebook, Twitter, etc.), a wrapper around MySQLdb, and templating. All in all this makes it a formidable web framework in its own right, especially if you're looking for something that's light and FAST.

In production, I'm running 4 Tornado instances per server behind nginx.

One issue not addressed out of the box was daemonizing the Tornado instance. I added PID file management and the ability to daemonize as follows (pid.py module follows):

# capture stdout/err in logfile
log_file = 'tornado.%s.log' % options.port
log = open(os.path.join(settings.log_path, log_file), 'a+')

# check pidfile
pidfile_path = settings.PIDFILE_PATH % options.port
pid.check(pidfile_path)

# daemonize
daemon_context = daemon.DaemonContext(stdout=log, stderr=log, working_directory='.')
with daemon_context:
    # write the pidfile
    pid.write(pidfile_path)

    # initialize the application
    http_server = tornado.httpserver.HTTPServer(application.app)
    http_server.listen(options.port, '127.0.0.1')

    try:
        # enter the Tornado IO loop
        tornado.ioloop.IOLoop.instance().start()
    finally:
        # ensure we remove the pidfile
        pid.remove(pidfile_path)

And now the pid.py module:

# pid.py - module to help manage PID files
import os
import logging
import fcntl
import errno

def check(path):
    # try to read the pid from the pidfile
    try:
        logging.info("Checking pidfile '%s'", path)
        pid = int(open(path).read().strip())
    except IOError, (code, text):
        pid = None
        # re-raise if the error wasn't "No such file or directory"
        if code != errno.ENOENT:
            raise

    # try to kill the process
    try:
        if pid is not None:
            logging.info("Killing PID %s", pid)
            os.kill(pid, 9)
    except OSError, (code, text):
        # re-raise if the error wasn't "No such process"
        if code != errno.ESRCH:
            raise

def write(path):
    try:
        pid = os.getpid()
        pidfile = open(path, 'wb')
        # get a non-blocking exclusive lock
        fcntl.flock(pidfile.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
        # clear out the file
        pidfile.seek(0)
        pidfile.truncate(0)
        # write the pid
        pidfile.write(str(pid))
        logging.info("Writing PID %s to '%s'", pid, path)
    except:
        raise
    finally:
        try:
            pidfile.close()
        except:
            pass

def remove(path):
    try:
        # make sure we delete our pidfile
        logging.info("Removing pidfile '%s'", path)
        os.unlink(path)
    except:
        pass

I'm going to follow up this post another on how I added a simple concept of "models" and an easy way to perform MySQL transactions. Let me know if you have any specific questions!

Related posts:

1. Python data sharing in the multiprocessing module
2. Tornado 1.0 Released
3. Async DNS Resolution in Tornado’s AsyncHttpClient (curl multi, c-ares)

Why is a high-level deployment infrastructure important? Deployment is something that should be simple, accessible, and repeatable. It should be as close to a "single click" as possible. Previously, for me, it was a bash script that exported some SVN branches. While this worked fine, as projects progress, you want some accountability, history, and the ability to roll back mission critical applications when something goes wrong with a deploy.

Capistrano is an open source, command line, deployment tool that provides all of these features. It's written in Ruby. You leverage a variety of built in "recipes" (Capistrano's term for a deployment script) that execute certain procedures to deploy an app. Out-of-the-box it's ideally built to deploy a Rails app. However, after some minor tweaks it can deploy most anything and do it well. It can restart servers, update symlinks, change permissions - pretty much anything. It assumes you access your POSIX compliant server via SSH via the same password (or have ssh keys setup).

Webistrano is an open source web front-end for Capistrano. It's a convenience layer that abstracts the command line away and provides an interface to perform the same tasks. This interface shows history as well as providing a convenient GUI for creating new deployment projects, stages, and recipes. Highly recommended.

Let's get down to business. This post makes a few assumptions about things you've already installed and used previously.

• Ruby 1.8.5+
• RubyGems
• MySQL 5.0+
• Apache 2+
• Subversion and a repository containing the "production" branch of your app.

Installing Capistrano

Well, this is an easy one (you probably want to do this as root):

gem install capistrano

Installing Webistrano

Also fairly easy, with a little splash of configuration.

# wget http://labs.peritor.com/webistrano/attachment/wiki/Download/webistrano-1.4.zip
# unzip webistrano-1.4.zip
# mv webistrano-1.4 /path/to/where/you/want/webistrano

Setup the database tables and create a new webistrano user (obviously be conscious of your security preferences for access to your database in the host and password portions):

# mysql
mysql> CREATE DATABASE `webistrano`;
mysql> CREATE USER 'webistrano'@'localhost' IDENTIFIED BY 'password';
mysql> GRANT ALL PRIVILEGES ON `webistrano`.* TO 'webistrano'@'localhost' WITH GRANT OPTION;

Now, in the directory where you placed webistrano you're going to want to copy config/database.yml.sample to config/database.yml. Edit this file, in the production area, to match your database settings. By default the file expects a socket to connect, you can chase this by specifying host: and port:. (Keep in mind Webistrano is simply a Rails app).

You should now be able to have Rails migrate the new database you created. In the webistrano directory:

# RAILS_ENV=production rake db:migrate

Finally, copy config/webistrano_config.rb.sample to config/webistrano_config.rb and edit according to your preferred mail settings.

We can now test to see if webistrano is working properly by serving it via mongrel:

# ruby script/server -d -e production -p 3000

This starts a single mongrel daemon, using the production environment, listening on port 3000. You should now be able to hit http://127.0.0.1:3000/ and get the Webistrano login prompt. If this is working, kill that mongrel instance.

For longer term serving I decided to go with Phusion Passenger (essentially mod_rails for Apache). It's a nearly zero configuration solution for serving a rails app and will feel at home to anyone with experience serving PHP apps via Apache and mod_php.

Installing Phusion Passenger

Again, as root:

# gem install passenger
# passenger-install-apache2-module

The second command will invoke an installer which compiled Passenger and provides instructions on integrating it into your Apache config. Essentially, edit your httpd.conf as follows (these were specific to my install, make sure to use the ones provide by the installer for you):

LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.2.9/ext/apache2/mod_passenger.so
PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.2.9
PassengerRuby /usr/bin/ruby

Now you can simply add VirtualHost entries to your httpd.conf for any of your Rails apps. Let's add one for Webistrano:

<VirtualHost *:80>
ServerName webistrano.mydomain.com
DocumentRoot /path/to/webistrano/public
</VirtualHost>

Yes, Passenger makes it that simple. Add configuration directives as needed for your environment.

Now Webistrano should be serving from the VirtualHost you specified, seamlessly, via Passenger.

Deploying A Non-Rails App

Now the fun stuff.

Capistrano breaks things down into projects, stages, and recipes. Each app you want managed by capistrano should be it's own project. Each project should have a stage for at least production and optionally staging and development.

Hosts are added globally and form the targets of a deploy for any given project. Hosts can include web, app, and database servers.

Deployments in Capistrano are done to a child directory under "releases" named via the date and time of the deployment. By default 5 releases are kept and available to rollback to. Upon successful deployment a symlink (default is called "current" and can be modified via the current_path configuration variable) is updated to that release directory. It is this symlink that should be targeted by your webserver (your DocumentRoot in Apache).

Capistrano also creates a "shared" directory that is symlinked to in each release useful for storing logs and other data that should be maintained through each deployment.

For non-rails apps you'll use the "Pure File" project type when creating your new project. Upon project creation you can add configuration variables specific to your project. I recommend using :export instead of :checkout for deploy_via for production subversion deployments as this doesn't expose .svn directories. Use an SSH user that has enough permissions to create directories where your deploy will occur or, specify use_sudo to true and create a new configuration variable admin_runner and set it to the same user as runner.

Add a stage to your new project for "production". In the "Manage Hosts" page add a new host for each of your application servers. Then add each host as a target of your "production" stage of your project.

At this point you should be able to execute the "Setup" task for your "production" stage. This is a one time task that simply creates the directories.

Assuming this went successfully, try doing a "Deploy" and see if that finishes without error. You might have to play around with permissions and other minor issues - post a comment if you have any specific questions.

For my PHP framework there are a couple specific tasks I wanted to run in addition to the default Capistrano tasks. You do this by creating custom recipes in the "Manage Recipes" page in Webistrano. Recipes are simply procedures written in ruby. Here's what my recipe looks like:

namespace :deploy do
	task :setup, :except => { :no_release => true } do
		dirs = [deploy_to, releases_path, shared_path]
		dirs += shared_children.map { |d| File.join(shared_path, d) }
		run "#{try_sudo} mkdir -p #{dirs.join(' ')} && #{try_sudo} chmod g+w #{dirs.join(' ')}"
		run "chmod 777 #{shared_path}/log"
	end

	task :finalize_update, :except => { :no_release => true } do
		run "mkdir -p #{latest_release}/app/tmp"
		run "chmod -R 777 #{latest_release}/app/tmp"
		run "rm -rf #{latest_release}/app/logs"
		run "ln -s #{shared_path}/log #{latest_release}/app/logs"
		run "cp #{latest_release}/public_html/.htaccess-production #{latest_release}/public_html/.htaccess"
		run "cp #{latest_release}/app/config/config-production.php #{latest_release}/app/config/config.php"
		run "cp #{latest_release}/app/config/db-default.php #{latest_release}/app/config/db.php"
		run "cp #{latest_release}/app/config/memcache-default.php #{latest_release}/app/config/memcache.php"
	end
end

If you're not familiar with Ruby - what this code is essentially doing is overwriting two tasks in the :deploy namespace with my custom code.

The first, :setup, simply duplicates the base :setup functionality discussed above (creating the releases and shared directories) and chmods the shared log directory to be writable.

The second, :finalize_update, performs a variety of configuration tasks for a PHP app built with my framework. Also, you'll notice that I'm removing my app's logs directory and symlinking to the shared log directory. This way all releases will log to the same directory, consistently.

In my case all of these procedures are command line instructions. Alternatively, you can do a variety of things leveraging the full breadth of the Ruby language and any gem you'd like to introduce. Things such as accessing your CDN API to clear image, JS, or CSS caching, etc.

Deploying Django Apps

First off it's worth noting that I serve my Django apps via mod_wsgi. To make the deployment process easier here's what my app.wsgi script looks like:

import os
import sys

appdir = os.path.normpath(os.path.join(os.path.realpath(os.path.dirname(__file__)), '..'))
sys.path.insert(0, appdir)
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
os.environ['PYTHON_EGG_CACHE'] = os.path.join(appdir, '.python-eggs')
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

This code allows us to avoid having to hardcode paths in the wsgi script (and thus avoid having to change them when we deploy). It assumes the following directory structure:

.python-eggs (egg cache)
apps (apps path is added to python system path in settings.py)
public (where your .wsgi script resides)
site_media
templates
settings.py
settings-production.py (used for deploy)
urls.py
...

If you follow this convention, the following Capistrano recipe works great:

namespace :deploy do
	task :setup, :except => { :no_release => true } do
		dirs = [deploy_to, releases_path, shared_path]
		dirs += shared_children.map { |d| File.join(shared_path, d) }
		run "#{try_sudo} mkdir -p #{dirs.join(' ')} && #{try_sudo} chmod g+w #{dirs.join(' ')}"
		run "chmod 777 #{shared_path}/log"
	end

	task :finalize_update, :except => { :no_release => true } do
		run "rm -rf #{latest_release}/logs"
		run "ln -s #{shared_path}/log #{latest_release}/logs"
		run "cp #{latest_release}/settings-production.py #{latest_release}/settings.py"
		run "mkdir -p #{latest_release}/.python-eggs"
		run "chmod 777 #{latest_release}/.python-eggs"
	end
end

Fin

This should give you a nice intro to leveraging Capistrano via Webistrano. Feel free to comment with questions, suggestions, or anything else!

Related posts:

1. Installing Ruby Enterprise Edition with Phusion Passenger
2. Improved deploy:cleanup for capistrano
3. 8 Books To Get A Developer For The Holidays

The Pragmatic Programmer: From Journeyman to Master

Write better, cleaner, more maintainable code. Learn how to manage your projects and focus on shipping your product. With insight that covers the gamut of software development from low level to management
this one is a must have for anyone involved in this industry.

The Passionate Programmer: Creating a Remarkable Career in Software Development

Highly recommended! Read my full review.

Code Complete: A Practical Handbook of Software Construction

Another classic "software construction" book. Sharpen your saw with timeless information that can be applied to any project in any language. Less bugs, more productivity, more programmer happiness.

Coders at Work

This one is different. Written as a set of interview transcripts with 15 legendary industry giants, this book is a fantastic insight into how some of the great minds think. It's inspiring to hear it from the source, must have!

Programming Clojure

A developer should learn at least one new language a year. This year that language should be Clojure. Clojure is a dynamic, general purpose, language targeting the Java virtual machine and designed for multi-threaded use. It's growing popularity, ability to leverage the Java standard library, and its multi-threaded nature make this a must have.

The Mythical Man-Month: Essays on Software Engineering

Another classic. Primarily discusses project management from the perspective of Fred Brooks and his experiences at IBM. Brooks' Law states that "adding manpower to a late software project makes it later".

Don't Make Me Think: A Common Sense Approach to Web Usability

Web developers should always keep in mind the user of the product their creating. Usability becomes increasingly important as applications move to the web. The design and usability of your app can make or break its success. This classic is a must read.

Design Patterns: Elements of Reusable Object-Oriented Software

This classic known most commonly as the "gang of four" book is the definitive reference on design patterns. Covering all of the most common cases and time and time again serving as an invaluable source of information.

Related posts:

1. Be Language Agnostic – Solve the Problem!
2. Deployment Using Capistrano / Webistrano via Rails / Phusion Passenger
3. Django 1.0 Template Development: Sample Chapter “Serving Multiple Templates”

The biggest reason why I chose to go with Python 2.5 at the time was because the MySQL Python (MySQLdb) package didn't support Python 2.6. The 1.2.3c1 release does so that roadblock is lifted.

The instructions are identical - nothing has really changed in that regard. Just change the references from Python 2.5 to 2.6. Here are the links to the versions I'm using successfully:

Python 2.6.4: http://www.python.org/ftp/python/2.6.4/Python-2.6.4.tgz

setuptools 0.6c11: http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c11-py2.6.egg#md5=bfa92100bd772d5a213eedd356d64086

MySQLdb 1.2.3c1: http://sourceforge.net/projects/mysql-python/files/mysql-python-test/1.2.3c1/MySQL-python-1.2.3c1.tar.gz/download

mod_wsgi 2.6: http://modwsgi.googlecode.com/files/mod_wsgi-2.6.tar.gz

Django 1.1.1: http://www.djangoproject.com/download/1.1.1/tarball/

Related posts:

1. Setup Python 2.5, mod_wsgi, and Django 1.0 on CentOS 5 (cPanel)
2. Ruby On Rails and SliceHost Part 1: Initial Setup
3. Getting Started with Django and Python – First Impressions

I'm sure many have used PHP's default session handling capabilities. By default, PHP uses the filesystem to store session data naming files with their session id # and putting them in /tmp.

This is done for the sake of simplicity. On a single-server, low load website, this particular setup works fine. It's when you start having multiple simultaneous requests from a single client (identified by a session) that the problems begin to show their ugly heads. Utilizing AJAX multiple simultaneous requests might be the norm, even for a low load website.

Essentially, in order to prevent race conditions PHP internally uses a lock to maintain exclusive access to the file containing the session data for the client connection. This means that as soon as a single request acquires exclusive access to that session file, no other request can access the file until the original request completes. What happens when that second request asks to start the session? It waits.

The Solution

Fortunately there's a solution to all this. Implementing your own custom session handler and moving your session storage backend to another technology (such as a MySQL database or memcache) affords you the ability to handle simultaneous requests in a thread-safe manner. Remember, it's a good thing that PHP prevents race conditions by locking the session file. What we're looking to do is increase the granularity of the lock to the level of individual session data key => value pairs.

For this post we're going to stick to storing sessions in the database with our own custom session save handler. Perhaps in another post I'll talk about doing the same in memcache. It's the theory we're concerned about, not necessarily the exact storage mechanism implementation.

Implementing your own custom session handler is simply a matter of calling session_set_save_handler() with the appropriate callback methods for handling the following scenarios:

• open
  Open function, this works like a constructor in classes and is executed when the session is being opened. The open function expects two parameters, where the first is the save path and the second is the session name.
• close
  Close function, this works like a destructor in classes and is executed when the session operation is done.
• read
  Read function must return string value always to make save handler work as expected. Return empty string if there is no data to read. Return values from other handlers are converted to boolean expression. TRUE for success, FALSE for failure.
• write
  The "write" handler is not executed until after the output stream is closed.
• destroy
  The destroy handler, this is executed when a session is destroyed with session_destroy() and takes the session id as its only parameter.
• gc
  The garbage collector, this is executed when the session garbage collector is executed and takes the max session lifetime as its only parameter.

There's a well known trick for situations like this that allow you to pass a class method (static or instance) for callbacks. Let's take a look at a simple example. This correctly implements the required methods but obviously doesn't do much:

class MySession
{
	public function __construct()
	{
		session_set_save_handler(
						array('MySession', 'sess_open'),
						array('MySession', 'sess_close'),
						array('MySession', 'sess_read'),
						array('MySession', 'sess_write'),
						array('MySession', 'sess_destroy'),
						array('MySession', 'sess_gc')
						);

		ini_set('session.auto_start',					0);
		ini_set('session.gc_probability',				1);
		ini_set('session.gc_divisor',					100);
		ini_set('session.gc_maxlifetime',				604800);
		ini_set('session.referer_check',				'');
		ini_set('session.entropy_file',					'/dev/urandom');
		ini_set('session.entropy_length',				16);
		ini_set('session.use_cookies',					1);
		ini_set('session.use_only_cookies',				1);
		ini_set('session.use_trans_sid',				0);
		ini_set('session.hash_function',				1);
		ini_seT('session.hash_bits_per_character',		5);

		session_cache_limiter('nocache');
		session_set_cookie_params(0, '/', '.mydomainname.com');
		session_name('mySessionName');
		session_start();
	}

	public static function sess_open($save_path, $session_name)
	{
		return true;
	}

	public static function sess_close()
	{
		return true;
	}

	public static function sess_read($id)
	{
		return '';
	}

	public static function sess_write($id, $sess_data)
	{
		return true;
	}

	public static function sess_destroy($id)
	{
		return true;
	}

	public static function sess_gc($maxlifetime)
	{
		return true;
	}
}

SPL and Storing Session Data In MySQL

Adding support for MySQL to this class is fairly trivial. Let's start off by creating a table to store our session data:

CREATE TABLE `sessions` (
  `sesskey` char(32) NOT NULL,
  `timestamp` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP,
  `varkey` varchar(128) NOT NULL,
  `varval` longtext NOT NULL,
  PRIMARY KEY  (`sesskey`,`varkey`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

PHP's SPL provides the ability to create objects that behave as though they were arrays. You can make objects that iterate, respond to accessing them via array notation ($object['key']), and lots of other interesting things.

We're going to enhance the MySession class with a couple SPL interfaces that will allow the object, when instantiated, to behave like an array. We can then override the $_SESSION superglobal with an instance of our new MySession class. It will provide an identical interface to access session data while internally storing session data to the database. Also, internally it will use methods for row-level locking via MySQL's advisory locks (GET_LOCK() and RELEASE_LOCK()).

The next step is to implement the required methods for the interfaces we're using. These methods allow the object to behave as though it's an array.

class MySession implements Countable, ArrayAccess, Iterator
{
	private $index;
	private $curElement;
	private $locks = array();
	private $sessionName = 'sessionId';
	private $serialize = 'serialize';
	private $unserialize = 'unserialize';
	private $session_id = null;

	public function __construct()
	{
		session_set_save_handler(
						array('MySession', 'sess_open'),
						array('MySession', 'sess_close'),
						array('MySession', 'sess_read'),
						array('MySession', 'sess_write'),
						array('MySession', 'sess_destroy'),
						array('MySession', 'sess_gc')
						);

		ini_set('session.auto_start',					0);
		ini_set('session.gc_probability',				1);
		ini_set('session.gc_divisor',					100);
		ini_set('session.gc_maxlifetime',				604800);
		ini_set('session.referer_check',				'');
		ini_set('session.entropy_file',					'/dev/urandom');
		ini_set('session.entropy_length',				16);
		ini_set('session.use_cookies',					1);
		ini_set('session.use_only_cookies',				1);
		ini_set('session.use_trans_sid',				0);
		ini_set('session.hash_function',				1);
		ini_seT('session.hash_bits_per_character',		5);

		session_cache_limiter('nocache');
		session_set_cookie_params(0, '/', '.mydomainname.com');
		session_name('mySessionName');
		session_start();
	}

	public function destroy()
	{
		$sessionName = session_name();
		$cookieInfo = session_get_cookie_params();
		$cookieExpires = time() - 3600;
		if((empty($cookieInfo['domain'])) && (empty($cookieInfo['secure']))) {
			setcookie($sessionName, '', $cookieExpires, $cookieInfo['path']);
		} elseif(empty($cookieInfo['secure'])) {
			setcookie($sessionName, '', $cookieExpires, $cookieInfo['path'], $cookieInfo['domain']);
		} else {
			setcookie($sessionName, '', $cookieExpires, $cookieInfo['path'], $cookieInfo['domain'], $cookieInfo['secure']);
		}
		unset($_COOKIE[$sessionName]);

		$dbo = DBO::getInstance();
		$q = "DELETE FROM `sessions` WHERE `sesskey` = '".$this->session_id."'";
		$dbo->query($q);

		session_destroy();
	}

	private function lockName($k)
	{
		return 'sesslock'.$this->session_id.$k;
	}

	public function locked($k)
	{
		$k = $this->lockName($k);

		return isset($this->locks[$k]);
	}

	public function acquire($k, $timeout = 0)
	{
		$k = $this->lockName($k);

		if(!isset($this->locks[$k])) {
			$dbo = DBO::getInstance();
			$q = "SELECT GET_LOCK('".$k."', ".$timeout.")";
			$rs = $dbo->query($q);
			$this->locks[$k] = $dbo->result($rs, 0);
			$dbo->fr($rs);

			return $this->locks[$k];
		}

		return false;
	}

	public function release($k)
	{
		$k = $this->lockName($k);

		unset($this->locks[$k]);

		$dbo = DBO::getInstance();
		$q = "SELECT RELEASE_LOCK('".$k."')";
		$rs = $dbo->query($q);
		$ret = $dbo->fetch($rs);
		$dbo->fr($rs);

		return true;
	}

	public function count()
	{
		$dbo = DBO::getInstance();
		$q = "SELECT COUNT(*) FROM `sessions` WHERE `sesskey` = '".$this->session_id."'";
		$rs = $dbo->query($q);
		$ret = $dbo->result($rs, 0);
		$dbo->fr($rs);

		return $ret;
	}

	public function rewind()
	{
		$this->index = 0;
		$this->getCurElement();
	}

	private function getCurElement()
	{
		$dbo = DBO::getInstance();
		$q = "SELECT `varkey`, `varval` FROM `sessions` WHERE `sesskey` = '".$this->session_id."' LIMIT ".$this->index.",1";
		$rs = $dbo->query($q);
		$row = $dbo->fetch($rs);
		$dbo->fr($rs);
		if(is_array($row) && (count($row) == 2)) {
			$this->curElement = $row;
		} else {
			$this->curElement = array(null, null);
		}
	}

	public function key()
	{
		return $this->curElement[0];
	}

	public function current()
	{
		return call_user_func($this->unserialize, $this->curElement[1]);
	}

	public function next()
	{
		$this->index++;
		$this->getCurElement();
	}

	public function valid()
	{
		return ($this->curElement[0] !== null);
	}

	public function offsetSet($k, $v)
	{
		$dbo = DBO::getInstance();
		$q = "REPLACE INTO `sessions` (`sesskey`, `varkey`, `varval`) VALUES ('".$this->session_id."', '".$k."', '".$dbo->sanitize(call_user_func($this->serialize, $v))."')";
		$dbo->query($q);
	}

	public function offsetGet($k)
	{
		$dbo = DBO::getInstance();
		$q = "SELECT `varval` FROM `sessions` WHERE `sesskey` = '".$this->session_id."' AND `varkey` = '".$k."'";
		$rs = $dbo->query($q);
		if($ret = $dbo->result($rs, 0)) {
			$ret = call_user_func($this->unserialize, $ret);
		}
		$dbo->fr($rs);

		return $ret;
	}

	public function offsetUnset($k)
	{
		$dbo = DBO::getInstance();
		$q = "DELETE FROM `sessions` WHERE `sesskey` = '".$this->session_id."' AND `varkey` = '".$k."'";
		$dbo->query($q);
	}

	public function offsetExists($k)
	{
		$dbo = DBO::getInstance();
		$q = "SELECT `varval` FROM `sessions` WHERE `sesskey` = '".$this->session_id."' AND `varkey` = '".$k."'";
		$rs = $dbo->query($q);
		$ret = $dbo->result($rs, 0);
		$dbo->fr($rs);

		return (bool)$ret;
	}

	public static function sess_open($save_path, $session_name)
	{
		return true;
	}

	public static function sess_close()
	{
		return true;
	}

	public static function sess_read($id)
	{
		return '';
	}

	public static function sess_write($id, $sess_data)
	{
		return true;
	}

	public static function sess_destroy($id)
	{
		return true;
	}

	public static function sess_gc($maxlifetime)
	{
		$dbo = DBO::getInstance();
		$q = "DELETE FROM `sessions` WHERE `timestamp` < '".date('Y-m-d H:i:s', time() - $maxlifetime)."'";
		$dbo->query($q);

		return $dbo->query($q);
	}
}

The $dbo object is just an example of an interface to the database through the use of a singleton. Replace the $dbo object with your preferred mysql database interface and you'll be set to go!

Starting your session is now as simple as:

$_SESSION = new MySession;

Related posts:

1. Mutex with PHP and mySQL
2. PHP Parallel Web Scraper
3. PHP Simple Profiling Class

By using hotcopy this can be performed on a live subversion repository and will produce a pristine backup.

#!/bin/bash
REPOS_PATH=/var/repos
mkdir -p /backups/weekly
rm -rf /backups/tmp
mkdir -p /backups/tmp/repos
for i in $(ls $REPOS_PATH); do
        /path/to/svnadmin hotcopy $REPOS_PATH/$i /backups/tmp/repos/$i
done
FN=svn.weekly.`date '+%Y%m%d'`.tar.gz
tar -czf /backups/weekly/$FN -C /backups/tmp .

Related posts:

1. Automated Backups – A 5 Minute Script To Safer Data
2. SVN Keyword Substitution svn:keywords
3. Setup Python 2.5, mod_wsgi, and Django 1.0 on CentOS 5 (cPanel)

The development server that I mess around on has recently accumulated a lot of data that I really don't want to lose, so I whipped up a small script that will help to ensure this never happens.

#!/bin/bash
rm -f /path/to/backups/tmp/*
mysqldump --opt --host=localhost --user=USERNAME --password=PASSWORD --all-databases > /path/to/backups/tmp/dbBackup.sql
tar -czvf /path/to/backups/database/mysql.`date '+%Y%m%d%H%M%S'`.tar.gz /backups/tmp
service httpd restart

Line by line, what's happening here is:

• delete the .sql file generated by the previous backup
• dump every MySQL table to a file called dbBackup.sql.  You don't have to go this far; check out MySQL's documentation on mysqldump for all the options
• tar up the dbBackup.sql file, include a timestamp in the filename
• restart Apache

I want this script to run every night at 1am, so I added it to the crontab:

0 1 * * * /path/to/backup.daily.cron

Obviously this is a very simple script that can grow in complexity fairly quickly. For example, you can clear out old logs (a must for some Rails apps), and do other housekeeping functions. A good idea would also be to have a syncing program grab that .tar and upload it to your home machine, or even an S3 bucket.

When your MySQL instance gets corrupted or otherwise obliterated, you'll thank yourself for taking 5 minutes to set this up.

Related posts:

1. mysqldump usage
2. Backing Up Subversion Repositories Using svnadmin hotcopy
3. Ruby On Rails and SliceHost Part 1: Initial Setup