I'm sure many have used PHP's default session handling capabilities. By default, PHP uses the filesystem to store session data naming files with their session id # and putting them in /tmp.

This is done for the sake of simplicity. On a single-server, low load website, this particular setup works fine. It's when you start having multiple simultaneous requests from a single client (identified by a session) that the problems begin to show their ugly heads. Utilizing AJAX multiple simultaneous requests might be the norm, even for a low load website.

Essentially, in order to prevent race conditions PHP internally uses a lock to maintain exclusive access to the file containing the session data for the client connection. This means that as soon as a single request acquires exclusive access to that session file, no other request can access the file until the original request completes. What happens when that second request asks to start the session? It waits.

The Solution

Fortunately there's a solution to all this. Implementing your own custom session handler and moving your session storage backend to another technology (such as a MySQL database or memcache) affords you the ability to handle simultaneous requests in a thread-safe manner. Remember, it's a good thing that PHP prevents race conditions by locking the session file. What we're looking to do is increase the granularity of the lock to the level of individual session data key => value pairs.

For this post we're going to stick to storing sessions in the database with our own custom session save handler. Perhaps in another post I'll talk about doing the same in memcache. It's the theory we're concerned about, not necessarily the exact storage mechanism implementation.

Implementing your own custom session handler is simply a matter of calling session_set_save_handler() with the appropriate callback methods for handling the following scenarios:

• open
  Open function, this works like a constructor in classes and is executed when the session is being opened. The open function expects two parameters, where the first is the save path and the second is the session name.
• close
  Close function, this works like a destructor in classes and is executed when the session operation is done.
• read
  Read function must return string value always to make save handler work as expected. Return empty string if there is no data to read. Return values from other handlers are converted to boolean expression. TRUE for success, FALSE for failure.
• write
  The "write" handler is not executed until after the output stream is closed.
• destroy
  The destroy handler, this is executed when a session is destroyed with session_destroy() and takes the session id as its only parameter.
• gc
  The garbage collector, this is executed when the session garbage collector is executed and takes the max session lifetime as its only parameter.

There's a well known trick for situations like this that allow you to pass a class method (static or instance) for callbacks. Let's take a look at a simple example. This correctly implements the required methods but obviously doesn't do much:

class MySession
{
	public function __construct()
	{
		session_set_save_handler(
						array('MySession', 'sess_open'),
						array('MySession', 'sess_close'),
						array('MySession', 'sess_read'),
						array('MySession', 'sess_write'),
						array('MySession', 'sess_destroy'),
						array('MySession', 'sess_gc')
						);

		ini_set('session.auto_start',					0);
		ini_set('session.gc_probability',				1);
		ini_set('session.gc_divisor',					100);
		ini_set('session.gc_maxlifetime',				604800);
		ini_set('session.referer_check',				'');
		ini_set('session.entropy_file',					'/dev/urandom');
		ini_set('session.entropy_length',				16);
		ini_set('session.use_cookies',					1);
		ini_set('session.use_only_cookies',				1);
		ini_set('session.use_trans_sid',				0);
		ini_set('session.hash_function',				1);
		ini_seT('session.hash_bits_per_character',		5);

		session_cache_limiter('nocache');
		session_set_cookie_params(0, '/', '.mydomainname.com');
		session_name('mySessionName');
		session_start();
	}

	public static function sess_open($save_path, $session_name)
	{
		return true;
	}

	public static function sess_close()
	{
		return true;
	}

	public static function sess_read($id)
	{
		return '';
	}

	public static function sess_write($id, $sess_data)
	{
		return true;
	}

	public static function sess_destroy($id)
	{
		return true;
	}

	public static function sess_gc($maxlifetime)
	{
		return true;
	}
}

SPL and Storing Session Data In MySQL

Adding support for MySQL to this class is fairly trivial. Let's start off by creating a table to store our session data:

CREATE TABLE `sessions` (
  `sesskey` char(32) NOT NULL,
  `timestamp` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP,
  `varkey` varchar(128) NOT NULL,
  `varval` longtext NOT NULL,
  PRIMARY KEY  (`sesskey`,`varkey`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

PHP's SPL provides the ability to create objects that behave as though they were arrays. You can make objects that iterate, respond to accessing them via array notation ($object['key']), and lots of other interesting things.

We're going to enhance the MySession class with a couple SPL interfaces that will allow the object, when instantiated, to behave like an array. We can then override the $_SESSION superglobal with an instance of our new MySession class. It will provide an identical interface to access session data while internally storing session data to the database. Also, internally it will use methods for row-level locking via MySQL's advisory locks (GET_LOCK() and RELEASE_LOCK()).

The next step is to implement the required methods for the interfaces we're using. These methods allow the object to behave as though it's an array.

class MySession implements Countable, ArrayAccess, Iterator
{
	private $index;
	private $curElement;
	private $locks = array();
	private $sessionName = 'sessionId';
	private $serialize = 'serialize';
	private $unserialize = 'unserialize';
	private $session_id = null;

	public function __construct()
	{
		session_set_save_handler(
						array('MySession', 'sess_open'),
						array('MySession', 'sess_close'),
						array('MySession', 'sess_read'),
						array('MySession', 'sess_write'),
						array('MySession', 'sess_destroy'),
						array('MySession', 'sess_gc')
						);

		ini_set('session.auto_start',					0);
		ini_set('session.gc_probability',				1);
		ini_set('session.gc_divisor',					100);
		ini_set('session.gc_maxlifetime',				604800);
		ini_set('session.referer_check',				'');
		ini_set('session.entropy_file',					'/dev/urandom');
		ini_set('session.entropy_length',				16);
		ini_set('session.use_cookies',					1);
		ini_set('session.use_only_cookies',				1);
		ini_set('session.use_trans_sid',				0);
		ini_set('session.hash_function',				1);
		ini_seT('session.hash_bits_per_character',		5);

		session_cache_limiter('nocache');
		session_set_cookie_params(0, '/', '.mydomainname.com');
		session_name('mySessionName');
		session_start();
	}

	public function destroy()
	{
		$sessionName = session_name();
		$cookieInfo = session_get_cookie_params();
		$cookieExpires = time() - 3600;
		if((empty($cookieInfo['domain'])) && (empty($cookieInfo['secure']))) {
			setcookie($sessionName, '', $cookieExpires, $cookieInfo['path']);
		} elseif(empty($cookieInfo['secure'])) {
			setcookie($sessionName, '', $cookieExpires, $cookieInfo['path'], $cookieInfo['domain']);
		} else {
			setcookie($sessionName, '', $cookieExpires, $cookieInfo['path'], $cookieInfo['domain'], $cookieInfo['secure']);
		}
		unset($_COOKIE[$sessionName]);

		$dbo = DBO::getInstance();
		$q = "DELETE FROM `sessions` WHERE `sesskey` = '".$this->session_id."'";
		$dbo->query($q);

		session_destroy();
	}

	private function lockName($k)
	{
		return 'sesslock'.$this->session_id.$k;
	}

	public function locked($k)
	{
		$k = $this->lockName($k);

		return isset($this->locks[$k]);
	}

	public function acquire($k, $timeout = 0)
	{
		$k = $this->lockName($k);

		if(!isset($this->locks[$k])) {
			$dbo = DBO::getInstance();
			$q = "SELECT GET_LOCK('".$k."', ".$timeout.")";
			$rs = $dbo->query($q);
			$this->locks[$k] = $dbo->result($rs, 0);
			$dbo->fr($rs);

			return $this->locks[$k];
		}

		return false;
	}

	public function release($k)
	{
		$k = $this->lockName($k);

		unset($this->locks[$k]);

		$dbo = DBO::getInstance();
		$q = "SELECT RELEASE_LOCK('".$k."')";
		$rs = $dbo->query($q);
		$ret = $dbo->fetch($rs);
		$dbo->fr($rs);

		return true;
	}

	public function count()
	{
		$dbo = DBO::getInstance();
		$q = "SELECT COUNT(*) FROM `sessions` WHERE `sesskey` = '".$this->session_id."'";
		$rs = $dbo->query($q);
		$ret = $dbo->result($rs, 0);
		$dbo->fr($rs);

		return $ret;
	}

	public function rewind()
	{
		$this->index = 0;
		$this->getCurElement();
	}

	private function getCurElement()
	{
		$dbo = DBO::getInstance();
		$q = "SELECT `varkey`, `varval` FROM `sessions` WHERE `sesskey` = '".$this->session_id."' LIMIT ".$this->index.",1";
		$rs = $dbo->query($q);
		$row = $dbo->fetch($rs);
		$dbo->fr($rs);
		if(is_array($row) && (count($row) == 2)) {
			$this->curElement = $row;
		} else {
			$this->curElement = array(null, null);
		}
	}

	public function key()
	{
		return $this->curElement[0];
	}

	public function current()
	{
		return call_user_func($this->unserialize, $this->curElement[1]);
	}

	public function next()
	{
		$this->index++;
		$this->getCurElement();
	}

	public function valid()
	{
		return ($this->curElement[0] !== null);
	}

	public function offsetSet($k, $v)
	{
		$dbo = DBO::getInstance();
		$q = "REPLACE INTO `sessions` (`sesskey`, `varkey`, `varval`) VALUES ('".$this->session_id."', '".$k."', '".$dbo->sanitize(call_user_func($this->serialize, $v))."')";
		$dbo->query($q);
	}

	public function offsetGet($k)
	{
		$dbo = DBO::getInstance();
		$q = "SELECT `varval` FROM `sessions` WHERE `sesskey` = '".$this->session_id."' AND `varkey` = '".$k."'";
		$rs = $dbo->query($q);
		if($ret = $dbo->result($rs, 0)) {
			$ret = call_user_func($this->unserialize, $ret);
		}
		$dbo->fr($rs);

		return $ret;
	}

	public function offsetUnset($k)
	{
		$dbo = DBO::getInstance();
		$q = "DELETE FROM `sessions` WHERE `sesskey` = '".$this->session_id."' AND `varkey` = '".$k."'";
		$dbo->query($q);
	}

	public function offsetExists($k)
	{
		$dbo = DBO::getInstance();
		$q = "SELECT `varval` FROM `sessions` WHERE `sesskey` = '".$this->session_id."' AND `varkey` = '".$k."'";
		$rs = $dbo->query($q);
		$ret = $dbo->result($rs, 0);
		$dbo->fr($rs);

		return (bool)$ret;
	}

	public static function sess_open($save_path, $session_name)
	{
		return true;
	}

	public static function sess_close()
	{
		return true;
	}

	public static function sess_read($id)
	{
		return '';
	}

	public static function sess_write($id, $sess_data)
	{
		return true;
	}

	public static function sess_destroy($id)
	{
		return true;
	}

	public static function sess_gc($maxlifetime)
	{
		$dbo = DBO::getInstance();
		$q = "DELETE FROM `sessions` WHERE `timestamp` < '".date('Y-m-d H:i:s', time() - $maxlifetime)."'";
		$dbo->query($q);

		return $dbo->query($q);
	}
}

The $dbo object is just an example of an interface to the database through the use of a singleton. Replace the $dbo object with your preferred mysql database interface and you'll be set to go!

Starting your session is now as simple as:

$_SESSION = new MySession;

Related posts:

1. Mutex with PHP and mySQL
2. PHP Parallel Web Scraper
3. PHP Simple Profiling Class

I've been working on a project management tool suite in Django. It's been a great side project to really experiment with Django in real-world scenarios.

Forms

At times I feel like I fight with newforms. In particular, it lacks the ability to specify basic class or style attributes for a given form field from within the template. I'd like to be able to more finely tune the display of the field, directly within the template, with a style attribute or a class. Is it suggested you write your own custom form field widget for a single element? I've been getting around this by doing the following:

<input type="text" name="{{ todo_form.item.name }}" style="width: 720px;"/>

This gets more complicated if you want to set a style attribute for a form field that's a select box (for a ForeignKey model field, for example).

<label for="category">Category:</label> <select name="{{ category_form.project.name }}" style="width: 221px;">
{% for choice_val, choice_label in category_form.project.field.choices %}
	<option value="{{ choice_val }}">{{ choice_label }}</option>
{% endfor %}

Is this a good use case for template tags? I feel like I'm missing something here.

On the positive side, it was an absolute pleasure to work with multiple forms on a single page submitted to and processed by a single view. This is primarily thanks to prefixes. Excellent, that's how easy it should be.

Ternary Operator

Update: It's been pointed out in comments (thanks!) that Python 2.5 introduced a ternary operator. It's syntax is as follows:

label = "true" if booleanVariable else "false"

I also ran into a minor Python syntax issue. I love the ternary operator in languages that offer it. It's a concise, one-line, syntax for an if-else clause. Consider the following PHP code:

$label = $booleanVariable ? 'true' : 'false';

// the above is identical to the following block:
if($booleanVariable) {
   $label = 'true';
} else {
   $label = 'false';
}

Python unfortunately lacks this syntactic sugar. Fortunately, however, you can effectively accomplish the same thing by doing this:

label = (booleanVariable and 'true' or 'false')

# the above is equivalent to the following block:
if booleanVariable:
   label = 'true'
else:
   label = 'false'

Sessions

Django has built in support for sessions. By default, sessions last longer than the lifecycle of the user's browser. I personally think it should be the other way around. It's easily changed though (in your settings.py):

SESSION_EXPIRE_AT_BROWSER_CLOSE = False

Views

In one of my views I wanted to test whether a filtered result set was empty or not. I was curious whether this was the "pythonic" way to accomplish this:

account = get_object_or_404(Account, pk=account_id)
if account.useraccount_set.filter(user__exact=request.user) != []:

Also, with respect to views and passing context to the response, sometimes it's an efficient shortcut to use locals() instead of explicitly typing out all the variables you'd like to expose. locals() returns a dictionary of all variables defined within the local scope.

def myview(request, id):
    account = get_object_or_404(Account, pk=id)
    new_account_form = NewAccountForm()

    return render_to_response('myview.html', locals())

More soon!

Related posts:

1. Django URL Parameter Passing and Python Strings
2. Django and Python First Impressions – Part II
3. Getting Started with Django and Python – First Impressions

I've been on a pretty long optimization kick for a lot of the production applications that are used on a daily basis. Most applications use an "extended database listing" class that extracts raw data from a mySQL table and provides interfaces to add columns using that base data and sort, format, and highlight the data. The process works in two phases - the second being an AJAX request to actually deliver the HTML output to the browser. In choosing to go this route (the originating page now finishes execution faster and displays faster) it's necessary to pass the list object from the originating script to the AJAX display helper script. It is this object that, depending on the size and data in the list, can become enormous.

Initially I had been using the tried and true serialize() / unserialize() on the object and storing it in the $_SESSION superglobal. As soon as the object becomes anywhere "unusually" large, the unserialize call will take an exponentially increasing # of seconds to complete. Not satisfied with this situation I searched and searched for a solution.

One solution I've come across that cuts execution time by 50% is the use of var_export() instead. var_export() will, by default, output a php parsable representation of the variable passed to it. In my case, the object being exported was a class. PHP > 5.1.0 changed the way var_export worked on class objects. You now have to implement the magic function __set_state() in your class in order to correctly handle a class exported with var_export(). On the other end of the process, where you would normally use unserialize(), now all you do is include the file you exported to.

Here is my implementation of the __set_state() magic function:

public static function __set_state($array)
{
    $obj = new cListExtDB;
    foreach($array as $field => $val) {
        $obj->$field = $array[$field];
    }
    return $obj;
}

The following example illustrates the exporting side of this procedure:

<?php
$_SESSION['_list_hash'] = $hash;
file_put_contents('_list'.$hash.'.dat', '<?php $__data = '.var_export($object, true).'; ?>');
?>

And in the AJAX helper script, the following example illustrates the importing side of this procedure:

<?php
$fname = '_list'.$_SESSION['_list_hash'].'.dat';
if(file_exists($fname)) {
    include($fname);
    unlink($fname);
    $list = $__data;
}
// proceed to operate with $list as it existed when exported in originating script
?>

Related posts:

1. PHP Error Log Viewer
2. PHP Daisy Chain Class Method Calls
3. PHP Custom MySQL Session Handler