It’s been a little over six months since I first began to use Kohana framework for all PHP development, and I thoroughly love the system. The only issue I’ve had with Kohana is the lack of built in, intuitive support for dynamic themes. I was unable to find an easy method to set up a theme that automatically wraps itself around all pages, while remaining dynamic enough to change certain elements on a page to page basis.

Over the past few days, I have been exploring methods to do this, with the following requirements:

• Keep all theme files separate from the application
• Require no changes to existing controllers and views to implement or change a theme
• Allow multiple applications on the same Kohana build to use the same theme
• Provide support for cascading themes

After a few failed attempts, I have found a solution that covers all four requirements and is pretty easy to implement. Of course, it is easier to implement this from the start, but if you have an existing Kohana site, it shouldn’t be too difficult to integrate this theming setup into the site. The concept of the theme is simple: create a theme as a module and then use hooks to implement the theme elements as appropriate.

Step 1 – Enable hooks

Without hooks, this theming set up won’t be able to execute at the proper time, and timing is essential. In order to enable hooks, open up application/config/config.php and change $config['enable_hooks'] to TRUE. With hooks enabled, all files within a hooks directory are included. This will be the starting point for inserting the theme content around the template.

Step 2 – Create a theme folder

One of my stated goals for the theme was to keep all theme files inside a single folder. This encapsulation allows for very easy modification, copying and replacing of themes. Make a folder called themes, and a subfolder named after the theme. This theme folder will operate just like a module folder, containing config, controllers, hooks, views, etc. My directory tree now looks like this:

/httpdocs
    /application
    /modules
    /system
    /themes
        /visual77
            /config
            /controllers
            /css
            /hooks
            /images
            /views
    /index.php

Step 3 – Add the theme folder to the list of enabled modules

In order to use the theme folder just as a module folder, you need to register the folder in the list of modules being used by your application. Open up application/config/config.php and add the theme folder to your list of modules at the end. This lets Kohana make full use of the directory.

$config['modules'] = array
(
    DOCROOT.'themes/visual77',
);

Step 4 – Create a hook for adding theme content

The most important step is creating a hook to add your theme content around your template content. You want to let your template proceed as normal, and after completion, add your theme data.

To create hooks in Kohana, just create a file inside your hooks directory. All files in that directory will be loaded automatically. For instance, make a file called theme.php inside themes/<theme-name>/hooks.

In order to do the theming, the ideal hook to use is system.post_controller. The previous hook, system.post_controller_constructor, is before the controller has executed; the following hook, system.send_headers, is after system.execute, and therefore can’t manipulate the controller.

The goal is then to create a hook on system.post_controller and manipulate the controller to take the template data and move it within the overall theme. To access the controller, call Kohana::$instance and begin manipulating. My hook file looks like this:

<?php defined('SYSPATH') or die('No direct access allowed');

// set my hook up to execute at the appropriate time
event::add('system.post_controller', 'visual77_theme');

function visual77_theme() {
    // grab the controller
    $controller = Kohana::$instance;

    // pull the original template aside
    $temp = $controller->template;
    // replace the template with my theme template
    $controller->template = new View('visual77/index');
    // put the original template into the content section of my theme
    $controller->template->content = $temp;
    // free up some memory by unsetting the temp variable
    unset($temp);
}

Step 5 – Begin building the theme

Now that the hook is in place, the only thing left to do is build your theme. In my example, create an index.php file in themes/<theme-name>/views/<theme-name>. This index file will contain the wrapper HTML for your theme, leaving the original template to drop neatly inside the content section that you define. When you begin adding css and images, you can keep it inside the theme directory. The usual .htaccess restriction on modules only applies to the module directory, which is why I put the themes outside of the module folder.

While building, remember to keep all theme related content, such as views and controllers, inside the theme directory. This makes the theme ultra portable and easily removed. To change themes, just change the name of the theme from Step 3 to the new theme directory.

In a system like this, you can even cascade themes. One great example of cascading themes is including barebones files like CSS resets and jQuery files. On a current project, I have to build an internal client administration and an external company website. Both use certain shared libraries and both need a CSS reset and jQuery files. Using cascading themes and module sharing, I can blend these two sites very neatly. The file structure for this project looks like this:

/httpdocs
    /internal_application
    /external_application
    /modules
        /shared_libraries
    /system
    /themes
        /internal_theme
        /external_theme
        /default
    /index.php

Both internal and external applications include modules/shared_libraries and themes/default. However, internal_application also includes themes/internal_theme and external_application includes themes/external_theme. A switch statement in index.php dynamically switches the application based on the hostname. In the end, I  have two sites running smoothly in parallel on a single Kohana build.

Final Thoughts

Kohana is a fantastic system, but really needs better support for dynamic theming. In my build process for this theming pattern, I tried to emulate the wordpress theme convention, where all theme files are kept clustered and are easily swapped out for a new theme.

To see a full theme example, you can download my Sample Kohana Theme and then modify it to fit your needs. Just remember to enable hooks and include this theme as a module!

In a recent Kohana project, I came across a somewhat odd PHP scenario – I had a multidimensional array that I needed to compress to a single dimensional array, but retaining all of the non array values. Basically, I need to make this change:

Array
    (
        to => Array
            (
                0 => stevep
                1 => bobw
            )
        bcc => Array
            (
                0 => paulj
            )
    )

Array
    (
        0 => stevep
        1 => bobw
        2 => paulj
    )

I looked around a bit for an existing PHP snippet to do just this, but they were all overly complex and used recursive callback functions. I had a feeling I could pull this off in a much more clean fashion. I toyed around a bit with the array functions and the ArrayObject library, and this is what I settled with:

$new_array = new ArrayObject();
array_walk_recursive($old_array, array($new_array, 'offsetSet'));
$flattened_array = array_keys($new_array->getArrayCopy());

The big drawback is the way ArrayObect::offsetSet() accepts its parameters, compared to how array_walk_recursive() sends the callback parameters; they are reversed. This causes the values to become keys, which has the effect of canceling out duplicate values. If your script needs duplicate values to remain intact, then this solution won’t work for you. Otherwise, it should work out just fine.

How many times have you been presented with a character encoding glitch on a PHP site that you thought was working properly? You finished the site months ago with no issues, but now you are being told some weird character is showing up on that page? Chances are, that character is either Ã, Â or Ä, possibly followed by some other random character. If I’m right, keep reading. If not, I offer my deepest condolences, for you are knee deep in a character encoding issue that I cannot help you with.

So what is it that causes that à to show up? In short – you have UTF-8 encoded data set to display as ISO-8859-1. But the more important question is how to prevent this from happening in the future. The easy answer to that – store all data in the same format as the final display. Convert the encoding as soon as you receive the data, either through an input field, reading a file or any other method. Don’t put anything in your database unless you are sure it is in the right format.

First, an introduction to the three important character encodings that you are likely to deal with – ASCII, UTF-8 and ISO-8859-1. Character encoding is, in a nutshell, the exact method that characters are stored and displayed by a computer. Since computers only deal in binary, there must be some way to convert from binary into a character. Character encodings come in two flavors (at least, two flavors I’ll be discussing) – single byte and multi byte encodings. Single byte encodings, such as ASCII and ISO-8859-1, use a single byte to represent a character. For instance, the letter “M” in ASCII is represented as 0x4D, whereas the number 4 is 0×34.

A single byte is limited to only 256 possible values, which rapidly becomes a limitation when you want to use characters not typically seen in English, such as “ç” or “ß”, and when you begin thinking about Asian languages, you can quickly see there simply isn’t room for all characters. Multi byte encodings use one or more bytes to represent a character. Each byte still only has 256 possibilities, but 2 bytes have 65,536 possibilities. But, what if there is confusion as to whether or not you are using single or multi byte encoding? Is that short string “0x3C 0xA5″ supposed to be the UTF-8 character å or the ISO-8859-1 string ÃÑ? And there we are – a sudden bug has shown up.

ASCII is the oldest of the three and the most basic. The ASCII character set started out as just 128 characters, and very English-centric. You can see that the basic ASCII table is very restricted to the English speaking world. The extended ASCII table goes off into some weird directions and is rarely used. The full ASCII table is only 256 characters, and thus fits neatly into a single byte, and so it is a single byte encoding.

ISO-8859-1 is also a single byte encoding, but you can see that it is a bit better in the extended range to support many Latin based languages. Unlike ASCII, ISO-8859-1 can handle the majority of Latin based languages without too much issue. It fails when you go outside of the Latin based languages, however.

The third character encoding is UTF-8, which is designed to support over 1.1 million characters, more than enough for all existing languages (and a few dead ones). Unlike ASCII and ISO-8859-1, UTF-8 is not limited to a single byte, but can vary between 1 and 4 bytes, depending on the character being used.

One thing to take note of is that the first 128 characters of all three encodings are identical. This is why a character encoding issue on an English website can take so long to spot. You can go months not knowing that you have a character encoding issue because the only characters you used were the same in all 3 encodings – 0x4D is an M in ASCII, UTF-8 and ISO-8859-1.

On to the meat of this article – dealing with character encodings in PHP. Namely, how to detect and convert encodings.

The magic bullet of dealing with character encodings in PHP can be summed up in three functions – mb_detect_encoding(), mb_detect_order() and iconv(). The mb libraries are designed for working with multibyte languages and iconv is a function to convert from one encoding type to another. One nonobvious restriction to the multibyte languages is the lack of native support for ISO-8859-1, since that is a single byte language. mb_detect_order() can be used to specify the order of encodings to scan when attempting to detect an encoding on a string.

The basics of dealing with character encoding conversion in PHP is a simple three step process:

1. Set the encoding types to scan (ASCII, UTF-8, ISO-8859-1). Luckily, this only has to be done once.
2. Identify the encoding of the string to convert.
3. Convert to the desired encoding type (I prefer UTF-8).

But, let’s cut to the crap. Sample code!

// tell the multibyte library which encodings to use
mb_detect_order('ASCII, UTF-8, ISO-8859-1');

// tell the browser that we are using UTF-8
header('content-type: text/plain; charset=UTF-8');

$string = "Fün wîth èñcøding!";

// UTF string displays properly
echo $string . "\r\n";

// convert to ISO-8859-1, which will confuse the browser
$string = iconv(
    mb_detect_encoding($string),
    'ISO-8859-1',
    $string
);
echo $string . "\r\n";

// convert back to UTF-8, which the browser understands
$string = iconv(
    mb_detect_encoding($string),
    'UTF-8',
    $string
);
echo $string . "\r\n";

When using the PHP socket functions to pull data from another website, you’ll often find yourself dealing with chunked data. Chunked data often looks like this:

1e
this line is 30 characters lon
2c
g, while this line is 44 characters long.
an
21
d this line is 33 characters long

As you can see, each chunk is preceded by a hex string. This hex string corresponds to the string length of the following chunk. Line breaks are included in the chunk, as you can see in the second chunk. So to unchunk the string, you have to find every hex string that is preceded by, and followed by, a line break (or start of end of string), and if the following block of code is equal in length to the hex value, strip out the hex value.

I accomplished this with a regular expression and preg_replace_callback. In essence, it finds each block of code that follows this pattern:

1. start of string / hex string on it’s own line
2. anything
3. hex string on it’s own line / end of string

Once it has found all of those patterns, it replaces only the ones where the hex value of #1 is equal to the string length of #2.

My unchunk function:

function unchunk($result) {
    return preg_replace_callback(
        '/(?:(?:\r\n|\n)|^)([0-9A-F]+)(?:\r\n|\n){1,2}(.*?)'
        .'((?:\r\n|\n)(?:[0-9A-F]+(?:\r\n|\n))|$)/si',
        create_function(
            '$matches',
            'return hexdec($matches[1]) == strlen($matches[2]) ?
                 $matches[2] :
                 $matches[0];'
        ),
        $result
    );
}

As part of the Nightlife Project, you will have to store some data through binary representation in the payment_rates table. While doing the writeup on the MySQL schema, I decided that this section was too large to easily fit inside of that article and it should be its own article. The goal of this article is to explain how database schemas employ the principles of binary notation to store a sequence of on/off flags in a single field.

Imagine this scenario: You are the head of maintenance at an office and are trying to determine which lightbulbs get the most usage and should be replaced with high efficiency bulbs. In order to determine which gets the most usage, you walk through the building in the morning, at lunch and again at night and make a note of which bulbs are on and which are off. Once you’ve gathered your data, you store the information in a database and after 2 months, you will view the results and determine the top used bulbs and replace them with higher efficiency bulbs to save electricity.

After each walk through the building, you have a list of lights and a corresponding on/off value for each one. Your list may look something like this:

The obvious way to store this data is to create a table called ‘lights’ and give it 10 fields – a timestamp (we need to know when this walk through occurred, after all) and a field for each of the 9 rooms. However, this obvious way has a few nasty shortcomings. What if you decide to add additional lights to be checked, such as desk lamps or the front walkway? Adding additional fields rarely ends well. This is also extremely inefficient, as you now have an 10 field table when a 2 field table would suffice.

Instead of storing it through this obvious, but inefficient method, instead you can convert that list of on/off switches into a binary string. Using the same data above, the binary string would look like this: 100011010. Each of the lights that are on is a 1 and each light that is off is a 0. Each character represents a room – the first position is reception, the second position is break room, etc. Converting this binary string to decimal, 100011010 becomes 282 (why?).

The key thing behind storing binary information in a database is this: no other combination of binary will equal that number. There is simply no other set of on/off values that will add up to 282. This lets us store an infinite amount of on/off values in a single integer slot, without ambiguity and always available for reading. As far as how you will read it, look into your language’s “bitwise and” and “bitwise or” operators. An example in PHP would be:

$user_permission = 6; // binary 0110

$view            = 1; // binary 0001
$edit            = 2; // binary 0010
$create          = 4; // binary 0100
$delete          = 8; // binary 1000

if($user_permission & $view)
    echo 'user can view';

if($user_permission & $edit)
    echo 'user can edit';

if($user_permission & $create)
    echo 'user can create';

if($user_permission & $delete)
    echo 'user can delete';

You can use a system like this for storing user permissions(which is how *nix permissions work), which days of the week an event occurs(the nightlife project will be doing this for storing payment rates), or any other piece of information that can be summed up in a series of on/off switches. Using binary storage is extremely efficient and flexible. Remember, however, that any additional fields to be added should be added to the left side of the string to preserve old data.

“A carelessly planned project takes three times longer to complete than expected; a carefully planned project takes only twice as long.”

The Nightlife Project is a novice PHP/MySQL tutorial. While you don’t have to know a whole lot to get started, you should understand the basics of PHP and MySQL. Knowledge of variables, functions and flow control is required. If the following block of code makes sense to you, you are ready to start the project:

<?php
repeat('foo', 15);
function repeat($string, $count) {
     for($i = 0; $i < $count; ++$i) {
          mysql_query("INSERT INTO `data` VALUES ('$string')");
     }
}
?>

Now, an explanation is due.

You work for Vision Nightlife, a nightlife promotion company working out of Las Vegas. Your company hires promoters to hand out nightclub flyers for the various clubs that have hired your company. These promoters give flyers to tourists for things like free entry to Domi Lounge, a free drink at Club Septuro, etc. Each flyer is stamped with a unique code identifying the promoter that drove the tourists to the club. The club will then pay your company, based on how many people your promoters drove to their club. The amount is based on how many people and what day of the week. For example, Hoodoo Lounge pays $1 per person on Friday or Saturday nights, if you bring between 1 and 10 people. If you bring between 11 and 20 people, it is $1.50 per person, and 21 or more people is $2.25 per person. On a Wednesday night, those amounts are cut in half. Your company wants you to write software to keep track of all of this information.

Your software needs to track clubs, promoters, payment rates, referral amounts and payout amounts.

This is just an introduction to the nightlife project. Part 2 will begin looking at the database structure and writing the ideal schema for each table.

I’ve been working on a solution the Kohana problem of the Validation library being unable to validate cleanly inside multidimensional arrays, and the root of this problem is passing a mask to the Validation::add_rules() method to identify deeply nested elements. In the course of trying to work out this problem, I wrote a function that accepts a mask and an array and returns the filtered array. While it doesn’t get me any closer to my actual goal, I do think it is an interesting function nonetheless and could be quite useful in the right circumstances. In the future, I might tweak this to allow you to invert the results, whereby it deletes anything that matches the mask, rather than preserves those that match the mask.

An example of a mask would be “messages/*/timestamp” would only keep elements of the array that matched $array['messages'][*]['timestamp'] with * being a wildcard.

function array_mask_filter($mask, $array, $ci = false) {
    if(!is_string($mask)) {
        throw new exception('filter mask must be a string');
    }

    if(!is_array($array)) {
        throw new exception('variable to be filtered must be an array');
    }

    $mask_chunks = explode('/', $mask);

    $this_mask = array_shift($mask_chunks);

    if($this_mask != '*') {
        foreach(array_keys($array) as $key) {
            $key = $ci ? strtolower($key) : $key;
            $this_mask = $ci ? strtolower($this_mask) : $this_mask;
            if($key !== $this_mask) {
                unset($array[$key]);
            }
        }
    }

    foreach(array_filter($array, 'is_array') as $key=>$element) {
        $array[$key] = array_mask_filter($element, implode('/', $mask_chunks));
        if(empty($array[$key])) {
            unset($array[$key]);
        }
    }

    return $array;
}

Back in July, I wrote an article about converting an object to array in PHP and mentioned the difficulties I was having with filtering private and protected keys to remove undesirable characters. I found a solution not long after, but never got around to writing a follow up. Here is my current function library to convert an object to an array, recursively, while maintaing nice keys.

< ?php

function convertObjectToArray(&$element) {
    // the recursive call can't operate through objects, so they
    // must be handled specially
    if(is_object($element)) {
        // typecast the object to an array, and clean up private and
        // protected keys
        $element = convertObjectToArray_keyCleanup((array)$element);
        // begin the recursion again to go through this object-turned-array
        // this is not strictly necesary, as removing it will cause the
        // recursion to happen in convertToXml, but putting it here makes it
        // more readable and ever so slightly faster.
        array_walk_recursive(
            $element,
           'convertObjectToArray'
        );
    }
    return $element;
}

function convertObjectToArray_keyCleanup($array) {
    // find every invalid key (private and protected member properties)
    foreach(array_filter(array_keys($array), 'convertObjectToArray_invalidKey')
        as $invalidKey) {
        // change the key name by copy / delete / create
        $data = $array[$invalidKey];
        unset($array[$invalidKey]);
        // find out the correct key name by getting the last chunk that
        // is only ascii 32 - 126, the standard set of printable characters
        // User�Types => Types
        $key = preg_replace(
            '/^.*[^\x20-\x7E]([\x20-\x7E]*)$/',
            '\\1',
            $invalidKey
        );
        $array[$key] = $data;
    }

    return $array;
}

function convertObjectToArray_invalidKey($key) {
    // a key is invalid if it has any characters that are outside
    // of the ascii range 32 - 126, which is the standard set of printable
    // characters
    return preg_match('/[^\x20-\x7E]/', $key);
}

?>

While working on some code recently, I realized that mysql_insert_id fails when using insert ignore. When using insert, mysql_insert_id returns the primary key of the newly inserted row. However, nothing is returned with insert ignore if a key conflict prevents a record from being inserted. If you are wanting to get the key that a conflict was just hit against, in as dynamic of a way as possible, you can use this script to find the primary key when insert ignore does not enter a record. This is useful if you are setting up a many-to-many pivot table and don’t want duplicate data on either side. When you attempt to insert a new record, it either gives you the new key or the key of the one that already existed.

$db->query($sql)

// if an insert id exists, use it
if ($db->insert_id != 0) {
    $id = $db->insert_id;
// if there is no insert id and there was no error and insert ignore was ued
} elseif($db->insert_id == 0 &&
    empty($db->error) &&
    preg_match('/^\s*insert\s+ignore/si', $sql)) {

    // find the table that was queried
    preg_match('/^\s*insert\s+ignore\s+into\s+([-`a-zA-Z0-9_]+)/si', $sql, $extract);
    $table = trim($extract[1], '`');

    // change insert ignore  to insert
    $Sql = preg_replace('/^\s*insert\s+ignore/si', 'insert', $sql);

    // query and scan the error for the key conflict
    $db->query($sql);
    $error = $db->error;
    preg_match('/^Duplicate entry \'(.*)\' for key (\d+)$/', $error, $extract);

    $value = $extract[1];
    $key = $extract[2];

    // in the case of multi column keys, figure out what the keys actually are
    if(strstr($value, '-') && !strstr($sql, $value)) {
        $values = explode('-', $value);
        $finished = false;
        while(!$finished) {
            foreach($values as $k=>&$v) {
                if(strstr($Sql, $v.'-'.$values[$k+1])) {
                    $values[$k] = $v.'-'.$values[$k+1];
                    unset($values[$k+1]);
                    $values = array_values($values);
                    break;
                }
                $finished = true;
            }
        }
        $value = $values;
    }

    // look up all keys on the table, isolating the primary key
    $keySql = "show keys from `$table`";
    $keyResult = $db->query($keySql);
    $keys = array();
    while($row = $keyResult->fetch_assoc()) {
        if(strtolower($row['Key_name']) == 'primary') {
            $primary = $row['Column_name'];
        }
        $keys[$row['Key_name']][] = $row;
    }

    // build a where clause to find the primary based on key conflicts
    $keys = array_values($keys);
    if(!is_array($keys[$key-1])) {
        $unique = $keys[$key-1]['Column_name'];
        $where = "`$unique` = '".$db->real_escape_string($value)."'";
    } else {
        foreach($keys[$key-1] as $key) {
            $whereParts[] = "`{$key['Column_name']}` = '".$db->real_escape_string($value[$key['Seq_in_index']-1])."'";
        }
        $where = implode(' and ', $whereParts);
    }

    // get the primary key that conflicted with the insert
    $sql = "select `$primary` from `$table` where $where";
    if(is_object($result)) {
        $result = $result->fetch_assoc();
        $id = $result[$primary];
    }
}

echo $id;