PHP_CodeSniffer == JS_CodeSniffer?

There has been one item on my PHP_CodeSniffer todo list for a long time; to get PHP_CodeSniffer to enforce coding standards for JavaScript files. I've finally found the time to get going with this project and I've made good progress over the last couple of days.

My local copy of PHP_CodeSniffer now uses the file extension to determine which tokenizer to use to parse each file. I've moved the tokenizing code out of PHP_CodeSniffer_File into a PHP tokenizer and have added a new JS tokenizer. Once the file is tokenized, all existing sniffs can be run on it, but obviously most of the existing ones are only going to work correctly for PHP files.

The next step is to look at all the existing sniffs and identify those that are PHP only and those that would also work for JS files. Once that happens, each will be flagged using a protected member var so developers of coding standards know which tokenizers a sniff supports. I'll only be doing this for the Squiz and Generic standards to start with as I'm sure PEAR has no desire to start writing a JavaScript coding standard.

For developers who have written their own sniffs, you don't need to change anything to keep your existing sniffs and standards working. You only need to make changes if you want to start checking JavaScript files as well.

My overall goal is to get PHP_CodeSniffer to check all file types that PHP web developers would commonly use, including JavaScript, CSS, HTML and XML. It's going to take a while to get there, but JavaScript files are easily the most complex files to parse in that list, so I'm well on the way. I'm hoping to get this change into CVS sometime next week and get a release candidate out after I've written a decent number of JS sniffs.

When PHP string comparions go wrong

I'm a little ashamed to admit that I didn't know about this before, but I've just never come across it. When PHP compares two strings it converts them to integers if they both appear to be numbers, unless you use the === operator to compare types as well. The comparison operator docs state:

If you compare an integer with a string, the string is converted to a number. If you compare two numerical strings, they are compared as integers. These rules also apply to the switch statement.

Keeping that in mind, take a look at this code:

if ('000E00080001' == '000E0008') {
    echo 'equal';
}

This statement evaluates to TRUE, but I wasn't sure why.

Even reading the comparison operator docs doesn't make it clear, but it does link off to a very important piece of information about strings. In particular, the section on string conversion to numbers states:

The string will evaluate as a float if it contains any of the characters '.', 'e', or 'E'. Otherwise, it will evaluate as an integer.

And the truth shall set you free!

The letter "e" in my two strings is considered part of the exponent of a number. In both cases, the number is "0" followed by an exponent. When PHP compares those two strings, it compares them as "zero to the power of ..." because they are both considered numbers. So in this case, it is comparing zero with zero and finding it is TRUE.

This also works with decimal points. This evaluates to TRUE as well:

if ('0000.' == '0000.0') {
    echo 'equal';
}

And just to confirm what is going on, adding a character that is not considered part of a number results in the expected behavior. This evaluates to FALSE because PHP no longer considers these values numbers:

if ('000E000F0001' == '000E000F') {
    echo 'equal';
}

And, if the initial portion of our number is not zero, the exponent modifies the value. We are no longer comparing "zero to the power of ..." (which is always zero) in the following example; we are comparing "one to the power of ...", so it evaluates to FASLE:

if ('001E00080001' == '001E0008') {
    echo 'equal';
}

So what is the solution? You can either use strcmp() to compare your strings, or use the === operator to compare types as well. When PHP compares types, it will not try and convert strings to numbers.

Notch one up for using the === equal operator, which we use exclusively in MySource4. We even go as far as banning type-insensitive and implicit comparisons using PHP_CodeSniffer. If you want to do the same, you can get PHP_CodeSniffer from PEAR and use the included Squiz standard.

PHP_CodeSniffer 1.0.1 released

I've just uploaded PHP_CodeSniffer version 1.0.1, which contains 6 bug fixes and adds some new sniffs into the Squiz and MySource standards after some lobbying by the MySource4 team. The coding standard we use for MySource4 now has 94 different sniffs, each containing up to 5 different checks, so it is getting fairly strict. The code should be pretty enough to frame and stick on your wall!

One important change has been made to the PEAR standard in this release. A recent RFC that asked PEAR developers to vote on forcing protected member vars to be prefixed with an underscore got me looking for the relevant sniff in PHP_CodeSniffer. (I didn't see the call for votes, but I would have voted not to prefix protected vars with an underscore.) While PEAR decided against adding this new standard, I did realise that PHP_CodeSniffer was not currently enforcing the existing standard. The 1.0.1 release adds a new sniff to the PEAR standard to ensure only private members vars are prefixed with an underscore.

You can view the full changelog, and download the release, on the package download page.

Aleks Bochniak writes... How to make a tag cloud (using Matrix)

Apart from being better at Matrix implementation than me by building his blog using MySource Matrix, Aleks has also written a great tutorial on how to build a tag cloud using Matrix. He uses metadata for tagging, an asset listing to show the tags and some JavaScript to make the magic happen.

Well worth a read if you're looking to implement a tag cloud on your own site, or if you just want to see an example of different parts of Matrix coming together to produce something beyond what they were designed to do.

MySource4 caching and keywords - I think I'm in love!

Sometimes things just fall into place.

When designing the sub-systems for MySource4, even when designing seemingly basic copies of MySource Matrix functionality, I often find myself surprised by the way we've managed to make slight design changes (often forced by the MySource4 architecture) but come out with systems that are significantly better than they are in MySource Matrix. Sometimes, design decisions made months ago suddenly start to pay off in ways I don't expect. The caching and keyword systems are a perfect example; they are a match made in heaven and I'm deeply in love with both of them!

Caching was to be a fairly simple copy of the MySource Matrix caching system. One of the strengths of MySource4 is the ability to replace core systems (like caching) with a system that is better designed for a site's specific needs. Knowing this, we started thinking about a basic caching system that Matrix users could migrate to easily, with plans to write other caching systems and allow users to pick the one they wanted. What we ended up with is something I am very proud of. Not only because of the feature set, but also because of the way it fits so nicely into my MySource4 vision. This is why we designed MySource4 like we did, and it is really starting to pay off.

The MySource4 caching system is made possible by a simple but critical change to the way content is printed. In MySource Matrix, each asset type and design area grabs attribute values from the database and prints them when required. When designing the keyword system (very early on) we made a decision to always print keywords rather than real values. So instead of a menu printing asset names and links, it would print keywords that would later be replaced by the keyword system. This provides two immediate benefits; assets and design areas take less time to print and keywords can be batch replaced in a page. Both those are performance-based, but now that decision has enabled us to make a fantastic caching system.

By utilising the MySource4 channels architecture, the caching system is able to hook into the frontend painting code and cache a copy of the page after static keywords have been replaced and before dynamic keywords have been replaced. Dynamic keywords are things like the contents of a custom form or the name of the current user, and should not be cached. The next time around, the caching system can serve up a copy of the page with just the dynamic keywords left to be replaced. Note that this is a full page cache, unlike Matrix which constructs the page each time using small cache blocks. That means that the MySource4 caching system is already faster than the MySource Matrix caching system and we haven't even got to the bonus features!

One of the biggest problems with the Matrix caching system is that we can never really know which assets are cached on which URLs. For example, we know that the name of the "Contact Us" page has changed but we have no idea which asset listings to clear the cache of or which pages have the old page name in the menu. We can't know this in Matrix because we don't have a consistent way of printing data. That is solved in MySource4 because the keyword system forces us to print keywords rather than real data. The result is that the MySource4 caching system knows exactly which URLs the name of the "Contact Us" page appears on and it can clear them for you.

That is a very powerful feature. You can now go to any asset and see exactly what parts of the asset have been cached and which URLs they are cached on. You can make an informed decision about clearing the cache; do you really want to clear 400 URLs because the name of the "Home Page" has changed?

We also have some other statistics that we are trying to decide how to use best. We know how many times a page's cache has been served and how many times it was generated (cache hits and misses). We also know how long each page generation took (on average), the static keywords used on the page and the dynamic keywords used.

We can obviously calculate how much time was saved by the caching of a page by looking at the hits and misses. That is a nice figure to have available, but it's not going to really help you in any way. So we thought about looking through the cache data and showing you how much performance improvement can be gained by removing a dynamic keyword. For example, we could calculate the time saving gained by removing the name of the current user from the design. Those sort of figures can help you modify your content and designs to improve performance.

I'd love to hear any suggestions for other ways these statistics can be used. Please leave a comment if you have some ideas.

MySource Matrix 3.18.0 RC1 released with PHP5 support

MySource Matrix 3.18.0 RC1 was released today; the first release that supports PHP5. It's been quite tough moving such a large project from PHP4 to PHP5 for a couple of reasons.

Changes to reference handling has been a big one. We've been changing so many lines of code to fix E_STRICT reference errors that we inevitably end up with some variables being copied rather than passed by reference.

Moving from PEAR::DB to PDO has also been a big deal. Not only have we moved to PDO, but we've also moved to the MySource4 DB abstraction layer, called DAL. DAL gives us the ability to write a single XML-based query for both Oracle and PostgreSQL, and native support for bind variables (an often requested feature). Learning the differences in syntax between PEAR::DB and DAL took time, as did DAL changes that were required to support the more complex queries that Matrix contains. In the end, we have a better, faster and more flexible DB abstraction layer that will see Matrix through until the MySource4 release and beyond. We also get the added benefit of testing such a key component of MySource4 in the real world.

On a more personally note, the large number of changes and testing required for this release got me more involved in Matrix development than I have been in a few years. Instead of sitting back and watching, I had enough time to dig in and convert part of the core, some packages and the unit testing system.

Speaking of unit testing, I couldn't imagine going through a process like this without an extensive unit test suite and some dedicated testers. Day-to-day, it is easy to forget why you write tests as you rarely get failures if you are careful enough. During a large core change like this, using the failing tests as a todo list gives you some guidance and, of course, confidence in your changes.

Besides PHP5 support, 3.18 has a couple of new features and improvements, some from the feature bazaar at the MMIUC 07. Many of the improvements are performance-based, with HIPOs and paint layouts getting special attention. All performance improvements in 3.18 have been road-tested in MySource4 first, so even though MySource4 has not been released, MySource Matrix users can be confident that they are benefiting indirectly from the project even at this early stage.

MySource Matrix 3.18.0 RC1 is not ready for production, but you can download a copy of the GPL version to give it a test on PHP 5.1.6 and PostgreSQL. Grab it from the MySource Matrix download page.

Baby Sherwood arrives

Gerred Charles Sherwood was born into the world at 8:51am on Sunday the 30th of December 2007, just in time to spend new year's in hospital learning how to be a dad. He weighed 2755 grams at birth and is 47cm long, a fair bit smaller than we were expecting. Mum is thankful!

Here are some pics: