Wednesday, 17 December 2008

PHP_CodeSniffer reporting improvements

Spurred on by some feature requests and at least one annoying colleague (you know who you are), I've made some fairly good improvements to the way PHP_CodeSniffer reports the errors and warnings it finds. I've added support for the concept of error sources, allowing you to see which sniff generated an error and also filter out the messages for that sniff.

The XML, CSV and CheckStyle report formats all now include a source attribute in their output. This will allow continuous integration platforms to call PHP_CodeSniffer and produce a report of the most common errors rather than just display a list of all errors or the number of errors for each file.

You can also expose this information in the full and summary reports, although it is off by default to avoid clutter. Use the new command line argument -s to show sources in these reports. Instead of something like this:

FILE: /home/squiz/PHP_CodeSniffer/temp.php
--------------------------------------------------------------------------------
FOUND 6 ERROR(S) AND 0 WARNING(S) AFFECTING 5 LINE(S)
--------------------------------------------------------------------------------
2 | ERROR | Missing file doc comment
3 | ERROR | Object operator not indented correctly; expected 4 spaces but
| | found 0
3 | ERROR | Object operator must be at the start of the line, not the end
4 | ERROR | TRUE, FALSE and NULL must be lowercase; expected "true" but found
| | "TRUE"
6 | ERROR | Object operator not indented correctly; expected 4 spaces but
| | found 1
9 | ERROR | TRUE, FALSE and NULL must be lowercase; expected "false" but found
| | "FALSE"
--------------------------------------------------------------------------------
You get this:
FILE: /home/squiz/PHP_CodeSniffer/temp.php
--------------------------------------------------------------------------------
FOUND 6 ERROR(S) AND 0 WARNING(S) AFFECTING 5 LINE(S)
--------------------------------------------------------------------------------
2 | ERROR | Missing file doc comment (PEAR.Commenting.FileComment)
3 | ERROR | Object operator not indented correctly; expected 4 spaces but
| | found 0 (PEAR.WhiteSpace.ObjectOperatorIndent)
3 | ERROR | Object operator must be at the start of the line, not the end
| | (PEAR.WhiteSpace.ObjectOperatorIndent)
4 | ERROR | TRUE, FALSE and NULL must be lowercase; expected "true" but found
| | "TRUE" (Generic.PHP.LowerCaseConstant)
6 | ERROR | Object operator not indented correctly; expected 4 spaces but
| | found 1 (PEAR.WhiteSpace.ObjectOperatorIndent)
9 | ERROR | TRUE, FALSE and NULL must be lowercase; expected "false" but found
| | "FALSE" (Generic.PHP.LowerCaseConstant)
--------------------------------------------------------------------------------

There is also a totally new report format that shows a summary of your error sources. You use the command line argument --report=source to print a report like this:
PHP CODE SNIFFER VIOLATION SOURCE SUMMARY
------------------------------------------------------------
STANDARD CATEGORY SNIFF COUNT
------------------------------------------------------------
PEAR White space Object operator indent 3
Generic PHP Lower case constant 2
PEAR Commenting File comment 1
------------------------------------------------------------
A TOTAL OF 6 SNIFF VIOLATION(S) WERE FOUND IN 3 SOURCE(S)
------------------------------------------------------------
You can also get it to use official source names using the -s command line argument:
PHP CODE SNIFFER VIOLATION SOURCE SUMMARY
------------------------------------------------------------
SOURCE COUNT
------------------------------------------------------------
PEAR.WhiteSpace.ObjectOperatorIndent 3
Generic.PHP.LowerCaseConstant 2
PEAR.Commenting.FileComment 1
------------------------------------------------------------
A TOTAL OF 6 SNIFF VIOLATION(S) WERE FOUND IN 3 SOURCE(S)
------------------------------------------------------------

The idea with these source IDs is that you can then use them to filter your (potentially) long list of error messages and target a particular rule that you want to fix all in one go. So in this report, we have 3 errors generated from PEAR.WhiteSpace.ObjectOperatorIndent. To find out what they are, add the command line argument --sniffs=PEAR.WhiteSpace.ObjectOperatorIndent to get this:
FILE: /home/squiz/PHP_CodeSniffer/temp.php
--------------------------------------------------------------------------------
FOUND 3 ERROR(S) AND 0 WARNING(S) AFFECTING 2 LINE(S)
--------------------------------------------------------------------------------
3 | ERROR | Object operator not indented correctly; expected 4 spaces but
| | found 0
3 | ERROR | Object operator must be at the start of the line, not the end
6 | ERROR | Object operator not indented correctly; expected 4 spaces but
| | found 1
--------------------------------------------------------------------------------

Another related feature that was added is the ability to write the report output to a file in addition to writing it to the screen. This solves the problem where PHP_CodeSniffer comes across some code it cannot parse and throws PHP notices before XML output, creating invalid XML. Writing to a file is easy; just use the --report-file=filename command line argument.

All these features are currently in CVS and will be released in 1.2.0a1 very soon.

2 comments:

steki said...

thanks for the new release.

while using CodeSniffer for a large project, i got two thinks i would like to see in the next release:

- when using CS on a large lib with no coding standards so far, i got a very long list of errors. so i began to fix all these typos, but most of them where things like:
* Perl-style comments are not allowed
* Line indented incorrectly

so i like to see an option:

do all tests but without (and now a list of test to skip)
--skiptest=PEAR.WhiteSpace.ScopeIndent,
PEAR.Commenting.InlineComment

or an external file with sniffs to skip

so i can concentrate on importent things.

using the new option --sniffs =... would not help as i had to type in every single test.

i dont want to create my own sniff rules, as the goal is that the library should be compatible with PHP, PEAR, ... at the end

- the second wish (a small one):
could the report be sorted by files in alphabetical order?

best regards
stefan

Greg Sherwood said...

Hi Stefan,

If you post your feature requests to the PEAR bug tracker, I won't forget about them :)