It's time for another mtrack update; here's what's new:

• Add "Snippets" feature; works like pastebin, but allows for comments to be supplied in wiki syntax in addition to the code or text snippet that you're pasting
• Add Custom Field support. This is implemented by modifying the schema (custom fields always have an "x_" prefix).
• Improvements to the "custom ticket query" screens, including ability to select which columns (including custom fields) are included in the results
• Fix an issue with sorting the "Remaining" time column
• Fix some IE compatibility issues
...

Continue reading on wezfurlong.org

In which I present some analysis of what Brooks' Law implies for scheduling on late projects. Read on for an overview of Brooks' Law, its components, how to model its effects on your schedule, and some caveats and conclusions to the analysis.

Today Facebook released a HipHop l33t shizznizzle of a tool for PHP but sadly it’s not the PHPVille I was hoping for. PHPVille is probably a game that encourages PHP community members to work on the PHP project (over at php.net). Much like FarmVille, users simply cannot stop playing! Possible game features: Each SVN commit [...]

Paul has written a few posts about members of the PHP community and this inspired one by me. It’s about a guy few hear about but all benefit from and his name is Felipe Pena. Somewhat anonymous in the PHP community, he’s a true superstar within the PHP.net project itself. His path began with several [...]

While exploring ways to improve how the PHP project finds and teaches new contributors, I wondered how other projects do it so started the process of evaluating several. So far I’ve fallen in love with the Ubuntu approach because they offer a ton of information and promote it well. The PHP project does neither of [...]

I stumbled upon a nice source of PHP information today (I’ve apparently been hiding under a rock) and think other PHP geeks will be interested. The audio version of the Zend Conference Talks are online, meaning, you may freely listen to them at your leisure. So go there now, download the talk slides, then press [...]

PHP Logos Welcome to the [almost complete] history of PHP logos. There have been a few changes over the years, so let’s explore them now: How this was created PHP 4.0.0 added main/logo.h which contains the logos themselves (as text (a bunch of numbers (magic))) so I checked out every version of this file from [...]

In response to a recent post by Hannes Magnusson titled “8 reasons why you should *not* write for the php.net manual“, I offer about 8 reasons why you should. Although at times subtle, Hannes shares good points regarding some of the frustration that comes with working on a successful (and old) Open Source project like [...]

While reminiscing what PHP was like back in the early early years, I stumbled upon a little historical nugget from the old website. Do you know what the acronym PHP stands for? Many of us do, or think we do, but just in case take a guess from: PHP: Hypertext Preprocessor PHP: Hypermedia Preprocessor PHP: [...]

Here are 20 possible reasons why PHP functions lack consistent names and parameters. Learning the definition for every PHP function is truly an amazing feat and I doubt this has been attempted or accomplished by anyone. At least, by any sane human. And references are named references because they are designed to be referenced, right? [...]

Today (or yesterday, or maybe tomorrow) marks a big day in my life because it’s the day I became editor of the PHP Manual. The request to take over as editor was not easy because it’s a position that requires work and responsibility… and consistent time. Working on the manual (something I’ve done since 2001) [...]

It’s been a long road and exactly five years (35 releases) since the much discussed and highly controversial PHP directive register_globals has been disabled by default in PHP. After sifting through the mailing list archives, the following set of information has been compiled. Feel free to make additions, corrections, and report register_globals memories! First, a [...]

For a while now, I've been keen to build a html Sanitisation solution for PHP. Where else would I end up putting it other than in Zend Framework? As I've explored in past articles [1] [2], html Sanitisation in PHP is a very inconsistent practice. Sanitisers like htmlPurifier are very secure out of the box but undeniably slow and resource intensive while others based on regular expression powered html parsing are much faster but tend to lose out a lot in the security stakes. Isn't it possible to create a sanitiser that is both secure by default and performs well?

This was the core of the idea that became Wibble, my prototype for Zend\Html\Filter. Wibble borrowed sanitisation routines from a few programming languages to ensure secure operation, but relied entirely on PHP DOM and html Tidy for speed and html parsing. The resulting prototype was benchmarked [1] which proved that while Wibble could be faster than even regular expression based sanitisers (in scenarios where html was being manipulated) it most definitely would be faster than htmlPurifier - without sacrificing security. Thus Wibble is capable of the best of both worlds - security and performance. The existing tradeoff in current solutions no longer applies.

You may read and comment on the proposal here: http://framework.zend.com/wiki/pages/viewpage.action?pageId=25002168. The proposal is up for review for Zend Framework 2.0.

Collecting Garbage: Cleaning Up

London, UK Monday, September 6th 2010, 09:23 BST

This is the second part of three-parts column that was originally published in the May 2009 issues of php|architect.

Part one is here.

In this second part of the three part column on the new garbage collecting mechanism in PHP 5.3, we'll dive into a solution to the problem with circular references. If we look quickly back, we found that by using code like the following, an in-request memory leak is created:

Traditionally, reference counting memory mechanisms such as PHP uses, fail to address those circular reference memory leaks. Back in 2007 while looking into this issue, somebody pointed me to a paper by David F. Bacon and V.T. Rajan titled "Concurrent Cycle Collection in Reference Counted Systems" . Although the paper was written with Java in mind, I started to play around with it to see if it was feasible to implement the synchronous algorithm as outlined in the paper in PHP. At that moment I didn't have a lot of time, but along came the Google Summer of Code and we put up the implementation of this paper as one of our ideas. Yiduo (David) Wang picked up this idea and started hacking on the first version as part of the Summer of Code project.

Explaining how the full algorithm works goes slightly too far for this column, but I will try to explain the basics. First of all we have to establish a few ground rules. If a refcount is increased, it's still in use and therefore not garbage. If the refcount is decreased and hits zero, the zval can be freed. This means that garbage cycles can only be created when a refcount argument is decreased to a non-zero value. Secondly, in a garbage cycle it is possible to discover which parts are garbage by checking whether it is possible to decrease their refcount by one, and then check which of the zvals have a refcount of zero.

To avoid having to call the checking of garbage cycles with every possible decrease of a refcount the algorithm instead puts all possible roots (zvals) in the "root buffer" (marking them "purple"). It also makes sure that each possible garbage root ends up in the buffer only once. Only when the root buffer is full, the collection mechanism starts for all the different zvals inside. This diagram shows this in step A.

In step B runs the algorithm a depth-first search on all possible roots to decrease the refcounts of each zval it finds by one, making sure not to decrease a refcount on the same zval twice (marking them as "grey"). In step C the algorithm runs again a depth-first search from each root node, to check the refcount of each zval again. If it finds that the refcount is zero, the zval is marked "white" (blue in the diagram). If it's larger than zero, it reverts the decreasing of the refcount by one with a depth-first search from that point on and they are marked "black" again. In the last step (D) the algorithm walks over the root buffer removing the zval roots from there, and in the mean while checks which zvals have been marked "white" in the previous step. Every zval marked as "white" will be freed.

Now that you have a slight understanding of how the algorithm works, we will look back on how this weaves in with PHP. By default, PHP's garbage collector is turned on. There is however a php.ini setting that allows you to change this: zend.enable_gc.

When the garbage collector is turned on, the cycle finding algorithm as mentioned above is executed whenever the root buffer runs full. The root buffer has a fixed size of 10.000 possible roots (although you can change this easily by re-compiling PHP and changing a constant). When the garbage collector is turned off, the cycle finding algorithm will never run. However, possible roots will always be recorded in the root buffer, no matter whether the garbage collection mechanism has been activated with this configuration setting. If the root buffer becomes full with possible roots while the garbage collection mechanism is turned off, further possible roots wil

Truncated by Planet PHP, read more at the original (another 2337 bytes)

Ran into a bug yesterday, where http://pecl.php.net/uuid in combination with http://pecl.php.net/imagick yielded a segfault when using uuid_create(). GDB backtrace looks like this (without the exact place where it happens in libuuid, as there is unfortunatly no libuuid1-dbg-package in current Ubuntu versions):

gdb --silent --ex run --args php -r "var_dump(uuid_create());" #0 0xb6e85321 in ?? () from /lib/libuuid.so.1 #1 0xb6e862bf in uuid_generate () from /lib/libuuid.so.1 #2 0xb6bcc67a in zif_uuid_create (ht=0, return_value=0xbffff1e8, return_value_ptr=0x0, this_ptr=0x0, return_value_used=1) at /usr/src/pecl-uuid-trunk/uuid.c:182 #3 0x0835d26a in zend_do_fcall_common_helper_SPEC (execute_data=0x894ed4c) at /build/buildd/php5-5.3.2/Zend/zend_vm_execute.h:313 #4 0x08333d8e in execute (op_array=0x891c464) at /build/buildd/php5-5.3.2/Zend/zend_vm_execute.h:104 #5 0x082fe283 in zend_eval_stringl (str=0xbffff998 "var_dump(uuid_create());", str_len=24, retval_ptr=0x0, string_name=0x871f2fc "Command line code") at /build/buildd/php5-5.3.2/Zend/zend_execute_API.c:1172 #6 0x082fe422 in zend_eval_stringl_ex (str=0xbffff998 "var_dump(uuid_create());", str_len=24, retval_ptr=0x0, string_name=0x871f2fc "Command line code", handle_exceptions=1) at /build/buildd/php5-5.3.2/Zend/zend_execute_API.c:1214 #7 0x082fe4a3 in zend_eval_string_ex (str=0xbffff998 "var_dump(uuid_create());", retval_ptr=0x0, string_name=0x871f2fc "Command line code", handle_exceptions=1) at /build/buildd/php5-5.3.2/Zend/zend_execute_API.c:1225 #8 0x083a0579 in main (argc=3, argv=0xbffff854) at /build/buildd/php5-5.3.2/sapi/cli/php_cli.c:1235

The interesting thing is, the crash happens in libuuid, but only if imagick is enabled. Let’s see what Valgrind says:

valgrind -q php -r "var_dump(uuid_create());" ==25103== Invalid write of size 2 ==25103== at 0x5517321: ??? (in /lib/libuuid.so.1.3.0) ==25103== by 0x55182BE: uuid_generate (in /lib/libuuid.so.1.3.0) ==25103== by 0x57D0679: zif_uuid_create (uuid.c:182) ==25103== by 0x835D269: zend_do_fcall_common_helper_SPEC (in /usr/bin/php5) ==25103== by 0x8333D8D: execute (/build/buildd/php5-5.3.2/Zend/zend_vm_execute.h:104) ==25103== by 0x82FE282: zend_eval_stringl (/build/buildd/php5-5.3.2/Zend/zend_execute_API.c:1172) ==25103== by 0x82FE421: zend_eval_stringl_ex (/build/buildd/php5-5.3.2/Zend/zend_execute_API.c:1214) ==25103== by 0x82FE4A2: zend_eval_string_ex (/build/buildd/php5-5.3.2/Zend/zend_execute_API.c:1225) ==25103== by 0x83A0578: main (/build/buildd/php5-5.3.2/sapi/cli/php_cli.c:1235) ==25103== Address 0x30 is not stack'd, malloc'd or (recently) free'd ==25103== ==25103== ==25103== Process terminating with default action of signal 11 (SIGSEGV) ==25103== Access not within mapped region at address 0x30 ==25103== at 0x5517321: ??? (in /lib/libuuid.so.1.3.0) ==25103== by 0x55182BE: uuid_generate (in /lib/libuuid.so.1.3.0) ==25103== by 0x57D0679: zif_uuid_create (uuid.c:182) ==25103== by 0x835D269: zend_do_fcall_common_helper_SPEC (in /usr/bin/php5) ==25103== by 0x8333D8D: execute (/build/buildd/php5-5.3.2/Zend/zend_vm_execute.h:104) ==25103== by 0x82FE282: zend_eval_stringl (/build/buildd/php5-5.3.2/Zend/zend_execute_API.c:1172) ==25103== by 0x82FE421: zend_eval_stringl_ex (/build/buildd/php5-5.3.2/Zend/zend_execute_API.c:1214) ==25103== by 0x82FE4A2: zend_eval_string_ex (/build/buildd/php5-5.3.2/Zend/zend_execute_API.c:1225) ==25103== by 0x83A0578: main (/build/buildd/php5-5.3.2/sapi/cli/php_cli.c:1235) ==25103== If you believe this happened as a result of a stack ==25103== overflow in your program's main thread (unlikely but ==25103== possible), you can try to increase the size of the ==25103== main thread stack using the --main-stacksize= flag. ==25103== The main thread stack size used in this run was 8388608. Segmentation fault

Not really any more helpful. After two hours debugging the issue with the help of Mikko and Pierre we found out, that pecl/imagick is linked against libuuid too:

ldd /usr/lib/php5/20090626+lfs/imagick.so (...) libuuid.so.1 => /lib/libuuid.so.1 (0xb7086000) (...)

For whatever reason this is happening, this is most likely the root cause of the issue.

Solution (sort of)

pecl/uuid was loaded by /etc/php5/conf.d/uuid.ini and pecl/imagick by /etc/php5/conf.d/imagick.ini. As they are loaded in there alphabetical order, imagick initialized before uuid. Renaming /etc/php5/conf.d/uuid.ini to /etc/php5/conf.d/00-uuid.ini fixed the issue, as uuid is than initialized before imagick and the segmentation fault was gone.

Not sure about that, but maybe it would be a good idea to check in PHP_MINIT(uuid) in pecl/uuid if pecl/imagick has been initialized before and warn the user about it?

I recently began working with an organization and we are in the midst of making some changes. They were not far enough along that any changes now make that much of a difference, but it was an odd set of changes nonetheless.

Initially, the client had started with CodeIgniter (during a phone call earlier, there was interest in ZF2, but when I started they’d gone with CI). One of the first things I tried to do was to get unit testing set up with CI, and the CIUnit failed out of the box. We’re using PHP5.3, and apparently CIUnit doesn’t work well with PHP5.3. I was on the fence about whether we should try to use a vanilla phpunit setup, but I counseled against it (was that wise?). CI was not hurting anyone, but it’s also PHP4 compatible (still), which strikes me as a drag. I can’t quite put my finger on it, and I know that ‘our’ code can still take advantage of PHP5 stuff. However, with things like split() being deprecated in 5.3, I think we might start seeing more warnings if/when we started exercising more of CI. I might be wholly wrong though. I had a bad experience (multiple) with Kohana, so don’t go there.

I’ve had zfkit in place for a while now, and suggested we look at that. However, zfkit has doctrine1 integrated, and the code the client had was already built around doctrine2. That took a little bit of cajoling to get working – once I’m a bit more comfortable with it, I plan to roll in doctrine2 support in to zfkit. I was a bit surprised to see some of the more useful aspects of doctrine1 were removed – the ‘acts as timestampable’, for example. The authors disagreed that something like that is fundamental, and offer some ways around it, but those workarounds are fairly inelegant, imo. The more I work with doctrine2, the clumsier this sort of stuff feels in PHP. Yes, I know, the language itself is the impediment, and that’s why many of my recent projects have been in Grails instead of PHP. For basic work, GORM is a lifesaver compared to doctrine and other PHP ORM tools. That said, phpunit is miles faster than unit testing in Grails, so there’s always tradeoffs. Also, this client has chosen to do their doctrine definitions in php docblock annotations. They irk me.

Still, we’ll have managed to port over the base of the existing app, have a replicatable(?) data story (drop/update schema, load test data) and the start of unit tests (possibly automated later with phpundercontrol) in less than a week’s time. This doesn’t feel too bad, all in all.

I did run in to a ‘problem’ (well, for me anyway) in that the client’s Doctrine models are namespaced. Referencing PHP’s DateTime object from inside a namespaced class requires it to be referenced as \DateTime().

<?php namespace foo;   class bar { public function demo() { $now = new \DateTime("now"); } }

That was a bit new to me, and I didn’t see any specific references to that online, but I may have missed it somewhere. Would have been nice for PHP to drop up to a global scope, or understand references to its own built-in objects, rather than having to do this workaround.

Note: for some reason, an earlier posted tagged ‘business’ made it in to planet-php’s feed. It should only be pulling ‘PHP’ tags. Apologies if it seemed I was spamming the aggregator.

The central tension in the software process comes from the fact that we must go from an informally identified need that exists in-the-world to a formal model that operates in-the-computer. From “Beyond Programming” by Bruce Blum, as quoted in “The Design of Design” by Frederick P. Brooks Jr.

Just saw a comment on a forum that ‘Blockbuster has a dead business model’, and I’ve noticed that one branch of our local video store chain closed. The other store, while still big, is feeling empty.

The rise of Netflix predicted the end of video stores years ago, but it didn’t happen. At least, not quickly. The rise of ‘video on demand’ also was seen as a nail in the coffin for video stores, but it didn’t happen right away either. ‘Video on demand’ is something we’ve had via Tivo for 3 years from Amazon, but I’ve only used it once. It’s not that convenient, perhaps due to a slow internet connection?

Cable services’ “on demand” options have probably hurt video stores too, but those have been around for a long time, and it didn’t seem to hurt video stores that much.

The advent of redbox and similar services, allowing quick, cheap rental of popular videos in convenient places – this is likely hurting video stores too.

It does seem the local video store may be in danger, and I’ll offer a few suggestions for local video stores who want to stay in business, and perhaps thrive.

1. Offer dropoff service with various local merchants. If I could drop off video-store X videos at the local grocery, subway, dry cleaners, gym, etc., even if I couldn’t rent another one, it would save me a trip and add to the convenience factor of dealing with your store.

2. Offer in-store systems to give me recommendations. I’ve been surprised that in 10 years, I’ve never seen a kiosk in any store that would let me hit IMDB or something similar. I can turn to the net and get movie recommendations, reviews, etc, but I’ve never been able to do that *at the moment I’m trying to make a decision*. Yes, many people come get the new releases and that’s it, but many stores have a large vast back catalog of items that just sit unwatched because people don’t know anything about them. Yes… it might be that videos people find on the web aren’t in stock. So… keep track of the movies people are looking up and… *get them* to rent.

3. Offer meal/movie deals with local eateries. Our local video store has a Burger King, Dairy Queen, PIzza Hut, Subway and private Greek and Chinese restaurants in the same shopping plaza – 1 minute walk from store to store. Yet none of them have ever paired up. I sometimes order my Chinese, walk down to the video store, rent a movie, come back and pick up the Chinese. I can’t be the only one doing this, but I bet if more people were offered $1 off a movie rental with a Chinese takeaway order, they’d be back. Or just a $20 “dinner/movie” package – here’s your coupon, go pick up the movie, and dinner’ll be ready in 10 minutes.

Eventually – years from now, even more people will live a digital lifestyle, and most movies will be available at the click of a button. But… I suspect it’ll be newer releases for a long time. Local video stores can survive for many years by offering a better shopping experience and catering to the niche interests.

I love LaTeX for any kind of text writing (actually typesetting), simply because it creates so nice looking and consistent layouts. And, of course, because I can write it in my favorite text editor. We use LaTeX especially for presentation slides at Qafoo, since the beamer package provides such a convenient environment. Combined with listings package, presenting source code snippets with nice syntax highlighting has never been easier. However, there was one problem we did not solve, yet, until some days ago: Highlighting certain source code lines of a listing on different slides.

Jos Poortvliet did an interview with me for dot KDE in this summer's aKademy and it has been online for a while now. In it we discuss things like Midgard as a storage engine for desktop applications, and Maemo's open QA process for Downloads applications. Some excepts:

At maemo.org we have an appstore for FOSS applications on the Maemo platform. This appstore is enabled by default on all Nokia N900s so we wanted to have some quality control. We had to create our own appstore approval process, compatible with the FOSS philosophy. Now any developer can submit an app, and anyone can test and vote. The whole process is completely transparent, auditable and visible. And it also provides a feedback channel from testers and users to the developers!

...

Midgard is a data storage service. Whether you write desktop or web applications, instead of coming up with your own file format, you just use Midgard. You can work more easily and object-based. Users have many different devices these days, so Midgard has strong replication features to synchronize between different systems. Midgard is built on top of GObject; we provide bindings to a bunch of different languages so developers can choose the tools they like - PHP, Python, Javascript. Currently (as in now, while we're talking) Qt bindings are being developed here at Akademy.

Read the whole interview.