PHP - Leonid Mamchenkov

ORM Designer

Here is a tool that might help you with your MVC framework, like CakePHP, Symfony, and others – ORM Designer. In essence, it is a graphical user interface for drawing a visual representation of your project (such as an Entity Relationship Diagram (ERD)) and than converting it into the code. You can specify which framework and which ORM you want to use and it will generate the appropriate bits and pieces. What’s even more interesting is that it has import functionality, which means that you can start using it with an existing project. Here is the video that shows and explains more.

[youtube=http://www.youtube.com/watch?v=FNlmU6zX5Ug]

Of course, I got excited about it, downloaded and installed. Two things that disappointed me were:

It’s a native Windows application, which runs on Linux through the wine emulator. While it works fine, I’d much prefer a native application that I could integrate with the rest of my development environment.
CakePHP import is not supported at this time.

Other than that though, it looks very promising. I’ve seen quite a few applications that help with database design, and ORM Designer stands well in that row. You can create entities, define fields, specify indexes, and associate entities with each other using relationships. Many-to-many relationships are supported, as are entity inheritance. While inheritance does make it for a bit more complicated structure of the project (with app/models/base/ folder for CakePHP), it’s very nice to have such support for bigger, more complex projects.

The project is commercial, with a 14 days evaluation version available for download. If you like it enough to buy, the price is very reasonable – 99 EUR per license.

Try and see if you like it, and provide some feedback to the guys who are developing it. ORM Designer has all the chances of becoming an extremely useful tool and since it is still in its early development, your feedback would be of the most value.

The saga of a lost connection

I came across an interesting problem at work a couple of days ago. One of our CakePHP-based applications has a scheduled task (cronjob) that runs every minute and imports mail from a number of IMAP mailboxes into the database. All of a sudden the script broke. In a very particular way. Not a single line of code was change in the script itself, and there were no significant changes in the rest of the application that the script is using. But one day, for some reason, it started to import emails from only half of the mailboxes, completely ignoring the rest.

A lengthy midnight drunk debugging session helped to realize the problem. Apparently, in the list of mailboxes to check there were a few old mailboxes, which were not used anymore for any email. The script was checking them but they were always empty. Server administrator removed those mailboxes during a scheduled maintenance window. Now, the script couldn’t connect to the mailbox anymore. No problem, there was error handling for this case. But was error handling didn’t take into account is the combination of time it takes timeout on the non-existing mailbox check and the database connection timeout.

It so happened, that these few old mailboxes were right in the middle of the list of all mailboxes. The script was importing mail from the first few mailboxes just fine. But then while it was taking a long time to timeout on a few non-existing mailboxes, the database connection got closed due to the inactivity timeout. This wasn’t handled properly. The script was only checking for the existing database connection at the beginning, when it was opening the connection – not later on. As the result, nothing could have been imported from the rest of the mailboxes.

As always, once you know the problem, you also know how to fix it or work around it. But troubleshooting a problem like this is tricky. Seeing the queries working for one mailbox and not working for another is disorienting at least. Having no recent changes to the application doesn’t help either. And the problem being not in the code itself, but in the resources is also not very obvious.

So, here you go. If you ever have a weird problem like that – check that your resources (database connection, network connection, file handler) are still available to you.

CakePHP + GraphViz = making sense of a numerous models

NOTE: THIS IS VERY MUCH OUTDATED! Read about the update or go directly to GitHub repository for the new version.

I have a task at hand. I have to re-introduce myself to a rather large codebase. It’s a project that migrated to CakePHP a couple of years ago and haven’t seen since. There was a whole team of people working on it sense then, and now I need to make sense of all those changes that were done and help reorganize and refactor them a bit. When I looked into CakePHP’s models/ folder, I was surprised to find 50+ models there. Each and every one of them links to other models. And documentation is practically non-existing. How do I go about it? I hack up a little script to help me out.

There is a really elegant and beautiful tool for graphing things – GraphViz. If you haven’t heard of it, you need to drop whatever is that you are doing and familiarize yourself with GraphViz. Right now. Right. This. Second. You are missing out a whole universe until you do so. I’ll wait.

Now that you are back, I just want to mention a very slick tool, which is a part of GraphViz package – dot. It is a simple language in which you can describe graphs. Sort of like “A goes to B, which goes to C”. You specify your graph in a very human readable format in a text file, and then dot will transform that text file into an image format of your choice (PNG, JPEG, GIF, etc). The primary beauty of this is that those text files can be generated automatically by using pretty much any programming language.

So here is what I did. I assumed the following:

Project documentation should be in app/docs/ folder. That’s where I’ll put the script and that’s where it will generate the dot configuration, dot later will generate the graph of all my models and their relationships.
Main project application folder is app/. Models are stored in app/models/ folder.
Project can have a number of plugins, which can have their own models, which I still want to know about. Plugins are in app/plugins/ and if plugin xyz has models, they are stored in app/plugins/xyz/models.
My project is under version control. Specifically I use Subversion, but it’s easy to adjust the script to support other systems.
I can get current project revision by running a command in shell. For Subversion that is /usr/bin/svnversion.

I probably assumed a whole bunch of other things, but you can get an idea of how simple the setup is from the above ones.

Here is how I generate a graph of all models and their dependencies:

cd app/docs/
php -f graph.php > graph.dot
dot -Tpng graph.dot > graph.png

Obviously, I can’t show you the full graph from that system (it’s not open source, it’s not mine, and it will drive you insane in a matter of seconds), but here is how a small part of that image looks like.

There is a different colour for each type of model relationship ($belongsTo, $hasMany, and $hasAndBelongsToMany). Each model folder is in a separate sub-graph cluster. There is a legend graph on the image. The current time stamp and version control revision are also imposed on the image for easier referencing.

And here is the source for the graph.php script. Feel free to modify any way you like. If you spot any major bugs or better ways of doing things, please let me know in the comments.

Continue reading CakePHP + GraphViz = making sense of a numerous models

Whatever happened to programming

Via this Slashdot post I came across an excellent blog rant – Whatever happened to programming (and the follow-up). Subject in focus – modern programming, and how boring it have become (mostly).

Today, I mostly paste libraries together. So do you, most likely, if you work in software. Doesn’t that seem anticlimactic? We did all those courses on LR grammars and concurrent software and referentially transparent functional languages. We messed about with Prolog, Lisp and APL. We studied invariants and formal preconditions and operating system theory. Now how much of that do we use?

Of course, when a subject like that is brought up, it’s pretty much guaranteed that the web will respond with numerous discussions on if and how much of it is true, how did we get here, and how we can get out, and anything else remotely or not at all related. And that’s just what happened. You can read Slashdot or Reddit comments or Google for more. But I think, if you do programming for living, you’d probably agree with the main point of the article. And even if you won’t, it’s still fun to read. Like this bit for example:

Especially, I have learned that anything that has “Enterprise” in its name is so incredibly boring that the people who use it had to shove the name of the Star Trek ship into its title just to keep themselves awake.

On the serious note though, working with mainly two programming languages – Perl and PHP, I see that there is indeed a difference to the “being boring” degree. PHP is way more boring than Perl. Surprisingly even with Perl being so well known for its CPAN – a huge archive of modules and libraries to use. I guess it has something to do with There Is More Than One Way To Do It – motto of Perl.

Monitoring PHP errors, warnings, and notices

There are a number of ways to monitor PHP errors, warnings, and notices. You can have your application code trigger some error handling, you can use PHP built-in methods, you can have some scripts running in the background analyzing logs, etc. While you already probably do some of it, here is something that you’ll find handy.

First of all, don’t log all PHP noise into a single file. You can easy make separate logs for each project. Somewhere at the top of your project, when it only starts loading, add the following configuration settings:

ini_set('error_reporting', E_ALL);
ini_set('log_errors', '1');
ini_set('error_log', '/path/to/project/logs/php_errors.log');
ini_set('display_errors', '0');

This will enable logging of all errors, warnings, and notices into a file that you specified. And, at the same time, it will disable the display of all the logs to your visitors (something that you should definitely do for a production server).

One you’ve done that, you’ll notice another problem. If your application is of any considerable size and/or if it uses a lot of third-party code, you’ll get buried in all those warnings and notices. The file will quickly become very large and boring and leave your attention span. Not good. While you can fight the size of the file with a tool like logrotate, the boredom is a more serious problem. The same notices and warnings appear over and over and over. You’ll fix some of them and the others will stay there forever. What you need as a way to have a quick overview of what is broken and what is noisy.

Today I wrote a quick cronjob to do just that. Here it is in all its entirety.

#!/bin/bash

# This script parses the project PHP errors logs every hour, creates the summary of all
# errors/warnings/notices/etc and emails that summary to the email specified below.

EMAIL="me@here.com"
SUBJECT="here.com PHP errors summary for the last hour"
PHP_ERRORS_FILE="/path/to/project/logs/php_errors.log"

# The log starts with timestamp like [01-Mar-2010 12:48:56]. Timestamp + 1 stamp occupy about 24 bytes
ONE_HOUR_AGO=`date +'[%d-%b-%Y %H:' -d '1 hour ago'`

# We only need that double backslash because date pattern uses square bracket
grep "^\\$ONE_HOUR_AGO" $PHP_ERRORS_FILE | cut -b 24- | sort | uniq -c | sort -n -r | mail -s "$SUBJECT" $EMAIL

You can drop this file into /etc/cron.hourly/report_php_errors.sh, change permissions to executable, and wait for the next run of hourly scripts. If you’ve updated the variables inside the script to reflect the correct email address and path to log file, you’ll get an email every hour which will look something like this:

From: cron@your.host
To: me@here.com
Subject: here.com PHP errors summary for the last hour

  14 PHP Notice:  Use of undefined constant PEAR_LOG_DEBUG - assumed 'PEAR_LOG_DEBUG' in /some/path/to/some/file.php on line 17
    12 PHP Notice:  Undefined index:  is_printed in /path/to/something.php on line 2035
     9 PHP Notice:  Undefined index:  blah in /some/foo/bar.php on line 42
     7 PHP Notice:  Undefined offset:  1 in /some/verifier/script.php on line 120

The email will not be limited to 3 or 4 lines. It will actually contain each and every individual notice, error, and warning that occurred during the last hour in your project. The list will be sorted by how often each warning occurred, with the most frequent entries at the top.

With this list you can start fixing your most frequently seen problems, and you can also notice weird activity much faster than just checking the log file and hoping to catch it with your own eyes.

Enjoy!