Yesterday was a somewhat slow day, so I spent some time on the housekeeping of my Firefox browser.  Somehow I managed to accumulate a lot of extensions, themes, plugins, bookmarks, bookmarklets, and what not.  It felt like a good time to clean the mess up a bit.

I spent about two hours going through the list of all installed pieces, upgrading outdated versions, changing old solutions to the modern alternatives, getting new tools, and so on.   I have to say that after that effort my Firefox works faster, and it suits me better now.  One of the biggest changes from my previous setup became the use of Tab Kit extension.  Among its many features, it has the one that I’ve been passively looking for for a long time now - tab bar on the right side, instead of top, but not as a part of the sidebar, and with a tree view.  Combined with Aging Tabs extension, the result is exactly the way I wanted it.

(it is better in real life than it is on the screenshot)

Now when I open links from the site in the new tabs, these new tabs are organized in a tree like structure.  Tabs that I haven’t yet visited are highlighted in green.  Current tab is highlighted in blue, as usual.  And the rest of the tabs are coloured in different shades of grey, depending on how long ago I last viewed them.  Also, because the tab bar is separate from the sidebar, I can get an additional panel on demand, with an application that I need the most at the moment, without sacrificing my precious tabs.

And just in case you are wondering which other extensions I am using, continue reading for the list of all extensions, which was generated by one of the extensions on that list.

Application: Firefox 3.0.1 (2008071615)

Operating System: Linux (x86-gcc3)

Total number of items: 42

• Adblock Filterset.G Updater 0.3.1.3Synchronizes Adblock with Filterset.G
• Adblock Plus 0.7.5.5Ads were yesterday!
• Aging Tabs 0.7.1Makes unused tabs fade with age and highlights the selected tab.
• All-in-One Sidebar 0.7.6Sidebar control with award-winning user experience!
• Better Flickr 0.3A compilation of useful Greasemonkey scripts for Flickr. All scripts copyright their original authors. Click on the script homepage in the Help tab for more information.
• Better GCal 0.3Enhances Google Calendar with a compilation of user scripts and styles. All scripts copyright their original authors. Click on the script homepage in the Help tab for more information.
• Better Gmail 2 0.6.1Enhances Gmail with a compilation of user scripts and styles. All scripts copyright their original authors. Click on the script homepage in the Help tab for more information.
• Better GReader 0.3Enhances Google Reader with a compilation of user scripts and styles.
• Callout 0.3.2.1Notification Services for Javascript
• CustomizeGoogle 0.75Enhance Google search results and remove ads and spam.
• CyberSearch 0.9.10Adds advanced Google Search capabilities to the Firefox 3 address bar.
• Delicious Bookmarks 2.0.95Access your bookmarks wherever you go and keep them organized no matter how many you have.
• Download Statusbar 0.9.6.3View and manage downloads from a tidy statusbar
• Dummy Lipsum 2.3.0Generate “Lorem Ipsum” dummy text.
• Extended Copy Menu 1.5Provides the option to copy selection as plain text or html.
• Extended Statusbar 1.2.8A Statusbar with Speed, Percentage, Time and loaded size (like Opera’s one)
• Extension List Dumper 1.14.1Dumps a list of the installed extensions.
• Find In Tabs 0.3.1.1Searches for text in all tabs.
• Firebug 1.1.0b12Web Development Evolved.
• Firefox Showcase 0.9.4.9Easily locate any tab you’ve opened in Firefox.
• FirePHP 0.1.1Firebug Extension for AJAX Development
• Flagfox 3.2.8Displays a flag depicting the location of the current server
• Flashblock 1.5.6Replaces Flash objects with a button you can click to view them.
• FoxClocks 2.4.91Displays local times around the world in the statusbar/toolbar. Supports virtually all time zones.
• Gmail Manager 0.5.5Gmail accounts management and new mail notifications.
• Google Gears 0.4.15.0These are the gears that power the tubes!
• Google Toolbar for Firefox 3.1.20080605LTake the power of Google with you anywhere on the Web!
• GUtil! 2.3.2Google® Utilities in your Firefox Menu and Toolbar
• IMDb Preview 0.5Preview movie & actor posters when hovering over their IMDb links.
• Linkification 1.3.5Converts text links into genuine, clickable links.
• Pencil 1.0.3Sketching, simplified
• Session Manager 0.6.1.13Saves and restores the state of all windows.
• Tab Kit 0.4.3Tab grouping, multi-rows, tree view, and other tweaks.
• Tabs Open Relative 0.3.3Tabs open to the right of the current one.
• TimeTracker 1.2.4Track the time you spend browsing.
• TwitKit 1.1An extension that adds a Twitter sidebar to your browsing experience!
• TwitterFox 1.6.1This extension lets you know twitter statuses.
• Update Notifier 0.1.5.4Notifies you when updates are available for your extensions and themes.
• URL Link 2.02.2Allow navigation to broken/unlinked URLs
• View Dependencies 0.3.3.0Adds a tab listing dependencies and their sizes in the Page Info window.
• Web Developer 1.1.6Adds a menu and a toolbar with various web developer tools.
• YSlow 0.9.5b2Make your pages faster with Yahoo!’s performance lint tool.

Since we started working on SugarCRM in the office, one of the hardest tasks that we had was solving the deployment issue.  On one hand, SugarCRM comes with some really nice GUI tools, such Studio and Module Builder.  On the other hand, the system is large and complex and should be developed and tested in a separate, non-production environment.

We’ve spent a lot of effort over the last couple of month trying to solve the puzzle.  The problem is that there is a tricky combination of files updates and database changes, some of which can be just copied over while others have to be executed from the destination machine’s administration.

So, what we did first was complete separation of environments.  Each developer had his own machine on which he could install and configure as many instances of SugarCRM as he saw fit.  Also, each developer had a separate branch in the Subversion, so that he could work on his own stuff without being afraid to run into conflict with anyone.

After that, we created a development server with a checkout of common trunk.  For extra insurance, we did a checkout from a system user, who does not have any write permissions in the repository.  In this case, even if someone will accidentally try to commit from the development server, we would be sure that it fails.

Now, each developer had to merge his changes into trunk, and then test them on the development server.  This procedure is very similar to the production deployment and consisted of two parts.  Firts part was updating all the relevant files (a bit more on this in a moment) with svn update.  Second part was logging into SugarCRM and doing Admin -> Repair -> Quick Repair and Rebuild.

The graphical tools that come with SugarCRM are powerful, but a bit confusing.  The biggest confusion for me was (and maybe still is) between Module Builder and Studio.  Studio can be used to customize core modules that are shipped with SugarCRM.  The results of these customizations are stored in custom/modules directory, and when loaded into the database, can be observed in _cstm tables (for example, accounts_cstm).  This is where new custom fields and things like that are going.  Module Builder is a tool which can help you customize existing modules or build the new ones. The confusion here is because both of these tools can be used to do the same things.  But with Module Builder you’d be working closer to the core system and modifying “original” functionality.  You can build your own modules too, by the way.  The results of the Module Builder work with go into modules/ directory, and changes in the database will take place in the original tables.  One thing to remember though, is that you’ll need to push Save & Deploy button every time you are finished with changes in Module Builder.  This is like compiling and building a module.  If you forget this step, then your module will hang in its source somewhere around custom/modulebuilder directory.

Another thing to keep in mind is the sillyness of the machine trying to figure out another machine.  Meaning that Subversion will often have issues trying to figure out the changes from the last commit, and these issues would be often caused by a lot of automatically generated code by SugarCRM.  In most of these problematic cases, Subversion will just merge the changes, and this would often result in a broken system.  I’ve found at least two reasons behind these: small context size that Subversion uses (3 lines or so) confuses it sometimes, bringing it to a wrong place in the file to do the merging; and rather messy automatically generated stuff by SugarCRM - unnecessary reordering and mixed (DOS and UNIX) ends of lines in a single file.  These problems are mostly related to vardef files (vardef.php and anything *def.php) and language dictionaries (anything with *en_us*php, or whatever your locale is).  The solution we are using at the moment is simple, although a bit heavy on the manual work - instead of merging the changes and checking them every time we simply remove old versions of files and add the new ones in two separate commits.  These way Subversion treats the files as completely different ones and real removes and re-creates them instead of trying to merge.

We follow exactly the same procedure now to deploy to the production server.  We just merge code from trunk/ to branches/stable , commit, then update the files on the production server, and then do the Quick Repair and Rebuild.

The thing about Quick Repair and Rebuild is that it takes the update definitions of your forms and layouts and rebuilds compiled templates.  It also compares the structure of the database with the update definitions in the files and, if needed, updates the database scheme too.  Sometimes you’d get an error of missing table (usually custom tables with _cstm suffix) - just create an empty table manually.  Put a couple of standard fields like id_c, date_modified, and date_entered.  After that, field modifications should be OK.  In case you run into a problem with updates to several fields at once, make sure that SugarCRM put a semicolon (;) at the end of each SQL statement that it shows you in a popup window.  For some weird reason, sometimes it just works, and sometimes it tries to execute several queries without separating them one from another.

So far the setup seems to be working for us just fine, but I’m sure that we’ll have a few changes here and there.  I’ll let you know once we find any better way of doing things.  In the meantime, here are some links that might help your development efforts:

• SugarCRM Model-View-Controller (MVC)
• SugarCRM SugarBean
• SugarCRM Vardefs
• SugarCRM Performance Tweaks for Large Systems

Point Nine is looking for some talanted people.  They are working in the financial sector, and seeking Python and C++ programmers, as well as Linux system administrators.  They seem to be an interesting company, judging both by their technology stack and hiring techniques.   For example, I’ve spotted their job ad on The Daily WTF.  Cool stuff.  If I wasn’t working where I am working, I would have applied for the job already.

I’ve been a bit quiet for the last couple of month.  That’s because I was leading an ambicious project at my new job - migration of a Microsoft Dynamics CRM version 3 to SugarCRM Community Edition version 5.0.0.  There were only three people involved, non of us could afford to work full time on the project, and we only had three weeks to do it.

Read on for a story on why it took us longer, how we did, and if it was a success at all.

First of all, a little background.  The company that I work for now is in financial services sector.  CRM is one of the two vital systems, with trading platform being the second one.  All processes are tied into CRM - registrations of new clients, incoming and outgoing money, email tracking, and so on and so forth.  The system has been installed about a year ago and has accomodated data for a few thousands of clients.

Another important bit to know is that CRM is used quite a lot 24 hours a day, 5 days a week.  Some data is coming in over the weekends too, but there are no real users who access the system.  Also, since the company is working with global clientele, weekend is really between 23:00 of Friday and 23:00 of Sunday.

By now you probably have this question in your head: “Why migrate?”.  What is so wrong with the existing system that it needs to be replaced so fast and so furious?

The company was preparing for a huge expansion.  Firtst of all, there was a take over of another company scheduled later that month.  That meant an almost sudden expansion of a few thousands of clients to a few tens of thousands of clients.  Secondly, a huge marketing campaign was starting up.  Thirdly, a few key processes were about to be change (the way new clients were registered).  Fourthly, there was a huge need for automation and extended functionality.

With all that, there was a very limited in-house expertise for the existing system.   SugarCRM had this nice advantage of being implemented with LAMP stack (Linux, Apache, MySQL, PHP), which was way more familiar to most of the IT people in the company.  And all the source code was there.

Another big reason for a change, was realization of how tied in the MS CRM is into company’s network.   While considering the upgrade of the MS Dynamics CRM to version 4, most of the people were horrified of the upgrade path.  In order to upgrade the CRM, we first had to upgrade our MS Exchange server.  Which meant that we had to upgrade the operating system of the machine.  Which meant that we had to upgrade the hardware of the server.  And MS Exchange upgrade was strongly suggesting upgrades of quite a few other bits and pieces of software and hardware.  Seemed like a really complicated and dangerous thing to do.

One thing I wanted to note separately, was how easy it was to sell SugarCRM to the management.  An open source stack meant that we were could be in full control of the system.  Independent software pieces meant that we could replace and upgrade them separately.  Fully web based technology aligned very well with many other company tools.  The brief demonstration of SugarCRM got them totally convinced that the system is mature enough and can handle everything the company needs, and more.  The fact that SugarCRM could be easily installed on a laptop in just a couple of minutes helped.  The nice look and selection of themes in default out of the box installation helped.  The selection of modules in out of the box installation helped.  And the possibility to install the system with plenty of sample data helped too.  Overall, it took me less than two hours to convince the management that the system is good enough.  And I wasn’t even trying hard.

So, what’s with the timing?  We were given three weeks.  In fact, it was two weeks for migration and one week for user training.  We had to create a full replica of the existing system.  Migrate all the data and implement all the functionality to support all the company processes.  It took us three weeks to do it.  Because we had no time to train the users, and because the company was on the doorstep of a large take over, we had to suspend the CRM migration efforts.

It took the company about three weeks to integrate all the new data.  Also, quite a few processes were changed.  And the company grew plenty of new staff in all departments.

When the question of the CRM migration was important once again, we had to face quite a few changes.  Much much much more data than we originally tested with.  A lot of changes in the ways things were handled.  More users.  And more automation was needed.  The replica of the old CRM was of no value anymore.

It took us about three more weeks to adjust our installation of SugarCRM to fit the new requirements.  Even now that the migration is officially over, we still have quite a few things to fix and improve.  But that’s going to be a live process from now on.

The project was very challanging for the company, the team, and me personally.  It was a complex thing technically, politically, and socially.  For me personally, it was also quite a new management experience.  I’ve been doing some project management before, but never had to deal with such magnitude, and especially in such tight deadlines.  The good thing though was that I had to pick my own team.

Here are a few things that I’ve learned and a few things that I think are important to share.

Technology-wise, thanks heavens for CSV (file format for comma separated values).  This was the backbone of the whole migration process.  Here is how it works.  MS Dynamics CRM has a very limited exporting functionality.  Getting things directly out of the database don’t make sense at all.  For example, our specific installation had more than 220 tables, all linked to each other.  On top of that there were almost 400 views and something around 60 stored procedures.  There was no way that we could analyze these structures in any meaningful period of time.  So we did an export with entity by entity method.  Gladly, there weren’t that many entities.  There is a way to list all entities in tabular form, with columns upon columns of entity values.  The only sensible format to save that list was in HTML.

Because we had huge sets of data, MS CRM couldn’t handle the listing properly.  So we had to come up with search criterias to separate the listings into smaller chunks of about 10,000 records each.  Now, I’ll give you a second to imagine an HTML file with a table, which has around 10,000 rows and anywhere from 100 to 500 columns.

The way to handle these HTML files was to load them up in MS Excel.  It should also better be a newer version of Excel, that can handle more than 256 columns.  MS Excel is wonderful at pasing those HTML tables, but it doesn’t offer much help in saving them in CSV.  CSV in MS Excel is weird.  So, what we ended up doing was using MS Excel to convert HTML tables into older version of Excel, so that we could open them up in OpenOffice.  OpenOffice is wonderful at saving CSV.  You can pick and choose delimeters, quoting symbols, etc.  The point is to make CSV files which can be parsed by PHP.

Here are two things to remember.  First, older Excel formats can’t work with more than 256 columns.  The trick we were using was the sheets.  Copy data of the extra columns into another sheet of the same file, and it will be much easier to process later.  Second, if you will do several exports of data from MS CRM, remember that columsn won’t necessarily be in the same order.  In fact, chances are that columns won’t be in the same order.  The way go about it is to use hashes, or, sorry, associative arrays.  Take the first row, of data, which has the column titles, build an index array out of it, and then go through data rows, assigning values to appropriate keys.  Don’t use indexes for data, or you will have to inspect the order and adjust your SugarCRM importing scripts on every export.

Another technical issue is the deployment process of SugarCRM.  We are still in the mess with this and haven’t solved it properly.  We used Subversion for the version control of all the files.  But it’s not enough for SugarCRM, since some of the modules are bulit with its own Studio, and loaded with its own Module Loader.  When this happens, the database is modified, and a few local files are changed also.  We sort of have it working now, with plenty of magic and strict procedure involved, but it’s far from perfect, requires too much manual labour, and is very prone to problems still.  If you have any suggestions for this one, please let me know.

Something else worth noting, is a very unpleasant behavior of SugarCRM in terms of usage tracking.  Surely, there is an option to not send anything, but somehow I find it always sending way too much stuff out.  And the worst part is that these things are hidden, and written in a way that makes them hard to find.  have a look in include/utils/mvc_utils.php and in include/utils/tracker_utils.hp .  These are the two ugliest places.  There is another one which deals with update notifications, which I think I found, but didn’t have time to cut out yet.  I am really surprised to see these in an open source project.  And then again, I’m glad to have all the sources and being able to remove this stuff.  Who knows what goes on in closed source applications…

In terms of project management, I’ve learned to talk to people a lot.  Talking to all the people all the time was the bigger part of my participation in the project actually.  I think I’ve spent close to 80% of time just talking.  I was talking to the members of the team, trying to figure out the best ways to approach problems.  I was talking to people who were supporting MS CRM installation, trying to get as much as possible from them regarding the details of the running system.  I was talking to each user and each user group trying to see what they were using, how they were using it, and what they wanted and needed from the new system.  I was talking to the management a lot, trying to figure out where the company was going and what would be the changes in the near, intermediate and far away future.  I was also talking to a lot of  people outside of the company, trying to see how similar problems were solved there, how they managed projects of this magnitude, how they were dealing with their users, developers, and managers.

I know that many projects take a lot of talking, but in this one we had practically no documentation of the existing system and process, and we had very tight deadlines, so talking ended up being a very effecient way of getting information.

Also, talking helped a lot in extending deadlines where needed, and in keeping users calm.  Somehow, people are taking delays and system limitations very much easier if they know why those limitations are in and when there are problems will be solved.  Sometimes I had to go into technical details explaining why certain things weren’t working or were working different from the previous system.  Sometimes I had do a totally non-technical talk showing to people which other departments exist in the company and how they interact, and how things are prioritiezed and why.

As I said, I did a lot of talking.  More than during any other project that I ever handled.  Of course, I was blessed with excellent teammates who were handling the technical parts of the project.  One person was doing everything and anything about SugarCRM.  Another one was working on all the exports, figuring out other systems and getting stuff off of them.  He also helped a lot with integration of SugarCRM with the other company tools.  Great guys both of them!

Now, this pretty much covers the migration process.  But since the SugarCRM saga in our company is just started, expect more stuff posted.  In the mean time, if you have any questions regarding any aspect of this migration, feel free to ask in the comments.

If you ever mention that your web application uses database to store files, you risk being flamed into oblivion.  Indeed, in most cases, it is a bad idea, since file system is more effecient when it comes to files.  However, there are cases when it makes sense to have files saved in the database.

Maybe I am doing something wrong, but in the last six month, I had to develop at least three systems that used MySQL for file storage (uploaded files that have to be synchronized across several hosts, etc).  Yesterday, for the third time I stumbled across the same problem, that almost drove me insane.

MySQL has four data types for storing binary data - TINYBLOB, BLOB, MEDIUMBLOB, and LONGBLOB.  Somehow I always forget about these and use BLOB.  BLOB works just fine, but it has a limit on size, which is rather low - 64 KBytes.  The mean thing here is that it will work just fine with most of the test data - text files, short PDFs, and small pictures.  Once the application is tested and put into production, the corrupted files will start coming in.  Re-writing all parts that deal with uploading, moving, cleaning, escaping, and encrypting binary data takes time.  Going through file reading and writing routines is boring too, and it won’t help either.

By the time, the issue is discovered and all fields are changed to LONGBLOB, it is often very late, and you’ve lost your weekend, as well as a lot of large files. This post is an attempt to save my (and your) sanity.

Reminder: use LONGBLOB instead of BLOB for file storage, unless you are absolutely sure about the maximum size of incoming data.

One of the frequent things that I hear about programmers is that it doesn’t matter which language the person is using and which language you need him to use, because if he is any good he’ll learn and catch up pretty fast.  In other words, if you take a decent Java programmer and push him to write PHP code for you, you’ll only have issues for a few days.  Or weeks, at most.

I understand the reasons for this statement, but I don’t agree with it.  At least not completely.

Firstly, the reasons.  They are rather obvoius, but I’d rather stagte them anyway.  Computer Science is not specific to any programming language.  The concepts and approaches are more or less the same everywhere.  Flow control, data structures, and algorithms are not language specific.  Each language has its own best practices and recommended variations, but a bubble sort in PHP will be very similar to bubble sort in Java.   Then you need some common sense, which is also not laguage bound at all.

Secondly, the disagreement.  I think that the Computer Science theory and common sense aren’t the only things that make up a programmer.  What makes a lot of difference is experience.  Programming languages, in their practical applicatoin, are just collections of software - compilers, linkers, debuggers, libraries, IDEs, etc.  Like any other software, programming language software has bugs, undocumented features, and Days When Things Don’t Just Work.  It’s the experience with the language that teaches the programmer how to handle the issues of each software piece.  And that experience is priceless (almost).

Even if you’d manage to push a Java programmer into writing PHP code, that would a waste of resources.  A Java programmer is a Java programmer, not PHP programmer.  He will, of course, learn PHP nuances with time, but, he’ll probably lose a part of his priceless (almost) bagage.  Sounds a lot like misuse of resources.

Another part of my disagreement is not so much reasoned as emotionalized.  I’ve seen a few C and Java developers switch to Perl and PHP for their new positions.  Not that I was forcing them to or anything, but they did.  And the switch was moslty painful to say the least.  Here are some of the areas that I noticed as being hard to comprehend.

Compiling vs. interpreting. Those people who were used to their compilation process were missing something for the first few days.  Some needed as much as a week to adopt, even though write-save-reload browser was done a few hundred times a day.

Debugging. There are two major camps here.  In the first one are all those people who live in the debugger.  They know all the keyboard shortcuts and they have their highlighting customized.  In another camp are people of the simpler nature, those who use print() and die() for most of their debugging needs.  It seems that most people coming from C and Java prefer the debugger way.  Most of the interpretted languages do have either a standalone debugger or a built in debugging tool, but it seems that the majority of interpretted language crowd use the print() and die() approach.

Sigils. If you don’t know what a sigil is, read this Wikipedia page.  Because you do know what it is.  Many strong type language don’t use any sigils.  Most of the loosely typed languages do.  Furthermore, when both the language from which you are changing and the language to which you are changing use sigils, chances are there will still be a difference.  PHP, for example, uses $ for both scalars and arrays.  In Perl though, you’ll get a $ for scalar, @ for array, and % for hash.  Perl’s sigils are extremely helpful when figuring out someone else’s code. I remember the pain of having just a $ in PHP, when I was learning it.  And I can’t even imagine how confusing it is for people who are used to non-sigilized programming languages.

Types. As already mentioned above, strong typed language programmers can be often confused with the fact that variables can change their type on the fly, and that they don’t even need to be declared before use.  Loosely typed language programmers will often complain about the requirement to define their types.  Three of the most common questions that I’ve heard regarding this matter were:

• “How do I define an array of elements of a certain type of a certain length?”
• “Is this line a piece of non-sense or does it really do something:   $sum += 0; ?”
• “What’s wrong with writing:  int amount; amount = 2.5; ?”

There are, of course, more areas than just those - include pathes, include files, OOP, database abstraction, loops (”What the heck is foreach?”), memory management, libraries, and so on and so forth.

Even the list of the resources for each programming language takes time to build.   Yes, time.  And time is one thing that’s always against us.  Everything else we ca handle.

This is a quick follow-up to yesterday’s post - “Where did all the PHP programmers go?“.

First of all, let me take the moment and say “Wow!”.  Somebody submitted the post to Reddit and it made it to the front page and got an unbelievable amount of comments.  Almost 500, and still coming.  Thank you all.

Secondly, the comments on this blog are fixed finally.  Murphy’s Law in action - they got broken just before the wave came in and they got fixed shortly after.

Thirdly, I should clear up a few things.  My apologies for getting you guys confused.  I never asked any candidate to compare sorting algorithms, much less to implement them.  I asked to sort an array.  I was expecting one of those PHP function calls in return.  But I only got it a few times.  Many candidates didn’t know how to sort an array (apparently they use MySQL to sort an array).  A few suggested “bubble sort”.  Probably thinking that the tasks for testing sorting algorithms.  One even went as far as implementing a bubble sort in PHP.  With pen and paper.  This one was the toughest to decide about, by the way.

Fourthly, the correction.  The language is indeed called Ruby, not Ruby on Rails. I am aware of that.  I was just trying to catch a thought.  Thanks for pointing it out though.

Fifthly, explanation for the pen and paper.  Yes, I know that programmers are used to typing code.  I know that they are used to their tools and online references.  But.  This is an interview.  My time is limited and I have to make a decision.  If I give all the tools and references to my mother, she will be able to solve the problem I am giving in reasonable time.  She is not a PHP developer.  She has no experience with PHP.  But she has enough of common sense to do it.  If I take everything away - she won’t be able to do that.  But any semi-decent programmer will do.  Further on, I am not feeding the resulting paper into the machine.  The only parser that sees that code is the one embedded in my brain.  And I assure you it is very tolerant to minor syntax errors and missing parameters.  I want to see the process.  The approach. Some data structures and algorithms.  A bit of style in variable names, indentation, and empty lines, if I am lucky.  That’s all.

Sixthly, on the exercise itself.  I like to think that I am pretty flexible with answers.  For this particular exercise, a Perl programmer inside me thinks associative array is the best data structre.  (And yes, before you start bashing further, I know that associative arrays in PHP aren’t the same as hashes in Perl.)  I can accept an OOP solution just fine.  What I find hard to accept is a single dimensional array with hopping over a pre-defined number of fields per record.

Seventhly, this post, once it got to reddit and then furthermore to other news streams, generated more candidates and hints to where to find them, then all of my prevoius efforts.  Thanks to all of you who sent me resumes, links, and pointers.  My inbox is a bit overwhelmed right now, but I’ll reply to everyone over the next few days.

Thanks a lot to all of you.

During the last six month or so, I’ve been looking to hire a PHP programmer for at least three companies.  I have spoken to quite a few people on the phone, reviewed a bunch of resumes, and even interviewed a few.  Out of all those candidates I recommended to hire exactly zero.

Before you start bashing my high standards, let me explain.  I wasn’t looking for a rocket scientist or anything remotely similar.  Not even a senior PHP developer.  Someone with enough knowledge to take over maintenance of a couple of projects, both of which are based on famous open source software - CakePHP and WordPress.

I can understand that not everyone have worked with or even heard of CakePHP or WordPress.  I can understand that getting used to that source code and going through documentation might need some time.  I can understand that not everyone is familiar with open source software development model and that not everyone has worked in groups, so familiarity with version control software, documentation tools, and bug tracking was never a requirement.

What I cannot understand is why a person who have (according to him) developed more than two dozens of web projects with PHP and MySQL cannot write the simplest piece of code with pen and pencil.  What I cannot undertand is how a “senior web developer” with years of PHP experience and team leading becomes useless when his Dreamweaver is taken away.  What I cannot understand is why people with more than one Bachelor Degree in Computer Science recommend using bubble sort.  What I cannot understand is why programmers start teaching the potential employer about the interviewing process instead of answering technical questions.  And what I don’t understand is why technical people with years of team work, get pissed off or burst into tears when you ask them a technical question, and a simple one at that, during the job interview.

If you are wondering what sort of questions I’ve been asking, here is an example.  A simple questions would be something like: “What is the difference between the stack (also known as FILO) and the queue (also known as pipe, also known as FIFO)?“.  Most of the answer is already in the questions, isn’t it?

Those of the candidates who were boasting about their years of experience and prevoius projects, were given a simple programming task, which could be something like: “Using PHP programming language, create a list to store information about people.  For each person you’ll need to store name, age, and gender. Populate the list with three sample records.  Then, print out an alphabetically sorted list of names of all males in that list. Bonus points for not using the database.“.  Each candidate was given a piece of paper, a pen, and unlimited amount of time.  And in the last six month I haven’t seen one candidate who could write the code to solve that problem.

We’ve been through all job sites, newspapers, local and foreign forums, and recruiting agencies, trying to find the candiate.  We haven’t found even one.  At least three are needed right now.  More will be needed in the nearest future.

Hopefully, by now you will agree with me on that the situation with the human resources on the island of Cyprus is disastrous.  There is more demand than there is supply, and it’s not getting any better.

Those of you who argue in favour of Cyprus being a small, unimportant country in the middle of technological nowhere, might want to wait.  Last year I’ve been in Greece at the Greek Blogger Camp.  This year I’ve been in Amsterdam at The Next Web Conference.  At both events I’ve chatted with a lot of people from all over Europe and the USA.  I’ve also been all over forums and job web sites both local and foreign.  And the feeling I’ve got is that the problem is not Cyprus specific, although, of course, Cyprus has it a bit worse than others, due to its position in the technology world, as well as geographical location.

While still spending a lot of time looking for a PHP programmer, I was thinking about the roots of the problem.  PHP seems to be quite a popular language.  So, why is it such a problem to find a good PHP programmer? (note: “good”, not “great” or even “very good”) Thinking about the roots of the problem, I got this theory, which isn’t even a theory yet, but rather a raw chain of assumptions and conclusions.  Here is how it goes.

PHP is an ugly language

I know a few good programmers personally.  I also read blogs and comments of a few more good programmes on the Web.  And even though many of them use PHP often, or even on a daily basis, I don’t remember anyone of them every saying that they enjoy PHP.  If given the choice of a programming language for a new project, they’ll pick anything - Java, C, Python, Perl, Ruby, Haskell… Anything, but not PHP. PHP has its pros, but being a beautiful or convenient language is not one of them.

PHP is newbie safe

One of the reasons for why PHP is so popular is because it is newbie safe.  You don’t need to know much about anything to start programming in PHP.  Most of the hosting companies will provide you with a PHP enabled hosting account for just a few dollars a month.  You can write PHP in any text editor, so you won’t need a high end machine or expensive IDE.  PHP.net web site has all the documentation and examples that you’ll ever need, so you don’t need to study hard in college or pay for subscription to developers’ network.  All of these make PHP very attractive to beginner programmers.

PHP avoidance

Most of the good programmers that I know, have learned PHP to some degree.   Most of the bad programmers that I know, have also learned PHP to some degree.  But for good programmers PHP was either not the first programming language under their belt, or they’ve moved forward to some other programming language.  Most of the bad programmers that I know, only know one programming language - PHP - and they don’t know it good enough.  So, for good programmers, learning and using PHP is more like a temporary state, while for the bad programmers using PHP is more like a constant state.

PHP is rich with secondary reasons

There are many reasons for why PHP is so popular.  It is free.  It is open source.  It is easy to setup.  Most hosting companies offer PHP-enabled packages, as well as a lot of PHP software pre-installed.

With primary technical reasons (execution speeds, required resources, development speed, etc) not being very different from many other programming languages, PHP wins a lot of popularity with its secondary powers.

PHP is getting mature

PHP started off as a handy Perl library for web development.  It grew and expanded over time.  And so did the projects which were written in PHP.  If, before, most of PHP scripts were doing the simplest of things, such as contact and registration forms, visitor counters and some templating, then now most projects are closer to full scale applications with user management, financial operations, high availability and load balancing setups, etc.

The moment of conflict

And here comes the moment of conflict.  The complexity of PHP applications is growing higher and higher (see above).  And the language is not beautiful enough to attract good programmers and make them stay (see above).  The result?  More and more applications are written by underqualified programmers, and it becomes harder and harder to find qualified personnel (the complexity of your own projects are growing too).

Questions?

How can we attract good programmers to PHP development?  What are really the reasons for using PHP all that often, if it shares the biggest problem with the other languages - impossibility of finding qualified personnel.  Is there any other programming language that can solve this problem?  Is there any solution at all?

Solutions

These, of course, I don’t have, as usual.  But.  I am looking with interest at hosted application services.  The ones like from Amazon and Google.  I think these will mature of the next few month and years.  And there will be a few more (Yahoo, Microsoft, and IBM maybe?).

The way I see hosted application services is like this.  They will split the programmers into two categories.  The first category will be all those novice programmers, who don’t know how or don’t have the resources to take care of everything.  They’ll be using hosting, databases, libraries, and programming interfaces provided by hosting application services. (Of course, good programmers will be using these too, but they will have a choice, not like the newbies).  Hosted application services will (not yet though) make it easy to cover the ignorance and help to make a few bucks here and there.  Exactly like PHP has been doing it for years now.  The good programmers though will mostly participate in in-house projects and customization developments, which won’t be fitting into hosted application services, and will require additional knowledge and experience.

Summary

If you are a PHP developer looking for a job in Cyprus, please let me know.

I’ve recently posted some of my thoughts on PHP.  If you enjoy the topic, here is an excellent post over at Coding Horror, which reminds why PHP sucks so badly and also why it will stick around for some time to come.

I’ve heard plenty of positive buzz about PHP 6 in the last few weeks.  Yes, it’s coming out.  Yes, it brings quite a few improvements, including better Unicode support, better security, and more help for larger projects through namespaces.  However, I hope that it won’t be the last PHP release, since there are so many other things that need fixing.

Here is a good overview, as compared to the best programming language ever - Perl.  But this probably reminds you of a famous Euro-English joke, no?   But I do miss sigils and proper hashes.  I’d love to see better memory management when programming objects.  I’d love to see improved database interfaces with prepared statements and database abstraction layer.  I would really welcome a cleanup in function names and return values. I … I … I … I hope that PHP 6 is not the end of the road, and that PHP 7, PHP 8, and PHP 9 will follow.