RT initialdata and Perl’s nested map

Request Tracker (aka RT) comes with a very powerful, yet not too widely known tool – initialdata.  This helps with automating configuration of the new system and data migration.  Combined with the power of Perl’s map() function, some really awesome things can be done in a jiffy.

Here is a snippet I’ve used recently, to set a list of access rights to a list of queues:

push @ACL, map {
  my $queue = $_;
  map {
      GroupDomain => 'SystemInternal',
      GroupType => 'Everyone',
      Queue => $queue,
      Right => $_,
  } qw(
} qw(

Extract, Transform, Load

I’ve been doing all kinds of data migrations and system integration for years now.  But only yesterday I’ve learned that there is a very specific term linked to the process.

In computing, extract, transform, and load (ETL) refers to a process in database usage and especially in data warehousing that:

  • Extracts data from outside sources
  • Transforms it to fit operational needs, which can include quality levels
  • Loads it into the end target (database, more specifically, operational data store, data mart, or data warehouse)

ETL systems commonly integrate data from multiple applications, typically developed and supported by different vendors or hosted on separate computer hardware. The disparate systems containing the original data are frequently managed and operated by different employees. For example a cost accounting system may combine data from payroll, sales and purchasing.

Download your Gmail and Google Calendar data … soon or now

I am a well known Google fan.  But even those who call it an Evil Corporation and a Global Spy, can’t argue with the awesomeness of these news:

Starting today we’re rolling out the ability to export a copy of your Gmail and Google Calendar data, making it easy to back up your data or move to another service.

You can download all of your mail and calendars or choose a subset of labels and calendars. You can also download a single archive file for multiple products with a copy of your Gmail, Calendar, Google+, YouTube, Drive, and other Google data.

gmail data export

Most of the 20 GB of data I store on Google Drive is actually my email archive.  I’ve imported email into my Gmail from as early as 1998 – much, much earlier than Gmail was even born.  Having a way to export them all out in one go, without using clunky POP or IMAP is much appreciated.

How To Survive a Ground-Up Rewrite Without Losing Your Sanity

How To Survive a Ground-Up Rewrite Without Losing Your Sanity

Developers tend to spectacularly underestimate the effort involved in such a rewrite (more on that below), and spectacularly overestimate the value generated (more on that below, as well).

Convert WordPress tag to existing category

By now most people have figured out what is the difference between categories and tags, and which ones to use where.  But that doesn’t necessarily mean that the problems are over.  Once in a while there is a need to convert from one to another.

WordPress provides the tool to convert from categories to tags.  But I needed something for the reverse process.  In fact, I had a slightly more complicated situation – I already had both a tag and category with exactly the same name, and a whole lot of posts which were tagged, or categorized, or both.  Messy!  I was looking for a quick way to add all posts tagged with specific tag to the category with the same name.  Once that is done, I could easily delete the tag.  After a few Google searches, nothing straight-forward came up, so I decided to not waste anymore time and wrote my own little script.

Below are the pros and cons of my approach.


  • Standalone script, not a WordPress plugin.  It is not utilizing any of the WordPress configuration or functionality, but raw PHP and direct connection to the database.  You can do a dump of your database, restore it on a totally different machine and play with this script until you are sure you are getting the expected results.  No worries about screwing up the live database.
  • Requires both the tag and the category to be present.  It will not proceed unless it will find both.
  • Safe.  While there is no undo for the effects of the script, it still works pretty safe.  Nothing is ever deleted or updated in the database.  Only the new records are added – for posts to appear under the selected category.  Nothing else!


  • Standalone script, not a WordPress plugin.  It is not utilizing any of the WordPress configuration or functionality.  You will have to specify your database credentials by editing the script.  You will also might need to change SQL queries to reflect your WordPress tables prefix.
  • Runs from the command line.  If you don’t have access to or don’t know how to use the command line – tough luck!
  • Code quality.  I wrote this as a very quick and dirty solution to my problem.  It is not intended to be executed frequently.  Neither it is not intended to be executed by someone other than myself.  Yeah, I know, this will probably change, but I don’t care at this stage.

Now, you’ve been warned, so I won’t be holding you off from the source code any longer.

Continue reading “Convert WordPress tag to existing category”

Upcoming content move

Since most of the preparations in the new blog have been finished, it’s time to move the worthy content from the old site and into this database.  As I mentioned before, I don’t want to just copy things over, but edit, update, and fix them a bit.  Because of that, I won’t be moving 4,000+ posts in one go, but rather in large chunks of a few dozens or hundreds at a time.  Posts that will be moved to the new database will be deleted (as in “permanently removed”) from the old site.  I hope my redirections will handle it all properly.

Anyway, I am totally ready to do the first chunk of the posts.  These will be my movie reviews.  And I am thinking of moving them during the end of this week (think Thursday, Friday, Saturday, and Sunday).

There are two side effects to these batch moves.

Firstly, the RSS feed will get quite polluted.  So if you can’t handle many posts at once or hate seeing content that you have been through before, then you might want to temporarily unsubscribe from the RSS feed of this site.

Secondly, there is no sane way for me to distribute these posts over dates, so I’ll probably publish them all under the same date or so.

Once the content is moved and re-organized I will have more control over it and will be able to place it better.  If that will be needed or not at all is remained to be seen.

With that I apologize for all inconvenience caused and ask you to bare with me during this bumpy ride.

GMail migration – done

I have finished my migration to GMail process. I have all my email archives uploaded, marked as read, and most of the messages labeled. Of course, as with any migration, there were some rough edges and few messages just didn’t make it through, while a few others were dumped into archives without being properly categorized. But overall I am happy. Those few messages that didn’t get through were probably useless anyway. And categorization is not vital anymore with Google search engine behind the inbox.

GMail usage for 15/08/2006

‘All Mail’ suggests that I have slightly more than 20,000 discussions. In terms of space usage, I am at 42%1156 MB used out of 2754 MB. That’s not too bad. My current new mail rate is not that high, and GMail is constantly increasing the size of the mailbox. I think it’ll be enough. If nothing else, I know how have a lot of stuff that I can delete to clean up some space. Some of the messages date as far back as 1999 – surely there’s something that I won’t even miss.

Quick GMail update

I am doing the biggest email move ever. I am migrating all of my email to GMail. That includes archives, which, in some cases, have threads dating back to 1999. That’s A LOT OF email.

Obviously, my GMail account is a huge mess right now. I’ll try to clear it up as soon as possible, but for the meantime, if I miss an email from you, please, stay with me. I’ll get it al in order ASAP. Note thought, that just the upload process will take around two to three days.

The script I’m using for this process is Mark Lyon’s GMail Loader. I’m usin gthe older .4 version, because I didn’t have enough Python librarires for the GUI. Neither did I need it. The script works like a charm.