Mailtrap – safe email testing for development teams

Mailtrap is a handy service for any developer or team that needs to test outgoing emails from test and staging environments, without spamming real users.  It’s basically a fake SMTP server with some controls of where and how to forward messages, and ways of analyzing, sharing, and testing of HTML rendering.

Mailtrain – self-hosted Open Source alternative to MailChimp

Mailtrain is a self-hosted Open Source alternative to MailChimp and other similar mass-mailing and newsletter services.  Of course, being self-hosted, it can go only so far – you’ll still need to deal with mail delivery, SPAM filtering, and other modern days “benefits” of delivering mass mailings.  But it looks pretty complete in terms of features, so if you want to have full control and don’t mind investing a bit of time in setting up, check it out – you might get a better or at least cheaper alternative.

Mautic – Open Source Marketing Automation

Mautic is an Open Source marketing automation solution.  It features contact management, social media marketing, email marketing, forms, campaigns, reports, and pretty much everything else you’d expect from a tool like this.

If you are lost between a gadzillion online tools available for marketing automation, and/or don’t trust third-party providers and want to have a system of your own, give it a try.

Don’t Design Your Emails

I do hate HTML emails with passion.  They are always too heavy, often bloated, render horrible, and just plain annoying.  I miss the old good days, when email clients were warning users that their signature was too long, spanning more than 4 lines.  Today, everybody is sending out HTML emails whether they need to or not.  Whether it’s for the signatures, corporate branding, or the “marketing value” or the “professional look”.

Finally, there is someone on my side of the fence, who actually tested the effects of HTML emails and suggests that plain emails are more efficient even for the marketing purposes.  Read the whole thing – “Don’t Design Your Emails“, especially if you are involved with email marketing.

The plain email—which took no time to design or code—was opened by more recipients and had 3.3x more clicks than the designed email.

[…]

The plain, unstyled emails resulted in more opens, clicks, replies, and conversions, every time.

Replies to welcome emails were tripled. Cold emails were getting 30-35% open rates and 3% conversion rates, which is incredible.

MailChimp vs. Amazon SES + Mailwizz

Here’s an interesting story of moving away from MailChimp to a combined setup of Amazon SES and MailWizz, which resulted in overall 92% reduction of the monthly bill.  Given it’s not the same functionality, but if you are technical enough and your requirements are simpler than all the functionality of the MailChimp, this looks like a good alternative.

Smart Reply in Gmail

Google Blog announces that Smart Reply feature is coming to Gmail on both Android and iOS.  It has been introduced earlier in other products like Google Inbox and Allo, and is apparently so useful that about 12% of all replies in those applications are done with the help of Smart Reply.

There’s also a link to the Google Research Blog article which has more details on the technology behind (machine learning, artificial intelligence, native language processing, and all the usual suspects).

Morphos – morphological solution in PHP for English and Russian

If you ever had to deal with morphology in English, you probably found one or two libraries to help you out.  But if you had to do that for Russian, than I’m sure you are missing a few hairs, and the ones that you still have are grayer than they used to be.  I’ve got some good news for you though, now there is Morphos (GitHub repository).

Morphos is a morphological solution written completely in the PHP language. Supports Russian and English. Provides classes to decline First/Middle/Last names/nouns and generate cardinal numerals.

Just look at this beauty!

var_dump($dec->getForms($user_name, $dec->detectGender($user_name)));
/* Will produce something like
  array(6) {
    ["nominativus"]=>
    string(8) "Иван"
    ["genetivus"]=>
    string(10) "Ивана"
    ["dativus"]=>
    string(10) "Ивану"
    ["accusative"]=>
    string(10) "Ивана"
    ["ablativus"]=>
    string(12) "Иваном"
    ["praepositionalis"]=>
    string(15) "об Иване"
  }
*/

Just this alone can make user interfaces and emails so much better.  But there is more to it than that.

Exporting messages from Gmail with fetchmail and procmail

One of the projects that I am involved in has a requirement of importing all the historical emails from a number of Gmail accounts into another system.  It’s not the most challenging of tasks, but since I spent a bit of time on it, I figured I should blog it here too, just in case a similar need will arise in the future.

In my particular case, I need two different solutions.  One for exporting all of the messages from all folders of all Gmail accounts in question (Gmail for Work).  And the other is for exporting only the messages from the “Sent Mail” folder, which were sent on specific dates.

The solution that I derived is based on the classic tools for this purpose – fetchmail and procmail.  Fetchmail is awesome at fetching emails using all kinds of protocols.  Procmail is amazing at sorting, filtering, and otherwise processing the email messages.

So, here we go.  First of all, we need to tell fetchmail where to get the messages from.  I didn’t want to create to separate configurations for each of my tasks, so I left only the options common between them in the configuration file, and the rest I will be passing as command line arguments, depending on scenario.

Note that I’ve been running these tests from a dedicated environment, where I only had the root user.  You don’t have to run it as root – it’ll work as any other just fine.  Also, keep in mind that I used “/root/fetchmail-test/” folder for my test runs.  You might need to adjust the paths if you have it any different.

Here’s my fetchmail.rc file, which I used to test a single mailbox.  A new “poll” section will go into this file later, for each mailbox that I’ll need to export.

poll imap.gmail.com proto imap:
  username "someuser@gmail.com" is root here
  password "somepass"
  fetchall
  keep
  ssl

If you are not root, you might need to adjust the second line, replacing “root” with your username. Also, for testing purposes, you can use “fetchlimit 1” instead of “fetchall“.

Now, we need two configuration files for procmail.  The first one is super simple – I’ll use this for simply pushing all downloaded messages into a single giant mbox file.  Here’s the procmail-all.rc:

VERBOSE=0
DEFAULT=/root/fetchmail-test/fetchmail.all.mbox

As you can see, it only defines the verbosity level and the default mailbox.  The second configuration file is a bit more complicated.  I’ll use it for the sent items only.  The sent items folder limit will be done with fetchmail.  But I want to do further is disregard all messages, which were not sent on a specific date.  Here is my procmail-sent.rc:

VERBOSE=0
DEFAULT=/dev/null
:0
* ^Date: .*28 Jul 2016.*|\
  ^Date: .*27 Jul 2016.*
/root/fetchmail-test/fetchmail.sent.mbox

Again, we have the verbosity level and the default mailbox to save messages to.  Since I want to disregard them unless they match a certain condition, I specify /dev/null.   Then, I specify my condition, which is simply a bunch of regular expressions for the Date header.  Usually, Date header is a not very reliable as different MUAs (Mail User Agents) use different formats, time zones, etc.  In this particular case test results seemed consistent (maybe Gmail fixes the header), and I didn’t have any other more reliable criteria to use.

As you can see, I use a very basic condition for date matching. So, if the Date header matches either “28 Jul 2016” or “27 Jul 2016“, the message is saved in the mbox file, rather than being thrown into the default mailbox.

Now, all I need is a way to tie fetchmail and procmail together, as well as provide some additional options.  For that I created the two one-liner shell scripts, just so that I won’t need to figure out the command line arguments if I look at this whole thing six month later.

Here is the check-all.sh script (multi-line for readability):

#!/bin/bash
fetchmail -f fetchmail.rc \
          -r "[Gmail]/All Mail" \
          --mda "procmail /root/fetchmail-test/procmail-all.rc"

and here is the check-sent.sh script (multi-line for readability):

#!/bin/bash
fetchmail -f fetchmail.rc \
          -r "[Gmail]/Sent Mail" \
          --mda "procmail /root/fetchmail-test/procmail-sent.rc"

If you run either one of these scripts, you’ll see the output similar to this:

$ ./check-all.sh 
fetchmail: WARNING: Running as root is discouraged.
410 messages for someuser@gmail.comat imap.gmail.com (folder [Gmail]/All Mail).
reading message someuser@gmail.com@gmail-imap.l.google.com:1 of 410 (446 header octets) (222 body octets) not flushed
reading message someuser@gmail.com@gmail-imap.l.google.com:2 of 410 (869 header octets) (230 body octets) not flushed
reading message someuser@gmail.com@gmail-imap.l.google.com:3 of 410 (865 header octets) (230 body octets) not flushed
...

Here are a few resources that you might find helpful:

Emails, WordPress, and lots of Archives

I’ve been running this blog for a very long time now.  The Archives page links back to all the months and years (all the way to the first post back on October 21, 2001) of all kinds of posts – random rants, movie reviews, technical posts, and day summaries.  But who does read the archives ever, right?

Well, if you are running a WordPress site with lots of content, and you want to rediscover some of your old gems, there is an excellent plugin that helps with that – “This Day in History“.  I have a widget, powered by that very plugin, both on the front page of the site (showing posts from the same day in previous years), and on every post page (showing posts from the same day of the post in different years).

Today I found this short post about email and Microsoft Outlook:

There was a time, when I used to love email.  I loved receiving email, and reading it.  Replying to email.  Or just writing up some new email.  Occasionally, forward email.  I loved searching through email.  Or categorizing it.  Or archiving email.  I loved quoting email.  And I loved email with attachments.  But now, I pretty much hate all of that.  Thank you, MS Outlook.

Which made me think of the IT Crowd TV series, the very first episode of the very first season, where Jen was going through the interview:

I’ve always been a big fan of IT Crowd, in particular for its accurate take on the corporate culture.  Obviously, I thought of myself more like the Roy character, not Jen:

Given that the post was written in 2012, and this episode came out in 2006, I was probably mocking it, but I don’t remember for sure. Anyways, it’s fun.

Oh, and by the way, if you were wondering what’s a better email client, here is the post just for you.