BitBucket Pipelines and Docker for PHP Developers

I’ve been meaning to look into Docker for a long while now.  But, as always, time is the issue.  In the last couple of days though I’ve been integrating BitBucket Pipelines into our workflow.  BitBucket Pipelines is a continuous integration solution, which runs your project tests in a Docker container.  So, naturally, I had to get a better idea of how the whole thing works.

Docker for PHP Developers” article was super useful.  Even though it wasn’t immediately applicable to BitBucket Pipelines, as they don’t currently support multiple containers – everything has to run within a single container.

The default BitBucket Pipelines configuration suggests the phpunit/phpunit image.  If you want to run PHPUnit tests only, that works fine.  But if you want to have a full blown Nginx and MySQL setup for extra bits (UI tests, integration tests, etc), then you might find smartapps/bitbucket-pipelines-php-mysql image much more useful.  Here’s the full bitbucket-pipelines.yml file that I’ve ended up with.

MySQL, PHP and “Integrity constraint violation: 1062 Duplicate entry”

Anna Filina blogs about an interesting problem she encountered with when working on a PHP and MySQL project:

MySQL was complaining about “Integrity constraint violation: 1062 Duplicate entry”. I had all the necessary safeguards in my code to prevent duplicates in tha column.

I gave up on logic and simply dumped the contents of the problematic column for every record. I found that there was a record with and without an accent on one of the characters. PHP saw each as a unique value, but MySQL did not make a distinction, which is why it complained about a duplicate value. It’s a good thing too, because based on my goal, these should have been treated as duplicates.

She also mentions two possible solutions to the problem:

My solution was to substitute accented characters before filtering duplicates in the code. This way, similar records were rejected before they were sent to the database.


As pointed out in the comments, a more robust and versatile solution would be to check the collation on the column.

I’m sure this will come in handy one day.

How Google Uses and Contributes to Open Source

Here is a good Open Source story – “How Google Uses and Contributes to Open Source“, which goes into some detail and history of how Google is working with Open Source community.

I’ve seen this before:

“There are companies and people who just take the software and say, “I didn’t have to pay for it. I can do anything I want. The license file is a big blob of text. I’m not going to read that,” Merlin said.

And I’ve this (quite a few times actually):

Back in its early days, around 1998, Google was a small company. It was using open source just like any other small company. While Google was abiding by licences, they were not giving back much due to several reasons. “Some of it was just run fast and make sure that we have money next month to pay everyone’s salary,” said Merlin.

Having been involved in open sourcing companies’ projects new and old, this is what I firmly believe now is the best strategy:

Go open source from the beginning

Google changed that by writing a lot of things from the ground up as open source or to be open source ready. That was a good lesson that they learned, and that’s a problem many companies face when they want to open source their stuff but can’t because the code was not designed to be open source from the beginning.

This, I think, is an interesting approach too (if  you are too small of a company to have research papers and algorithms, consider blog posts, tips and tweaks, case studies, and the like):

Even if Google can’t open source certain code, they found a way to bring that work to the public. “We wrote papers talking about the magic algorithm that we used. We can’t give you the code for the reason I just explained, but we’re giving you the way they work so you can rewrite them,” said Merlin. Google has published hundreds of such papers and people are using it to create projects based on those ideas.

This bit on Android is mind blowing:

Now virtually all of Google’s open source code is on GitHub, except for Android. “The Android distribution is so big and it gets released in big chunks. So, when it gets released, everyone wants to sync that,” Merlin said. “It’s so huge that if we put it on GitHub, it would completely kill GitHub. We use our own mirrors for that, to help out.”

A word of caution for the companies using Open Source software:

Companies have to be extremely careful when using open source. Different projects use different licenses, and you need to be in compliance with them.


Things become complicated when you have projects that you ship. In the case of open source, you need to list the projects that you use and their licenses. In the case of BSD and MIT, you need to list the name and the copyright of the person you got that project from.

You’ll probably need a set of tools to deal with issues like this.  For PHP-based projects, composer is indispensable.  You can run “composer licenses” command and instantly get information about the project’s license, as well as licenses for each and every dependency in use (thanks to this blog post).

There is a good section on Contributor License Agreements (CLAs).  I am slightly familiar with the subject (I signed a few myself), but my experience is limited, especially from the company perspective.  I found this part useful, for that distant time when I’ll need to set it up:

Google uses the Apache foundation ICLA, without modifying it or putting anything special in it. CLAs ensure that companies like Google “can re-license your code under a different open source (license) if we need to. Sometimes we need to merge with other projects and that’s what the CLA allows us to do,” said Merlin.

These are just bits and pieces which I found interesting.  I wish more companies shared their practices and experiences – in particular those larger businesses, with years of history and a wide variety of challenges.

git: history of a source code line

git is one of those tools that no matter how much you know about it, there is an infinite supply of new things to learn.  Here’s a handy bit I’ve discovered recently, thanks to this StackOverflow comment:

Since Git 1.8.4, git log has -L to view the evolution of a range of lines.


And you want to know the history of what is now line 155.

Then, use git log. Here, -L 155,155:git-web– means “trace the evolution of lines 155 to 155 in the file named git-web–“.

Absolutely brilliant!  I used to suffer through this via an iteration of git blame and git show to the point of custom bash scripts.

400,000 GitHub repositories, 1 billion files, 14 terabytes of code: Spaces or Tabs?

Here is an interesting bit of research – do people prefer tabs or spaces when programming the most popular languages?

Tabs or spaces. We are going to parse a billion files among 14 programming languages to decide which one is on top.

The results are not very surprising and somewhat disappointing (for all of us, tab fans):

tabs vs. spaces

As far as PHP goes, I’m sure the choice of spaces has to do with the PSR-2 coding style guide, which states:

Code MUST use 4 spaces for indenting, not tabs.

On a more technical note, I think this is also related to the explosion of editors and IDEs in the recent years, which, as good as they are, aren’t as good as Vim.  Vim allows for a very flexible configuration, where your code can be formatted and re-formatted any way you like, making tabs or spaces a non-issue at all.

Regardless of the results of the study, what’s more interesting is the method and tools used.  I’ve had my eye on the Google Big Query for a while now, but I’m too busy these days to give it a try.  The article gives a few insights, into how awesome the tool is.  1.6 terabytes of data processed in 864.6 seconds:

That query took a relative long time since it involved joining a 190 million rows table with a 70 million rows one, and over 1.6 terabytes of contents. But don’t worry about having to run it, since I left the result publicly available at [fh-bigquery:github_extracts.contents_top_repos_top_langs].


Analyzing each line of 133 GBs of code in 16 seconds? That’s why I love BigQuery.

If you enjoyed this article, also have a look at “Analyzing GitHub issues and comments with BigQuery“, which works with a similar-sized data, trying to figure out how to write bug reports and pull request comments, so that they would be acted upon faster.

Kids react to Metallica

This video made me feel very, very old.  Ancient almost.  I grew up with Metallica and it is shocking to realize that kids these days don’t even know the band.  Apart from that, the reactions are funny.  New generation …

By the way, there are more videos in that YouTube channel (see the Windows 95 for example).   Radio Nostalgie … :)

Quora: if programming languages were countries …

If programming languages were countries, which country would each language represent?” over Quora is hilarious!  Here are a few bits to get you started:

CRussia. Everything has to be done in a backwards way, but everything is possible, and there’s a lot of legacy.

C++USA. Powerful, but more and more complicated, unreadable, error-prone. Tends to dominate and influence everything.

Haskell Monaco. Not many people, but very rich, so they don’t have to consider lower classes’ problems.

Java Sweden. Comfortable, but has its own king and currency.

JavaScript China. Developing really fast and can do lots of surprising stuff. A lot of users.

PHPBangladesh. Poor, but numerous, and it’s found all over the web.

PascalGermany. Strict rules, good performance. And there are many people who just don’t like the language.

BashSwitzerland. Not very big in itself, but pulls the strings of the others.

Awk: North Korea. Stubbornly resists change, and its users appear to be unnaturally fond of it for reasons we can only speculate on.

Page builders and multilingual WordPress websites, the web home of the WordPress Multilingual Plugin runs this blog post about the upcoming support for WordPress page builders.  Apart from the good news themselves, there are some insightful results of the survey that the team did, trying to understand who uses page builders and how.  I found the stats on which page builder solutions people use the most interesting:


At work we are primarily using Divi (when we are not building our own themes), but we’ve also done a few sites with Enfold.  I’ve also seen Avada in the wild.  But I can’t tell you which ones are better, because when it comes to using page builders, I’m mostly not involved.  These tools are so awesome these days that they can be easily used by a non-technical person.  Which is exactly what we do ;)

StackOverflow: Docker vs. Vagrant, with project authors’ comments

There is this discussion over at StackOverflow: Should I use Vagrant or Docker for creating an isolated environment? It attracted the attention of the authors of both projects (as well as many other smart people).  Read the whole thing for interesting insights into what’s there now and what’s coming.  If you’d rather have a summary, here it is:

The short answer is that if you want to manage machines, you should use Vagrant. And if you want to build and run applications environments, you should use Docker.

3 serious (but common) misconceptions about software testing

QA Symphony looks at 3 serious (but common) misconceptions about software testing:

  1. Testing is a Cost Center
  2. Legacy Tools are Good Enough
  3. Testing Is Easy

These are indeed very common.

Let me just briefly focus on the last one.  Consider how quickly the complexity escalates.  You are a building a simple website – nothing fancy, just some modern design and a few pages of content to represent your company.  Let’s say you have just four pages: front page, about us, services, and contact us.  You can quickly check how these four pages look in your browser.  Easy right?

But wait.  People use different browsers to access the web.  So just checking it in your favorite one is not that enough.  Which browsers should you test for?  Let’s say we take the major ones – Google Chrome, Firefox, Microsoft Internet Explorer, and Safari.  These should about cover us.  Until now, your entire test was 4 page loads.  Now you are multiplying it by 4 browsers.  That’s 16 page loads.  So far so good?

Not really.  Each browser has multiple versions, which render pages differently.  Not everyone is using the latest version.  And new versions are released continuously.  Let’s say you decided to support the latest two major versions of each browser.  So now, that’s each browser times two.  So 4 pages by 8 browser comes up to 32 page loads.

But we were just talking about the desktop browsers.  You do want your site looking good on tablets and mobiles, right?  Of course you do.  What does that mean in terms of testing?  Let’s say we support only iOS and Android – two major platforms, both for mobile and tablets.  And we only support the default browser on each of those.  That adds 4 more browsers.

By the way, when people browse on mobiles and tablets, sometimes they view your page holding a device vertically, and sometimes horizontally.  You should probably test for those things as well.  You know, just to make sure, nothing breaks through to the right, or requires too much scrolling down.

Remember that we are still talking about the simplest of all the websites here.  Nothing fancy.

Let’s throw in an additional language.  Here in Cyprus, for example, most websites are in English and Greek.  Some add Russian.  Some have a few languages.  I’ve worked in the companies supporting over a dozen languages on the website.  Each language means that you need to do more tests.  In each of those browsers and on each of the supported devices.

You get my drift.

Something else.  So far our tests were simple page loads, just to have a quick look whether or not the page looks fine.  What if we have multiple tests per page.  You want the page to look fine.  But you also don’t want to have any syntax or grammar errors in the content.  Especially in those foreign languages, that you don’t speak yourself.  And you want the page to be optimized for search engines.  And you want it to be fast (performance testing).

By now, you’ll probably give and agree that testing is not easy.  Not even for the simplest of sites.  Consider just how much more complicated it can get if your site is slightly more complex than that.

Remember that contact us page.  That thing was just showing the company mailing address and the phone number.  Now you want an integrated Google Map and a contact us form.  Nothing wrong with that, right?

Well.  Google services are banned in many countries.  Will your contact us page still work if the user cannot access the Google Maps service?  You’ll need to test for that.  What about your contact form?  Does it actually work? You can’t be sure from it just appearing on the page.  You need to test it now too.

We are still in the domain of simple websites.  And this is getting too long.  Let me just through a few more things at you:

  • more content (imagine a website with dozens or hundreds of pages … this very blog has almost 10,000 blog posts, which are organized into categories, tags, date archives, etc).
  • more functionality (fancy things, dynamic page loads, form validations, etc)
  • even more functionality (online shop, user-driven content, etc)
  • more integrations with other services (Google, social networks, company CRM/ERP/etc)
  • spreading the site across multiple servers for performance (multiple web servers, caching servers, application servers, database servers, etc)

With each and every bit that you throw into your website, the testing gets exponentially more complex.  You have to consider functionality testing, user interface testing, performance testing, security testing, and so on and so forth.

When you get to all of that, you probably have multiple people working in several teams.  They all need to communicate and coordinate.  Which is another layer of complexity.

I’ll stop here.

Next time you think testing is easy, think again.

Update (September 10, 2016): here are a few more misconceptions, which are as common as the ones above.