400,000 GitHub repositories, 1 billion files, 14 terabytes of code: Spaces or Tabs?

Here is an interesting bit of research – do people prefer tabs or spaces when programming the most popular languages?

Tabs or spaces. We are going to parse a billion files among 14 programming languages to decide which one is on top.

The results are not very surprising and somewhat disappointing (for all of us, tab fans):

tabs vs. spaces

As far as PHP goes, I’m sure the choice of spaces has to do with the PSR-2 coding style guide, which states:

Code MUST use 4 spaces for indenting, not tabs.

On a more technical note, I think this is also related to the explosion of editors and IDEs in the recent years, which, as good as they are, aren’t as good as Vim.  Vim allows for a very flexible configuration, where your code can be formatted and re-formatted any way you like, making tabs or spaces a non-issue at all.

Regardless of the results of the study, what’s more interesting is the method and tools used.  I’ve had my eye on the Google Big Query for a while now, but I’m too busy these days to give it a try.  The article gives a few insights, into how awesome the tool is.  1.6 terabytes of data processed in 864.6 seconds:

That query took a relative long time since it involved joining a 190 million rows table with a 70 million rows one, and over 1.6 terabytes of contents. But don’t worry about having to run it, since I left the result publicly available at [fh-bigquery:github_extracts.contents_top_repos_top_langs].

and:

Analyzing each line of 133 GBs of code in 16 seconds? That’s why I love BigQuery.

If you enjoyed this article, also have a look at “Analyzing GitHub issues and comments with BigQuery“, which works with a similar-sized data, trying to figure out how to write bug reports and pull request comments, so that they would be acted upon faster.

Kids react to Metallica

This video made me feel very, very old.  Ancient almost.  I grew up with Metallica and it is shocking to realize that kids these days don’t even know the band.  Apart from that, the reactions are funny.  New generation …

By the way, there are more videos in that YouTube channel (see the Windows 95 for example).   Radio Nostalgie … :)

Quora: if programming languages were countries …

If programming languages were countries, which country would each language represent?” over Quora is hilarious!  Here are a few bits to get you started:

CRussia. Everything has to be done in a backwards way, but everything is possible, and there’s a lot of legacy.

C++USA. Powerful, but more and more complicated, unreadable, error-prone. Tends to dominate and influence everything.

Haskell Monaco. Not many people, but very rich, so they don’t have to consider lower classes’ problems.

Java Sweden. Comfortable, but has its own king and currency.

JavaScript China. Developing really fast and can do lots of surprising stuff. A lot of users.

PHPBangladesh. Poor, but numerous, and it’s found all over the web.

PascalGermany. Strict rules, good performance. And there are many people who just don’t like the language.

BashSwitzerland. Not very big in itself, but pulls the strings of the others.

Awk: North Korea. Stubbornly resists change, and its users appear to be unnaturally fond of it for reasons we can only speculate on.

Page builders and multilingual WordPress websites

WPML.org, the web home of the WordPress Multilingual Plugin runs this blog post about the upcoming support for WordPress page builders.  Apart from the good news themselves, there are some insightful results of the survey that the team did, trying to understand who uses page builders and how.  I found the stats on which page builder solutions people use the most interesting:

q2-which-page-builder

At work we are primarily using Divi (when we are not building our own themes), but we’ve also done a few sites with Enfold.  I’ve also seen Avada in the wild.  But I can’t tell you which ones are better, because when it comes to using page builders, I’m mostly not involved.  These tools are so awesome these days that they can be easily used by a non-technical person.  Which is exactly what we do ;)

StackOverflow: Docker vs. Vagrant, with project authors’ comments

There is this discussion over at StackOverflow: Should I use Vagrant or Docker for creating an isolated environment? It attracted the attention of the authors of both projects (as well as many other smart people).  Read the whole thing for interesting insights into what’s there now and what’s coming.  If you’d rather have a summary, here it is:

The short answer is that if you want to manage machines, you should use Vagrant. And if you want to build and run applications environments, you should use Docker.

3 serious (but common) misconceptions about software testing

QA Symphony looks at 3 serious (but common) misconceptions about software testing:

  1. Testing is a Cost Center
  2. Legacy Tools are Good Enough
  3. Testing Is Easy

These are indeed very common.

Let me just briefly focus on the last one.  Consider how quickly the complexity escalates.  You are a building a simple website – nothing fancy, just some modern design and a few pages of content to represent your company.  Let’s say you have just four pages: front page, about us, services, and contact us.  You can quickly check how these four pages look in your browser.  Easy right?

But wait.  People use different browsers to access the web.  So just checking it in your favorite one is not that enough.  Which browsers should you test for?  Let’s say we take the major ones – Google Chrome, Firefox, Microsoft Internet Explorer, and Safari.  These should about cover us.  Until now, your entire test was 4 page loads.  Now you are multiplying it by 4 browsers.  That’s 16 page loads.  So far so good?

Not really.  Each browser has multiple versions, which render pages differently.  Not everyone is using the latest version.  And new versions are released continuously.  Let’s say you decided to support the latest two major versions of each browser.  So now, that’s each browser times two.  So 4 pages by 8 browser comes up to 32 page loads.

But we were just talking about the desktop browsers.  You do want your site looking good on tablets and mobiles, right?  Of course you do.  What does that mean in terms of testing?  Let’s say we support only iOS and Android – two major platforms, both for mobile and tablets.  And we only support the default browser on each of those.  That adds 4 more browsers.

By the way, when people browse on mobiles and tablets, sometimes they view your page holding a device vertically, and sometimes horizontally.  You should probably test for those things as well.  You know, just to make sure, nothing breaks through to the right, or requires too much scrolling down.

Remember that we are still talking about the simplest of all the websites here.  Nothing fancy.

Let’s throw in an additional language.  Here in Cyprus, for example, most websites are in English and Greek.  Some add Russian.  Some have a few languages.  I’ve worked in the companies supporting over a dozen languages on the website.  Each language means that you need to do more tests.  In each of those browsers and on each of the supported devices.

You get my drift.

Something else.  So far our tests were simple page loads, just to have a quick look whether or not the page looks fine.  What if we have multiple tests per page.  You want the page to look fine.  But you also don’t want to have any syntax or grammar errors in the content.  Especially in those foreign languages, that you don’t speak yourself.  And you want the page to be optimized for search engines.  And you want it to be fast (performance testing).

By now, you’ll probably give and agree that testing is not easy.  Not even for the simplest of sites.  Consider just how much more complicated it can get if your site is slightly more complex than that.

Remember that contact us page.  That thing was just showing the company mailing address and the phone number.  Now you want an integrated Google Map and a contact us form.  Nothing wrong with that, right?

Well.  Google services are banned in many countries.  Will your contact us page still work if the user cannot access the Google Maps service?  You’ll need to test for that.  What about your contact form?  Does it actually work? You can’t be sure from it just appearing on the page.  You need to test it now too.

We are still in the domain of simple websites.  And this is getting too long.  Let me just through a few more things at you:

  • more content (imagine a website with dozens or hundreds of pages … this very blog has almost 10,000 blog posts, which are organized into categories, tags, date archives, etc).
  • more functionality (fancy things, dynamic page loads, form validations, etc)
  • even more functionality (online shop, user-driven content, etc)
  • more integrations with other services (Google, social networks, company CRM/ERP/etc)
  • spreading the site across multiple servers for performance (multiple web servers, caching servers, application servers, database servers, etc)

With each and every bit that you throw into your website, the testing gets exponentially more complex.  You have to consider functionality testing, user interface testing, performance testing, security testing, and so on and so forth.

When you get to all of that, you probably have multiple people working in several teams.  They all need to communicate and coordinate.  Which is another layer of complexity.

I’ll stop here.

Next time you think testing is easy, think again.

Update (September 10, 2016): here are a few more misconceptions, which are as common as the ones above.

What is risk management?

risk management

O’Reilly runs a nice and simple article on what is risk management.  They look at it from the perspective of a web application, but the suggestions are generic enough to be applied universally.  The highlights are:

  • Managing risk
  • Identifying risk
  • Remove worst offenders
  • Mitigate
  • Review regularly

I particularly liked this paragraph from the identifying risks section:

You will likely find that there are obvious entries in the list, but there should also be entries that surprise you. This is good. You want to uncover as many of your risk vulnerabilities as possible, and if some of them don’t come as a surprise to you, you probably haven’t dug deep enough.

Micro – a modern and intuitive terminal-based text editor

micro-solarized

Micro is a modern console based text editor, written in Go.  Version 1.0.0 has been recently released.  It’s cross-platform (installs as a single binary) and supports a variety of features:

  • Easy to use and to install
  • No dependencies or external files are needed — just the binary you can download further down the page
  • Common keybindings (ctrl-s, ctrl-c, ctrl-v, ctrl-z…)
    • Keybindings can be rebound to your liking
  • Sane defaults
    • You shouldn’t have to configure much out of the box (and it is extremely easy to configure)
  • Splits and tabs
  • Extremely good mouse support
    • This means mouse dragging to create a selection, double click to select by word, and triple click to select by line
  • Cross platform (It should work on all the platforms Go runs on)
    • Note that while Windows is supported, there are still some bugs that need to be worked out
  • Plugin system (plugins are written in Lua)
  • Persistent undo
  • Automatic linting and error notifications
  • Syntax highlighting (for over 75 languages!)
  • Colorscheme support
    • By default, micro comes with 16, 256, and true color themes.
  • True color support (set the MICRO_TRUECOLOR env variable to 1 to enable it)
  • Copy and paste with the system clipboard
  • Small and simple
  • Easily configurable
  • Common editor things such as undo/redo, line numbers, unicode support…

Although not yet implemented, I hope to add more features such as autocompletion, and multiple cursors in the future.

If you are looking for a new editor, give Micro a try.

Troubleshooting with /dev/tcp and /dev/udp

Imagine you are on a freshly installed Linux machine with the minimal set of packages, and you need to test network connectivity.  You don’t have netcat, telnet, and your other usual tools.  For the sake of the example, imagine that even curl and wget are missing.  What do you do?

Well, apparently, there is a way to do this with plain old bash.  A way, which I didn’t know until today.  You can do this with /dev/tcp and /dev/udp. Here is an example verbatim from the Advanced Bash-Scripting Guide:

#!/bin/bash
# dev-tcp.sh: /dev/tcp redirection to check Internet connection.

# Script by Troy Engel.
# Used with permission.
 
TCP_HOST=news-15.net       # A known spam-friendly ISP.
TCP_PORT=80                # Port 80 is http.
  
# Try to connect. (Somewhat similar to a 'ping' . . .) 
echo "HEAD / HTTP/1.0" >/dev/tcp/${TCP_HOST}/${TCP_PORT}
MYEXIT=$?

: <<EXPLANATION If bash was compiled with --enable-net-redirections, it has the capability of using a special character device for both TCP and UDP redirections. These redirections are used identically as STDIN/STDOUT/STDERR. The device entries are 30,36 for /dev/tcp: mknod /dev/tcp c 30 36 >From the bash reference:
/dev/tcp/host/port
    If host is a valid hostname or Internet address, and port is an integer
port number or service name, Bash attempts to open a TCP connection to the
corresponding socket.
EXPLANATION

   
if [ "X$MYEXIT" = "X0" ]; then
  echo "Connection successful. Exit code: $MYEXIT"
else
  echo "Connection unsuccessful. Exit code: $MYEXIT"
fi

exit $MYEXIT