Rundeck – Job Scheduler and Runbook Automation

Rundeck-EditJob

Rundeck is yet another one of those services that I want to get my hands on but haven’t yet got the time to.  The simplest way to describe it is: cron on steroids.

Rundeck allows one to define the commands and then allow for execution on those commands manually, periodically or based on a certain trigger.  Imagine, for example, a deployment command that needs to run across some servers to which you are not comfortable giving access to developers, or even non-technical users.  You can create a command in Rundeck and give access to certain users to execute it, via clicking a button or two in a user friendly web interface.

A side benefit to using Rundeck versus cron are the metrics.  Rundeck collects metrics like successful and failed executions, execution times, etc.  So it makes it easier for you to see that certain jobs are getting progressively slower or fail on specific weekdays, etc.

The best part is that Rundeck is Open Source and self-hosted, so you don’t need to give sensitive access to some external web service.

Amazon Linux AMI 2016.09

amazon ami 2016.09

AWS Blog lets us know that Amazon Linux AMI 2016.09 is now available.  It comes with a variety of updates, such as Nginx 1.10, PHP 7, and PostgreSQL 9.5 and Python 3.5.  Another thing that got quite a bit of improvement is the boot time of the Amazon Linux AMI instances.  Here’s a comparison chart:

amazon-linux-ami-launch-time-2016-09-whiteboard

Read about all the changes in the release notes.

P.S.: I’m still stuck with Amazon AMI on a few of my instances, but in general I have to remind all of you to NOT use the Amazon AMI.  You’ve been warned.

Vim 8.0 Released!

The team behind the greatest text editor of all times has release the new major version – Vim 8.0.  It’s the first major release in 10 years!  Brief overview of the changes:

  • Asynchronous I/O support, channels, JSON
  • Jobs
  • Timers
  • Partials, Lambdas and Closures
  • Packages
  • New style testing
  • Viminfo merged by timestamp
  • GTK+ 3 support
  • MS-Windows DirectX support

For a more complete list and details, have a look here.

The TL;DR summary: Vim provides a lot more power now to plugin developers, so we’ll be seeing a boost in both new functionality and old ways getting better.

Here is a mandatory Slashdot discussion with your usual Vim vs. Emacs flame.

P.S.: Emacs has recently released a major update too …

Top 13 Amazon Virtual Private Cloud (VPC) Best Practices

Cloud Academy Blog goes over top 13 Amazon VPC best practices – particularly good for those just starting up with the platform.  The article discusses the following:

  1. Choosing the Proper VPC Configuration for Your Organization’s Needs
  2. Choosing a CIDR Block for Your VPC Implementation
  3. Isolating Your VPC Environments
  4. Securing Your Amazon VPC Implementation
  5. Creating Your Disaster Recovery Plan
  6. Traffic Control and Security
  7. Keep your Data Close
  8. VPC Peering
  9. EIP – Just In Case
  10. NAT Instances
  11. Determining the NAT Instance Type
  12. IAM for Your Amazon VPC Infrastructure
  13. ELB on Amazon VPC

Overall, it’s a very handy quick list.

GitHub Improvements

GitHub has recently announced a whole lot of improvements to their service and functionality.  For me personally the following bits were super exiciting.

Improvements to the code review

reviews popover

Approve or require changes

You’re no longer left on your own to figure out if a comment was important. Or if that emoji means “Go ahead, looks great!” Or “Please no, this is likely going to bring the site down!”

With Reviews, you can leave your comments as suggestions, approve the changes, or request additional changes—on any pull request.

Improvements to the GitHub Profile page

github profiles

See what’s behind your green squares

GitHub profiles show your life and career as a developer. We’ve taken the contribution graph to new heights with your GitHub timeline—a snapshot of your most important triumphs and contributions.

But there is more – projects, notes, comment drafts, etc.  Check the full announcement.

BitBucket Pipelines and Docker for PHP Developers

I’ve been meaning to look into Docker for a long while now.  But, as always, time is the issue.  In the last couple of days though I’ve been integrating BitBucket Pipelines into our workflow.  BitBucket Pipelines is a continuous integration solution, which runs your project tests in a Docker container.  So, naturally, I had to get a better idea of how the whole thing works.

Docker for PHP Developers” article was super useful.  Even though it wasn’t immediately applicable to BitBucket Pipelines, as they don’t currently support multiple containers – everything has to run within a single container.

The default BitBucket Pipelines configuration suggests the phpunit/phpunit image.  If you want to run PHPUnit tests only, that works fine.  But if you want to have a full blown Nginx and MySQL setup for extra bits (UI tests, integration tests, etc), then you might find smartapps/bitbucket-pipelines-php-mysql image much more useful.  Here’s the full bitbucket-pipelines.yml file that I’ve ended up with.

MySQL, PHP and “Integrity constraint violation: 1062 Duplicate entry”

Anna Filina blogs about an interesting problem she encountered with when working on a PHP and MySQL project:

MySQL was complaining about “Integrity constraint violation: 1062 Duplicate entry”. I had all the necessary safeguards in my code to prevent duplicates in tha column.

I gave up on logic and simply dumped the contents of the problematic column for every record. I found that there was a record with and without an accent on one of the characters. PHP saw each as a unique value, but MySQL did not make a distinction, which is why it complained about a duplicate value. It’s a good thing too, because based on my goal, these should have been treated as duplicates.

She also mentions two possible solutions to the problem:

My solution was to substitute accented characters before filtering duplicates in the code. This way, similar records were rejected before they were sent to the database.

and

As pointed out in the comments, a more robust and versatile solution would be to check the collation on the column.

I’m sure this will come in handy one day.

git: history of a source code line

git is one of those tools that no matter how much you know about it, there is an infinite supply of new things to learn.  Here’s a handy bit I’ve discovered recently, thanks to this StackOverflow comment:

Since Git 1.8.4, git log has -L to view the evolution of a range of lines.

[…]

And you want to know the history of what is now line 155.

Then, use git log. Here, -L 155,155:git-web–browse.sh means “trace the evolution of lines 155 to 155 in the file named git-web–browse.sh“.

Absolutely brilliant!  I used to suffer through this via an iteration of git blame and git show to the point of custom bash scripts.

400,000 GitHub repositories, 1 billion files, 14 terabytes of code: Spaces or Tabs?

Here is an interesting bit of research – do people prefer tabs or spaces when programming the most popular languages?

Tabs or spaces. We are going to parse a billion files among 14 programming languages to decide which one is on top.

The results are not very surprising and somewhat disappointing (for all of us, tab fans):

tabs vs. spaces

As far as PHP goes, I’m sure the choice of spaces has to do with the PSR-2 coding style guide, which states:

Code MUST use 4 spaces for indenting, not tabs.

On a more technical note, I think this is also related to the explosion of editors and IDEs in the recent years, which, as good as they are, aren’t as good as Vim.  Vim allows for a very flexible configuration, where your code can be formatted and re-formatted any way you like, making tabs or spaces a non-issue at all.

Regardless of the results of the study, what’s more interesting is the method and tools used.  I’ve had my eye on the Google Big Query for a while now, but I’m too busy these days to give it a try.  The article gives a few insights, into how awesome the tool is.  1.6 terabytes of data processed in 864.6 seconds:

That query took a relative long time since it involved joining a 190 million rows table with a 70 million rows one, and over 1.6 terabytes of contents. But don’t worry about having to run it, since I left the result publicly available at [fh-bigquery:github_extracts.contents_top_repos_top_langs].

and:

Analyzing each line of 133 GBs of code in 16 seconds? That’s why I love BigQuery.

If you enjoyed this article, also have a look at “Analyzing GitHub issues and comments with BigQuery“, which works with a similar-sized data, trying to figure out how to write bug reports and pull request comments, so that they would be acted upon faster.