Infrastructure update : GitHub, BitBucket, HipChat, TeamworkPM and Redmine

It’s been a while since I posted an update on our infrastructure tools, so here goes one.  (I know, ideally, it should be on our company’s blog, but we haven’t finished that part of the site yet).

First things first – migration from GitHub to BitBucket.  I have said many times that I love GitHub dearly.  And I do.  But it gets expensive pretty fast for the type of company that we are running.  GitHub subscription price is based around the number of private repositories.  Of which we have plenty as we are dealing with client’s projects mostly.  BitBucket subscription strategy is around the team size.  We have a small team and it probably won’t grow significantly in any distant future.  So where we are spending $100 a month on GitHub, and hitting a limit, a $10 per month BitBucket subscription will do for a long time.

Gladly, with git being distributed and decentralized, migrating from one to another is super easy.  All you have to do is this:

# Remove GitHub remote
git remote remove origin

# Add BitBucket origin
git remote add origin git@bitbucket.com...

# Update remotes
git remote update

And just to be clear, we are not moving away from GitHub completely.  We are only moving our private repositories over.  All our Open Source repositories will remain at GitHub, as it provides a larger exposure.  And also, we are keeping private repositories for projects where we work with clients’ developer teams, as GitHub seems to be the standard de facto.

We’ve finished the migration last week and now we are on a $10 BitBucket subscription and $25 GitHub subscription, getting the best of both worlds.

In terms of functionality, both services offer pretty much the same tools – git repositories, code browsers and review tools, pull requests, API and web hooks, etc.

GitHub offers a handy feature called “Releases“, which extends the functionality of git tags into something more useful – release notes and attached files (like built binaries).  BitBucket doesn’t have it yet, but it’s been requested and a lot of people voted for, so there is hope.

Also, GitHub seems to work a bit faster – BitBucket sometimes gets a bit clunky when your push to it or when you create a Pull Request.

But BitBucket has a neat Approve button, which works much like Facebooks’ Like.  Instead of all those :+1: and me too comments, one just clicks Approve and BitBucket shows the list of people who approved the pull request.  Oh, and you can also ping people by assigning them to be the reviewers.  So if you want somebody in particular to look at your changes, there is an interface for it, rather than a free form @-mentions.  Which also work by the way.

For us, the most difficult part of this migration was the fact that we had automated deployment logic built around GitHub actions.  So, every time a pull request was merged into a master branch, we’d automatically deploy the branch to the test server (using GitHub web hooks).  And every time a release was created, it would get automatically deployed to the live server.

BitBucket provides similar web hooks, but without releases, which we could replace with tags.  But that would mean that we had to support two different deployment scenarios, which sounded like too much.  Also, there were a few issues related to that logic in the first place.  For example:

• Deployment trigger was in one place, while deployment feedback was in another (HipChat, emails and logs).
• Re-deploying the same version wasn’t supported.  You had to do dummy pull requests or dummy releases to re-deploy something.
• Multiple pull requests in a single deployment weren’t possible – each merged pull request was a separate deployment.
• Deployment of projects which consisted of more than one repository was not supported.

So instead of supporting two separate scenarios – one for BitBucket and one for GitHub – and fixing these issues later, we’ve changed to …  

HipChat.  I know that everybody and their mother is crazy about Slack these days, but we’ve started using HipChat slightly before all that hype and we love it.  I’ve also tried Slask with some Open Source projects rooms, and I can’t say that I am a big fan of it.  It just doesn’t click with me, and all those animated GIFs don’t help either.

Anyways.  HipChat works really well for us.  It has a really nice user interface.  It works as a web app, and has both desktop and mobile clients for anything you can image.  It has public and private rooms.  It has attachments and links.  It has popup notifications.  It has an API.  And it’s free.  The pro version for $2/month/user adds support for video calls and screen sharing, which are not the features that we use frequently.

And there’s one more advantage when compared to Slack.  Slack is built by the company that only (?) works on Slack.  It does mean that they are more focused on it, but it also means that if it fails as a business they will just go away.  HipChat is built by Atlassian, which is the company building a whole array of developer tools – Jira, BitBucket, Confluence, and more.  HipChat is not the primary source of income for them, but rather yet another tool they probably use internally.

We use HipChat extensively.  We have rooms for everybody, for particular groups of users, rooms for different project ideas, and each project gets a dedicated room as well.  We’ve integrated HipChat with GitHub and BitBucket (appropriate HipChat rooms show notifications of new pull requests, comments, etc).  We’ve integrated HipChat with Nagios and Zabbix (monitoring notifications).  We’ve integrated HipChat with Twitter and RSS feeds, to get updates on things around us.

And now we’ve shifted out automated deployments from GitHub and BitBucket to HipChat.  So, instead of relying on a button press or a new tag, we issue a command in one of the rooms, and the project gets deployed.  And while the deployment is happening, all the logs and output of the deployment is shown in the same room.  And everybody can see what’s going on.  It’s beautiful and very very simple.

The last but not least, is the migration from TeamworkPM to Redmine.  We’ve used Teamwork for slightly more than a year now and it’s a great tool.  It just doesn’t fit our culture as well as a self-hosted Redmine setup.  Teamwork is a feature-rich tool, with projects, contacts, task lists, sub-tasks, due-dates, estimations and time tracking, files, notebooks, links, comments, and more.  But we needed something different.  Here are the reasons that we moved:

• Tasks don’t have numbers.  I mean, they do.  You can see them in the URL.  But they are not easy to reference, especially for the non-technical people.  In Redmine, every task is assigned a unique number, which is the point of reference and makes it easy to find or spell it out on the phone.
• Watcher notifications is sub-par.  As a manager, I want to get all notifications on all changes on all projects and activities.  I have no problem cutting through a thousand emails a day, if I need to.  But in Teamwork, there seems to be a way for people to remove me from notifications and completely bypass certain actions.  In Redmine, it’s much more flexible.
• Sub-projects.  In TeamworkPM every project is standalone.  It’s organized under the client company, but we need a level more.  Sometimes we do larger projects, which consist of smaller ones. We need estimation and billing summaries, which get tricky.
• Task statuses.  This is probably the largest issue we had with TeamworkPM.  Tasks are either open or closed.  There’s nothing in between.  Our workflow is not that simple.  We have a lot of client communication for example, which might stall the task for quite a bit.  We have approvals and other things that benefit greatly from custom task statuses (new, assigned, in progress, stalled, feedback, pending deploy, etc).
• Task types.  Redmine speak for this is “tracker”.  It’s like a different task type (“bug”, “feature”, “support request”, “user story”, etc).  You can define as many of them as you want.  Each one can have a separate access control and custom statuses.  You can make it as simple or as complex as you need.  And the best thing is that you can mix and match different trackers for different projects.
• Custom fields.  I think a lot of things could have been solved and improved if custom fields were supported.  I want my projects, tasks, users, and companies to have custom fields.  I want to have API access to those custom fields.  If I don’t – things get difficult and quite limited.
• Disk space.  Not the biggest of issues, but still a concern.  With self-hosted tools you get as much disk space as you need.  With TeamworkPM, your quota is part of the subscription plan.  We didn’t get close in year, but maybe because we were careful.  Uploading large PSD and video files will quickly get you overboard.  But if you are not involved in these kinds of projects, you are probably fine.

As I said, we loved TeamworkPM and we think it’s a great tool.  It’s just Redmine is a better fit for us. By the way, if you still insist on using TeamworkPM, have a look at Hubstuff, which extends TeamworkPM with a few handy features.

So, these are the news from the fronts.  Coming soon is another update on our Amazon AWS setup, puppet configuration manager, monitoring switch from Nagios to Zabbix, and, hopefully blog and portfolio sections on the company website.