Software Engineering at Google

Fergus Henderson, who has been a software engineer at Google for 10 years, published the PDF document entitled “Software Engineering at Google“, where he collects and describes key software engineering practices the company is using.

It covers the following:

  • software development – version control, build system, code review, testing, bug tracking, programming languages, debugging and profiling tools, release engineering, launch approval, post-mortems, and frequent rewrites.
  • project management – 20% time, objectives and key results (OKRs), project approval, and corporate reorganizations.
  • people management – roles, facilities, training, transfers, performance appraisal and rewards.

Some of these practices are widely known, some not so much.  There are not a lot of details, but the overall summaries should provide enough food for thought for anyone who works in the software development company or is involved in management.


move fast & break nothing

An interesting talk by GitHubber Zach Holman on code, teams and process – “move fast & break nothing“.  It covers everything from DO’s and DONT’s, tools, and even Blue Angels jet fighter flying squad.   (Check the link above for slides and transcript, if video is not your thing).


Flexible Feature Control at Instagram


Flexible Feature Control at Instagram” article describes how Instagram controls the release of new features to groups of users.

I’ve implemented a very simple feature control mechanism before, but nothing to the sounds of this one.  Rolling out to groups of users, conditional control, geo-tagging, and more.  On top of it, non-technical users seem to be able to use for tuning the groups.  This sounds quite impressive, especially when you think of the Instagram’s user base (400,000,000+ users).

How Complex Systems Fail

How Complex Systems Fail – a very concise, yet complete paper on how complex systems fail.  It’s not system or industry specific.  Here are just the bullet points:

  1. Complex systems are intrinsically hazardous systems.
  2. Complex systems are heavily and successfully defended against failure.
  3. Catastrophe requires multiple failures – single point failures are not enough…
  4. Complex systems contain changing mixtures of failures latent within them.
  5. Complex systems run in degraded mode.
  6. Catastrophe is always just around the corner.
  7. Post-accident attribution accident to a ‘root cause’ is fundamentally wrong.
  8. Hindsight biases post-accident assessments of human performance.
  9. Human operators have dual roles: as producers & as defenders against failure.
  10. All practitioner actions are gambles.
  11. Actions at the sharp end resolve all ambiguity.
  12. Human practitioners are the adaptable element of complex systems.
  13. Human expertise in complex systems is constantly changing.
  14. Change introduces new forms of failure.
  15. Views of ’cause’ limit the effectiveness of defenses against future events.
  16. Safety is a characteristic of systems and not of their components.
  17. People continuously create safety.
  18. Failure free operations require experience with failure.


Alex Stamos : AppSec is Eating Security

I’m throwing this into the pile of arguments for “security and privacy are little but myths” discussions.  If top of the top companies, with multi-million budgets and hundreds or thousands of top security professionals get compromised, how realistic is it for the average Joe to protect his business?  I say – not very.

I think 80% of problems can be prevented with the 20% time and effort investment: minimize attack surface by removing and disabling everything you don’t need or use and limiting access to everything else, use layered defense where possible, use encryption where possible and strong passwords if you have to, don’t rely on security through obscurity, have log analyzers and/or intrusion detection system installed, etc.  But most importantly, make peace with the fact that being compromised is not the question of “if”, but “when”.  Prepare yourself.  Have an offsite backup and know how to restore your services in a completely new environment, if necessary.

And as far as your privacy goes, if you put anything private on the Internet, as well, prepare for it to be stolen and leaked.  If it never happens, consider yourself lucky.  Otherwise, just learn to deal with it.  It’s very unpleasant in a variety of ways, but seldom deadly.

Via EtherealMind.

Git branch names support forward slashes

Oh. My. God!  I’ve been using git for years now and I only learned this today – git branch names support forward slashes! How awesome is that?!  You can do things like this:

$ git checkout -b feature/foobar

Grouping branches like this is much easier indeed!

I came across this while reading CakeDC’s CakePHP Plugin Standard.  Searching around to find more details, I see that there are a few potential issues with this naming convention, as tools occasionally break (composer, IDEs, etc).   However, these problems are fixed by tool vendors.  Looking into it even further, I found the following description in the book “Version Control with Git: Powerful tools and techniques for collaborative software development” (Google Books preview):

Dos and Don’ts in Branch Names

Branch names must conform to a few simple rules.

  • You can use the forward slash (/) to create a hierarchical name scheme.  However, the name cannot end with a slash.
  • The name cannot start with a minus sign (-).
  • No slash-separated component can begin with a dot (.).  A branch name such as feature/.new is invalid.
  • The name cannot contain two consecutive dots (..) anywhere.
  • Further, the name cannot contain:
    • Any space or other whitespace character
    • A character that has special meaning to Git, including the tilde (~), caret (^), colon (:), question mark (?), asterisk (*), and open bracket ([).
    • An ASCII control character, which is any byte with a value lower than \040 octal, or the DEL character (\177 octal)

These branch name rules are enforced by the git check-ref-format plumbing command, and they are designed to ensure that each branch name is both easily typed and usable as a filename within the .git directory and scripts.

So, as you can see, you aren’t even limited to the single forward slash. Even things like this work just fine:

$ git checkout -b Leonid/ideas/feature/foobar

But remember, just because you CAN do something, doesn’t necessarily mean you SHOULD.  Have a look at this StackOverflow discussion about git branch naming best practices for more understanding on what you should and shouldn’t do.

The Architecture of Open Source Applications

The Architecture of Open Source Applications

Architects look at thousands of buildings during their training, and study critiques of those buildings written by masters. In contrast, most software developers only ever get to know a handful of large programs well—usually programs they wrote themselves—and never study the great programs of history. As a result, they repeat one another’s mistakes rather than building on one another’s successes.

Our goal is to change that. In these two books, the authors of four dozen open source applications explain how their software is structured, and why. What are each program’s major components? How do they interact? And what did their builders learn during their development? In answering these questions, the contributors to these books provide unique insights into how they think.

If you are a junior developer, and want to learn how your more experienced colleagues think, these books are the place to start. If you are an intermediate or senior developer, and want to see how your peers have solved hard design problems, these books can help you too.

There is also the fourth book in the works – 500 lines or less.

Why is software engineering exciting?

Quora runs the question.  There are some really inspiring and insightful replies.  Have a look.  Here are some bits to get you started.  Robert Love said:

Software is the most malleable of media. With just bits—which are nothing, really—a software engineer can build castles out of thin air. Entire businesses, industries even, are created with nothing physical at all. Software’s substrate is the stuff of pure thought.

Other engineering disciplines are constrained by the surly bonds of the physical world. To design a new plane, the aerospace engineer may spend years designing a model. A model! A software engineer can go from idea to reality in a day. As an intellectual pursuit, software is enormously rewarding.

Martin Sarsale:

It’s like playing with Lego but the blocks are product of your mind.

Noam Lerner:

Creation. It’s my way to express my creativity. Some people create music, movies, paintings or pottery. I create software.

Briggitte Jellinek:

It’s exaclty like magic:   with my spells (that are completely incomprehensible to muggles) I can make stuff happen!

And there’s more, of course.