pushd/popd vs. cd

My shell of choice and circumstance for most of my Linux life was Bash.  So, naturally, in my head, shell pretty much equals Bash, and I rarely think or get into situations when this is not true.  Recently, I was surprised by a script failure, which left me scratching my head.  The command that failed in the script was pushd.

pushd and popd, it turns out, are built into Bash, but they are not standard POSIX commands, so not all the shells have them.  My script wasn’t setting the shell explicitly, and end up executing with Dash, which I haven’t even heard of until this day.  The homepage of Dash says the following:

DASH is not Bash compatible, it’s the other way around.

Mkay… So, I’ve done two things:

  1. Set /bin/bash explicitly as my shell in the script.
  2. Switch to “cd folder && do something && cd –“, instead of pushd/popd combination, where possible.

I knew about “cd –” before, but it was interesting to learn if there are any particular differences (hint: there are) between the this approach and the pushd/popd one that I was using until now.  This StackOverflow thread (ok, ok, Unix StackExchange) was very helpful.

Bulletproof Bash : Stop script on error

The other day I’ve been puzzled by the results of a cron job script.  The bash script in question was written in a hurry a while back, and I was under the assumption that if any of its steps fail, the whole script will fail.  I was wrong.  Some commands were failing, but the script execution continued.  It was especially difficult to notice, due to a number of unset variables, piped commands, and redirected error output.

Once I realized the problem, I got even more puzzled as to what was the best solution.  Sure, you can check an exit code after each command in the script, but that didn’t seem elegant of efficient.

A quick couple of Google searches brought me to this StackOverflow thread (no surprise there), which opened my eyes on a few bash options that can be set at the beginning of the script to stop execution when an error or warning occurs (similar to use strict; use warnings; in Perl).  Here’s the test script for you with some test commands, pipes, error redirects, and options to control all that.


# Stop on error
set -e
# Stop on unitialized variables
set -u
# Stop on failed pipes
set -o pipefail

# Good command
echo "We start here ..."

# Use of non-initialized variable
echo "$FOOBAR"
echo "Still going after uninitialized variable ..."

# Bad command with no STDERR
cd /foobar 2> /dev/null
echo "Still going after a bad command ..."

# Good command into a bad pipe with no STDERR
echo "Good" | /some/bad/script 2> /dev/null
echo "Still going after a bad pipe ..."

# Benchmark
echo "We should never get here!"

Save it to test.sh, make executable (chmod +x test.sh), and run like so:

$ ./test.sh || echo Something went wrong

Then try to comment out some options and some commands to see what happens in different scenarios.

I think, from now on, those three options will be the standard way I start all of my bash scripts.


CakePHP 3 : Remove Shell Welcome Header

CakePHP 3 has an excellent support for command line Shells, Tasks, and Console Tools.  There are a few that are bundled with the framework itself, and that come from a variety of plugins.  And, of course, you can have your own commands, specific to your application.

$ ./bin/cake

Welcome to CakePHP v3.4.3 Console
App : src
Path: /home/leonid/Work/cakephp_test/src/
PHP : 7.0.16
Current Paths:

* app:  src
* root: /home/leonid/Work/cakephp_test
* core: /home/leonid/Work/cakephp_test/vendor/cakephp/cakephp

Available Shells:

[Bake] bake

[DebugKit] benchmark, whitespace

[Migrations] migrations

[CORE] cache, i18n, orm_cache, plugin, routes, server

[app] console

To run an app or core command, type `cake shell_name [args]`
To run a plugin command, type `cake Plugin.shell_name [args]`
To get help on a specific command, type `cake shell_name --help`

There is one tiny little annoyance though.  Sometimes, it’s useful to get an output of the CakePHP Shell and use it in another script.  For example, you might need to get a list of all loaded plugins and loop over them, performing another action, outside of CakePHP.  Say, in a bash script.  Getting a list of loaded plugins is easy with the bundled shell like so:

$ ./bin/cake plugin loaded

Welcome to CakePHP v3.4.3 Console
App : src
Path: /home/leonid/Work/cakephp_test/src/
PHP : 7.0.16

But, as you can see, the output is not very useful for machine processing. The welcome header is in the way.  Sure, you can parse it out with regular expressions, or even a simple line count.  But that lacks elegance.  Is there a better way?  I thought there was.

My first approach was to use the –quiet option, which, I thought, would leave me with just the needed output.  It turns out, that’s not what it does.  It strips out all the output, and there is no list of plugins at all.

The second approach worked out better.  I learned about it from this thread.  The solution is to extend the needed CakePHP shell and overwrite the protected _welcome() method.  Here’s the content of the newly created application level shell in src/Shell/PluginShell.php:

namespace App\Shell;

use Cake\Shell\PluginShell as Shell;

class PluginShell extends Shell
     * Silence the welcome message
     * @return void
    protected function _welcome()

And now running the same command as before produces a cleaner output:

$ ./bin/cake plugin loaded

This now can be easily used in other scripts without any need for regular expressions and other trimming techniques.

awless – a Mighty CLI for AWS

awless is a command line interface to the Amazon AWS.  While Amazon AWS already has its own set of tools for command line interface, awless makes things even simpler, with the following features:

  • run frequent actions by using simple commands
  • easily explore your infrastructure and cloud resources inter relations via CLI
  • ensure smart defaults & security best practices
  • manage resources through robust runnable & scriptable templates (see awless templates)
  • explore, analyse and query your infrastructure offline
  • explore, analyse and query your infrastructure through time

How To Use Git to Manage your User Configuration Files

There is probably a gadzillion different ways that you can manage and synchronize you configuration files (aka dotfiles) between different Linux/UNIX boxes – anything from custom symlink scripts, all the way to configuration management tools like Puppet and Ansible.  Here are a few options to look at if you are not doing it already.

Personally, I’m using Ansible and I’m quite happy with it, as it allows me to have multiple playbooks (base configuration, desktop configuration, development setup, etc), and do more things than just manage my configuration files (install packages and tools that I often need, setup correct permissions, and more).

Recently, I came across this tutorial from Digital Ocean on how to manage your configuration files with git.  Again, there are a few options discussed in there, as even with git, there’s more than one way to do it (TMTOWTDI).

The one that I’ve heard about a long time ago, but completely forgot, and which I think is quite elegant is the approach of separating the working directory from the git repository:

Now, we do things a bit differently. We will start by specifying a different working directory using the core.worktree git configuration option:

git config core.worktree "../../"

What this does is establish the working directory relative to the path of the .git directory. The first ../refers to the ~/configs directory, and the second one points us one step beyond that to our home directory.

Basically, we’ve told git “keep the repository here, but the files you are managing are two levels above the repo”.

I guess, if you stick purely to git, you can offload some of the additional processing, such as permission changes and package installation, into one of the git hooks.  Something like post-checkout or post-merge.

Defensive BASH Programming

If you write any Bash code that lasts more than a day, you should definitely read “Defensive BASH Programming” and follow the advice, if you haven’t already.  It covers the following:

  • Immutable global variables
  • Everything is local
  • main()
  • Everything is a function
  • Debugging functions
  • Code clarity
  • Each line does just one thing
  • Printing usage
  • Command line arguments
  • Unit Testing

All that with code examples and explanation of importance.


SC-IM – Spreadsheet Calculator Improvised

Here is an interesting project – SC-IM, or Spreadsheet Calculator Improvised, which is an ncurses spreadsheet program for terminal.  Here are some of the features:

  • UNDO / REDO.
  • 65.536 rows and 702 columns supported. (The number of rows can be expanded to 1.048.576 if wished).
  • CSV / TAB delimited file import and export.
  • XLS / XLSX file import.
  • Key-mappings.
  • Sort of rows.
  • Filter of rows.
  • Cell shifting.
  • 256 color support – screen colors can be customized by user, even at runtime.
  • Colorize cells or give them format such as bold or underline.
  • Wide character support. The following alphabets are supported: English, Spanish, French, Italian, German, Portuguese, Russian, Ukrainian, Greek, Turkish, Czech, Japanese, Chinese.
  • Implement external functions in the language you prefer and use them in SC-IM.
  • Use SC-IM as a non-interactive calculator, reading its input from a external script.
  • More movements commands implemented !
  • Input and Output was completely rewritten.

A combination of interactive and non-interactive interface seems to be particularly useful.

dotfiles – your unofficial guide to dotfiles on GitHub

Warning: you will lose a lot of sleep if you follow the link below. :)

No matter how well you know Vim, bash, git, and a whole slew of other command line tools, I promise you, you’ll find something new, something you had no idea existed, something that will help you save hours and hours of your life by shaving off a few seconds here and there on the tasks you perform on a daily basis, in the repositories link to from this site.

I think I’ve spent most of my Sunday there and my dotfiles are so different now that I’m not sure I should commit and push them all in one go.  I think I might need to get used to the changes first.

Some of the things that I’ve found for myself:

  • PHP Integration environment for Vim (spf13/PIV).
  • myrepos – provides a mr command, which is a tool to manage all your version control repositories.
  • bash-it – a community Bash framework.
  • Awesome dotfiles – a curated list of dotfiles resources.

… and a whole lot of snippets, tips, and tricks.

P.S.: Make sure you don’t spend too much time on these things though :)