command line

Bash trick: Repeat last command until success

More and more often I come across a scenario where I need to repeat the shell command until it succeeds. Here are a couple of examples:

Reboot a server. Try to remotely login to it via ssh. This fails until the server actually boots up. Keep trying until connected.
Start an application that writes to the log file. Run “tail -f some.log” to watch the log messages. This fails if the log file does not exist yet. Keep trying until the application creates the log file and writes something into it.

Sure, I can always press the up arrow key and Enter, to repeat the last command from the history. But it is a tiny bit annoying.

Today I came across this little trick, that solves the problem. Add the following function to your .bashrc:

rpt() {
  CMD=$(fc -ln | tail -n 2 | head -n 1)
  echo "repeating until success: $CMD"
  until $CMD
  do
    sleep 1
  done
}

Now you can run “rpt” to repeat the latest command until it succeeds.

Handy!

40x speed up of ls on large dirs

If you ever tried listing a directory with a lot (10,000+) of files in it, I’m sure you know how annoyingly slow ‘ls’ can be. Turns out there is a simple way to make it better. Have a look at the “When setting an environment variable gives you a 40x speedup” blog post.

Or just do the following:

export LS_COLORS='ex=00:su=00:sg=00:ca=00:'

This will disable colors on the executable files, setuid/setgid bits, and capabilities.

htrace.sh – HTTP/HTTPS troubleshooting and profiling tool

htrace.sh is a handy command-line tool for HTTP/HTTPS troubleshooting and profiling. It also integrates with a number of other security tools, like nmap, SSL Labs, subfinder, etc.

A Deep Dive into Iptables and Netfilter Architecture

It’s been a while since I had to dive into the iptables and netfilter. These days I mostly have to do some basic configuration here and there, with occasional adjustments or troubleshooting (less and less so, thanks to Amazon AWS). But if drilled on the details, I quickly lose my confidence. In an effort to refresh my memory, I looked around for a blog post or an article that is short and simple, yet deep enough for me to brush some rust of. I found “A Deep Dive into Iptables and Netfilter Architecture” very helpful.

Turns out, the bit I needed the most was this one:

Chain Traversal Order
Assuming that the server knows how to route a packet and that the firewall rules permit its transmission, the following flows represent the paths that will be traversed in different situations:
* Incoming packets destined for the local system: PREROUTING -> INPUT
* Incoming packets destined to another host: PREROUTING -> FORWARD -> POSTROUTING
* Locally generated packets: OUTPUT -> POSTROUTING

Technical documentation is so much easier these days. I remember the old days of manual pages and HOWTO guides, and I think we’ve made a lot of progress.

Dotfile madness

“Dotfile madness” is an excellent look at the problem of hidden data and configuration files that seem to be multiplying lately in the users’ home directories:

We are no longer in control of our home directories.
My own home directory contains 25 ordinary files and 144 hidden files. The dotfiles contain data that doesn’t belong to me: it belongs to the programmers whose programs decided to hijack the primary location designed as a storage for my personal files. I can’t place those dotfiles anywhere else and they will appear again if I try to delete them. All I can do is sit here knowing that in the darkness, behind the scenes, they are there. Waiting in silence. Some of those programmers decided to additionally place some normal files and directories in the same place. Those are clearly visible every time I execute ls in my home directory.

While there is no easy centralized solution to this problem, as each application’s developer decides for himself, the article proposes a better way of doing things, reminding us about the XDG Base Directory Specification. This spec allows for a much finer control of where things go via the XDG_* environment variables.

Nice one!