Sysadmin

SQL Server in a Fedora Docker Container

MS SQL Server and Docker

It’s a well known fact that I am not the greatest fan of Microsoft and their technologies. I’ve been bitten many a time through the years. And not even them becoming a Platinum Partner in the Linux Foundation can change my attitude towards them. It’s just been too much pain, and scars, and tears, and sweat.

But the way life is, once in a while, I just have to work with or around them. Recently, for example, at work, we’ve done a project that just had to use MS SQL Server and there was no way to get around it. Gladly, I managed to find just the right image on the Amazon AWS Marketplace, and spin a new EC2 instance for testing. The local development was difficult, but at least we had a place to test stuff before sending it off to the customer.

If such a need arises in the future, I think I’ll give the MS SQL for Linux a try. And that’s when this article from Fedora Magazine might come in handy. MS SQL + Docker + Fedora. Hmm.

Monitoring the monitoring : keeping Zabbix server service up

After our recent MySQL migrations, I started getting a weird issue – Zabbix server process was crashing periodically (several times a day).

8395:20161109:175408.023 [Z3005] query failed: [2013] Lost connection to MySQL server during query [begin;]
8395:20161109:175408.024 [Z3001] connection to database 'zabbix_database_name_here' failed: [2003] Can't connect to MySQL server on 'zabbix_database_host_here' (111)
8395:20161109:175408.024 Got signal [signal:11(SIGSEGV),reason:1,refaddr:(nil)]. Crashing ...

Digging around for a bit, it seems like a widely reported issue, related Zabbix server using the same database connection as one of its agents is monitoring (here is an example bug report).

Not having enough time to troubleshoot and fix it properly, I decided for the time being to use another monitoring tool – monit – to keep an eye on the Zabbix server process and restart it, if it’s down. After “yum install monit“, the following was dropped into /etc/monit.d/zabbix:

check process zabbix_server with pidfile /var/run/zabbix/zabbix_server.pid
    start program = "/sbin/service zabbix-server start" with timeout 60 seconds
    stop program = "/sbin/service zabbix-server stop"

Start the monit service, make sure it also starts at boot, and watch it in action via the /var/log/monit:

[UTC Nov 20 20:49:18] error    : 'zabbix_server' process is not running
[UTC Nov 20 20:49:18] info     : 'zabbix_server' trying to restart
[UTC Nov 20 20:49:18] info     : 'zabbix_server' start: /sbin/service
[UTC Nov 20 20:50:19] info     : 'zabbix_server' process is running with pid 28941

The chances of both systems failing at once are slim, so I think this will buy me some time.

Fixing “InnoDB: Error: log file ./ib_logfile0 is of different size”

For the last few days I’ve been moving MySQL databases around at work. Being a bit in a rush and overconfident (I have backups!), I was simply detaching the /var/lib/mysql volume on one host (running Amazon AMI and MySQL) and attaching it to another host (running CentOS 7 and MariaDB).

It’s not surprising that I got this error: “InnoDB: Error: log file ./ib_logfile0 is of different size“. Gladly, this ServerFault thread provided enough hints for me to solve the problem. In a nutshell:

Temporarily comment out the InnoDB log file size setting (e.g.: innodb_log_file_size = 64M) in /etc/my.cnf.
Set innodb_fast_shutdown to 0 (read more).
Restart the MySQL service once or twice.
Uncomment the log file size setting.
Set InnoDB fast shutdown back to default or remove it from your my.cnf altogether.
Celebrate!

Knowing how little I learn from my own mistakes, I’m sure I’ll find this post useful in the future.

Shell parameter expansion : default values for shell script parameters

When writing shell scripts, it’s often useful to accept some command line parameters. It’s even more useful to have some defaults for those parameters. Until now I’ve been using if statements to check if the parameter was empty, and if it was, to set it to the default value. Something like this:

#!/bin/bash

DB_HOST=$1
DB_NAME=$2
DB_USER=$3
DB_PASS=$4

if [ -z "$DB_HOST" ]
then
    DB_HOST="localhost"
fi

if [ -z "$DB_NAME" ]
then
    DB_NAME="wordpress"
fi

if [ -z "$DB_USER" ]
then
    DB_USER="root"
fi

echo "Connecting to the database:"
echo "Host: $DB_HOST"
echo "Name: $DB_NAME"
echo "User: $DB_USER"
echo "Pass: $DB_PASS"

It turns out there is a much more elegant way to do this with shell parameter expansion. Here is how it looks rewritten:

#!/bin/bash

DB_HOST=${1-localhost}
DB_NAME=${2-wordpress}
DB_USER=${3-root}
DB_PASS=$4

echo "Connecting to the database:"
echo "Host: $DB_HOST"
echo "Name: $DB_NAME"
echo "User: $DB_USER"
echo "Pass: $DB_PASS"

This is so much better. Not only the script itself is shorter, but it’s also much more obvious what is going on. Copy-paste errors are much less likely to happen here too.

I wish I learned about this sooner.