Puppet : Beginner’s Guide to Modules

I’ve been using Puppet here and there for about a year now.  In the last six month, I went rather heavy, and managed to accumulate quite a bit of modules, configs, hosts, etc.  Refactoring some of the code, and trying out new ideas, I’m reading through the Beginner’s Guide to Modules to help me with the best practices.  A couple of bits from there, I think, are worth quoting:

(Tip: If you describe the function of your module and you find yourself using the word ‘and’, it’s time to split the module at the ‘and’.)

This above is useful to people like me, who got comfortable throwing bits and pieces into modules, but don’t quite know yet where is that thin line for the module separation.  The above tip makes that crystal clear.  So, for example, “my module installs and configures something” no becomes two modules, where one “installs something” and another “configures something”.

It is standard practice for Puppet users to have upwards of 200 modules in their environment.

With this I know that my current setup is rather small and simplistic.  It’s not the numbers I am after, of course.  But, again, it’s difficult to say if I am doing too much of it, or not enough; if the system has been built to scale to the level where I am.  It turns out,  I have a long way to go.  A quick check shows that I’m at 127 modules in my current setup.  And that one is used for configuring pretty much everything – base installs, web / db / dev servers, desktops, etc.

Some of it will get expanded as I am replacing quite a bit of my own modules with third-party ones.  Interestingly enough, I wasn’t comfortable using other people’s code in the beginning and wanted things to be precisely my way (there were some time constraints).  It worked well, for a time.  But not a lot of it can be thrown away and replaced with other people’s code.  More features and less maintenance for the win!

On Amazon EC2 instances

I am staring at the t2.micro (the smallest available instance type) server running MySQL 5.5.40 (using the my-huge.cnf example configuration shipped with MySQL, which ironically matches t2.micro specs).  Here’s why (as reported by Nagios for the last few hours):

Queries per second avg: 12888.839

The number is fluctuating between about 12,500 and 13,500.  Server load is moving between 0.05 and 0.08.

I think this answers the question of whether or not I am happy with the Amazon EC2 instance performance with a “hell yeah!” bang.

MySQL view processing algorithms

I had a last work session last night, troubleshooting one of the project’s database performance issues.  Without giving more details (at least for now), I want to save the link to MySQL view processing algorithms for future me.

For UNDEFINED, MySQL chooses which algorithm to use. It prefers MERGE over TEMPTABLE if possible, because MERGE is usually more efficient and because a view cannot be updatable if a temporary table is used.

A reason to choose TEMPTABLE explicitly is that locks can be released on underlying tables after the temporary table has been created and before it is used to finish processing the statement. This might result in quicker lock release than the MERGE algorithm so that other clients that use the view are not blocked as long.

A particular heavy query, using views, kept going into “Copying to tmp table” state, locking up the server and slowing everything to a crawl.  Upon closer examination, the view was created without specifying the algorithm (UNDEFINED).  Changing the view to use TEMPTABLE made everything so much faster.

I knew there were reasons for me being against using views in MySQL, but I could never remember them.  This is one.  Views not supporting indexes is another.