Thoughts on technology, movies, and everything else
System administration is a special are of IT. It also has a special place in my heart. It is an interesting mixture of all the other disciplines, both common across the whole industry, and at the same time unique for each person, company, and geographical location. When I have something to say or share about system administration, I use this category.
For quite some time now I wanted to play around with the recently added JSON type in MySQL. Finally, I have a project where MySQL version is high enough to support it, and the requirements are such that this choice makes sense.
The first impression was great – JSON type is basically LONGTEXT type with a bunch of added functionality to manipulate JSON data. It took no time to setup tables and necessary queries to work with it.
The second iteration though raised a few questions. Large tables, with complex JSON structures were rather slow in some of the more complex queries. The first solution to look at was obviously indexes. Turns out, MySQL does not support indexing of the JSON fields. Bummer.
But there is a rather elegant work around. It involves another recently added feature, of which I haven’t heard about until today – GENERATED columns. Think of table views, but on the column level, not table level. And generated columns can be indexed.
In fact, there’s a whole lot that you can do with GENERATED columns in general, and JSON data in particular. This blog post – “MySQL for JSON: Generated Columns and Indexing” – provides a great starting point with examples and explanations, including a scenario with the primary key of the table being a generated column, with the data from the JSON-typed column.
Whether you are an experienced shell user, or just a newbie, have a look at this article for a collection of the great tools and examples of how to process text in the shell. It includes all the usual suspects: cat, head, tail, wc, grep, cut, paste, sort, uniq, awk, tr, fold, and sed. Great examples and real life scenarios for each are also provided, with the logic explained and more complex use cases broken down into steps.
k6 is a developer centric open source load and performance regression testing tool for testing the performance of your cloud native backend infrastructure: APIs, microservices, serverless, containers and websites. It’s built to integrate well into your development workflow and CI/CD automation pipelines.
This is one of the better tools that I’ve seen in a long time. Not only it does its job great, but it integrates brilliantly with your development and testing pipelines.
You can either build your tests from scratch, or you can convert import them from your existing tools. For example, Postman collections, environments, and tests can be converted to k6 with postman-to-k6. Here’s a blog post to get you started on that path.
Side note: if you hit the “EACCES: permission denied, mkdir ‘/usr/local/lib/node_modules/postman-to-k6/vendor’” durin the postman-to-k6 installation, then simply append “–unsafe-perm=true –allow-root” to the “npm install” command, as suggested in this GitHub thread.