PHP limit on maximum form fields

We had an interesting issue to debug at work today.  One of the screens in our application features a form with a whole lot of checkboxes.  It’s in the access control module, where the administrator of the system can manage user permissions for each module of the system.  Here’s the screenshot just to give you an idea.

This is not the pretties user interface (yet), but it sort of works.  Every module can be expanded or collapsed, and inside each modules all checkboxes can be checked and unchecked easily.

The problem that we had wasn’t user interface related.  It was something else.  The number of modules and permission checkboxes varies from system to system.  This is based on each particular system setup.  Now, on some of these systems, it was reported that some permissions are not being saved.

The problem is not new, but it was slowly escalating with more and more clients reporting it.  So, today we dived into it and found the cause of it.   Since PHP 5.3.9 there is a new runtime configuration setting: max_input_vars, with default value of 1.000:

max_input_varsinteger

How many input variables may be accepted (limit is applied to $_GET, $_POST and $_COOKIE superglobal separately). Use of this directive mitigates the possibility of denial of service attacks which use hash collisions. If there are more input variables than specified by this directive, an E_WARNING is issued, and further input variables are truncated from the request.

So, on the systems, where the form contains more than a thousand checkboxes, PHP was only bringing in the first thousand and skipping the rest, causing not all permissions being saved properly.

Increasing the value in runtime configuration is one way to solve it.  But since we have a rather dynamic system and don’t always control the runtime configuration (client hosting), we opted for a different solution.  As per this StackOverflow thread, it’s a much more future-proof solution to combine the values into a single field.  Either simple concatenate, or JSON-encode the values on form submit, and send them all as a single field value.  Then just split or JSON-decode on the server before processing and you are done.

P.S.: The extra bit that made the troubleshooting so much more difficult was that for some reason we were not seeing the PHP warning in logs.

 

On Semantic Versioning

First of all, you should know that versioning is important.  Even the worst versioning practices provide more value than no versioning at all.  At work, we are big fans of the Semantic Versioning, and we use it for all our projects, plugins, and libraries.  And I think, you should do too.

In general, Semantic Versioning works great for us.  But there were a few bumps recently, with more and more libraries dropping support for PHP 5.6 and requiring PHP 7.  I can’t blame them – after all PHP 5.6 has reached its end of life quite a while ago.

It’s not what the maintainers do, but how they do it that I have an issue with.  I’ve been thinking about writing a blog post on the subject for a few month now.  Never got to it.  And yesterday I came across this blog post by Paul Jones, which is so much better than whatever I was about to say.  Paul explains the problem in detail and suggests the “System” addendum to Semantic Versioning:

I opine that requiring a change in the public environment into which a package is installed is just as major an incompatibility as introducing a breaking change to the public API of the package. To cover that case, I offer the following as a draft addendum to the SemVer spec:

  • If the package consumer has to change a publicly-available system resource to upgrade a package, then the package upgrade is not backwards-compatible with the existing system, and the package SHOULD receive a major version bump.

Using “SHOULD” makes this rule somewhat less strict than the MUST of a major version bump when changing the package API.

Coming back to PHP 5.6 vs PHP 7, that would suggest that maintainers who drop support for PHP 5.6 SHOULD bump up the MAJOR release of the library.   And I wholeheartedly agree with that!

P.S.: For those of you who don’t or can’t use Semantic Versioning for whatever reason, checkout Paul’s blog post on Semantic Versioning vs. Romantic Versioning.

CakePHP with NightwatchJS on Travis CI

My colleague Andrey Vystavkin has been setting up a testing environment for our CakePHP projects recently.  We had one before, of course, using PHPUnit.  But this time we wanted to add Google Chrome headless browser with some form of JavaScript test suite, so that we could cover functional tests and a bit of front-end.  Andrey described the configuration of NightwatchJS on TravisCI in this blog post.  If you are more of a “show me the code on GitHub” person, have a look at this Pull Request (still work in progress) on our project-template-cakephp project.

Once we are happy with the TravisCI configuration, we’ll be bringing this setup to our BitBucket Pipelines environment as well.

The setup is also based around CakePHP framework, but it’s easy enough to adopt it to any other framework, PHP or not.

Clean Code SOLID principles applied to PHP

clean-code-php is an excellent set of examples for the SOLID principles as applied to PHP programming:

Software engineering principles, from Robert C. Martin’s book Clean Code, adapted for PHP. This is not a style guide. It’s a guide to producing readable, reusable, and refactorable software in PHP.

Not every principle herein has to be strictly followed, and even fewer will be universally agreed upon. These are guidelines and nothing more, but they are ones codified over many years of collective experience by the authors of Clean Code.

Inspired from clean-code-javascript

TNTSearch – a fully featured full text search engine written in PHP

TNTSearch – a fully featured full text search engine written in PHP.  Here’s also a blog post that shows how to use it with the Laravel framework.  Which shouldn’t be too difficult to adjust for any other PHP framework.

Integrated Package for better testing in CakePHP

Viraj Khatavkar wrote this blog post showing how to use Integrated Package for better testing in CakePHP.  Testing in general is not a simple subject, so anything to assist with it is very very welcome.

I’m sure we’ll be trying it at work in the next week or two.

Using php-fpm as a simple built-in async queue

Here’s an interesting solution for a poor man’s asynchronous queue using PHP-FPM:

PHP-FPM already acts as a queue for Nginx/Apache FastCGI clients. While your web-request is running you can just send another FastCGI request to the same PHP-FPM socket asynchronously and non-blocking. This request is immediately executed in another php-fpm process in parallel and you could wait for it to complete or just fire and forget.

Given the experimental nature of this approach, you probably won’t be running this in production.  And with many developers switching to the built-in PHP web server for the local development, this doesn’t work for those environments other.

But it makes me think what else can be used as a queuing mechanism.  After all, there are plenty of systems that rely on this already – email servers, printer spoolers, web and proxy servers, and probably more.

CakePHP Events System

Events are a great way to separate the business logic of your application and make things simpler and, often, faster.  CakePHP framework introduced an events system in version 2.1, and since then it got much better.  The official documentation covers current implementation pretty well.  But in this post I wanted to link to a few articles that provide more of a historical perspective.

First, goes this blog post by Martin Bean from back in 2013.  It shows how things were initially.  Even with all the improvements in version 3, the first implementation was still pretty useful.

Second, comes this review of the CakePHP events system (still in version 2), and some profiling of this new functionality.  These guys looked at all the details and eventually suggested some improvements.

Their effort didn’t go unnoticed.  Mark Story, one of the lead developers of CakePHP framework, wrote this blog post, explaining the upcoming (at the time) changes to the events system in CakePHP version 3.

As a result CakePHP 3 event system is a much simpler and cleaner implementation.  Have a look at this guide for a quick introduction.

I’m sure this is not the end of the road, as no software is ever perfect.  But it’s a good place to be.

pre-commit – a framework for managingmulti-language git pre-commit hooks

From the pre-commit homepage:

Git hook scripts are useful for identifying simple issues before submission to code review. We run our hooks on every commit to automatically point out issues in code such as missing semicolons, trailing whitespace, and debug statements. By pointing these issues out before code review, this allows a code reviewer to focus on the architecture of a change while not wasting time with trivial style nitpicks.

As we created more libraries and projects we recognized that sharing our pre-commit hooks across projects is painful. We copied and pasted unwieldy bash scripts from project to project and had to manually change the hooks to work for different project structures.

[…]

We built pre-commit to solve our hook issues. It is a multi-language package manager for pre-commit hooks. You specify a list of hooks you want and pre-commit manages the installation and execution of any hook written in any language before every commit. pre-commit is specifically designed to not require root access.

Have a look at the list of all supported hooks. There’s plenty!

Secure Headers – a PHP library for easier management of browser security features

Modern browsers offer a variety of security mechanisms for web developers.  Unfortunately, some of these aren’t so easy to manage.  One needs a deep understanding of the functionality as well as theory behind.  Secure Headers is a library that makes all that work a lot easier for PHP developers.  Here are some of the features:

  • Add/remove and manage headers easily
  • Build a Content Security Policy, or combine multiple together
  • Content Security Policy analysis
  • Easy integeration with arbitrary frameworks (take a look at the HttpAdapter)
  • Protect incorrectly set cookies
  • Strict mode
  • Safe mode prevents accidental long-term self-DOS when using HSTS, or HPKP
  • Receive warnings about missing, or misconfigured security headers