Autoptimize 2.8.2 update “mishap” postmortem

This morning I finally pushed Autoptimize 2.8.2 out of the gates which was a relatively minor release with misc. small improvements/ bugfixes. Only it proved not that minor as it broke some sites after the update, so here’s a quick postmortem.

Timeline

  • 7h33 CEST: I pushed out 2.8.2
  • 7h56 CEST: first forum post about a Fatal PHP error due to wp-content/plugins/autoptimize/classes/external/php/ao-minify-html.php missing
  • 7h58 CEST: second forum post confirming issue
  • 8h01 CEST: responded to both forum posts asking if file was indeed missing on filesystem
  • 8h04 CEST: I changed the “stable version” back to 2.8.1 to stop 2.8.2 from being pushed out.
  • 8h07 CEST: forum post replies confirming the file was indeed missing from the filesystem
  • 8h15 CEST: I pushed out 2.8.3 with the fix
  • 8h22 CEST: confirmed fixed by first user
  • 8h26 CEST: confirmed fixed by second user

Root cause analysis

One of the improvements was changing the classname of the HTML minifier to avoid W3 Total Cache’s HTML minifier being used. For this purpose not only small changes were made to the HTML minifier code, but the file was also renamed from minify-html.php into ao-minify-html.php. The file itself was present on my local filesystem, but I did *not* svn add it, so it was never propagated to the wordpress.org SVN server, resulting in it not being in the 2.8.2 zip-file causing the PHP Fatal “require(): Failed opening required” errors.

Conclusions

Every svn ci has be proceeded by an svn stat, always. I’ve updated my “go live” procedure to reflect that.
Additionally; I strongly advise against automatic updates for Autoptimize (and I don’t auto-update any plugin myself), not only for major f-ups like mine today, but also because any change to how (auto-)optimization works needs to be tested for regressions. And if you have a site that generates money somehow, you really should have a staging site (which can auto-update) to test updates on before applying on production.

WordPress dashboard fatal error; blame & tweak APC!

I upgraded APC a couple of days ago and subsequently encountered a fatal error when trying to load my WordPress dashboard (but not the other wp-admin pages);

Fatal error: Call to undefined function wp_dashboard_setup() in wp-admin/index.php on line 15

Google confirmed the obvious; APC and WordPress didn’t get along (again), and the most common solution (next to switching from APC to XCache or eAccelerator) is to alter wp-admin/index.php by replacing
require_once(ABSPATH . 'wp-admin/includes/dashboard.php');
with
require_once('includes/dashboard.php');
But I prefer not to change code that is bound to be overwritten at every WordPress upgrade and surely other PHP-apps suffer from the same problem, so instead I tweaked APC by adding the following line to my configuration:
apc.include_once_override=0
This tells APC not to optimize include_once (it’s complicated). Why I have to explicitly specify this, if “0” is the default value anyhow, is beyond me though.