You’re asked to update the SSL certificate for movingpackets.net on a load balancer. The requestor (me, I suppose) gives you the certificate, the private key and passphrase, and the intermediate bundle file provided by the certificate authority.

movingpackets.net.crt
movingpackets.net.key
movingpackets.net-intermediate-chain.crt

You faithfully go to the load balancer, upload the files, enter the passphrase, and create a client SSL profile referencing the cert/key/chain combination I provided, and all is well. The only thing is, you have 200 VIPs on the load balancer, mostly issued by the same certification authority (CA), so don’t they nominally share the same intermediate chain? (Hint: Almost certainly, yes)

Operational Annoyance

Here is the operational annoyance. The fact that the same intermediate certificate/chain has been uploaded 200 times with different names doesn’t stop things working, but it does seem rather inefficient. As far as I can determine, the F5 LTM load balancers (for example) actually concatenate all the uploaded certificates into a single bundle file and search the bundle when a certificate is referenced. I have no idea if there’s a huge performance gain here (unlikely), but it seems logical to want to minimize that file size regardless. On other systems it may simply provide a large number of files to manage, cluttering the view and making management more time-consuming.

The lack of visibility into the intermediate chain’s content (the name “movingpackets.net-intermediate-chain.crt” tells you nothing about the CA) also means that when the requestor doesn’t provide an intermediate cert, it’s common to see engineers just re-using the intermediate cert that was used on the soon-to-expire certificate, which – 9 times out of 10 – is not valid for the new certificate. So what to do?

Operational Solutions

Use OpenSSL to tell you who signed the server certificate, perhaps. That’s pretty easy if you can remember the syntax; I posted about that specific topic, showing how to decode an SSL Base64 certificate so you can see the content). However in order to be usable for people who don’t have their head firmly stuck up OpenSSL’s, uh, command line options, I realized that something simpler was going to be needed.

The second element of the problem is that even once the certificate is decoded, many people’s head seems to spin at the names. “Symantec Class 3 Secure Server G4” is easily confused with “VeriSign Class 3 Secure Server CA – G3”; it seems that the names and numbers get twisted in our heads and we get confused. I’m as guilty as anybody of going mad trying to follow the chain of certificate issuers sometimes. What is needed then is a solution that will pull out the issuer, then tell the engineer which intermediate certificate bundle to use on the load balancer. The engineer doesn’t upload a new chain cert, they reference a standard certificate that I have uploaded to the load balancers for this very purpose.

Scripty Magic

I am not rewriting OpenSSL in perl/php/ruby/whatever; there’s no return on that investment for me. On the other hand, since I know how to interpret the output of openssl commands, what I can do is to wrap them and present the results nicely, and reference the standard certificates I’ve created. My logic is that if I tell the engineer what to do, it minimizes the chance of an error.

Since this is yet another ‘quick and dirty’ script born of laziness, I find myself once more using Perl to do my dirty work, in turn calling openssl to do the truly difficult stuff. The script is called “checkcert.”

checkcert

Give checkcert a certificate, and not only will it identify the right intermediate chain to use, it also – for additional usefulness in my current environment – generates the A10 CLI commands necessary to upload the cert/key files and build a new base client-ssl profile. Let’s see:

# ./checkcert.pl microsoft.com.crt

checkcert:
           Filename: microsoft.com.crt
            Subject: microsoft.com
             Issuer: Microsoft IT SSL SHA2
 Intermediate Chain: STD-Baltimore_CyberTrust_Root
               File: /sslrepo/STD-Baltimore_CyberTrust_Root

------------------------------------------------------------------------------
# CLI Config. SSH to the relevant A10 and:

conf
slb ssl-load certificate microsoft.com-Jun2015 use-mgmt-port scp://[email protected]/Users/john/microsoft.com.crt
slb ssl-load private-key microsoft.com-Jun2015 use-mgmt-port scp://[email protected]/Users/john/microsoft.com.key

slb template client-ssl microsoft.com-Jun2015
   cert microsoft.com-Jun2015
   chain-cert STD-Baltimore_CyberTrust_Root
   key microsoft.com-Jun2015
   template cipher STD-A10-SSL-Template
   disable-sslv3
!

------------------------------------------------------------------------------

In case you’re wondering, yes that really is the result for the microsoft.com certificate; it seemed like a handy example to use. The file path listed in the headers is simply for reference; the intermediate cert chain (bundle) is already present on the load balancers. The file upload commands given above automatically pull the IP of the host and the path where the command is being run (and thus where the files are) in order to make this as ‘ready to roll’ as it can be. So far I haven’t had to edit any of the commands given; they just work, which is quite a time saver.

Once the commands above have been issued, all that remains is to edit the service-group (or VIP, depending on your load balancer platform) and point to the new client-ssl profile we just created, and the certificate has been upgraded.

Operational ease. Oh yes. Take it a step further and let requestors upload their certificate and key file via a web portal, and the engineer can be presented with the configuration to enter, without having had to touch the certificate files. In fact, since creating the client-ssl profile is a non-disruptive operation as it’s not applied until somebody goes and switches the profile being referenced by the VIPs, this whole process could be automated and the engineer could be presented with the name of the new profile, already built on the device. Yes, I’m always thinking of ways to be lazy save time and effort, and reduce human error (I much prefer computer error).

Think Lazy™

I think this should be my new mantra. I have neither the time nor the job remit to write the kind of large scale strategic automation projects I’d like to see in place; I leave that to those people who use a programming language that doesn’t cause shame when mentioned, and those with a budget to fund it. My focus is by definition purely on tactical wins for the engineer on the ground, not least because it benefits me as well. This one is looking so far like a winner; I’ve used it a number of times already and have been delighted with how fast and simple this has made things.

GitHub

Yes, but I need some time to sanitize the code first. Hang in there; your laughter can wait a little.