Education Center - What Causes Outages?
What is the Issue with Outages?
Organizations are reliant on digital communications secured using X.509 certificates for their day-to- day operations. Certificates are used for user authentication, secure communications using Secure Sockets Layer (SSL), program-to-program and machine-to-machine communications, digital signature, and code signing. Many unseen internal and external services performing their daily duties are authenticated and trusted based on a relatively simple process involving the verification that an issued certificate is still active.
Expired X.509 certificates result in a number of system maladies, ranging from a simple error message on a screen to an abrupt termination of service. This can lead to abandoned e-commerce transactions or the loss of trust in a company's Web presence. Many organizations that have an unplanned certificate expiry typically focus on other systemic causes, such as hardware/software issues, long before they begin to consider an expired X.509 certificate as the source of troubles. This typically results in significant delays in identifying and resolving the root cause of a system outage.
During the past few years, another form of X.509 certificate-related problem has come forward in security news — that of the compromised public certificate issuing authority. Because of threatened or actual compromises at DigiNotar, Comodo and others, previously issued certificates from these public providers have had to be revoked, either individually or, at worst, en masse, with an entire certificate issuance infrastructure losing complete trustworthiness. This latter case invalidated all currently issued and active X.509 certificates from that certificate authority, and necessitated a complete reissuance of X.509 certificates from an alternate issuer. Organizations should be explicitly aware of the potential for significant impact on their operations should they be associated with such an incident. Knowing the specific provenance of each and every X.509 certificates in use within an organization is critical in ensuring the timely reissuance of certificates, thus minimizing downtime. It is also necessary to have a suitable management interface to remotely installed certificates, and to trust root certificate lists so they too can be replaced if compromised (see "Certificate Authority Breaches Impact Web Servers, Highlighting the Need for Better Controls"). Many organizations scramble to remove a trusted root when a compromise takes place. Organizations need to understand their internal process for removing a trusted certificate or root from browsers and applications. While browsers are relatively easy to fix by waiting for the browser patch with the removed/deleted/revoked root, applications typically involve more work and, thus, more planning.
One of the first steps in identifying compromised certificates is to identify ALL certificates and evaluate each relying server for certificate validity. However, many organizations do not have a good understanding of their inventory of digital certificates or where they are. They may rely on spreadsheet-based manual methods to track certificates, but the spreadsheet method is deficient in that it does not record or track anything that is not entered into it. The commercial certificate authorities and public key infrastructure (PKI) software vendors do provide some rudimentary certificate management. For example, they may notify the original individual corporate purchaser of an imminent certificate expiry, but not be able to handle cases where that individual has changed roles or left the organization.
X.509 Certificate Management Functions
Several commercial certificate authorities, PKI and certificate management solution providers offer Certificate Management System (CMS) software that can be used to discover, identify, track, notify, and ultimately automatically renew and audit the installation of new X.509 certificates. In general, the functions of this software are:
Discovery:
- Discovery is a primary function of a CMS. It scans the network, systems and applications; logs all instances of X.509 certificates; and may include all or some of the following: SSL, Transport Layer Security, Secure Socket Shell (SSH), Pretty Good Privacy and others. Filters can be set to limit "noise" in discovery, but overall discovery should be able to support a deep dive understanding of where cryptographic keys are stored, their strength, the issuer, validity period and expiry date. There is a potential problem with discovery "crawlers" that may occasionally cause a target platform to malfunction. Further, some keys, such as those supporting Microsoft's Encrypting File System, are stored in the registry, meaning the discovery agent cannot access them unless the local user is logged on.
Ownership:
- Identify who the certificate owners are for a given certificate and the approval structure for the issuance and renewal. Billing and chargeback processes can also be associated for a certificate as part of this activity.
Validate:
- A CMS may eventually have a feature that regularly checks certificates against certificate revocation lists (CRLs) and Online Certificate Status Protocol (OCSP) responders to recertify trust both inside and outside the context of a live interaction between servers, or between a browser and a server. Currently, validation is a function of the client (such as a browser), but is not typically turned on for performance or other reasons. Ultimately, CRL and OCSP functionality will become a mature CMS feature.
Renewal/provisioning:
- Some CMSs can support the automatic renewal of a certificate within a prescribed period prior to expiry. What is important to note is that the renewal process from most certificate issuers and PKI vendors typically favor their own brands, meaning that certificates slated for renewal will be renewed using only their own certificate authority. At this time, only Venafi supports renewal using any certificate authority (internal or external).
Audit:
- When a certificate is renewed, it is critically important to ensure that the new X.509 certificate itself was both installed and rendered active within the target system, otherwise the previous X.509 certificate can remain active even after a new certificate has been installed.
- Verify the actual in-use certificates against those charged by a third party or an internal a provider list.
A limited number of certificates can be managed manually using a spreadsheet or other basic tools, but many features, such as discovery, will be missing. If this method is chosen, specific individuals or roles need to be assigned to managed certificates on groups of machines, and scheduling reminders set for certificate renewal before the installed certificates expire.
