Highly Available, Secure, and Convenient: Leveraging Azure Blob Storage for your PKI Needs, Part 1

For as long as public key infrastructure (PKI) has existed, it has relied on certificate revocation lists (CRLs) and authority information access (AIA). CRLs are a fundamental part of most PKIs that allow clients to be 100% sure a certificate has not been revoked. AIA is a certificate extension that provides information on how to retrieve information about the certificate’s issuer.

AIA data and CRLs are most commonly served from an HTTP server that allows clients using those certificates to verify they are not revoked. A common Active Directory Certificate Services (ADCS) setup involves a separate Windows server with the Internet Information Services (IIS) role installed, allowing it to receive new CRLs automatically (usually through local file share) and immediately make them available for clients. CRLs are also configured to be reachable via LDAP when configuring a new ADCS Certification Authority (CA) by default.

Although these methods are tried and true, there is room for improvement. Some concerns with this configuration are:

• A separate secure server must be maintained for a relatively simple task. This includes patching, Windows license costs, the cost of hardware, and other considerations for any given server.
• With a single IIS site, you have a single point of failure. When the server running IIS needs to be patched or rebooted, clients cannot verify CRLs or AIA. Even if technologies such as multiple IIS sites or load balancers are in use, they still must be maintained and properly managed, leading to additional overhead.
• If CRLs/AIA need to be publicly available, external networking must be configured. This creates an additional security vector and increased operational overhead.
• LDAP CRLs can only be used by devices joined to a domain. For organizations that rely on their PKI being used offsite or from mobile devices, LDAP CRLs are insufficient and provide no value over HTTP CRLs.

If only there were some globally available service that was tailor-made to handle transferring the data we need to reference with our Certificate Distribution Points (CDPs) and AIAs….

Thankfully, there is! Enter Azure Blob Storage. Azure Blob Storage is Microsoft’s solution to allowing files to be served worldwide; it is backed by Microsoft’s highly available infrastructure. An administrator could adjust their ADCS configuration to use Azure Blob Storage for serving CRLs and AIA files. This allows administrators to take advantage of Microsoft’s infrastructure to cost-effectively reduce their organization’s on-premises footprint and management tasks.

Please note that this blog is meant to be a reference for understanding and testing Azure Blob Storage-based CRLs. Performing these steps directly in a production environment is not recommended without rigorous testing!

Requirements

The requirements to configure Azure Blob Storage-based CRLs and AIA are:

• An existing PKI
• The ability to create Azure Blob Storage accounts in Azure Active Directory (AD)
• The ability to modify issuing and root CA CDP/AIA information
• The ability to create and modify internal and external DNS records

Creating an Azure Blob Storage Account

To store CRL information in Azure Blob Storage you must first create a storage account. Although an existing storage account can be leveraged, it is recommended to compartmentalize your storage accounts and other services related to PKI. To create a new storage account, follow these steps:

1. Log in to portal.azure.com.
2. Navigate to Storage Accounts.
3. Select Create.
4. Enter a valid Subscription and Resource Group.
5. OPTIONAL: Create a new Resource Group specifically for PKI if one does not exist.
6. Enter a new meaningful Storage account name; for this example, we will use labgrassapki.
7. Choose the region closest to where your clients will most commonly check for CRLs. Any location may be used, but high latency can have a negative impact on CRL revocation check times. For this example, we will use (US) Central US.
8. Review Microsoft’s Storage Account Overview documentation for a detailed explanation on types of storage accounts. For most situations, Standard performance is sufficient. CRLs are critical infrastructure, and any downtime can result in client impact. For this reason, carefully consider Redundancy options. For this example, we will be configuring Standard performance with Geo-zone-redundant storage (GZRS), with the Make read access to data available in the event of a regional unavailability option.
9. Click the Review button.
10.  You may review the Advanced, Networking, Data protection, Encryption, and Tags panes, but we will make no changes to these settings from the default.
11. Click Review.
12. Review the configuration; once satisfied, click Create.
13. You may need to wait a few minutes until the deployment is complete. Once completed, click Go to resource.

Configuring Azure Blob Storage Networking

Now that you have a storage account, you need to configure the networking settings.

1. Navigate to Networking > Custom Domain.
2. Enter the custom domain name for your CRLs. For this example, we will use azurepki.labgrassa.com.
3. Note the CNAME DNS record recommendations. For our example, we will create a CNAME record in our internal and external DNS provider pointing from azurepki.labgrassa.com to labgrassapki.blob.core.windows.net.
4. Wait a few minutes for DNS to propagate. This may take a variable amount of time depending on the DNS record TTL and your domain registrar.
5. Check the status of the DNS records by using the nslookup command in CMD. A correct configuration will show the blob aliases. Make sure both the internal and external DNS records are correct.

6. Once DNS has propagated, select Save.

Adjust Transfer Settings for the Storage Account

Although it seems counterintuitive, you must allow insecure (HTTP rather than HTTPs) traffic to your storage account. Because it’s a fundamental PKI requirement, if you do not allow HTTP traffic, CRL revocation checking will simply not work. Remember, public certificates and revocation lists are meant to be shared publicly. This is explicitly stated in RFC 5280, which notes that using HTTPS CRLs or AIA may result in unbounded recursion. You will not negatively impact your security posture by allowing HTTP traffic for your storage account, assuming only public certificates and CRLs are hosted there.

1. Navigate to Configuration.
2. Under Secure transfer required, ensure Disabled is selected.
3. Click Save.

Create CDP and AIA Storage Containers

Now that you have a storage container, you need to configure the storage account.

1. Navigate to Containers.
2. Click the + Container button.
3. Name the container aia.
   Note: We are using only lowercase letters and no spaces to simplify the naming convention. Azure Blob Storage is case-sensitive and has special character limitations. If your information does not match exactly, CRL/AIA checking will not work. Review the Microsoft documentation on Naming and Referencing Blobs to ensure that your blob and file names will be valid.
4. Set the Public access level to Blob (anonymous read access for blobs only).
5. Click Create.
6. Click the + Container button.
7. Name the container cdp.
8. Set the Public access level to Blob (anonymous read access for blobs only).
9. Click Create.

Configure Root and Issuing CA CDP and AIA Information

Now that you have a location to store your CRLs and CA certificates, the next step is to update the CA CDP and AIA information to inform clients of the new location and to check for CA public certificates and CRLs. For testing, you must use a strategy of configuring certificates to reference both the current CDP/AIA information and new CDP/AIA information, allowing clients to check for CRLs and CA certificates from either location.

In a large production environment, it would likely be more appropriate to plan a brief downtime in which the CA templates are temporarily unpublished, to allow for all changes to be made in a single cutover. During this downtime, the existing PKI would function normally but new certificates could not be issued. CDP and AIA information are configured at the CA level for clients, so it is essentially all or nothing. Once a certificate is issued with updated CDP and AIA information, there is no way to remotely update it. The certificate must be reissued if the CDP and AIA information needs to be updated.

1. Log in to the root CA.
2. Open the Microsoft Management Console (MMC) Certification Authority snap-in.
3. Right-click the CA name and select Properties.
4. Navigate to the Extensions tab.
5. Document the existing CDP and AIA information to serve as a backup in the event a rollback is required.
6. Ensure CRL Distribution Point (CDP) is selected and click Add.
7. Enter the URL that will be used to access the CRLs and click OK. Do not specify any variables other than <CRLNameSuffix> and <DeltaCRLAllowed>. For our example, we entered http://azurepki.labgrassa.com/cdp/labgrassa_root_ca_01<CRLNameSuffix><DeltaCRLAllowed>.crl.
8. Click the CDP location that was just configured and ensure that the following are selected:
   1. Include in CRLs (Clients use this to find the delta CRL locations.)
   1. Include in the CDP extension of used certificates
   1. Include in the IDP extension of issued CRLs
9. Ensure Authority Information Access (AIA) is selected and click Add.
10. Enter the URL that will be used to access the public CA certificates and click OK. Do not specify any variables other than <CertificateName>. For our example, we entered http://azurepki.labgrassa.com/aia/labgrassa_root_ca_01<CertificateName>.crt.
11. Click the AIA location that was just configured and ensure Include in the AIA extension of issued certificates is selected.
12. Click Apply.
13. You will be notified that ADCS needs to restart. Assuming you can restart ADCS, click Yes.

This process must also be completed on any issuing CAs you wish to migrate to Azure Blob Storage.

From this point forward, any certificate issued by the CA will have both your pre-existing CDP/AIA information and the newly configured CDP/AIA information. Only one or the other needs to work, allowing you to temporarily rely on the existing CRL serving method while you prepare to migrate entirely to blob storage. When you examine a newly issued certificate using the certutil -url command, you should see the existing CDP/AIA URLs working as expected, whereas the new CRP/AIA URLs will fail because you haven’t updated the information in Azure Blob Storage yet.

Manually Publish Root and Issuing CA Public Certificates and CRLs

Now that you have configured the new CDP and AIA information on the CA, you can manually upload your certificates and first new CRL.

1. Log in to the root CA.
2. Open the MMC Certification Authority snap-in.
3. Right-click Revoked Certificates and select All Tasks > Publish.
4. Select New CRL and click OK.
5. Assuming you are running ADCS defaults, the CA public certificate and CRL(s) will be updated at C:\Windows\System32\CertSrv\CertEnroll.
6. Copy the .CRL and .CRT file(s) to another workstation that can log in to Azure.
7. In the workstation and not on the CA, rename the .CRL and .CRT file(s) to match what was configured for the CDP and AIA information. For our example, we updated the file names to:
   labgrassa_root_ca_01.crt
   labgrassa_root_ca_01.crl
   .CRLs or .CRT files that have multiple CA certificates or include delta CRLs might look like:
   labgrassa_issuing_ca_01.crt
   labgrassa_issuing_ca_01.crl
   labgrassa_issuing_ca_01(1).crt
   labgrassa_issuing_ca_01(1)+.cr
8. Navigate to the Azure Blob Storage account you created and select Containers > aia.
9. Click Upload and select the .crt file.
10. Select Upload.
11. Confirm the upload was successful and navigate to Containers > cdp.
12. Click Upload and select the .crl file(s).
13. Confirm the upload was successful.

This process must also be completed on any issuing CAs you wish to migrate to Azure Blob Storage.

After waiting about a minute for the blob storage to propagate, you should see new results when running the certutil -url command against a newly issued certificate.

Bingo! You now have valid CRLs and AIA files being served from highly robust Azure Blob Storage. Surely, it’s time to retire your existing internal HTTP CRL server, right? Not so fast….

Final Notes

Although it’s great that we can take advantage of Azure Blob Storage for serving CRLs and CA public certificates, additional configuration is necessary to make this an enterprise-ready solution. We still must consider:

• How to handle automatic renaming of CRLs and certificates to match the CDP/AIA values
• How to automate CRL publication to Azure Blob Storage
• How to gracefully remove client reliance on old CRL locations
• Removal of old CRL checking infrastructure
• Secure management of the Azure Blob Storage account itself

These topics will be covered in future blog posts. If you’re looking for advice on configuring or managing your PKI, reach out to Ravenswood today! Our experts are here to help.