LDAP Degredation

‌

Date: Oct 8, 2025

Date of Incident: Oct 5, 2025

Description: RCA for LDAP authentication failures

‌

Summary:

On October 5, 2025, a number of customers experienced intermittent failures when attempting to authenticate to LDAP. Users and services attempting to authenticate received an error message indicating a failure to successfully establish a connection.

‌

Root Cause:

The incident was caused by a failure in our automated certificate renewal process.‌

1. Certificate Expiration: A critical internal Transport Layer Security certificate, used for secure communication within our infrastructure, expired.
2. Automation Failure: The automated system responsible for proactively renewing this certificate failed to run its scheduled update.
3. Cascading Affect: Because the core certificate was not renewed, dependent LDAP services could not renew their own certificates, leading to connection failures with our core database and security vault.

‌

The team manually executed the renewal script to update and deploy the expired certificate across all necessary servers, and restarted services on systems that did not pick up the new certificates immediately, restoring normal operation to the LDAP services.

‌

Corrective Actions / Risk Mitigation:

1. Immediately execute the renewal script and restart services - DONE
2. Implementing dedicated, proactive alerting on the expiration dates of these infrastructure certificates - IN PROGRESS.
3. Add automation checks that verifies the successful execution of the certificate renewal process - IN PROGRESS