Post-Mortem Report: Production Infrastructure Service Interruption
Date: July 27, 2026
Impact window: 09:25 AM – 12:00 PM UTC
Total disruption: ~2 hours 35 minutes
Impact
On July 27, between 09:25 AM and 12:00 PM UTC, the Kameleoon application (at app.kameleoon.com) went down and access was not possible. Kameleoon engine files (and SDK config files) were generally available, ensuring correct displaying of AB tests and tracking. However, we had some reports of individual engine files not being served correctly for some customers; in those cases live AB tests were not being delivered until a regeneration process completed.
No customer data was lost or compromised. The deployment was a routine infrastructure configuration update and was not expected to impact customer-facing services.
Cause
The incident occurred during the deployment of a routine production infrastructure configuration update.
During deployment, an incomplete network configuration unintentionally restricted communication between certain infrastructure components. This affected internal platform connectivity and interrupted one of the network paths responsible for serving static assets.
The issue became apparent only after the configuration was applied in the production environment and was not identified during pre-deployment validation.
Resolution
Our engineering teams worked to restore normal platform operation and validate infrastructure health.
Actions taken included:
All services were fully restored by 12:00 PM UTC.
Next Steps & Prevention
To reduce the likelihood of similar incidents in the future, we are implementing the following improvements: