Gadget - Notice history

We've identified the issue and have implemented some fixes to stabilize the system.

Automation: This incident has been resolved.

Automation: This incident has been resolved.

Automation: This incident has been resolved.

We've identified the issue and made the appropriate fix. We will be monitoring actively for the following hours.

We are continuing to monitor this, but the DB looks to be back to normal.

We're still investigating the reason for the spike of 503 responses, however everything has recovered and is operating smoothly.

Our systems have recovered. We're still investigating what happened.

We want to update you on a performance issue that affected our API services today. Starting at approximately 7 AM EST, we began experiencing slower response times and, in some instances, timeouts across our services. After thorough investigation, our team identified that the root cause was a failure in the connection to one of our Redis caches. This issue was pinpointed around 12 PM EST.

By 1 PM EST, we successfully deployed a fix to restore the connection and stabilize our services. We have monitored the situation closely and can confirm that the API is now performing normally.

We sincerely apologize for any inconvenience this may have caused and appreciate your patience as we worked to resolve the issue. Our team will continue to monitor our systems to prevent similar issues in the future.

Automation: This incident has been resolved.

We recently updated our NGINX settings to address an issue, but this unintentionally caused websocket connection timeout errors. We have now restored the original settings, and the websocket connection issues have been resolved.

Background actions are now processing at their usual speed.