Proxy & load-balancer upgrade global rollout
Scheduled Maintenance Report for Websolr
Postmortem

The new websolr proxy rollout completed successfully for our California and Virginia regions. There were some errors detected in our rollout in our Ireland region (details below), and we rolled back the update in that region.

During the maintenance window, users in our Ireland region may have seen a period of 10-15 minutes of elevated 503 errors. After detecting the elevated errors, our ops team rolled back the deployment in Ireland. Following further analysis, we've identified the issue and will will re-run the Ireland region rollout in our next maintenance window.

Also, users in our Virginia region utilizing our optional HMAC authentication scheme may have seen up to an hour of elevated error responses. After detecting error responses related to our HMAC authentication scheme in Virginia, our ops team quickly identified the issue and prepared and deployed a patch.

Posted Jan 12, 2017 - 12:33 UTC

Completed
The scheduled maintenance has been completed.
Posted Jan 12, 2017 - 05:00 UTC
In progress
Scheduled maintenance is currently in progress. We will provide updates as necessary.
Posted Jan 11, 2017 - 21:00 UTC
Scheduled
We've been working on some major enhancements to our routing and load-balancing layer. Following several successful test rollouts for Cobalt plans, we're proceeding with a global rollout that will affect all users using our standard regional load-balancers.

For any questions, please contact support@websolr.com
Posted Jan 11, 2017 - 11:27 UTC