The new websolr proxy rollout completed successfully for our California and Virginia regions. There were some errors detected in our rollout in our Ireland region (details below), and we rolled back the update in that region.
During the maintenance window, users in our Ireland region may have seen a period of 10-15 minutes of elevated 503 errors. After detecting the elevated errors, our ops team rolled back the deployment in Ireland. Following further analysis, we've identified the issue and will will re-run the Ireland region rollout in our next maintenance window.
Also, users in our Virginia region utilizing our optional HMAC authentication scheme may have seen up to an hour of elevated error responses. After detecting error responses related to our HMAC authentication scheme in Virginia, our ops team quickly identified the issue and prepared and deployed a patch.