The issue was traced to a partial outage in AWS. The root cause was a malfunction within a single AZ, which caused a handful of instances to erroneously be replaced by the ASG. This manifested in Websolr as indices suddenly returning HTTP 500-level errors. Users without replication additionally experienced some data loss. Further questions should be directed to support@websolr.com.
Posted Mar 07, 2019 - 17:11 UTC
Investigating
We are investigating some issues affecting servers in a single availability zone. We are responding to issues as they arise, and are actively investigating the root cause. Automatic recovery is happening slowly, but steadily.
Posted Mar 07, 2019 - 16:03 UTC
This incident affected: Region Health: AWS US East (Virginia).