"What got a lot of startups in trouble was architecting systems that were fault-tolerant across availability zones within the US-East region, but when one availability zone went down, everybody's apps started flooding the other availability zones"
If most startups implement this solution wouldn't the problem just replicate itself to regions instead of availability zones?
The problem is mathematical in nature and the dependent variables are uncontrolled by consumers of the AWS service. To be certain that you can failover and handle the load you need to run at 50% capacity. EC2 does not do this therefore you cannot solve the problem with certainty using only AWS resources.
Given this is a common problem that all tenants of the system face, I'd like to see Amazon offer a more holistic approach to faul-related instance migration.
Essentially, if Amazon offers that functionality rather than individual scripts, it can have a better chance of managing the resource as a whole, rather than everyone fight for resource with no overall management.
If most startups implement this solution wouldn't the problem just replicate itself to regions instead of availability zones?
I think the problem would be less severe across regions than across availability zones within a region and here's why. A major benefit of multiple availability zones is the low-latency connection between them. That encourages you to copy instances, EBS volumes, S3 buckets, and RDS data between them, most of the time en masse in the event of a failure. Across regions, you're sending data over a slower connection, so you have to take care of replication on an ongoing basis. So your instances, EBS volumes, S3 backups and RDS instances and replicas would already be in the secondary region when failure occurs in the primary. I'd compare it to having a vacation home and spare car in another state when an earthquake hits, instead of looking for a shelter near the disaster area.
If most startups implement this solution wouldn't the problem just replicate itself to regions instead of availability zones?
The problem is mathematical in nature and the dependent variables are uncontrolled by consumers of the AWS service. To be certain that you can failover and handle the load you need to run at 50% capacity. EC2 does not do this therefore you cannot solve the problem with certainty using only AWS resources.