Eventual Consistency…

Eventual consistency is eventually consistent eventually.

Working with Terraform to create/destroy a non-trivial deployment into AWS can be an exercise in patience. The primary reason isn’t necessarily the quirks of Terraform – and it has a few manageable ones – but instead, it’s those things that we have absolutely no actual control over like Amazon’s necessary Eventual Consistency.

Eventual consistency, of course, is necessary because AWS is a distributed infrastructure. It takes time for configurations to be passed around to the assorted systems involved and implemented.

It works a bit like this:

Requestor: “Hey, Amazon, could you create me a VPC?”

AWS: “Done. Your new VPC ID is vpc-1234567890”

Requestor: “Great, thanks – now in vpc-1234567890, create me a loadbalancer named ‘webservers’ and one called ‘databases’”

AWS: “Whoa, woah, woah, woah! You need a VPC to do that.”

Requestor: “To do what?”

AWS: “To create a loadbalancer. You need a VPC.”

Requestor: “Yes, vpc-1234567890 – on that one, which you just created, put those loadbalancers in there.”

AWS: “I have no record of a working VPC with that name.”

Requestor: “But you just created it! You said, ‘Done: Your new VPC ID is vpc-1234567890’!”

AWS: “I said I created it. I didn’t say it was ready for use.”

Requestor: gives up… just before vpc-1234567890 is ready for use.

Just because the result code said it was “complete”, it doesn’t mean “functioning”.

However, because of Eventual Consistency, Amazon could take, per their docs, up to five minutes to complete some tasks, as engineers, we need to be aware and compensate for it. In practice, I’ve observed it can be just a few seconds, or even much longer than five minutes for some tasks.

I know what you’re thinking. You’re thinking, “I’ll just add a 30-second wait here or there.”

It’s a trap!

Instead, get creative and design your functions so they do a few things:

  • First: submit your Create request to AWS and wait for the receipt
  • Next: poll AWS every few seconds with a Describe
  • Next: Compare the Describe to the initial Create request
  • Finally: If they do match, poll once more to confirm again that they match.
  • Yes, really.

Limitations of Humanity

Kurzgesagt outlines how far humanity will ever go in the universe. Ever.

Yep. Even with science fiction technology, humans – even the robots or species that will ever come after us – will never travel any further than the Local Group.

And yet the possibility of getting off of this planet and becoming a space-faring species seems almost an impossibility. Not because of the technical challenges – but the political ones.