- Define a path for error handling
- Have a plan for recovery
- Always set timeouts on remote system calls
- Retry on timeout
- Use circuit breaker
- Don't handle timeouts like a failure
- Don't invoke remote systems inside transactions
- Use smart batching
- All APIs MUST be idempotent
- Define response time and throughput SLAs explicitly and code to adhere to them
- Define and limit batch APIs
- Think about Observability up-front
- Cache aggressively
- Consider unit of failure
- Isolate external domain objects at the edge of the system
- Sanitize input at every edge
- Never commit credentials