Architecture · 6 May 2026
System design for a team of five, not a platform org
Atalaya Digital
I have inherited "reference architectures" with a lake, a mesh, a feature store, and three buses. The team was four people. Two of them were on support.
Architecture is what you can still explain on a whiteboard after an incident. Everything else is inventory.
Three planes, one warehouse
We keep the same shape whether the cloud is GCP or AWS:
- Ingest — scheduled jobs, files, CDC. Idempotent. Observed.
- Model — staging then marts. dbt-shaped SQL. Tests on the grain.
- Serve — APIs and boards on the marts. No hidden cube.
Object storage is raw and cheap. The warehouse is the contract. A lakehouse product in the middle is optional. Most clients do not need it in year one.
Boundaries that survive a hire
Services are boring: FastAPI or Cloud Functions in front of the warehouse, Next.js or a thin board for humans, Docker for the jobs. Angular shows up when the client already lives there. We do not rewrite a plant UI to prove a point.
Auth is one place. Secrets are not in the DAG. Environments are dev / prod, not twelve stages that nobody promotes.
If a new engineer cannot draw the request path in five minutes, the architecture is too clever.
Failure is a first-class path
Retries, late data, consent deletes, and "the file never came" are designed with the happy path. We do not add them after the first outage.
Queues beat chatty services when a dealer DMS or a HIS is slow. Synchronous "microservices" across a factory network are how you invent distributed locks you cannot see.
What we cut
A real-time stack for a daily close. Kubernetes for three batch jobs. A second warehouse "for AI." Event sourcing because a conference talk was good.
You can add those when the question demands them. Until then the system is: ingest, model, serve, observe. That is enough to move, and enough to leave.