From DevOps to AI Infrastructure: Extend, Don't Restart
Your platform team already has most of what production AI needs. A practical map from DevOps skills to AI infrastructure, and a checklist for putting RAG and agents into production safely.
Teams that want to run AI in production often assume they need a new kind of engineer. In most cases they need their existing platform team to extend what it already does. This guide maps the skills you have to the work AI adds, and gives a checklist for taking a retrieval-augmented (RAG) assistant or an agent into production. It is the practical side of our AI Infrastructure service.
The map
| What your team already does | What it becomes with AI |
|---|---|
| Monitoring and alerting | Evaluating answers: quality scores, regression tests on prompts, traces per request |
| Infrastructure as code | GPU nodes, model servers and gateways defined the same way, reviewed and repeatable |
| CI/CD | Prompt, model and index changes shipped through the same pipeline, with evaluation as a gate |
| DevSecOps | Guardrails: input and output checks, prompt-injection defence, limits on what an agent may call |
| IAM and secrets | Per-agent and per-tool permissions, so an assistant cannot read more than the person asking |
| Cost control | Cost per request and per token, measured and capped |
None of this starts from zero. The new part is the model itself: it is probabilistic, so “it deployed” no longer means “it works”.
What a production RAG assistant needs
- Ingestion: a repeatable job that cleans, splits and embeds your documents, and removes what was deleted at the source.
- A vector store with access control: results are filtered by who is asking, not only by similarity.
- An API layer: authentication, rate limits, timeouts and a fallback when the model is slow.
- Containers and a pipeline: the same build, test and deploy path as every other service.
- Evaluation: a fixed set of real questions with expected answers, run on every change to prompt, model or index.
- Prompt-injection defence: treat retrieved text and tool output as untrusted input. Never let it widen what the assistant is allowed to do.
- Tracing: every answer can be traced back to the prompt, the retrieved sources and the model version.
Where agents change the picture
An agent calls tools, so each tool is a new door into your systems. Expose them through an MCP server or gateway with authentication, narrow scopes and an audit log. For actions that are hard to undo, such as sending, deleting or paying, keep a person in the loop and make the approval step part of the workflow, not an afterthought.
A first week
- Day 1: list one real use case and the data it needs.
- Day 2 to 3: build the evaluation set from 20 real questions.
- Day 4 to 5: run the simplest version end to end behind your normal pipeline, and read the traces.
If the evaluation set is not ready, the system is not ready, however good the demo looks.
Next step
If you want a second pair of eyes on your first AI workload, book a call and we will walk through the map above with your setup.
Ready to simplify your platform?
We work with a small number of clients at a time. If you’re dealing with reliability problems, complexity, or a platform that’s hard to operate, let’s talk.