DevOps Engineer - AI Runtime Services

About this role

Not quite right? These jobs in Hyderabad match the same skills — apply to a few to improve your chances.

Software Engineer III (Python, C#, AWS, Snowflake/Databricks and/or Go)

FactSet

In OfficeHyderabad

Apply

Senior Specialist, Platform Analytics AI ( Python and AWS)

Novartis

In OfficeHyderabad

Apply

IT Recruiter

bTranz

In OfficeHyderabad

Apply

Software Engineer

ServiceNow

In OfficeHyderabad

Apply

Skills you'll use

What you'll do

• The LLM gateway. The single front door to every model provider we use — keys, routing, rate limits, failover, provider quotas. When it's down, every AI feature at Demandbase is down.
• Reliability, for real. SLOs, on-call, incident response, and the postmortems for the runtime. This is a genuine SRE ownership role, not "build it and let someone else run it."
• Cost control. Per-team and per-model attribution, budgets, quotas. LLM spend is the kind of line item that quietly triples; your job is to make it legible and then make it smaller.
• LLM observability. Traces, spans, prompt/response capture, ingest and billing visibility for every AI feature in the company.
• Evals and experimentation infra. The shared harnesses, datasets, and scoring plumbing product teams use to know whether a prompt or model change actually made things better.
• Caching and guardrails. Response and semantic caching to cut latency and redundant spend; input/output safety, PII handling, and policy enforcement at the gateway.
• Strong production infrastructure background: Kubernetes, AWS, Terraform, GitOps. You've operated systems that people were paged for, and you've been the one paged.
• Python, at a level where you're comfortable owning services and tooling in it — not just scripting.

Who can apply

• Real on-call and incident-response experience. You can talk about an incident you ran, what you got wrong, and what you changed afterward.
• Observability fluency beyond "we have dashboards": you've instrumented systems, chased cardinality and cost in a metrics/logging backend, and built alerts that fire when they should.
• Hands-on familiarity with how LLM systems actually work. You don't need production ML experience. But you need to have built something with these tools — an agent, a RAG pipeline, an internal tool — and to have used evals or experiments to decide whether it was any good. Tokens, context windows, prompt/response tracing, and why an eval suite is fundamental should all be familiar ground. Candidates who have only read about this are not a fit.
• An LLM gateway or proxy (LiteLLM or similar) in production.
• LLM observability tooling: Datadog LLM Observability, LangSmith, Braintrust, Arize, or similar.
• Eval frameworks and LLM-as-judge scoring in a real workflow, not a demo.
• FinOps instincts — cost attribution, showback, quota design.
• Guardrails, PII detection, or content-filtering systems.

Perks & benefits

About demandbase

Searching for “demandbase jobs” or “jobs at demandbase”? You’re in the right place.