Question 1

What is Amazon CloudWatch?

Accepted Answer

CloudWatch is AWS's monitoring and observability service: metrics with dashboards and alarms, Logs with search and Insights queries, Events via EventBridge, Synthetics for canaries, RUM for real-user monitoring, and Application Signals for automatic service-level monitoring. The framing to give is that it is the default telemetry layer every AWS service publishes into, so you get baseline metrics with no instrumentation. Application-level detail still needs custom metrics or an agent, which is the gap people underestimate.

Question 2

What is the difference between metrics, logs and traces?

Accepted Answer

Metrics are numeric time series — cheap to store, good for dashboards and alerting, but cannot explain a single request. Logs are discrete records with detail, good for investigating a specific event but expensive at volume. Traces follow one request across services, showing where time was spent. The way to use them together is the substance: alert on metrics because they are cheap and aggregate, use traces to find which service is slow, then logs to find out why. Reaching for logs first for every problem is a slow and expensive workflow.

Question 3

What is a CloudWatch alarm and what states does it have?

Accepted Answer

An alarm watches a metric or a metric math expression against a threshold over a number of evaluation periods, with states OK, ALARM and INSUFFICIENT_DATA. Actions can notify SNS, trigger Auto Scaling, stop or terminate an instance, or invoke Systems Manager automation. The INSUFFICIENT_DATA state is what catches people: a metric that stops being published — because the service stopped entirely — leaves the alarm not in ALARM. Configuring how missing data is treated is the setting that decides whether a total outage pages anyone.

Question 4

What is a composite alarm and why use one?

Accepted Answer

A composite alarm combines other alarms with boolean logic, firing only when a combination is true. It is used to reduce noise — page only when the error-rate alarm and the latency alarm are both firing — and to suppress downstream alarms while a known upstream alarm is active. Alarm suppression during a known dependency outage is the feature that prevents a single incident generating fifty pages. That noise reduction is what keeps the pager trusted, which is the real reliability benefit.

Question 5

What is a metric filter and an embedded metric format?

Accepted Answer

A metric filter extracts a numeric value or a count from log entries matching a pattern and publishes it as a CloudWatch metric. The Embedded Metric Format lets an application emit a structured log entry that CloudWatch automatically parses into metrics, so you get metrics and the surrounding log context in one write. EMF is the better modern approach because it avoids a separate PutMetricData call per metric — which is charged and adds latency — and keeps the dimensional context alongside the log line. Naming it rather than only metric filters shows currency.

Question 6

What is CloudWatch Logs Insights?

Accepted Answer

Logs Insights is a query language for CloudWatch Logs supporting filtering, parsing, aggregation, statistics and visualisation across log groups, charged per gigabyte of data scanned. It is the tool for ad hoc investigation across large log volumes. The cost model — per byte scanned — means narrowing the time range and log groups matters, exactly as it does with Athena. Naming that structured JSON logging makes queries far more reliable than regex parsing of free text is the practice that enables it.

Question 7

How do you control CloudWatch Logs cost?

Accepted Answer

Set retention on every log group rather than leaving it at never-expire; export archival copies to S3 which is far cheaper; filter out high-volume low-value logs such as health checks before ingestion; use log class Infrequent Access for logs you rarely query; and reduce chatty debug logging in production. Ingestion, not storage, is where most of the cost is, so filtering before it reaches CloudWatch saves more than shortening retention. Log groups left at infinite retention are the single most common source of quietly growing observability cost.

Question 8

What is AWS X-Ray?

Accepted Answer

X-Ray provides distributed tracing: it collects segments and subsegments from instrumented services, assembles them into traces showing the path and timing of a request across services, and produces a service map with latency, error and fault rates per node. The requirement that makes it work is context propagation — the trace header must be passed downstream, or traces fragment into disconnected pieces. Sampling rules are the cost control, since tracing every request at high volume is expensive.

Question 9

What is OpenTelemetry and how does it relate to AWS?

Accepted Answer

OpenTelemetry is the vendor-neutral standard for instrumenting applications to emit metrics, traces and logs. The AWS Distro for OpenTelemetry is a supported distribution that exports to X-Ray, CloudWatch, Amazon Managed Prometheus and third-party backends, so instrumentation is not locked to one vendor. The portability argument is the reason to prefer it over the X-Ray SDK: instrumentation is the expensive, invasive part, and doing it once against an open standard means changing backends later is a configuration change. That is a genuine architectural consideration.

Question 10

What is an SLI, an SLO and an SLA?

Accepted Answer

A service level indicator is a measurement of behaviour — request success rate, latency at a percentile. A service level objective is a target for that indicator over a window, such as 99.9% success over 28 days. A service level agreement is a contract with consequences, usually financial, and is normally looser than the internal SLO. The relationship to state is that the SLO should be stricter than the SLA so you detect degradation before breaching a contract. The other point is that an SLI must be measured from the user's perspective — server-side success rate misses failures that never reached your server.

Question 11

What is an error budget and how do you use it?

Accepted Answer

The error budget is the acceptable unreliability implied by the SLO — 99.9% over 30 days allows about 43 minutes of failure. It is a decision-making tool: while budget remains, the team ships features and takes risk; when it is exhausted, priority shifts to reliability work until it recovers. The organisational point is what matters: the error budget turns "are we shipping too fast" from an argument into a measurable, pre-agreed policy. Without a defined consequence for exhausting it, the budget is just a number on a dashboard.

Question 12

What is burn-rate alerting?

Accepted Answer

Burn rate measures how fast you are consuming the error budget relative to the rate that would exhaust it exactly at the window's end. A fast burn over a short window catches sudden outages quickly; a slower burn over a longer window catches gradual degradation. Combining both gives high precision with few false pages. The advantage over a static threshold is that burn rate accounts for both severity and duration, so a brief spike does not page anyone but a sustained smaller error rate does. Multi-window multi-burn-rate alerting is the standard SRE pattern.

Question 13

What is CloudWatch Application Signals?

Accepted Answer

Application Signals automatically instruments applications on EKS, ECS, Lambda and EC2 to produce standard service-level metrics — latency, error rate, request volume — plus a service map and built-in SLO tracking with burn-rate alerting, without you defining custom metrics. It closes the gap between AWS-native monitoring and the SLO-based practice teams previously had to build themselves with Prometheus and custom dashboards. Naming it, and that it uses OpenTelemetry underneath, shows current knowledge.

Question 14

What makes a good alert?

Accepted Answer

It is actionable, urgent and user-visible: it fires only when a human must act now, about something affecting users, with a runbook link and enough context to start. Anything informative but not urgent belongs on a dashboard or in a ticket queue, not on a pager. The anti-pattern to name is alerting on causes rather than symptoms — paging on high CPU when users are unaffected. Symptom-based alerting on SLIs is what keeps the pager trusted, and alert fatigue is why real incidents get missed.

Question 15

Why alert on percentiles rather than averages?

Accepted Answer

An average hides the tail: if 99% of requests take 50 milliseconds and 1% take 10 seconds, the average looks fine while one user in a hundred has a terrible experience. Percentiles show the distribution, and the tail is where user-visible problems live. The amplification point is worth adding: in a system where one page makes twenty backend calls, a p99 backend latency affects a large share of pages. That fan-out is why tail latency matters more than it first appears.

Question 16

What are the four golden signals?

Accepted Answer

Latency, traffic, errors and saturation. Latency is how long requests take, traffic is demand, errors is the rate of failed requests, and saturation is how full the most constrained resource is. Monitoring these four covers most user-visible failure modes. The detail worth adding is to measure successful and failed request latency separately, because fast failures can make average latency look excellent during an outage. That distinction is a classic SRE point interviewers listen for.

Question 17

What is CloudWatch Synthetics?

Accepted Answer

Synthetics runs canaries — scripted browser or API checks — on a schedule from AWS locations, verifying availability, latency, page load, broken links and complete user journeys, with screenshots and HAR files on failure. It provides black-box monitoring independent of your application's own instrumentation, which is what catches DNS failures, expired certificates and load balancer misconfiguration. Those are the outages where internal metrics look perfectly healthy.

Question 18

What is the difference between white-box and black-box monitoring?

Accepted Answer

White-box uses internal signals the system exposes — queue depth, garbage collection time, connection pool utilisation — good for diagnosis and prediction. Black-box probes from outside as a user would, good for detecting that something is genuinely broken from the user's perspective. The rule is to page on black-box and symptom signals and use white-box for diagnosis and capacity planning. Paging on white-box internals is how teams end up with alerts firing while users are perfectly happy.

Question 19

What is Amazon Managed Service for Prometheus and Managed Grafana?

Accepted Answer

Managed Prometheus is a managed, scalable Prometheus-compatible metric store with PromQL and long retention, removing the burden of running and sharding Prometheus yourself. Managed Grafana provides hosted dashboards with authentication via IAM Identity Center and connectors to many data sources. The reason teams use them over CloudWatch is the Kubernetes ecosystem: existing exporters, dashboards and PromQL knowledge transfer directly. Combining them with CloudWatch as a Grafana data source gives one pane over both.

Question 20

How do you correlate logs, metrics and traces for one request?

Accepted Answer

Propagate a trace identifier through every service and include it in structured log entries, so logs can be filtered by trace. X-Ray links traces to logs when the trace ID is present, and metrics carry the same resource dimensions. OpenTelemetry handles the propagation and exports all three consistently. Without propagation the three sources stay unlinked, and correlating by timestamp alone is unreliable under load — which is exactly when you need it. Logging a request ID and tenant on every line is the design decision that makes investigation possible at all.

Question 21

What is CloudWatch Contributor Insights?

Accepted Answer

Contributor Insights analyses log or metric data to identify top contributors — the busiest callers, the most-throttled keys, the noisiest sources — producing time series of the top N contributors so you can see who is driving a spike. It answers the question aggregate metrics cannot: which client, which key, which endpoint is responsible. Naming its use for DynamoDB hot keys or for identifying an abusive IP in load balancer logs makes it concrete.

Question 22

What is AWS Health Dashboard and why does it matter?

Accepted Answer

The AWS Health Dashboard reports service events and account-specific issues — scheduled instance retirements, certificate expiries, service degradations affecting your resources. Health events can be routed through EventBridge to trigger automation or notify a channel. Automating on health events is the mature use: an instance retirement notice can trigger a workflow that drains and replaces the instance before the deadline, rather than someone reading an email. That is the difference between reactive and operationalised.

Question 23

How do you monitor a Lambda function properly?

Accepted Answer

Alert on error rate, throttles, duration approaching the timeout, and concurrent executions approaching the limit. For poll-based sources, alert on iterator age. Use structured logs with request context, X-Ray for tracing, and Lambda Insights for enhanced runtime metrics such as memory utilisation. Duration approaching timeout is the leading indicator that warns you before invocations start failing. Memory utilisation matters because an OOM appears as an opaque error, and iterator age is the only signal that a stream consumer is permanently falling behind.

Question 24

How do you monitor a data pipeline as opposed to a request-serving service?

Accepted Answer

The user-facing property is freshness and completeness rather than latency and error rate. Monitor data freshness — how old the newest processed record is — record counts against expected volume, dead-letter and rejected-record counts, and reconciliation between source and destination totals. Reconciliation is what catches silent data loss, which is the characteristic pipeline failure: everything reports success while records are quietly dropped. Alerting on absence, so a job that never starts is noticed, is the other essential control.

Question 25

What is a blameless postmortem and why does it matter?

Accepted Answer

A postmortem focused on the systemic causes of an incident — what made the failure possible and why it was not caught — rather than on who made a mistake. It matters because blame suppresses information: people who fear consequences stop reporting near-misses, and you lose the data that prevents the next outage. The practice is asking "what about the system allowed this action to cause an outage" instead of "why did you do that". Action items with owners and deadlines are the other half, because a postmortem without follow-through is a document nobody reads twice.

Question 26

What is MTTR and what reduces it?

Accepted Answer

Mean time to recovery — from incident start to service restoration. It is reduced by fast, accurate detection; good diagnosis through dashboards, tracing, structured logs and runbooks; and fast, safe remediation via one-command rollback, traffic shifting, feature flags and tested failover. The most under-invested lever is fast rollback: if reverting takes forty minutes, every incident has a forty-minute floor. Investing in progressive delivery often reduces MTTR more than better monitoring does.

Question 27

How do you run an incident?

Accepted Answer

Declare it with a severity that determines who is involved; appoint an incident commander who coordinates rather than debugs; assign a communications lead and an operations lead; mitigate first and diagnose afterwards; keep a timeline; and follow with a blameless postmortem with tracked actions. Separating the commander from the people debugging is the structural point — the common failure is everyone investigating and nobody coordinating or communicating. Naming the roles rather than describing the technical steps is what an SRE interviewer is assessing.

Question 28

What is AWS Systems Manager Incident Manager?

Accepted Answer

Incident Manager provides on-call schedules, escalation plans, automated engagement of responders, incident records with a timeline, runbook automation, and post-incident analysis, integrated with CloudWatch alarms and EventBridge so an alarm can open an incident automatically. The automatic runbook execution on incident creation is the useful part: the first diagnostic or mitigation steps run before a human is even engaged. That directly reduces MTTR for known failure modes.

Question 29

What would you put on a service dashboard?

Accepted Answer

The SLIs at the top — success rate and latency percentiles with the SLO target and remaining error budget — then traffic volume, error breakdown by type, saturation of the constrained resource, dependency health and latency, and deployment markers so you can see which change coincided with a change in the graphs. Deployment markers are the underrated element: most incidents follow a change, and a dashboard overlaying deploy times answers "what happened" in seconds. Ordering by user impact rather than by infrastructure layer is the other principle.

Question 30

What is toil and how do you reduce it?

Accepted Answer

Toil is manual, repetitive, automatable work that scales with the size of the service and produces no lasting value — restarting a stuck process, approving a routine change, copying data by hand. Reduce it by automating the task, removing the need for it, or changing the system so it self-heals. The SRE convention worth citing is capping toil at roughly half of an engineer's time, with the rest on engineering that reduces future toil. Being able to name that boundary shows familiarity with the discipline rather than the job title.

Question 31

How do you handle alert fatigue?

Accepted Answer

Audit every alert against whether it is actionable and user-affecting; delete or downgrade those that are not; move cause-based alerts to dashboards; tune thresholds using burn-rate windows; use composite alarms to suppress downstream noise; and track pages per shift, treating a noisy alert as a defect with an owner. The cultural point is that tolerating noisy alerts is itself the risk: once responders assume a page is probably noise, the real incident is answered slowly. Framing alert quality as a reliability control is the mature position.

Question 32

What is chaos engineering and what does AWS provide?

Accepted Answer

Chaos engineering deliberately injects failure to verify resilience mechanisms work before a real incident tests them. AWS Fault Injection Service runs controlled experiments — terminating instances, injecting latency, failing an AZ, throttling API calls — with defined stop conditions tied to CloudWatch alarms. The stop condition is what makes it an experiment rather than an outage: the experiment aborts automatically if a guardrail alarm fires. Starting with a written hypothesis, in a non-production environment, is the discipline that keeps it useful.

Question 33

How do you monitor cost as a reliability concern?

Accepted Answer

Enable Cost Anomaly Detection for machine-learning-based alerting on unusual spend, set budgets with threshold alerts, build Cost Explorer or Athena dashboards over the Cost and Usage Report, and treat a runaway cost event — a recursive Lambda, an unbounded autoscale — as an incident with the same urgency as an outage. Framing cost as an incident class is the insight: a misconfigured pipeline can spend a quarter's budget in a weekend, and monthly budget alerts arrive far too late. Daily anomaly detection is the control that actually catches it.

Question 34

How would you investigate a latency regression after a deployment?

Accepted Answer

Compare latency percentiles before and after, split by version or deployment. Use X-Ray to compare trace span breakdowns between versions and find which operation grew. Check for new error groups, examine dependency metrics in case the cause is downstream, and check whether a database migration accompanied the release. Splitting metrics by version is the key technique and requires that dimension to exist, which means tagging metrics and traces with the deployment version. Without it, before-and-after comparison is guesswork.

Question 35

What is the difference between monitoring and observability?

Accepted Answer

Monitoring watches known signals for known failure modes — you decide in advance what to measure and alert on. Observability is being able to answer questions you did not anticipate, from data the system already emits, without shipping new code. High-cardinality structured events and traces enable it; a fixed set of pre-aggregated metrics does not. The practical consequence is designing for unknown-unknowns: log request ID, tenant, region and version on every event so you can slice by any of them later. A system where every investigation needs a new deployment to add a log line has monitoring but not observability.

Question 36

What is a runbook and what should it contain?

Accepted Answer

A runbook is the documented response for a specific alert: what it means, the user impact it implies, first diagnostic steps with exact queries or commands, known causes and remedies, escalation contacts, and how to verify recovery. It should be linked directly from the alert. The quality test is whether someone unfamiliar with the service can follow it at 3am. Runbooks that say "investigate the issue" are decorative; ones with copy-pasteable queries and decision points are what actually reduce MTTR.

Question 37

How do you set an SLO for a new service?

Accepted Answer

Start from what users need rather than what the system currently does, but validate against historical data so the target is achievable. Pick SLIs measured at the user boundary, choose a rolling window, define the error budget, and agree in advance what happens when it is exhausted. The mistake to avoid is setting 99.99% because it sounds good: each additional nine costs disproportionately more, and an unachievable target trains everyone to ignore the budget. A target you will actually enforce is worth more than a strict one you will not.

Question 38

What CloudWatch metrics would you alert on for a typical web application?

Accepted Answer

Load balancer 5XX rate and target response time percentiles; target group unhealthy host count; Auto Scaling group in-service capacity against desired; database CPU, connections and replica lag; queue age of oldest message; and the application's own SLO burn rate. Alert on symptoms, keep resource metrics for diagnosis. Unhealthy host count is the specific one that catches partial failures where the service still responds but with reduced capacity. Distinguishing load balancer 5XX from target 5XX is the diagnostic detail — the first can mean no healthy targets at all.

Question 39

Design the observability strategy for a platform of 40 microservices on AWS.

Accepted Answer

Standardise instrumentation with the AWS Distro for OpenTelemetry, exporting metrics to Managed Prometheus or CloudWatch, traces to X-Ray and structured logs to CloudWatch with trace correlation. Define SLIs and SLOs per user-facing service with error budgets and multi-window burn-rate alerting, paging only on symptoms. Provide a templated per-service dashboard including deploy markers, and a service map. Centralise CloudTrail, Config and security findings in a logging account, exclude health-check noise and set retention on every log group. Maintain runbooks linked from every alert, a severity model with defined incident roles, Incident Manager for on-call, and blameless postmortems with tracked actions. The closing scenario. The senior markers are standardising instrumentation so every service is observable the same way, paging only on user-visible symptoms, and treating the human process — severity, roles, postmortems — as part of the strategy rather than an afterthought.

AWS CloudWatch, X-Ray & Observability Interview Questions and Answers

How to use this set