On September 22nd, 2026, AWS launched Amazon CloudWatch Omni, a new observability experience that lives outside of the AWS console. It gives you a standalone web application for observability and an IDE extension for developing AI agents. The web experience uses CloudWatch telemetry, while the IDE extension can inspect local runs or connect to a cloud space. CloudWatch Omni can even use AgentCore Evaluations to score agent behavior.

But we already had CloudWatch, and it got a lot of love on the AWS Console in the past few years. Plus, AgentCore Observability already let us inspect agents. So why another experience?

The strongest reason is making investigation and iteration easier to perform together. You can take an agent's production interaction into a local test, inspect its execution while changing it, and compare the new behavior with the original failure. For applications, you can pursue questions with an assistant that sees the current view and retains the conversation alongside the evidence you opened.

Let's take a deeper look at CloudWatch Omni's features and how it can change your experience.

This article is sponsored by All Day AI

The demo worked. Production added runaway tool calls, weak permissions, missing evals, and an unexpected bill. All Day AI brings together practitioners who have worked through those failures inside real systems.

Agenda highlights:

- See what broke when agent swarms reached production.  

- Learn why prompt injection needs architectural controls.  

- Study how engineers design AI systems that fail safely.

Sessions also cover edge AI, trustworthy agents, governance workflows, and the path from prototype to a supportable product. Four tracks let developers, platform teams, security leaders, and product teams follow the material that matters to them.

October 22, 2026, from 6:00 a.m. ET. 122 speakers. No vendor pitches. Free. Online.

To configure CloudWatch Omni, just head over to the CloudWatch console and click on the banner:

That will take you to the setup page. You can set it up at the Organization level from your Organization's root account, or at the Account level for that specific account. The easiest way to log in to test it is by logging in to the AWS Console and signing in from there, but you can also configure login via IAM Identity Center, which in the long run is probably better.

You'll get a bunch of configurations, which you can just leave as default. Then click the orange button that says Create space. It's on the right, rather than at the bottom.

Then you wait, and when it's done you click Sign in. And you're taken to CloudWatch Omni.

It doesn't have your CloudWatch data by default, which is honestly a bit weird, but I'll forgive it since it's technically in public beta, and also sort of makes sense with spaces (you could have more than one app per account, and want to only see one app's data in a given space). Instead, it'll let you ingest your data from either apps or agents, or instrument your agents. This uses OpenTelemetry for apps and agents, and for agents you can also use OpenInference. This is a one-time setup per app, you just need to re-instrument it and then it automatically gets all of the data.

Omni is built on CloudWatch, rather than replacing its collection and storage system. Your existing console pages, APIs, alarms, and dashboards keep working (as long as you re-instrument everything). You still manage things such as log groups and retention through CloudWatch.

In the web application, you work in a space: a telemetry and access boundary associated with an AWS account and Region. The space brings application and agent telemetry into the same working environment. The IDE extension gives you the local development experience.

A trace records one request or agent run, as usual in observability. Its spans record operations inside that run, such as a service request, model invocation, or tool call. Seeing those operations lets you inspect what happened between receiving an input and returning an answer. This is kind of the same thing you could see in X-Ray, but in a much more comfortable view.

For agents, you can also see the results of evaluations, from AgentCore Evaluations. This is part testing and part monitoring I guess, but when dealing with agents it's a very important view to have. You could already see this in the AgentCore Evaluations console, so what CloudWatch Omni brings here is condensing all the information in a single place, instead of the dashboards sprawl that the CloudWatch console and the AgentCore experience have become.

The IDE extension serves to show you all of this information directly where you're doing your development work. No more switching to the AWS Console to find that log with the error you're trying to fix, you can check that information directly from the IDE. For agents it also provides full-agent local testing, experiments, and prompt management. You can inspect traces and work with evaluation datasets, effectively bringing all of that to the IDE as well (which in my opinion is less monitoring and more like an AgentCore Evaluations IDE extensions, but I love it).

AWS's Application Signals example follows an application whose availability drops during traffic spikes. The developer opens the unhealthy service, selects a point on its availability chart, and inspects a correlated fault trace. Following the request reveals errors from a DynamoDB operation. This shows you that CloudWatch could already take you from an application symptom to evidence about a failing dependency, and I wanted to include it here because it shows you the path you had to take with CloudWatch.

CloudWatch also already provided help beyond that first diagnosis. Logs Insights can generate and update queries from English. Configured CloudWatch investigations can gather further telemetry, propose hypotheses, retain findings and comments, and let colleagues continue the investigation.

CloudWatch Omni can also give you a route to a diagnostic. From the application map, you select the service and relevant time range, then use a chart point to search for traces.

Then you inspect the request and open its matching logs or service metrics, using the service view.

Following the Application Signals example, let's consider a follow-up question: do the DynamoDB failures also occur outside the traffic spikes?

With CloudWatch Omni, you can ask from the service or finding you're looking at. The Omni agent can use the on-screen context, write and run queries, and build views from the results. It shows the query it executed, so the answer has an inspectable route back to the data. Technically your Claude Code or Codex can do the same, but what CloudWatch Omni brings here is the ability to do everything in one place. It's not just a chat, and it's not just a graph or dashboard. It has all of those things in a single place, letting you visualize what's wrong and come up with better questions for the chat.

For example, the query or question itself can reveal a mistake in the investigation. If it only covers the peak-traffic interval, it hasn't answered whether failures also happen outside that interval. You can correct the question and continue, but to even know that it's wrong you need to look at the information, to trigger that “senior engineer instinct”.

All of this information stays together in a thread, which contains the conversation itself with all the messages you exchanged with the Omni agent (including the queries it ran and the evidence behind its answers), and all the canvases and tabs you opened while investigating (charts, log tables, maps, trace views). All of this is bundled as an Investigation, which sort of belongs to a thread (or rather, a thread is about an investigation), but can also be queried from other threads.

An investigation often involves several questions: where the failure occurred, whether it is recurring, which requests are affected, and what evidence rules out another explanation. Retaining the views helps you recover that reasoning without reopening every chart and remembering which query produced it. If that reconstruction is part of your current routine (and you know it is!), CloudWatch Omni significantly lowers that burden, and might help you avoid keeping 10 browser tabs open.

AWS's Omni walkthrough uses as an example a customer-support agent that answers a cooking question. The agent's response can be fluent, helpful, and technically successful, but the agent is supposed to handle support requests, not provide recipes.

A low error rate won't identify that problem. A general helpfulness check need not identify it either: the answer is actually helpful to someone who wants a recipe. You need a check against this agent's intended scope. You need evaluations.

AgentCore already gives you the necessary tools for investigating and measuring that behavior. Its observability experience exposes conversations and model/tool activity, and its evaluations supply quality checks. Its SDK offers an on-demand dataset runner that invokes an agent, collects spans, and calls the evaluation service. Dataset evaluation is currently in public preview, but you can use it. AWS also has AgentCore quality gates in GitHub Actions, which I highly recommend.

Basically, AgentCore already lets you do all the agent evaluations that you want. CloudWatch Omni's contribution here is how you move through that work while developing the fix. Again, developer experience.

You're seeing your agent fail, and you want to fix that and make sure it doesn't happen again (at least for this particular type of failure). So you find the interaction, read the response, and inspect the recorded calls. That tells you what happened in this run. To check a change, you also need the input and a definition of what should happen instead, and to bundle up all of that in a test case.

In CloudWatch Omni, you can turn a trace into a test case. The recorded input becomes the test input, and the observed response and tool calls can seed expectations for you to review. Again, nothing you couldn't do before with some good old copy-paste, but why copy-paste if you can do it much easier?

For the cooking request, the intended behavior would be to decline the off-topic question and return to support topics. You don't need to prescribe one exact sentence to express that requirement, but you do need to state the behavior you're trying to preserve.

Omni can also convert a session into a multi-turn case. Choose one case per session to retain the conversation together, and you can test the whole thing with the context needed to reproduce or interpret the failure, instead of using only the final question.

Once you have your test cases, you can publish a dataset version and download it into the IDE. A published version is a fixed set of cases used by a comparison. Each upload or download creates a new dataset at the destination.

This means that the interaction you took from that production trace now becomes something you can use repeatedly while making the fix. You aren't relying on a description of the complaint or rebuilding the conversation from memory for each attempt, you're actually testing for that specific failure.

If your existing process already preserves production examples as runnable cases, CloudWatch Omni adds convenient actions around it. If you currently stop at reading a trace, transitioning to retaining that as a test case is a much more substantial improvement, and one I highly recommend.

For a given agent that gives off-topic answers (such as the example above), a reasonable hypothesis is that a narrower system instruction will stop the off-topic answers. You can make that change, but now you need to test that change and see what the agent actually does.

The AgentCore SDK's on-demand dataset runner could already invoke a local agent through a function or HTTP request, without having to deploy everything to AgentCore Runtime first. Evaluation still uses the AgentCore service. Its default span collector reads CloudWatch, while a custom collector can support another path. You need to configure that execution and collection path and arrange how you'll inspect the results, likely through the AWS Console.

The Omni extension supplies that information in your local environment: it starts a collector, sets the telemetry environment, connects to the agent server, and exposes the resulting traces. You can inspect a local run without first sending its traces through CloudWatch.

That helps with the repeated work of debugging. Edit the instruction, run a fixed input, inspect the answer and recorded calls, and decide what to try next. The evidence remains close to the change you're making, all in your IDE. Having a trace viewer in the editor is useful because the collection and execution path is supplied with it, and you can see everything in the same place.

You also need to choose what you're testing. Omni's prompt playground distinguishes two approaches:

Prompt-only testing can quickly examine the wording hypothesis. Full-agent testing checks the actual execution path. A model refusing the cooking question in isolation doesn't tell you whether the changed support agent still looks up an order and handles a legitimate return.

Keep in mind that local trace collection doesn't mean offline execution. The agent calls its model provider, and Omni's evaluators use AgentCore Evaluations. Model access and scoring are separate from where the trace files are stored. CloudWatch Omni brings the observability experience to your local environment, but it doesn't change where inference and evaluations run.

For agentic AI development, I'd start with the extension in VS Code, Kiro, or Cursor. Open the project, verify instrumentation, configure the agent server, send a request, and inspect a useful trace. Follow the onboarding guide that pops up when you open CloudWatch Omni and you'll have your agents instrumented in a few minutes.

You don't need an Omni cloud space for local trace collection, but you do need credentials for the model your agent calls. I'd suggest starting with an existing project and an interaction you understand, so you can connect the recorded execution to its behavior, and figure out how CloudWatch Omni works and how you can work with it.

For applications, follow the CloudWatch Omni onboarding guide and try to do in Omni anything you'd do in the CloudWatch Console. I believe you'll find most things easier. For production environments, use an existing space or have an administrator set one up.

CloudWatch Omni’s web interface and IDE extension are free, but model inference, evaluation, and cloud telemetry might cost you 💰. Omni pricing includes ingestion, storage, and analysis charges, with agent evaluation billed at AgentCore rates.

CloudWatch Omni's appeal is the developer experience. It doesn't introduce new capabilities, it just redesigns how you consume the existing capabilities into an experience that better matches what you need.

For agents, it gives you a direct path from a real interaction to a retained test, local execution evidence, and an inspectable comparison. That is especially useful when you would otherwise need to manually reconstruct examples. For applications, the benefit is a workspace that keeps exploratory questions and working evidence together, and closer to where you'll use that information to fix the issues.

CloudWatch and AgentCore already provided many of these outcomes. Omni's value is making them easier to use together during the actual work of investigating and changing a system. Overall, it's an improvement on developer experience, and definitely something worth writing about.