Troubleshooting Copilot Studio Agents - Field Guide

1. Table of contents#

2. The overall workflow#

The ordering isn’t arbitrary. Each stage is cheaper than the one after it, and each one narrows the search space for the next. Jumping straight to KQL is the single most common time sink on these incidents - you end up staring at a dependencies table with no idea which conversation you’re looking for.

3. 1. Introduction#

4. Why bother with a process#

Copilot Studio agents fail in a way that traditional apps don’t. There’s no stack trace pointing at line 47. The agent “just answered wrong”, and the reason could be a trigger phrase, a badly worded instruction, a stale SharePoint index, an expired connection reference, a 429 from the model, or a downstream API that returned 200 OK with an empty body. All of those look identical to the user.

Without a process, people investigate whatever they’re most comfortable with. The .NET dev goes straight to exceptions. The Power Platform person re-publishes the agent and hopes. Both sometimes work, which is exactly what makes the habit hard to break.

The process below is ordered by cost of investigation, not by likelihood. That’s deliberate: a 30-second check that rules out 20% of causes beats a 30-minute KQL session that rules out 40%.

5. Start with the boring checks#

Before anything else, ask these four questions. In my experience they close a genuinely uncomfortable share of tickets:

1. Is the agent published? Changes in the authoring canvas don’t reach users until you publish. The test pane uses the unpublished version, so “it works in test” is expected behaviour, not evidence.
2. Which environment? DEV, ACC and PRO agents share a name and nothing else. Confirm the environment ID, not the display name - display names get reused.
3. When did it last work? If the answer is “this morning”, you’re looking for a change: a publish, a solution import, a connection expiry, a knowledge source re-index, a tenant policy update.
4. Is it one user or everyone? One user almost always means permissions, licence, or channel. Everyone means configuration or a dependency.

6. Common categories of failure#

7. Prerequisites#

Line these up before an incident, not during one. Chasing a Reader role at 22:00 is a bad time.

• Copilot Studio: maker access to the affected environment; ability to see the agent’s Settings > Advanced and Analytics/Monitor.
• Transcripts: the Bot Transcript Viewer security role. The Environment Maker role does not include it - an admin has to grant it explicitly, usually during agent sharing.
• Application Insights: at minimum Reader on the resource, plus the connection string / resource name written down somewhere findable.
• Power Platform admin centre: read access, for environment settings, DLP policies and (if you’re using it) the environment-level telemetry export config.
• Azure portal: read access to whatever the agent depends on - Azure OpenAI, APIM, Function Apps, Logic Apps.
• Dataverse: access to the environment if you want to query the conversationtranscript table directly.
• Knowledge of which telemetry mode your tenant uses - see section 5.1. This matters more than anything else in this guide and is the thing most teams get wrong.

Screenshot: the agent’s Settings > Advanced page showing the Application Insights section with the connection string field (redacted) and the four logging toggles.

8. 2. Reproducing the issue#

The goal of this stage is not to fix anything. It’s to turn “the bot is broken” into a set of identifiers you can search on. If you skip this, everything downstream becomes guesswork, because you’ll be scanning telemetry with no anchor.

9. What to capture#

Get all of this before you touch a setting. Every field here maps to something you’ll filter on later.

Getting the Conversation ID#

This is the field that turns a two-hour investigation into a ten-minute one. Three ways to get it:

• From the user’s session: in the test pane, or in a channel that supports it, type /debug conversationid. The agent returns the ID for the current conversation.
• From the test pane: use Download snapshot in the test panel toolbar. You get a ZIP with the conversation and its diagnostics - this is the single best artefact to attach to a ticket.
• From error cards: many Copilot Studio error responses embed Error code: ... Conversation ID: ... directly in the message the user sees. Ask users to screenshot the whole bubble, not just the first line.

Tip: add a support topic to your agents that echoes the conversation ID on request (“Something went wrong? Say support code”). Costs you ten minutes to build, saves hours per incident. Put it behind a phrase that users won’t hit accidentally.

If you can’t reproduce it#

That’s common and it’s fine. Change tactics rather than grinding:

• Widen the time window and search telemetry for any failure in that agent around the reported time.
• Check whether the reporter is in a different Entra group, has a different licence, or is on a different channel.
• Ask whether they were on VPN / mobile / an unmanaged device - Direct Line token issues cluster there.
• Look for the failure signature (see Appendix A) rather than the specific conversation.

Intermittent failures are usually one of: model throttling (429), a dependency timeout, token expiry mid-conversation, or an inconsistently indexed knowledge source. All four are visible in telemetry even when you can’t reproduce them.

10. Reusable intake template#

Copy this into a Confluence template and attach it to every agent incident.

## Incident intake

Reported by:            
Date/time reported:     
Time of failure (UTC):  
Environment (name/ID):  
Agent (name/ID):        
Published version:      
Channel:                
Affected user (UPN):    
Authenticated? (Y/N):   
Scope: single user / group / everyone
Conversation ID:        
Session ID:             

Exact prompt:
>

Expected behaviour:
>

Actual behaviour:
>

First occurrence:       
Last known good:        
Recent changes (publish / solution import / policy / connection):
>

Attachments: screenshots, test-pane snapshot ZIP, transcript CSV

11. Reproduction checklist#

• I know the environment ID, not just the name
• I know whether the user hit the published agent or the test pane
• I have a Conversation ID (or a five-minute UTC window and a user)
• I have the exact prompt text, copy-pasted
• I know if this affects one user or many
• I know what changed recently, or have confirmed nothing did
• Expected behaviour is written down and someone agrees with it

12. 3. Troubleshooting inside Copilot Studio#

Everything in this section is free - no KQL, no Azure portal, no waiting for telemetry ingestion. Work through it before you open Application Insights. Realistically, a large share of incidents die here.

13. 3.1 Agent validation#

Start with the dullest possible questions.

Is it published, and is the published version the one you think it is? Open Publish and check the timestamp of the last successful publish. Compare it to when your change went in and when the user hit the failure. A publish that ran after the failure doesn’t help you; a publish that failed silently is worse.

Are you in the right environment? Check the environment picker and the URL. If your DEV and PRO agents have the same display name - and they usually do - this is a real trap. Verify the environment GUID.

Is the agent actually available on the channel the user used? An agent can be published and still not enabled for Teams, or published to Teams but not submitted to the org catalogue. Check Channels and confirm the specific channel shows as configured.

Is the deployment intact? If the agent arrives via solution import (ALM pipeline), check the solution imported cleanly in the target environment. Partial imports are notorious: the agent lands, but connection references or environment variables don’t get set, and everything downstream fails with authentication errors that look like a permissions problem.

Environment health Environments in an admin operation (backup, restore, copy, capacity block) behave strangely. Check the environment state in the Power Platform admin centre before you spend an hour on the agent.

Screenshot: Publish page showing publish history with timestamps and status. Screenshot: Channels page with Teams enabled and its availability state.

Common mistake: testing your fix in the test pane, seeing it work, and closing the ticket. The test pane runs the draft. Publish, then test through the same channel the user used, ideally as a user without your permissions.

14. 3.2 Topics and routing#

Routing problems produce two symptoms: nothing fires (you get the fallback / “I’m not sure how to help with that”) or the wrong thing fires.

Trigger phrases#

• Open the topic and read the trigger phrases out loud. Do they sound like things your users say, or like things a product owner wrote in a workshop? “Submit annual leave request” versus “book holiday”.
• You want 5-10 varied phrases per topic, not 30 near-duplicates. Near-duplicates make topics compete with each other and produce unpredictable routing.
• Watch for phrases shared across topics. If two topics both have “help with my account”, the winner is effectively arbitrary from a user’s perspective.

Trigger priority and conditions#

• Where topics overlap, priority decides. Check the ordering explicitly rather than assuming.
• If the topic has a trigger condition (a variable check, an authentication requirement), verify the condition can actually be true for the affected user. A condition that reads Global.IsManager = true will silently never fire if the variable is never populated for that channel.
• System topics (Conversation Start, Escalate, Fallback, End of Conversation, Multiple Topics Matched) can pre-empt custom ones. If Multiple Topics Matched is firing, that’s your answer: your trigger phrases overlap too much.

Generative orchestration vs. classic routing#

This distinction matters a lot and is easy to miss:

• With classic topic triggering, routing is driven by trigger phrases and priority.
• With generative orchestration, the model picks topics and tools based on their descriptions and the agent’s instructions. Trigger phrases matter much less; descriptions matter enormously.

If your agent uses generative orchestration and a topic never fires, the fix is almost never “add more trigger phrases”. It’s “write a description that tells the model when to use this”. A description like “Handles leave” is far weaker than “Use when the user wants to check their remaining annual leave balance, request time off, or cancel an existing request.”

Screenshot: topic trigger configuration showing trigger phrases and the priority setting. Screenshot: a tool/topic description field in generative orchestration mode.

15. 3.3 Generative AI configuration#

Open Settings > Generative AI (and the agent’s Instructions).

Instructions Read them as if you were the model and knew nothing. Look for:

• Contradictions - “always answer from knowledge” plus “never speculate” plus a knowledge source that doesn’t cover the question means the agent has no legal move.
• Vague scope statements. “Be helpful and professional” doesn’t constrain anything.
• Instructions that reference variables or context the model doesn’t have at that point in the conversation.
• Sheer length. Very long instruction blocks dilute everything in them. If instructions have grown past a couple of screens, that’s usually where the inconsistency lives.

Grounding and the “answer from knowledge only” setting If the agent is set to answer only from knowledge sources and the sources don’t contain the answer, the correct behaviour is a refusal. Users report this as “the bot doesn’t know anything”. That’s a content problem, not a bug. Conversely, if general model knowledge is allowed, you’ll get plausible answers about things you never documented - reported as hallucination.

Model selection Agents can run on different models, and the model can change under you when your tenant gets updated defaults. If behaviour shifted with no change on your side, check which model the agent is using - this is one of the few genuinely invisible causes of “it changed overnight”.

Content moderation / safety Moderation levels trade off answer coverage against risk. A high setting will block borderline content and return a generic message. If users report the agent “refusing normal questions”, check moderation before you rewrite instructions. You can confirm it in telemetry - see the ContentFiltered query in Appendix B.

Screenshot: Generative AI settings showing knowledge grounding options and content moderation level.

16. 3.4 Knowledge sources#

Knowledge problems are the most common cause of “the answer is wrong” reports, and they’re rarely about the model.

Work through this order:

1. Is the source attached and enabled? Obvious, but check - sources get disabled during testing and never re-enabled.
2. Has it finished indexing? Newly added sources, and newly added files in existing sources, take time. Large SharePoint sites take considerably longer than people expect. If content was published this morning, indexing is your prime suspect.
3. Permissions. This is the big one for SharePoint and Graph-based sources. Copilot Studio can honour the user’s permissions - meaning the agent legitimately can’t see a document the maker can see. Always test with an affected user’s permission set, not yours.
4. Is the content actually in scope? A SharePoint knowledge source scoped to a site won’t reach a subsite or a different library. Check the exact URL.
5. Is the content findable? If SharePoint search can’t find the document, neither can the agent. Search for it in SharePoint first - a 20-second test that saves a lot of arguing.
6. File type and size. Scanned PDFs with no text layer, huge files, and password-protected documents contribute nothing.

Testing the retrieval, not the answer Ask a question whose answer appears verbatim in exactly one document, phrased as it appears in the document. If that fails, retrieval is broken. If it succeeds but paraphrased questions fail, the content is retrievable but poorly chunked or poorly titled - a content-authoring problem.

Watch out: when SharePoint is the knowledge source, conversation transcripts include the question and the retrieved source content (search_results) but the agent’s answer is written out as REDACTED. Don’t waste time hunting for the response text in the transcript - go to the telemetry or reproduce it live.

Screenshot: Knowledge page showing sources with their status/indexing state.

17. 3.5 Actions and tools#

Actions (Power Automate flows, connectors, custom connectors, prompts, MCP tools) fail in a small number of predictable ways.

Inputs

• Are required inputs actually populated at the point of the call? An empty required input often produces a generic failure rather than a validation message.
• With generative orchestration, the model fills the inputs from conversation context. If a tool receives rubbish, the fix is usually the input’s description - make it explicit about format (“Employee ID, 6 digits, no prefix”), not just its name.
• Type mismatches: dates as strings, numbers as text, empty arrays vs null.

Outputs

• Check the output schema matches what the underlying service returns. A flow that returns a different shape than declared will bind nothing and the agent will improvise around the gap.
• Nulls: a tool that returns 200 OK with no data is successful as far as telemetry is concerned. This is a classic invisible failure.

Authentication and connection references

• Confirm which auth model the tool uses: the maker’s connection, the end user’s connection, or a service principal.
• Connection references that came in via solution import frequently point at nothing or at the deployer’s connection. Open the connection reference and confirm it’s bound.
• Connections expire. If a tool worked for weeks and now doesn’t, and nothing changed, check the connection status first.
• If the tool requires the user to authenticate, that consent flow has to be possible in the channel they’re using. Some embedded channels block the popup.

Error handling Does the topic handle a failed action, or does it charge on and produce a confusing message? Add explicit error branches for anything user-facing - it makes future incidents diagnosable from the transcript alone.

Screenshot: an action node showing input/output mapping. Screenshot: the connection reference configuration for a custom connector.

18. 3.6 Connected agents and sub-agents#

Multi-agent setups add a layer where things get lost.

• Connectivity: confirm the child agent exists, is published, and is published in the environment the parent expects.
• Authentication: the identity flowing to the child agent isn’t always the end user. Confirm what the child sees and whether that identity has the permissions it needs.
• Descriptions: as with tools, the parent decides when to call a child agent based on its description. Weak description, no invocation.
• Timeouts: a child agent that calls its own tools can exceed the parent’s patience. Long chains produce truncated or empty responses at the top level.
• Response validation: does the parent do anything sensible with an empty or error response from the child? Usually not, by default.

In telemetry, sub-agent activity is traceable: the child inherits the parent’s conversation ID with a _<subConversationId> suffix, so you can reconstruct the whole tree. See B.7.

19. Copilot Studio validation checklist#

• Agent published, and the publish timestamp predates the failure
• Correct environment confirmed by ID
• Channel the user used is enabled and healthy
• Solution import (if any) completed without errors
• Trigger phrases / tool descriptions reviewed for the failing intent
• Trigger priority and conditions checked
• Instructions read end-to-end for contradictions
• Grounding mode and content moderation level noted
• Knowledge sources enabled, indexed, in scope, and permission-tested as an affected user
• Tool inputs/outputs and connection references verified
• Connected agents published and reachable
• Retested through the real channel after publishing

20. 4. Analysing the conversation transcript#

If section 3 didn’t find it, stop looking at configuration and start looking at what the agent actually did. The transcript is the cheapest source of truth you have: it shows the decisions, not the intentions.

21. Where to get transcripts#

Know the gaps before you rely on them: transcripts aren’t written for Dataverse-for-Teams environments, Dataverse developer environments, or Microsoft 365 Copilot agents. SharePoint-grounded answers come through REDACTED. The Copilot Studio CSV truncates each agent response at 512 characters - for full responses, go to Dataverse.

Screenshot: Analytics page with the Download Sessions pane open. Screenshot: test pane activity trace showing nodes with expand arrows.

22. Reading a transcript#

Work through it in this order. Resist the urge to jump to the error - the error is usually a consequence of something three steps earlier.

1. Timeline. Get the sequence of turns and the gaps between them. A four-second gap before a response is a tool call or a knowledge search. A twenty-second gap is a problem even if the answer was correct.

2. Topic selection / orchestration decision. Which topic or tool did the agent choose, and was it the right one? This single field resolves a large fraction of “wrong answer” tickets - the agent answered a different question correctly.

3. Trigger evaluation. What did the agent match on? If it matched a topic you didn’t expect, compare that topic’s trigger phrases with the user’s prompt and you’ll usually see the collision immediately.

4. Variable values. Check them at each step, particularly:

• Variables that should be populated by authentication (User.DisplayName, User.Id) - empty here means the user isn’t authenticated the way you assumed.
  • Variables passed into tools.
  • Variables set from tool outputs - an empty variable after a “successful” action is the invisible-failure signature.

5. User inputs as the agent received them. Not as the user thinks they typed them. Autocorrect, pasted formatting, and channel-level trimming all cause surprises.

6. Knowledge searches. Look at what was searched for and what came back. A search that returned nothing tells you a different story than one that returned three irrelevant documents. The first is a scope/permission/indexing problem; the second is a content problem.

7. Action execution. Inputs sent, outputs received, duration, result. Confirm the inputs match what you’d expect from the conversation.

8. Errors and fallback. Note where the fallback topic engaged. Fallback firing mid-conversation (rather than at the start) usually means an unhandled action failure.

23. Failure signatures#

Learn these five and you’ll diagnose most transcripts on sight:

24. Worked example - a successful conversation#

[10:04:12] User: what's my remaining leave for this year?
[10:04:12] Orchestration: selected tool "GetLeaveBalance"
                          reason: user asked about remaining annual leave
[10:04:12] Auth: User.Id = a1b2… (authenticated, Entra)
[10:04:13] Action GetLeaveBalance
           inputs  : { employeeId: "a1b2…", year: 2026 }
           outputs : { remainingDays: 12.5, carriedOver: 2 }
           duration: 840ms  result: Success
[10:04:14] Agent: You have 12.5 days of annual leave remaining for 2026,
                  including 2 days carried over from last year.

Everything you want is visible: the routing decision and why, the identity, the inputs, the outputs, the duration, the result.

25. Worked example - the same conversation failing#

[14:22:31] User: whats my remaining leave for this year
[14:22:31] Orchestration: selected tool "GetLeaveBalance"
[14:22:31] Auth: User.Id = (empty)
[14:22:32] Action GetLeaveBalance
           inputs  : { employeeId: "", year: 2026 }
           outputs : { }
           duration: 310ms  result: Success        <- note: Success
[14:22:32] Variable LeaveBalance = (null)
[14:22:33] Agent: I'm sorry, I couldn't find that information right now.

Routing was correct. The action “succeeded”. The real failure is on line 3: User.Id is empty, so the tool was called with a blank employee ID and the backend returned an empty result set with a 200. The user-visible message points at knowledge; the actual cause is authentication.

This is the single most valuable pattern in this guide: result: Success does not mean the step did its job. Always check the outputs, not the status.

A caveat worth knowing: in the environment-level telemetry preview, agent and tool execution errors aren’t always reflected correctly in trace statuses. Treat resultCode as a hint, not as ground truth, and verify against payloads.

Screenshot: annotated activity trace with the empty variable circled and an arrow to the downstream generic response.

26. Transcript analysis checklist#

• I have the transcript for the specific failing conversation
• I know which topic/tool the orchestrator chose and why
• I’ve checked authentication variables are populated
• I’ve checked tool inputs are sensible, not just present
• I’ve checked tool outputs, not just their success flags
• I’ve looked at knowledge search results, not just whether a search happened
• I’ve identified where the conversation diverged from expected - the first divergence, not the last
• If the cause still isn’t visible, I have the Conversation ID ready for section 5

27. 5. Application Insights troubleshooting#

This is the deepest layer, and the one where people waste the most time. Used well, it’s the only place that shows you the whole picture - the agent’s decisions, every downstream call, the failure, and the timing. Used badly, it’s a very expensive way to look at rows.

28. 5.1 First: know which telemetry you have#

Read this before you write a single query. Copilot Studio has two telemetry modes and they write to completely different tables with completely different schemas. Running a customEvents query against an environment-level export returns nothing, and you’ll conclude telemetry is broken when it’s working fine.

Don’t send both to the same Application Insights resource. Microsoft explicitly recommends against it, and in practice you’ll spend your incident trying to work out which schema a row belongs to. Pick one strategy per resource.

Two more things that will save you an hour:

• Environment-level export can take up to 24 hours to start flowing on a new configuration. If you just set it up and see nothing, that’s expected. Don’t debug it during an incident.
• During the current rollout, root agent invocations (invoke_agent) moved from requests to dependencies. You may still see agent root traces in the requests table until rollout completes in your region. If a query comes back empty, check the other table before concluding anything.

Quickest way to find out what you actually have:

union isfuzzy=true customEvents, dependencies, requests, traces, exceptions
| where timestamp > ago(1d)
| summarize Rows = count(), Sample = any(name) by itemType
| order by Rows desc

29. 5.2 How telemetry correlates#

Getting this model straight in your head is most of the battle.

Environment-level (OpenTelemetry spans):

• Each agent turn is one trace, sharing an operation_Id.
• The InvokeAgent span is the trace root. ExecuteTool and OutputMessages spans hang beneath it via operation_ParentId = the InvokeAgent span’s id.
• A conversation is many turns, so many traces. Thread them together with gen_ai.conversation.id.
• Sub-agents inherit the parent conversation ID with a _<subConversationId> suffix - split on _ to reconstruct the tree.
• OutputMessages spans don’t always have an InvokeAgent root, so orphaned single-node traces are normal, not a bug.

Agent-level (customEvents):

• session_Id groups a user session; customDimensions.conversationId identifies the conversation.
• Events arrive in sequence: TopicStart > Action/GenerativeAnswers/node events > BotMessageSend / BotMessageReceived > TopicEnd.
• customDimensions.designMode tells you whether it came from the test canvas. Filter this out in production queries or your metrics will be polluted by your own testing.
• Enable Node execution events in agent settings if you want per-node granularity - invaluable during an incident, noisy the rest of the time.

The schema will change under you#

Rather than trusting any documented field list (including this page), discover the live schema in your own resource:

// What columns exist?
dependencies
| getschema
| project ColumnName, ColumnType
| order by ColumnName asc

// What customDimensions keys are actually being emitted, and on which events?
dependencies
| where timestamp > ago(7d)
| mv-expand Key = bag_keys(customDimensions) to typeof(string)
| summarize Events = make_set(name), SampleValue = take_any(tostring(customDimensions[Key])) by Key
| order by Key asc

Run the second query once a month and paste the output into a child page. It’s the highest-value five minutes of documentation you can do for your team.

30. 5.3 Requests#

What they represent: inbound operations against the service. Historically this is where root agent invocations landed; with the environment-level rollout they’re moving to dependencies. Check both.

What to look at:

• success / resultCode - the blunt “did it work” signal.
• duration - end-to-end time for the operation. This is what the user feels.
• operation_Id - your join key into everything else.
• performanceBucket - pre-bucketed durations, handy for quick distribution checks.

// Failed requests in the last 24h, most common first
requests
| where timestamp > ago(24h)
| where success == false
| summarize Failures = count(), Sample = any(operation_Id) by name, resultCode
| order by Failures desc

// Duration distribution - are you slow always, or slow sometimes?
requests
| where timestamp > ago(24h)
| summarize p50 = percentile(duration, 50),
            p95 = percentile(duration, 95),
            p99 = percentile(duration, 99),
            Count = count()
  by bin(timestamp, 1h)
| render timechart

Read p95/p99, not the average. Averages hide exactly the tail that generates tickets. If p50 is 900ms and p99 is 28s, you don’t have a slow agent - you have a specific slow path, and it’s usually one dependency.

31. 5.4 Dependencies#

This is where you’ll spend most of your time. With environment-level telemetry, dependencies holds both the agent’s own GenAI spans (type == "GenAI") and the outbound calls the platform makes.

Things that show up here: Azure OpenAI / model calls, Dataverse operations, SharePoint and Graph queries, REST APIs via custom connectors, MCP servers, Work IQ, Power Automate flows, and your own APIs.

Separating agent spans from downstream calls#

dependencies
| where timestamp > ago(24h)
| summarize Calls = count(),
            Failures = countif(success == false),
            p95_ms = percentile(duration, 95)
  by type, target, name
| extend FailureRate = round(100.0 * Failures / Calls, 2)
| order by Failures desc, p95_ms desc

type == "GenAI" rows are the agent itself. Everything else is downstream. If failures cluster in one target, you’ve found the system to investigate in section 6.

Failed dependencies#

dependencies
| where timestamp > ago(24h)
| where success == false
| extend ConversationId = tostring(customDimensions["gen_ai.conversation.id"]),
         ToolName       = tostring(customDimensions["gen_ai.tool.name"]),
         ErrorType      = tostring(customDimensions["error.type"]),
         StatusMessage  = tostring(customDimensions["Status.message"])
| project timestamp, name, target, ToolName, resultCode, ErrorType, StatusMessage,
          duration, ConversationId, operation_Id
| order by timestamp desc

Interpreting resultCode:

Tool call detail#

The tool payloads are the most useful thing in the whole telemetry set, and they’re easy to miss because they live inside customDimensions:

dependencies
| where timestamp > ago(24h)
| where name == "ExecuteTool"
| extend ToolName   = tostring(customDimensions["gen_ai.tool.name"]),
         ToolType   = tostring(customDimensions["gen_ai.tool.type"]),
         Args       = tostring(customDimensions["gen_ai.tool.call.arguments"]),
         Result     = tostring(customDimensions["gen_ai.tool.call.result"]),
         ConvId     = tostring(customDimensions["gen_ai.conversation.id"])
| project timestamp, ToolName, ToolType, Args, Result, duration, resultCode, ConvId
| order by timestamp desc

Two things to look for:

1. Arguments that are empty or nonsense. The model builds these from conversation context. Bad arguments mean bad tool descriptions, not a bad model.
2. Results that are technically valid but empty. {"value":[]} with a success code is the invisible failure from section 4, and this is where you prove it.

Note: tool input arguments and output results are only captured when Log conversation details is enabled in the agent’s Application Insights settings. If gen_ai.tool.call.arguments is consistently empty across all rows, that’s a configuration gap, not a platform bug. Weigh it against your data-handling policy before enabling it in production - these payloads can contain personal data.

Slow dependencies#

dependencies
| where timestamp > ago(24h)
| summarize Calls = count(),
            p50 = percentile(duration, 50),
            p95 = percentile(duration, 95),
            Max = max(duration)
  by target, name
| where p95 > 3000
| order by p95 desc

32. 5.5 Traces#

traces carries diagnostic log records and, for some workloads, the richest lifecycle detail.

traces
| where timestamp > ago(24h)
| where severityLevel >= 2          // 2 = Warning, 3 = Error, 4 = Critical
| project timestamp, message, severityLevel, operation_Id, customDimensions
| order by timestamp desc

For real-time voice agents the whole picture lives here rather than in dependencies, keyed on customDimensions.Subject:

traces
| where timestamp > ago(24h)
| where customDimensions.Subject == "LlmInvocation"
| extend cl = parse_json(tostring(customDimensions.CallLifecycle))
| project timestamp,
          corrId  = tostring(cl.TrackingContext.CorrelationId),
          model   = tostring(customDimensions.Model),
          ttfabMs = toint(customDimensions.Duration_Ms),
          inTok   = toint(customDimensions.InputTokens),
          outTok  = toint(customDimensions.OutputTokens)
| order by ttfabMs desc

Voice event subjects worth knowing: CallLifecycle, DialogLifecycle (DialogDuration_Ms, EndReason), SpeechPipeline (AudioLatency_Ms), LlmInvocation, ToolExecution. The top-level message field is a placeholder - always read customDimensions.

Custom telemetry. If you emit your own events from Power Automate flows or custom APIs feeding the agent, put the Conversation ID in a custom dimension. It costs nothing at write time and makes your logs joinable to the agent’s, which is worth a great deal at 02:00.

33. 5.6 Exceptions#

exceptions
| where timestamp > ago(24h)
| summarize Count = count(), LastSeen = max(timestamp), Sample = any(operation_Id)
  by type, outerMessage
| order by Count desc

Then pivot from a specific exception to its whole trace:

let Op = "<operation_Id>";
union requests, dependencies, traces, exceptions
| where operation_Id == Op
| project timestamp, itemType, name, message = coalesce(tostring(message), tostring(outerMessage)),
          resultCode, duration, customDimensions
| order by timestamp asc

Reading exceptions:

• type groups the failure class. Two exceptions with the same type and different messages are usually the same bug.
• outerMessage is the human-readable one. details holds the stack, which matters mainly for custom code you own.
• Exceptions from platform components rarely give you a fixable stack trace. Their value is the timestamp and correlation, not the trace itself - use them to locate the turn, then read the dependencies.

Content-safety blocks show up as a specific signature rather than a classic exception:

customEvents
| where timestamp > ago(7d)
| where customDimensions contains "ContentFiltered"
| project timestamp, name, session_Id, user_Id, customDimensions
| order by timestamp desc

34. 5.7 Performance analysis#

Users report slowness as brokenness. Two very different problems hide behind it: consistently slow (an architecture issue) and occasionally very slow (a tail issue). Separate them first.

// Where does the time actually go? Break the turn down by span type.
dependencies
| where timestamp > ago(24h)
| where type == "GenAI"
| summarize Calls = count(),
            p50 = percentile(duration, 50),
            p95 = percentile(duration, 95),
            TotalMs = sum(duration)
  by name
| order by TotalMs desc

// The slowest individual turns, with the tool that dominated them
let SlowTraces =
    dependencies
    | where timestamp > ago(24h) and name == "InvokeAgent" and duration > 10000
    | project operation_Id, TurnDuration = duration, timestamp;
dependencies
| where timestamp > ago(24h)
| join kind=inner SlowTraces on operation_Id
| extend ToolName = tostring(customDimensions["gen_ai.tool.name"]),
         ConvId   = tostring(customDimensions["gen_ai.conversation.id"])
| summarize SlowestStep = arg_max(duration, name, ToolName)
  by operation_Id, TurnDuration, ConvId
| order by TurnDuration desc

Where the time usually goes, in rough order of frequency:

1. A single slow tool - an on-prem API behind a gateway, or a Power Automate flow doing a loop it shouldn’t.
2. Multiple sequential tool calls in one turn. Each one is fine; four in sequence is eight seconds.
3. Model latency under load, especially with long instructions and large retrieved context.
4. Knowledge retrieval over large or slow sources.
5. Sub-agent chains - the parent waits for the child, which waits for its own tools.

The duration caveat: duration isn’t populated for classic agent traces in the environment-level export. If durations are all 0, you’re looking at classic traces - fall back to timestamp deltas between spans, or use agent-level telemetry for that agent.

Timeouts. A dependency that ends around a suspiciously round number (30s, 60s, 120s) with no result code is a timeout, and the number tells you whose timeout: the connector’s, the gateway’s, or the platform’s. Find the component whose configured limit matches and you’ve found the layer to fix.

35. 5.8 End-to-end correlation#

This is the payoff. Given a Conversation ID, you can walk the entire lifecycle.

The one query to start every investigation with:

let ConvId = "<paste conversation id>";
dependencies
| where timestamp > ago(7d)
| where tostring(customDimensions["gen_ai.conversation.id"]) startswith ConvId
| extend Operation = tostring(customDimensions["gen_ai.operation.name"]),
         Agent     = tostring(customDimensions["gen_ai.agent.name"]),
         Tool      = tostring(customDimensions["gen_ai.tool.name"]),
         UserMsg   = tostring(parse_json(tostring(customDimensions["gen_ai.input.messages"]))[0].parts[0].content),
         AgentMsg  = tostring(parse_json(tostring(customDimensions["gen_ai.output.messages"]))[0].parts[0].content),
         ErrType   = tostring(customDimensions["error.type"]),
         StatusMsg = tostring(customDimensions["Status.message"])
| order by operation_Id asc, iff(name == "InvokeAgent", 0, 1) asc, timestamp asc
| project timestamp, name, Operation, Agent, Tool, UserMsg, AgentMsg,
          duration, resultCode, ErrType, StatusMsg, operation_Id, operation_ParentId

startswith rather than == is deliberate - it picks up sub-agent conversations, which carry the _<subId> suffix.

Once you have the operation_Id for the failing turn, the end-to-end transaction view in the Azure portal (click any row > End-to-end transaction details) renders the whole span tree as a waterfall. For timing problems it’s faster than any query you’ll write.

36. 5.9 Things that make telemetry lie to you#

Before you conclude “there’s no telemetry”, rule these out:

• Ingestion delay. Normally a couple of minutes. Up to 24 hours for a newly configured environment-level export.
• Sampling. If sampling is enabled on the resource, individual conversations can be genuinely absent. Check itemCount - values above 1 mean you’re seeing sampled data, and aggregate counts need multiplying.
• Local authentication disabled on the target Application Insights resource blocks the environment-level export entirely.
• The wrong resource. DEV and PRO agents pointing at the same App Insights instance is common, and so is the reverse.
• Retention. Investigating something from six weeks ago may simply be past your retention window. Confirm the configured retention on the resource.
• designMode == True rows are test-canvas traffic. If your “production incident” only appears in design mode rows, the incident is somebody testing.
• Export isn’t transactional. Small amounts of data loss during transient service events are expected. One missing span is not proof of a bug.

37. Application Insights checklist#

• I know whether this environment uses agent-level or environment-level telemetry
• I’m querying the right resource for the right environment
• I’ve filtered out designMode == True (agent-level)
• I’ve found the conversation and its operation_Ids
• I’ve checked dependencies for failures and for successes with empty results
• I’ve checked exceptions in the same operation_Id
• I’ve looked at durations at p95, not just averages
• I’ve viewed the failing turn in the end-to-end transaction view
• If telemetry is missing, I’ve ruled out delay, sampling, retention and wrong-resource

38. 6. Troubleshooting external systems#

By this point telemetry has told you which system failed. This section is about confirming it and finding out why. The general method is always the same: reproduce the call outside the agent. If a REST call fails in Postman with the same identity, the agent isn’t your problem.

39. Azure OpenAI / model layer#

40. Dataverse#

• Access: confirm the calling identity has the table privileges, not just environment access. Copilot Studio’s own data (transcripts, agent config) lives here too.
• Throttling: Dataverse enforces service protection limits per user per five minutes. Sudden 429s under load with no code change is the classic signature.
• Performance: unindexed contains filters on large tables are slow. If a Dataverse dependency sits at 4-8 seconds, look at the FetchXML/OData being generated, not the platform.
• Plugins: a synchronous plugin on the table adds its execution time to every call the agent makes, and its failures surface as opaque errors.

41. SharePoint#

Most common source of “the answer is wrong” and it’s nearly always one of four things:

1. Permissions. Retrieval is permission-trimmed for the user. Test as an affected user.
2. Indexing lag. New or recently changed content isn’t immediately retrievable.
3. Scope. The knowledge source URL points at a site, and the content is in a subsite or a different library.
4. File form. Image-only PDFs, huge files, and protected documents contribute nothing to retrieval.

Remember that SharePoint-grounded answers appear as REDACTED in transcripts - use telemetry or live reproduction instead.

42. Microsoft Graph#

• Consent: the app registration needs admin consent for the scopes it uses. Newly added permissions do not apply to tokens already issued - delete and recreate the connection to force a fresh token.
• Delegated vs application permissions: delegated calls only see what the user sees. A “missing data” report is often correct behaviour under delegated auth.
• Throttling: Graph throttles aggressively per-endpoint. Look for 429s with a Retry-After header.

43. Work IQ#

• Confirm the connection exists and is authorised for the calling user.
• Work IQ results are permission- and tenant-scoped; empty results for one user and not another are almost always permissions or licensing.
• In telemetry it appears as a tool with a recognisable gen_ai.tool.name - filter on it directly:

dependencies
| where timestamp > ago(24h)
| extend Tool = tostring(customDimensions["gen_ai.tool.name"])
| where Tool has "workiq"
| project timestamp, Tool, resultCode, duration,
          Result = tostring(customDimensions["gen_ai.tool.call.result"])
| order by timestamp desc

44. MCP servers#

MCP failures cluster into three types, and the debugging order matters:

1. No tools discovered. If the agent behaves as if the tools don’t exist:

• Is the server running and the endpoint correct?
  • Does tools/list actually return tools? Test with MCP Inspector outside Copilot Studio - this single step resolves most MCP tickets.
  • Many servers return no tools until the user has authenticated. A skipped or failed sign-in produces an empty tool list rather than an error.

2. Tools discovered but never called. That’s a description problem, not a connectivity problem. See section 3.5.

3. Tools called and failing.

• 401/403: token audience, expired consent, or the user lacking rights in the target system.
  • Cross-tenant: if the client and server app registrations live in different tenants, you’ll see errors about a missing service principal. A tenant admin has to provision it (az ad sp create --id <server_app_client_id>).
  • DLP: tenant policies can block custom connectors outright. The symptom is a creation or invocation failure that looks like a permissions bug.
  • Tenant isolation policies block cross-tenant data flow and need explicit exception rules.

45. REST APIs and custom connectors#

• Reproduce with the same identity and payload outside the agent. If it fails there, stop looking at Copilot Studio.
• Check the connector’s response schema matches reality. A mismatched schema binds nothing and produces empty variables with success codes.
• Watch for gateway and connector timeouts - see the round-number heuristic in section 5.7.
• API Management in front of the API adds its own policies, throttling and auth. Check APIM diagnostics separately; a 401 at APIM looks identical to a 401 at the backend from the agent’s side.

46. Authentication (cross-cutting)#

Work through it in this order - it goes from most to least likely:

1. Which identity is actually calling? End user, maker’s connection, or service principal. Getting this wrong invalidates everything after it.
2. Is the token fresh? Recently granted permissions need a new connection to take effect.
3. Is consent granted? Per-user, admin-wide, or pre-authorised in the server app registration.
4. Does the identity have rights in the target system? Authentication and authorisation fail identically to the user.
5. Can the channel complete the flow? Some embedded surfaces block sign-in popups; the same agent works in Teams and silently fails in a custom web chat.

47. Azure resources#

Before deep-diving, check the boring infrastructure facts: is the resource running, is it in the expected region, has a firewall or private endpoint changed, is there an active Azure Service Health advisory, and did anything deploy recently. Check the resource’s Activity log - an unannounced config change by another team is a genuinely common root cause.

48. External systems checklist#

• I’ve identified the specific failing dependency from telemetry
• I’ve reproduced (or failed to reproduce) the call outside the agent
• I know which identity makes the call
• I’ve checked the target system’s own logs, not just the agent’s
• I’ve checked for throttling (429) and timeouts specifically
• I’ve checked recent changes to the target resource
• I’ve checked Azure Service Health and Microsoft 365 service health

49. 7. Determining the root cause#

A root cause is not “the API failed”. It’s the answer to why the API failed, stated precisely enough that someone can prevent it happening again.

Test your candidate root cause against three questions:

1. Does it explain everything you observed - including the timing and the scope (one user vs. all)?
2. Could you reproduce the failure on demand by recreating this condition?
3. Does the fix follow from it obviously? If the fix feels like a guess, you have a symptom, not a cause.

If you can’t answer all three, keep going. Closing a ticket on a symptom means you’ll see it again in a fortnight with a different reporter.

50. Root cause analysis template#

Copy this into a Confluence template. Attach the completed version to the ticket and link it from the agent’s known-issues page.

## Root cause analysis - <short title>

**Incident:**            <ticket ref>
**Agent / environment:** 
**Detected:**            <UTC>          **Resolved:** <UTC>
**Impact:**              <who, how many, how long, what they couldn't do>
**Severity:**            

### Symptoms
What users experienced, in their words. Include the exact prompt and response.

### Evidence
| Source | What it showed |
|---|---|
| Transcript (ConvId …) | |
| App Insights (operation_Id …) | |
| External system logs | |
| Configuration | |

### Root cause
One or two sentences. Specific. Names the component and the condition.

### Contributing factors
What made this possible, or made it take longer to find than it should have.

### Resolution
What was changed, by whom, when, and in which environment(s).

### Validation
How we confirmed it's fixed - see section 8.

### Preventative actions
| Action | Owner | Due | Status |
|---|---|---|---|
| Alert on <condition> | | | |
| Add error handling to <topic> | | | |
| Document in known issues | | | |

51. Worked example#

Symptoms: From 14:10 UTC, roughly 40% of users asking about leave balances received “I couldn’t find that information right now.” Others got correct answers. No recent publish.

Evidence: Transcript for ConvId a1b2... showed the correct tool selected, User.Id empty, and GetLeaveBalance returning {} with a success result. Application Insights showed the same pattern across 213 turns, all from the custom web chat channel; Teams traffic was unaffected. The HR API logged 200s with empty result sets for blank employee IDs.

Root cause: The Direct Line token issued by the custom web chat host stopped including the user claim after a change to the host application’s token endpoint at 14:05 UTC. The agent’s tool therefore received an empty employee ID and the HR API returned an empty set with a 200, which the agent surfaced as a generic “not found”.

Contributing factors: (1) The tool had no validation on a required input. (2) The HR API returns 200 for unknown IDs rather than 400/404. (3) No alert existed for a spike in empty tool results.

Resolution: Host application token endpoint reverted at 15:40 UTC. Input validation added to the topic to fail explicitly when User.Id is empty.

Preventative actions: Alert on empty gen_ai.tool.call.result rate; explicit error branch on the topic; note added to the channel onboarding checklist.

Note what the root cause statement does: it names the component, the change, the time, and the mechanism by which a working system produced a wrong answer. “Authentication issue” would have been useless.

52. Root cause checklist#

• The cause explains the timing and the scope
• I could reproduce the failure deliberately
• The fix follows obviously from the cause
• Contributing factors are recorded, not just the trigger
• Evidence is linked, with IDs, not summarised from memory
• Preventative actions have owners and dates

53. 8. Validating the fix#

Deploying a change is not resolving an incident. Four things need to be true before you close.

54. 1. The original issue is resolved#

• Retest the exact original prompt, not your cleaned-up version of it.
• Retest through the original channel, as a user with the original permissions. Not the test pane.
• Retest against the published agent.
• If you can, have the original reporter confirm. Their definition of “fixed” is the one that counts.

55. 2. Nothing else broke#

• Test the two or three topics closest to the one you changed. Trigger phrase and description edits have side effects on routing that are invisible in the topic you edited.
• If you changed instructions or the model, run a short regression set of representative prompts. If you have agent evaluations configured, this is what they’re for - a stored test set beats ad-hoc prompting every time.
• If you changed a shared connection or connection reference, check every agent that uses it.

56. 3. Performance is acceptable#

// Before/after comparison - adjust the fix time
let FixTime = datetime(2026-08-06 09:00:00);
dependencies
| where timestamp > FixTime - 24h
| where type == "GenAI" and name == "InvokeAgent"
| extend Period = iff(timestamp < FixTime, "Before", "After")
| summarize Turns = count(),
            p50 = percentile(duration, 50),
            p95 = percentile(duration, 95),
            Failures = countif(success == false)
  by Period

57. 4. Telemetry is healthy#

Give it 24 hours of real traffic and check:

• The failure signature is gone - not merely reduced.
• No new exception types appeared after the change.
• Dependency failure rates are back to baseline (you do have a baseline written down somewhere, don’t you?).
• Conversation volume is normal. A sudden drop can mean users gave up, which is a worse outcome than a visible error.

58. Post-resolution checklist#

• Original prompt retested, in the original channel, against the published agent
• Reporter has confirmed
• Adjacent topics regression-tested
• Other agents sharing the changed component checked
• p95 duration is at or better than before
• 24 hours of clean telemetry - no new exception types
• RCA written and linked from the ticket
• Known-issues page updated
• Alert or monitoring gap closed, if this was found by a user rather than by monitoring

59. Appendix A - Common troubleshooting scenarios#

Pattern matching. Find the closest symptom, work the steps.

60. A.1 Topic not triggering#

Symptoms: User asks something the agent should handle; gets the fallback or a generic answer. The topic exists and looks correct.

Likely causes: trigger phrases too far from real user language, another topic winning on priority, a trigger condition that’s never true, agent not published, under generative orchestration, a weak topic/tool description.

Investigate:

1. Check the publish timestamp. (Yes, really. Do this first every time.)
2. Reproduce in the test pane and read the activity trace - which topic did fire?
3. If Multiple Topics Matched fired, you have overlapping trigger phrases.
4. Compare the user’s exact wording with the trigger phrases side by side.
5. If using generative orchestration, read the topic description as if you were the model deciding.

Resolution: add trigger phrases in the user’s actual language (classic), or rewrite the description to state when to use the topic (generative). Adjust priority where topics genuinely overlap. Publish and retest in the real channel.

61. A.2 Wrong topic selected#

Symptoms: Agent answers a different question, confidently and correctly.

Likely causes: overlapping trigger phrases, one topic with very broad phrases acting as a magnet, priority ordering, descriptions that overlap in scope.

Investigate:

1. Transcript > which topic fired and on what match.
2. Search all topics for the colliding phrase.
3. Look for a topic with generic phrases like “help”, “question”, “information” - that’s usually the magnet.

Resolution: narrow the greedy topic, remove duplicate phrases, and make descriptions mutually exclusive. Where two topics are genuinely close, consider merging them and branching inside.

62. A.3 Knowledge search returns incorrect answers#

Symptoms: Agent answers from the wrong document, or cites something outdated.

Likely causes: multiple documents covering the same topic (old and new both indexed), poor chunking on long documents, scope wider than intended, outdated content still live in SharePoint.

Investigate:

1. Transcript > what did the search return, and in what order?
2. Search SharePoint for the same terms and see whether the wrong document ranks first there too. If it does, it’s a content problem.
3. Check for duplicate or superseded documents in scope.

Resolution: archive superseded content, tighten source scope, improve document titles and headings. Model changes won’t fix a content problem - this is the one people most often try to solve with prompt engineering, and it never works.

63. A.4 Hallucinations#

Symptoms: Confident, plausible, wrong. Often about policies or numbers.

Likely causes: general model knowledge allowed when it shouldn’t be, knowledge gap plus instructions that push toward always answering, retrieval returning weakly relevant content, a question the content genuinely doesn’t cover.

Investigate:

1. Check whether the agent is restricted to knowledge sources only.
2. Transcript > did retrieval return anything relevant? A hallucination with an empty retrieval is a grounding configuration issue; one with good retrieval is an instruction issue.
3. Read the instructions for anything that discourages saying “I don’t know”.

Resolution: restrict to knowledge sources where accuracy matters more than coverage; add an explicit instruction to decline when the answer isn’t in the sources; fill the content gap. Accept that a stricter agent will refuse more often - that’s the trade, and it’s usually the right one for policy and HR content.

64. A.5 Authentication failures#

Symptoms: Works for you, fails for users. Or worked yesterday, fails today. 401/403 in dependencies.

Likely causes: expired connection, connection reference unbound after solution import, newly granted permissions not reflected in an existing token, delegated vs application permission mismatch, the channel can’t complete the sign-in flow.

Investigate:

1. dependencies | where resultCode in ("401","403") for the affected window.
2. Open the connection reference and confirm it’s bound and its connection is healthy.
3. Confirm which identity the tool calls with.
4. Test in a channel known to support the sign-in flow, to isolate channel from identity.

Resolution: rebind or recreate the connection (recreating forces a fresh token - necessary after permission changes), fix consent, correct the permission model. If the channel blocks the popup, that’s a host application fix, not an agent fix.

65. A.6 Action failures#

Symptoms: Agent says something went wrong, or gives a generic answer after a pause.

Likely causes: bad or missing inputs, output schema mismatch, downstream error, unhandled failure path.

Investigate:

1. Transcript > inputs sent and outputs received.
2. Telemetry > the ExecuteTool span: arguments, result, resultCode, duration.
3. If it’s a flow, open the flow’s own run history - it has the real error.

Resolution: fix input descriptions so the model populates them correctly, correct the schema, and add explicit error handling so the next occurrence is diagnosable from the transcript alone.

66. A.7 Action “succeeds” but nothing happens#

Symptoms: No error. The agent behaves as though it got no data - because it didn’t.

Likely causes: downstream returns 200 with an empty payload, output schema mismatch means nothing binds, a filter in the downstream query excludes everything.

Investigate:

1. gen_ai.tool.call.result in the ExecuteTool span - is it empty or {"value":[]}?
2. Check the arguments that were sent. Empty required arguments are the usual cause.
3. Run the same call manually with the same arguments.

Resolution: validate required inputs in the topic before calling; add an explicit branch for empty results with a useful message. Consider asking the API owner to return 404 rather than 200-with-nothing.

67. A.8 MCP server unavailable#

Symptoms: Tools that used to work aren’t called at all, or fail immediately.

Likely causes: server down or endpoint changed, authentication not completed (many servers return no tools until sign-in), DLP policy blocking the custom connector, cross-tenant service principal missing, tenant isolation.

Investigate:

1. Test the server directly with MCP Inspector. Does tools/list return tools?
2. Check the connection in Power Apps - does it need re-authorisation?
3. Check DLP policies in the Power Platform admin centre for the environment.
4. For cross-tenant setups, check for the “lacks a service principal” error.

Resolution: restore the server, re-authorise, add a DLP exception (or move the work to an environment where the connector is permitted), or have a tenant admin provision the service principal.

68. A.9 Work IQ connection issues#

Symptoms: Work IQ-backed answers empty for some users, fine for others.

Likely causes: licensing, connection not authorised for that user, permission-scoped data genuinely not visible to them.

Investigate: compare a working and non-working user’s licence and group membership; check the connection state; filter telemetry on the Work IQ tool name and compare results per user.

Resolution: fix licensing or permissions. If the data genuinely isn’t visible to that user, the agent is behaving correctly - say so in the ticket, and consider whether the agent should explain why it can’t answer rather than returning nothing.

69. A.10 Azure OpenAI timeout#

Symptoms: Long pause, then a generic error or a truncated response. Intermittent, worse at peak.

Likely causes: quota/TPM limits (429s), very large prompts, regional capacity pressure.

Investigate:

1. Dependencies filtered to the model calls: durations, resultCode, and specifically 429 counts over time.
2. Azure OpenAI resource metrics for the deployment.
3. Compare instruction and context size against the slow turns.

Resolution: request quota, spread load, shorten instructions, tighten retrieval so less context is passed. Add retry/backoff where you control the call.

70. A.11 Slow responses#

Symptoms: “The bot is really slow.” Sometimes correct answers, sometimes client-side timeouts.

Investigate:

1. Establish whether it’s consistent or tail - p50 vs p95 (see section 5.7).
2. Break the turn down by span and find the dominant one.
3. Count tool calls per turn. Sequential calls add up fast.

Resolution: parallelise or eliminate tool calls, cache stable reference data, reduce context size, push slow work into an async pattern with an acknowledgement message. Set expectations in the UI - a typing indicator changes the perception of six seconds considerably.

71. A.12 Permission issues#

Symptoms: Different users get different answers to the same question. Often reported as inconsistency, not as a permissions bug.

Investigate: compare group membership and licences between a working and non-working user; verify permission-trimming behaviour on the knowledge source; test as an affected user, not as yourself.

Resolution: fix the permissions at source, or accept and explain the behaviour - an agent that says “you don’t have access to that document” is far better than one that says “I couldn’t find anything”.

72. A.13 API failures#

Symptoms: Specific capabilities fail while everything else works.

Investigate: identify the failing target in dependencies; reproduce outside the agent with the same identity; check the API’s own logs and any gateway (APIM) in front of it; look at status codes over time to distinguish a hard failure from throttling.

Resolution: fix the API, the credentials, or the gateway policy - then add error handling in the agent so the next occurrence produces a useful message rather than a shrug.

73. A.14 “It works in test but not in production”#

Symptoms: Exactly what it says.

Almost always one of: the agent isn’t published, connection references point at the maker’s connections in DEV but nothing in PRO, environment variables didn’t carry across the solution import, the user has different permissions than the maker, the channel differs, a DLP policy applies in PRO but not DEV.

Investigate: compare the two environments component by component - publish state, connection references, environment variables, DLP policies, knowledge source URLs. In an ALM pipeline, check the deployment settings file: unset connection references are the number-one cause.

Resolution: fix the deployment configuration, and add a post-deployment smoke test so the pipeline catches it next time instead of a user.

74. Appendix B - KQL cookbook#

Run these in Application Insights > Logs. Placeholders are in <angle brackets>.

Before you start: confirm whether the environment uses agent-level (customEvents) or environment-level (dependencies) telemetry - section 5.1. Queries are marked [env] or [agent] accordingly. Unmarked ones work with either.

75. B.0 What telemetry do I have?#

Troubleshoots: “there’s no telemetry” reports. Use when: starting on an unfamiliar environment.

union isfuzzy=true customEvents, dependencies, requests, traces, exceptions
| where timestamp > ago(1d)
| summarize Rows = count(), Sample = any(name) by itemType
| order by Rows desc

Expect: rows concentrated in customEvents (agent-level) or dependency (environment-level). Nothing at all means a configuration problem, not an agent problem.

76. B.1 Find a recent conversation for an agent [env]#

Troubleshoots: no Conversation ID in the ticket. Use when: you have an agent name and a rough time.

let Window = 24h;
let AgentName = "<Agent name>";
dependencies
| where timestamp > ago(Window)
| where tostring(customDimensions["gen_ai.agent.name"]) == AgentName
| extend ConvId = tostring(customDimensions["gen_ai.conversation.id"]),
         UserName = tostring(customDimensions["user.name"]),
         Channel = tostring(customDimensions["microsoft.channel.name"])
| summarize Turns = dcount(operation_Id),
            Start = min(timestamp), End = max(timestamp),
            Failures = countif(success == false)
  by ConvId, UserName, Channel
| order by End desc

Expect: one row per conversation. Sort by Failures to jump straight to the broken ones. Variations: filter on UserName or user.email when you know the reporter.

77. B.2 Full trace for a Conversation ID [env]#

Troubleshoots: everything. Use when: you have a Conversation ID. This is the workhorse.

let ConvId = "<Conversation ID>";
dependencies
| where timestamp > ago(7d)
| where tostring(customDimensions["gen_ai.conversation.id"]) startswith ConvId
| extend Operation = tostring(customDimensions["gen_ai.operation.name"]),
         Agent     = tostring(customDimensions["gen_ai.agent.name"]),
         Model     = tostring(customDimensions["gen_ai.request.model"]),
         Tool      = tostring(customDimensions["gen_ai.tool.name"]),
         ToolArgs  = tostring(customDimensions["gen_ai.tool.call.arguments"]),
         ToolResult= tostring(customDimensions["gen_ai.tool.call.result"]),
         UserMsg   = tostring(parse_json(tostring(customDimensions["gen_ai.input.messages"]))[0].parts[0].content),
         AgentMsg  = tostring(parse_json(tostring(customDimensions["gen_ai.output.messages"]))[0].parts[0].content),
         ErrType   = tostring(customDimensions["error.type"]),
         StatusMsg = tostring(customDimensions["Status.message"])
| order by operation_Id asc, iff(name == "InvokeAgent", 0, 1) asc, timestamp asc
| project timestamp, name, Operation, Agent, Model, Tool, ToolArgs, ToolResult,
          UserMsg, AgentMsg, duration, resultCode, ErrType, StatusMsg,
          operation_Id, operation_ParentId

Expect: the conversation in order, root span before its children, one operation_Id per turn. Variations: == instead of startswith to exclude sub-agents; add | where resultCode != "OK" to see only failures.

78. B.3 Find a session [agent]#

Troubleshoots: agent-level equivalent of B.2. Use when: you have a session ID or user.

let SessionId = "<session id>";
customEvents
| where timestamp > ago(7d)
| where session_Id == SessionId
| extend Topic = tostring(customDimensions["TopicName"]),
         Kind  = tostring(customDimensions["Kind"]),
         Text  = tostring(customDimensions["text"]),
         Channel = tostring(customDimensions["channelId"]),
         DesignMode = tostring(customDimensions["designMode"])
| project timestamp, name, Topic, Kind, Text, Channel, DesignMode
| order by timestamp asc

Expect: TopicStart > node/action events > BotMessageSend > TopicEnd. Variations: swap session_Id for | where customDimensions contains "<conversationId>".

79. B.4 Failed requests#

Troubleshoots: platform-level failures. Use when: triaging a broad outage.

requests
| where timestamp > ago(24h)
| where success == false
| summarize Failures = count(), Sample = any(operation_Id), LastSeen = max(timestamp)
  by name, resultCode
| order by Failures desc

Expect: empty on a healthy day. Take Sample into B.6 to see one in full.

80. B.5 Failed dependencies#

Troubleshoots: the most common failure class. Use when: any tool or downstream issue.

dependencies
| where timestamp > ago(24h)
| where success == false
| extend ConvId    = tostring(customDimensions["gen_ai.conversation.id"]),
         Tool      = tostring(customDimensions["gen_ai.tool.name"]),
         ErrType   = tostring(customDimensions["error.type"]),
         StatusMsg = tostring(customDimensions["Status.message"])
| project timestamp, name, target, Tool, resultCode, ErrType, StatusMsg,
          duration, ConvId, operation_Id
| order by timestamp desc

Variations: | summarize count() by target, resultCode for the shape of the problem instead of the detail.

81. B.6 Everything for one operation (correlation)#

Troubleshoots: “what happened in this turn?” Use when: you have an operation_Id.

let Op = "<operation_Id>";
union isfuzzy=true requests, dependencies, traces, exceptions, customEvents
| where operation_Id == Op
| project timestamp, itemType, name,
          Message = coalesce(tostring(message), tostring(outerMessage), ""),
          resultCode, duration, customDimensions
| order by timestamp asc

Expect: the full span tree plus any exception, in order. This is the query that usually ends the investigation.

82. B.7 Root agent plus sub-agents [env]#

Troubleshoots: multi-agent handoffs. Use when: a connected agent is involved.

let Window = 7d;
let RootConvId = "<root conversation id>";
dependencies
| where timestamp > ago(Window)
| extend ConvId = tostring(customDimensions["gen_ai.conversation.id"]),
         Agent  = tostring(customDimensions["gen_ai.agent.name"]),
         Tool   = tostring(customDimensions["gen_ai.tool.name"])
| where ConvId == RootConvId or ConvId startswith strcat(RootConvId, "_")
| extend Depth = countof(ConvId, "_"),
         Role  = iff(ConvId == RootConvId, "root", "sub-agent")
| order by timestamp asc
| project timestamp, Role, Depth, Agent, name, Tool, duration, resultCode, ConvId, operation_Id

Expect: root spans plus a nested set per child agent. Known quirk: sub-agent spans currently parent to the invoking InvokeAgent span rather than to the InvokeAgent inside their own trace - the tree looks flatter than it logically is.

83. B.8 Slow turns#

Troubleshoots: “the agent is slow”. Use when: performance complaints.

dependencies
| where timestamp > ago(24h)
| where name == "InvokeAgent"
| extend ConvId = tostring(customDimensions["gen_ai.conversation.id"]),
         Agent  = tostring(customDimensions["gen_ai.agent.name"])
| where duration > 10000
| project timestamp, Agent, duration, ConvId, operation_Id
| order by duration desc
| take 50

Expect: the worst offenders with their operation_Ids. Feed those into B.6. Variations: replace the where with a percentile summary by hour to see whether it’s getting worse.

84. B.9 Slow dependencies#

Troubleshoots: which downstream call is the bottleneck.

dependencies
| where timestamp > ago(24h)
| extend Tool = tostring(customDimensions["gen_ai.tool.name"])
| summarize Calls = count(),
            p50 = percentile(duration, 50),
            p95 = percentile(duration, 95),
            Max = max(duration),
            Failures = countif(success == false)
  by type, target, coalesce(Tool, name)
| where p95 > 2000
| order by p95 desc

Expect: a short list. Usually one row dominates, and that’s your fix.

85. B.10 Model / Azure OpenAI calls#

Troubleshoots: throttling and model latency.

dependencies
| where timestamp > ago(24h)
| where type == "GenAI" or target has_any ("openai", "cognitiveservices")
| summarize Calls = count(),
            Throttled = countif(resultCode == "429"),
            Failed = countif(success == false),
            p95 = percentile(duration, 95)
  by bin(timestamp, 1h), tostring(customDimensions["gen_ai.request.model"])
| order by timestamp desc

Expect: Throttled should be zero. Any sustained non-zero value is a capacity conversation.

86. B.11 Dataverse operations#

dependencies
| where timestamp > ago(24h)
| where target has_any ("dynamics.com", "crm") or name has "Dataverse"
| summarize Calls = count(), Failures = countif(success == false),
            Throttled = countif(resultCode == "429"),
            p95 = percentile(duration, 95)
  by target, resultCode
| order by Failures desc

Watch for: 429s (service protection limits) and p95 above ~2s (query shape, or a synchronous plugin).

87. B.12 Responsible AI content filtering [agent]#

Troubleshoots: blocked or replaced responses. Use when: users report the agent refusing normal questions.

customEvents
| where timestamp > ago(7d)
| where customDimensions contains "ContentFiltered"
| project timestamp, name, itemType, session_Id, user_Id, cloud_RoleInstance, customDimensions
| order by timestamp desc

Variations: add | where customDimensions contains "<conversationId>" to confirm a specific conversation was affected.

88. B.13 Knowledge and SharePoint activity#

dependencies
| where timestamp > ago(24h)
| where target has_any ("sharepoint.com", "graph.microsoft.com")
       or tostring(customDimensions["gen_ai.tool.name"]) has_any ("sharepoint", "knowledge")
| extend Tool = tostring(customDimensions["gen_ai.tool.name"]),
         Result = tostring(customDimensions["gen_ai.tool.call.result"])
| project timestamp, Tool, target, resultCode, duration,
          EmptyResult = iff(isempty(Result) or Result in ("{}", "[]"), "YES", "no"),
          ConvId = tostring(customDimensions["gen_ai.conversation.id"])
| order by timestamp desc

Expect: EmptyResult = YES alongside a success code is the “found nothing” signature - a scope, permission or indexing problem.

89. B.14 Work IQ#

dependencies
| where timestamp > ago(24h)
| extend Tool = tostring(customDimensions["gen_ai.tool.name"])
| where Tool has "workiq"
| summarize Calls = count(), Failures = countif(success == false), p95 = percentile(duration, 95)
  by Tool, resultCode, tostring(customDimensions["user.email"])
| order by Failures desc

Expect: failures concentrated on specific users points at licensing or permissions, not the connector.

90. B.15 MCP tool activity#

dependencies
| where timestamp > ago(24h)
| extend Tool     = tostring(customDimensions["gen_ai.tool.name"]),
         ToolType = tostring(customDimensions["gen_ai.tool.type"])
| where ToolType has "MCP" or Tool has "mcp"
| summarize Calls = count(), Failures = countif(success == false),
            p95 = percentile(duration, 95), LastSeen = max(timestamp)
  by Tool, ToolType, resultCode
| order by Failures desc

Expect: if a known MCP tool has zero rows, it isn’t being discovered or isn’t being selected - see A.8.

91. B.16 Authentication failures#

dependencies
| where timestamp > ago(24h)
| where resultCode in ("401", "403")
| extend Tool  = tostring(customDimensions["gen_ai.tool.name"]),
         User  = tostring(customDimensions["user.email"]),
         ConvId= tostring(customDimensions["gen_ai.conversation.id"])
| summarize Failures = count(), Users = dcount(User), Sample = any(ConvId)
  by target, Tool, resultCode
| order by Failures desc

Read it this way: many failures across many users = configuration. Many failures for one user = that user’s permissions or consent.

92. B.17 User activity#

let UserEmail = "<[email protected]>";
dependencies
| where timestamp > ago(7d)
| where tostring(customDimensions["user.email"]) == UserEmail
| extend ConvId = tostring(customDimensions["gen_ai.conversation.id"]),
         UserMsg = tostring(parse_json(tostring(customDimensions["gen_ai.input.messages"]))[0].parts[0].content)
| where isnotempty(UserMsg)
| project timestamp, ConvId, UserMsg, resultCode
| order by timestamp desc

Use when: the reporter can’t remember what they asked, which is most of the time.

93. B.18 Performance trend#

dependencies
| where timestamp > ago(7d)
| where name == "InvokeAgent"
| summarize Turns = count(),
            p50 = percentile(duration, 50),
            p95 = percentile(duration, 95),
            FailureRate = round(100.0 * countif(success == false) / count(), 2)
  by bin(timestamp, 1h)
| render timechart

Use when: establishing a baseline (do this before your next incident) or proving a fix worked.

94. B.19 Top failures across everything#

union isfuzzy=true requests, dependencies, exceptions
| where timestamp > ago(24h)
| where success == false or itemType == "exception"
| extend Signature = strcat(itemType, " | ", coalesce(name, type), " | ",
                            coalesce(tostring(resultCode), tostring(outerMessage), ""))
| summarize Count = count(), FirstSeen = min(timestamp), LastSeen = max(timestamp),
            Sample = any(operation_Id)
  by Signature
| order by Count desc
| take 25

Use when: starting a morning triage, or after a release. Run it daily and you’ll spot regressions before users do.

95. B.20 Discover the live schema#

dependencies
| where timestamp > ago(7d)
| mv-expand Key = bag_keys(customDimensions) to typeof(string)
| summarize Events = make_set(name), SampleValue = take_any(tostring(customDimensions[Key])) by Key
| order by Key asc

Use when: a documented field returns nothing. The schema evolves; this is the only source of truth that can’t go stale.

96. Appendix C - Troubleshooting decision tree#

Why the order is what it is: each stage is cheaper than the next and narrows the search for it. Config checks cost seconds. Transcripts cost a minute. KQL costs ten. Chasing an external system owner costs a day. Work left to right and you’ll rarely need the expensive end.

97. Maintenance#

This page goes stale. Copilot Studio ships changes continuously and the telemetry schema is explicitly documented as evolving.

If you fix something that isn’t in Appendix A, add it. The appendix is the part of this guide that pays for itself.