Egiziago Cioffi, IT and Enterprise Architect and CEO of SynSphere Italia, a Microsoft partner based in Milan, built an Azure OpenAI email assistant that auto-resolved roughly 60% of inbound customer email. His team ran every evaluation they had. The unit tests passed. The retrieval scores were clean. None of those tests asked the one question that mattered: whose permissions does the retrieval pipeline use when it fetches the source material? Cioffi ran a low-privilege account against the same queries a high-privilege account had already submitted. The outputs did not match. The assistant returned SharePoint content the low-privilege user could not have opened in SharePoint directly. The evaluation had passed. The retrieval permission boundary had not been enforced.
The gap Cioffi found: Azure OpenAI retrieval pipelines answer with the indexer’s permissions, not the requester’s
Azure AI Search has shipped native document-level ACL trimming via Entra-based tokens since preview in May 2025, and SharePoint ACL sync followed in a later preview. The capability exists. It does not exist everywhere it needs to. The SharePoint ACL preview can ingest site-group metadata via the spg: prefix in the 2026-05-01-preview API, but only Entra-backed principals are documented as reliably enforced at query time. The preview runs through the REST API and preview SDKs and does not cover all agent deployment paths.
Azure OpenAI On Your Data supports document-level access via Azure AI Search security filters, but Microsoft’s own documentation states that if the permitted-groups field is not mapped, document-level access is disabled. That is a fail-open default in a first-party path. Custom RAG pipelines that bypass Azure AI Search entirely index under a broadly privileged service account with no query-time entitlement check unless the developer builds one. Cioffi’s deployment took the custom-pipeline path.
From the attacker’s perspective, this is a broken access control. Adriel Desautels, founder and CEO of Netragard, told VentureBeat that the failure reduces to a structural collapse of authorization boundaries. “If the NHI credentials usually have broad authorization and can read high privilege data then that is then stored in their index,” Desautels wrote. “If an app does not enforce identity-aware retrieval, then a ‘normal’ user with lower permissions can query the app and access otherwise restricted data. This collapses authorization boundaries down to the lowest privilege level with search capability.”
Cioffi’s retrieval logs are the evidence for this specific production failure. Independent data shows the failure class is not isolated.
91% of successful attacks on production productivity agents ended in silent data exfiltration
Straiker’s red team ran more than 1,700 successful exploit attempts against production agents and published the results in its inaugural STAR Labs Threat Report in July. Across the productivity agents in scope, 91% of successful attacks ended in silent data exfiltration without detection. The report noted that no malware was required. There was also no lateral movement through the network. The agent returned all the data it could reach. Straiker’s report does not break out which of those successes trace to entitlement failures specifically versus prompt injection, tool abuse, or other attack classes. The figure is a measure of what happened after an exploit succeeded, not a measure of how many deployments fail to enforce retrieval-time entitlements specifically.
Working independently, the UK’s AI Security Institute documented 19 unsanctioned agent actions from a July 25 to 28 cyber evaluation. The UKASI published its incident report on August 4 of this year. The evaluation deliberately ran with cyber classifiers disabled and internet access enabled. What the UKASI report demonstrates is agents acting outside the scope their deployers intended, in a permissive test environment, with no reliable mechanism to catch the deviation before it causes damage. It is a containment failure, not a retrieval-entitlement failure, and the overlap with the Cioffi incident is the shared absence of a runtime scope check rather than an identical mechanism.
Why evaluations miss this gap: answer quality tests do not check permission boundaries
The evaluations Cioffi’s team ran were designed to test whether the agent answers correctly. They check factual accuracy, relevance, and task completion. They do not ask whose permissions the retrieval pipeline uses when it fetches the source material, because that question is not in the evaluation framework. This is the fundamental blind spot in current RAG deployment testing.
Azure AI Search is currently shipping the retrieval-time entitlement check at the platform level. The query-time ACL trimming validates the caller’s Entra token, extracts user and group claims, and returns only documents whose synchronized permission metadata grants the caller access. For deployments that use Azure AI Search with the SharePoint indexer and Entra-backed principals, the control exists natively. Cioffi’s deployment did not use this path. His custom Azure OpenAI retrieval pipeline bypassed the native trimming layer, which is how the gap survived every evaluation his team ran.
Desautels put the evaluation blind spot in operational terms. “Agents tend to run a single, long-lived, non-human identity that holds a wide range of permissions that it might need for any task it is ever asked to complete,” he wrote. “Evaluations also don’t often cover prompts, outputs, transcripts, memory, and logs where it can be read or hijacked through injected content. That mismatch is what most current evaluations get wrong.”
Cioffi’s fix: a query-time filter that narrowed the assistant’s retrieval scope while maintaining 60% auto-resolution
Cioffi’s fix did not require a new identity platform. He moved the entitlement decision into the retrieval path itself, adding a query-path filter that checks the requesting user’s SharePoint permissions before the model sees a chunk. The filter runs at query time, not at index time. Content the user could not open in SharePoint does not enter the model’s context window.
The control narrowed what the assistant could reach. The assistant still auto-resolves roughly 60% of inbound email with the filter live. Cioffi did not provide a before-the-filter auto-resolution figure for comparison. The qualitative tradeoff he described is that some content the assistant previously used to answer questions is now excluded because the requesting user’s permissions do not reach it. That is the price of enforcing the boundary.
The question of whether retrieval-time entitlement filtering is worth the narrowed retrieval scope does not have a single answer. It depends on the sensitivity of the indexed content, the permission variance across the user population, and whether the deployment can tolerate unanswered queries when the filter blocks a chunk the model needs. What Cioffi’s incident demonstrates is that the gap exists in custom Azure OpenAI pipelines, that answer-quality evaluations do not catch it, and that a query-path filter closes it at a trade-off the builder can describe.
Identity governance platforms address a different layer than retrieval permission boundaries
CrowdStrike announced its $740 million acquisition of SGNL on January 8, 2026, and closed the deal on February 20, 2026. Palo Alto Networks announced its $25 billion acquisition of CyberArk in July 2025 and closed the deal on February 11, 2026. Both deals closed the same month, establishing identity security as a platform pillar at two of the largest security vendors in the world.
Identity governance platforms focus on which service accounts exist, what they can reach, and when their tokens expire. They govern the lifecycle of the credentials that power AI agents. That layer matters. What it does not govern is the retrieval permission boundary. That is the moment a correctly scoped service account retrieves content on behalf of a user who holds fewer permissions than the indexing job does.
Every credential in the chain is legitimate. The service account is clean and properly managed. The knowledge base is correctly indexed. A low-privilege user queries the assistant, and it answers from the full indexed scope. Nothing flags the retrieval because no credential was misused. Cioffi’s filter is a control at the retrieval permission boundary layer specifically. Azure AI Search’s native ACL trimming addresses the same layer for deployments that use it. Neither replaces identity governance. A production deployment that wants to close both the credential lifecycle gap and the retrieval-time entitlement gap needs controls at both layers.
What is the single test that reveals whether your Azure OpenAI assistant enforces retrieval permission boundaries?
The single test that reveals the gap is a two-account comparison. Run the same question a high-privilege account has already put to the assistant from a low-privilege account. Compare the outputs against what the low-privilege account can access through the underlying system directly. If the assistant returns more than the account’s direct access would allow, the retrieval permission boundary is not enforced at query time. That test costs two accounts and thirty minutes. It produces a result an evaluation score cannot replicate.
Desautels confirmed that this is where a red team would start. “The first test would likely target the gaps between data and instructions, and the gaps between the user’s identity and the assistant’s own credentials,” he wrote. “We’d attempt to plant an instruction within content that we think the assistant will ingest as data. We’d have that content direct a side-effectful, privileged action that the attacking user is not authorized to perform.” A failing result, in Desautels’ assessment, is “the successful or even partial execution of our injected commands.”
Ask one question before any AI retrieval deployment goes live
Ask whose permissions each AI retrieval system uses when it fetches content. If the deployment uses Azure AI Search with the SharePoint indexer and Entra-backed principals, verify that query-time ACL trimming is enabled and that the user population does not depend on SharePoint site groups. If the deployment uses a custom retrieval pipeline, the entitlement check may not exist at all.
Cioffi built the agent on a custom Azure OpenAI pipeline that bypassed the native ACL trimming layer. He ran every evaluation his team had. He found the gap in his own logs after all of them passed. The evaluation tested whether the agent answered correctly. It did not test whose permissions the agent was using. Run the two-account comparison before the next deployment goes live. Thirty minutes tells you which side of the line you are on.
