Loading page…
Its internal review shows the difficulty of separating cyber misuse from legitimate engineering. The findings are a development baseline, not a production accuracy estimate.
Listen to this story
Snowflake’s review points to a design requirement for agent security monitoring: assess intent, authorization and effects across an ordered session, rather than classify isolated tool calls. In its selected 610-session cohort, reviewers confirmed 21 detections, marked two as candidates and left 26 unresolved; the remaining 561 were authorized engineering or security work. Snowflake cautions that these figures are neither production-accuracy results nor a measure of misuse prevalence, and says the findings will inform future Cortex AI Gateway integration.
The detector targets cyber misuse attributable to a user, not general agent misalignment or errors on legitimate objectives.
Snowflake froze data, prompts, model routes and policy settings for each run, preserving raw responses and evidence links for review.
Privacy safeguards include session-specific aliases, automated redaction and blocking a session if a final scan finds residual sensitive data.
Of 610 AI-agent sessions flagged by Snowflake’s cyber-misuse detector, the company confirmed 561 as authorized engineering and security work. Its newly published findings show why checking an agent’s actions requires more than spotting suspicious techniques: reviewers also examined the original request, authorization and outcome before deciding what the activity meant.
Snowflake evaluated the detector’s sequence-analysis layer using a frozen snapshot of internal data. The review separated confirmed detections from possible cases, unresolved activity and authorized operations, rather than treating every alert as a security incident.
Much of the legitimate activity looked risky at the level of individual actions. It included defensive security reviews, environment troubleshooting, permission checks, telemetry analysis and optimization code. Snowflake says the mechanics of an action—and the model’s confidence in its judgment—were poor substitutes for understanding the surrounding context.
The confirmed cases included user-directed misuse and agent actions that exceeded the apparent scope of a request. Some attempts were blocked; others had observed effects. Examining actions in order helped reviewers distinguish who initiated an objective, whether it stayed within authorized boundaries and whether anything consequential happened.
These results are not a production accuracy or misuse-prevalence estimate. The detector selected the cohort, reviewers used a conservative rubric, and some cases remained unresolved. Snowflake describes the exercise as a baseline before production enforcement that would intervene during execution, not a measurement of performance across all agent sessions.
The system reconstructs an agent trajectory: an ordered sequence of objectives, decisions and actions. Its underlying trace contains messages, tool calls, results, timestamps and metadata. Findings are evaluated for the whole session, allowing evidence spread across a request, a plan and several tool interactions to inform the judgment.
Its stated target is narrower than every way an agent might go wrong. Snowflake says the current detector focuses on cyber misuse attributable to a user. It is not designed as a general-purpose detector for misalignment, where an agent exceeds or misreads an otherwise legitimate objective.
Snowflake’s privacy controls keep operational identity separate from the material sent to the model. The system uses validation, session-specific aliases and automated redaction. Authorized reviewers can still access identity for routing and investigation. If a final scan finds residual sensitive data, the session fails rather than sending partial input.
Long sessions are divided at event boundaries, with limited overlap between chunks. Original event identifiers remain attached so reviewers can trace a finding back to its evidence. For each run, Snowflake freezes the dataset, prompts, model routes and policy settings, while preserving raw responses and the links from evidence to results.
A classifier finding is an input to investigation, not a conclusion about a person, authorization, intent or misconduct.
Snowflake, engineering blog
Selected findings move from triage to a separate adjudication stage. Model errors, malformed output and inconsistent fields remain visible as failures rather than being classified as benign. Snowflake says a Gateway-integrated detector must likewise expose dropped inputs, empty responses and parsing errors, with retryable failures queued and exhausted work recorded as exceptions.
Reviewers can confirm a detection, identify authorized activity or request more context. Every decision requires a rationale and is recorded without overwriting earlier entries. Results from multiple chunks are combined into one session result with surrounding context preserved, rather than leaving reviewers to judge isolated fragments.
Integration with Cortex AI Gateway remains a future focus. The review is guiding that work toward preserving sequence and making ownership, authorization, requested scope and effects central evidence before escalation. Snowflake also plans to convert high-confidence findings into repeatable evaluation cases while reducing noise and balancing coverage, review speed and privacy.
Monitoring is not intended to replace hard access boundaries. Snowflake says it complements sandboxing, minimum necessary permissions, narrowly scoped tools, network controls and approval requirements. Those controls restrict what an agent can do; trajectory analysis connects objectives, actions and effects when a fixed rule cannot interpret a security-sensitive action on its own.
Loading discussion...
Join the conversation
Explain when automatic blocking would be justified, or when review should come first.
Be the first to share a perspective or an experience.
Reader comments
Newest comments first. Replies stay oldest first.