Building an AI-Assisted Incident Investigation Workflow

Incident investigation in Microsoft 365 environments is rarely blocked by a lack of telemetry. Audit logs, sign-in logs, mailbox activity, Defender alerts, Purview events — the data is there. The slow parts are stitching it together into a narrative, deciding what is relevant, and writing it down in a form that other humans (and regulators) can follow.

AI assistance is well suited to those slow parts. It does not, and should not, replace the human responder; it removes the friction that makes investigations take days instead of hours. This article outlines a workflow pattern we have been developing as part of CLEVERINA Incident Assistant (currently in Preview).

1. Triage with structured prompting, not free-form chat

Useful AI assistance starts from a structured input: an alert, a user identifier, a time window. The model summarizes recent activity for the entity, highlights anomalies relative to that entity's baseline, and proposes a small number of investigative questions. Free-form 'tell me what is happening' prompting produces narrative but rarely actionable triage.

2. Evidence gathering with explicit provenance

Every piece of evidence cited by the assistant should be linked back to its source — the log query, the timestamp, the record ID. An investigation narrative without provenance is not auditable. The pattern we use is: the assistant proposes a query, the system runs it, the results are stored, and the narrative references the stored result by ID. Hallucinated evidence is the single biggest risk in this category and provenance is the mitigation.

3. Narrative drafting with human sign-off

Once evidence is gathered, a draft narrative is genuinely useful: a chronological reconstruction of what happened, with cited evidence, suitable as the starting point for an incident report. The analyst reviews, edits and signs off. The assistant does not finalize anything — every output is a draft until a named human approves it.

4. Suggested actions, never executed actions

The assistant can propose containment actions — disable account, revoke sessions, block sender — but the execution path stays in the existing tooling with the existing approvals. Autonomous remediation is not appropriate at this maturity level and we deliberately do not design for it.

5. Audit trail by default

Every prompt, every model response, every action taken on the assistant's suggestion is logged. The audit trail is part of the deliverable — both for internal review and for any post-incident regulatory reporting.

An anonymized example

A simulated business-email-compromise scenario in a test tenant: a user clicks a phishing link, a malicious inbox rule is created, and outbound mail is forwarded to an external address. With an assistant following the pattern above, the analyst received within minutes a triage summary identifying the inbox rule, a chronological reconstruction citing the relevant audit log entries, and a suggested containment sequence. The analyst then executed containment through the existing Defender and Entra workflows. Time-to-narrative dropped from an estimated two to three hours of manual log review to roughly fifteen minutes of analyst time — none of which was spent typing the report from scratch.

Practical recommendations

• Design the workflow around evidence provenance from day one.
• Treat every AI output as a draft requiring human approval.
• Keep autonomous remediation out of scope until the pattern is proven across many incidents.
• Log everything — prompts, responses, actions — for audit and improvement.

Conclusion

AI-assisted investigation is most valuable when it is narrowly scoped, evidence-grounded and human-supervised. That is the pattern we are building into CLEVERINA Incident Assistant — currently in Preview / MVP, not production-ready. If the direction is relevant to your environment, the product page outlines where it is heading and the roadmap covers what comes next.