Spot the mistakes
Guess first, no penalty. You asked for "accounts with more than 20 failed sign-ins from one IP in the last hour". The AI's KQL query runs. Tap the line that makes it count the wrong thing.
The query, line by line
0 of 1 found
An event ID is the number Windows gives each kind of security event. Microsoft's reference pages name them exactly. Event 4625 is "An account failed to log on", and the page says it "is logged for any logon failure". Event 4624 is "An account was successfully logged on".
One digit sends a query after normal activity instead of an attack. It still runs, and it still returns results: the accounts that sign in a lot. That looks like an answer, which makes it worse than an error.
The same page lists status codes that tell you why a logon failed. 0xC000006A means a bad password. 0xC0000064 means a misspelled or bad username, and Microsoft notes that several in a row "can be a sign of a user enumeration attack" (someone guessing which accounts exist).
So check every event ID and value an AI uses against the vendor's reference page, before you trust a single result.
A worked example
Fixed query:
SecurityEvent
| where TimeGenerated > ago(1h)
| where EventID == 4625
| summarize Attempts = count() by Account, IpAddress
| where Attempts > 20
Check note: 4625 = "An account failed to log on" (Microsoft event reference).Fill the gaps
Fill the gaps.
Check note
Failed logons are event , per . A run of status 0xC0000064 can mean someone is .
Try it
Your steps, about 2 minutes
- 1.Pick one query you use and look up every event ID in it on Microsoft's reference pages. Add "check every event ID against the vendor reference" to your checklist.2 min