Every earlier chapter was about keeping attackers out. This one starts from the honest assumption that sometimes they get in anyway, and asks the next question: how would you know? The answer is not "collect more logs". It is collecting the right ones, keeping them honest, writing rules that fire on behavior rather than trivia, and doing the arithmetic that tells you whether a human can actually keep up with what those rules produce.
Common mix-up: a log is not a detection, and an ATT&CK cell colored green is not protection. Logs are a record; detection is someone or something reading the record in time. And a mapped technique only means a rule exists for it, not that the rule would catch the next attacker who uses it.
Read first: this chapter assumes you are comfortable reading the evidence a network leaves behind: packet captures, DNS lookups, connection tables. If not, start with The troubleshooting toolkit on networks.jelia.nyc. Here we turn that evidence into something that watches on its own.
A building can have a camera in every corridor and still be robbed in plain view, because recording is not watching. Most organizations that discover a break-in late already had the evidence on disk. What they lacked was the right evidence in one place, with honest timestamps, and something reading it.
So the first decision is not which tool to buy. It is which questions you will need to answer on the worst day, and which logs answer them. A useful test for any event is whether it records who did it, what they did, when (with a time zone), where (which host, from which address) and the outcome (success or failure). An event missing any of the five is much less useful when you need it.
Six sources give the most detection value for the effort, and appear in some form in almost every national guide on the subject:
| Source | Answers the question | What a good event contains |
|---|---|---|
| Authentication | Who logged in, from where, and did it work? | Account, source address, method (password, key, passkey, token), result and reason for failure. Failures matter as much as successes. |
| Process creation | What ran on this machine, and what started it? | Program, full command line, parent process, user, a hash of the file. The parent is what turns "PowerShell ran" into "a word processor launched PowerShell". |
| Network flows | Who talked to whom, how much, for how long? | The 5-tuple, bytes each way, start and end time. Not the content. Firewall logging itself is set up in Chapter 5. |
| DNS queries | Which names did each machine look up? | Client, name asked for, type, answer. Almost every outbound connection starts with a lookup, so this is a cheap map of intent. |
| Admin actions | Who changed the rules? | New accounts, group changes, permission grants, policy edits, and anything that turns logging off. |
| Cloud audit logs | Who called which API on which resource? | Caller identity, API action, resource, source address, result. In the cloud the control plane is the network, so this is often the most important log you have. |
Two things should stay out of logs. The first is secrets: passwords (even wrong ones, which are often a real password with one typo), session tokens, API keys, full card numbers. A log is copied to more places and kept longer than almost anything else, which makes it the worst possible place for a credential. The second is personal data you do not need. Log the account ID rather than the person's home address, and decide in advance who may search the logs, because a searchable record of everyone's activity is itself sensitive.
# illustrative: a log line that leaks and tells you little Login failed for jsmith, password was Summer2026x
# illustrative: structured, five Ws, no secret
{"time":"2026-10-22T09:52:14.120Z","event":"auth","host":"dc-01",
"user":"jsmith","src":"203.0.113.45","method":"password",
"outcome":"failure","reason":"bad_password"}
Syslog is the oldest common format for shipping log events between machines, and RFC 5424 (2009) is its current definition. Every message starts with a priority value in angle brackets, which packs two numbers into one: a facility (which part of the system is talking, 0 to 23) and a severity (0 Emergency to 7 Debug).
After the priority come a version number, a timestamp in the RFC 3339 format with an explicit offset, the host name, the application name, a process ID, a message ID, optional structured data (named key-value blocks in square brackets), and finally the free-text message. The structured part is what saves you later: a parser can read [auth outcome="failure"] without guessing, while free text needs a regular expression for every product's wording.
RFC 5424 also sets a floor on size: every receiver must accept messages up to 480 octets, should accept up to 2,048, and may accept more. Anything longer risks being cut, which matters when a command line is the most interesting part of the event.
An intrusion is a story told across many machines: a login on the domain controller, a process on a laptop, a lookup on the DNS server, a connection through the firewall. Read separately, each line looks routine. Read in order, on one timeline, they become a sentence. Everything in a detection pipeline exists to make that reading possible.
Collect. Get each event off the machine that produced it as soon as possible. A log that stays on a compromised host can be edited or deleted by whoever compromised it; one already forwarded cannot. Collection is usually an agent on each host, syslog from network gear, and API pulls from cloud services.
Normalize. One product calls it src_ip, another ClientAddress, a third puts it inside a sentence. Normalizing maps every source onto one schema, so that a rule written once for "source address" works across all of them. It is also where timestamps are converted to UTC.
Store. Keep events searchable for as long as you might need to look back. Attackers are often discovered weeks or months after they arrived, and the first question in incident response is always "when did this start?" If your logs are only kept for seven days, the honest answer is "we cannot know".
Detect, then triage. Rules run over the stream and raise alerts; a person looks at each alert and decides whether it is real. The arrow back from triage to detect is the most important arrow in the diagram, and the rest of this chapter is mostly about it.
A timeline is only as honest as its clocks. If the laptop runs three minutes slow and the domain controller two minutes fast, the process launch appears before the login that made it possible, and the story reads backwards. The fix is boring and essential: every device synchronizes to the same time sources with the Network Time Protocol (NTPv4, RFC 5905), ideally authenticated with Network Time Security (RFC 8915) so an attacker cannot simply move your clocks, and every event is stored with an explicit offset or in UTC. Scene 2 of the walkthrough above shows what a few minutes of skew does to a story.
Attackers know logs exist. ATT&CK has a whole tactic, Defense Impairment, for interfering with defenders' tools, including clearing Windows event logs. Defenses follow directly: forward events off-host immediately, over an authenticated, encrypted channel (RFC 5425 defines syslog over TLS); store them somewhere the source machine has no rights to modify, such as write-once storage; and alert on the absence of logs as well as their contents. A log source that suddenly goes quiet is an event.
Write-once storage stops changes; a hash chain lets you prove no changes happened. For each new record, compute a cryptographic hash (Chapter 2) over the record plus the previous record's hash, and store the result alongside it. Change, delete or reorder any record and every hash after it stops matching. Publish or separately store the latest hash now and then, and you have a checkpoint that an attacker on the logging host cannot quietly rewrite.
Anything an outsider can type may end up in a log: a user name, a URL, a browser's User-Agent header. If the logging code pastes that text in raw, an input containing a line break can start a second, fake line. MITRE classes this as CWE-117, Improper Output Neutralization for Logs. It is the same family of mistake as the injection bugs in Chapter 13: data crossing into a format where some characters have meaning.
# illustrative: raw concatenation; a user name containing a newline # can make the second line below appear to be a real event 09:52:14 login failed for user=guest 09:52:14 login succeeded for user=admin
# illustrative: structured logging encodes the input as data
{"time":"...","event":"login","outcome":"failure","user":"guest\nlogin succeeded..."}
The fix is the same as for every injection bug: never build the format by gluing strings. Use a structured logger that writes JSON or key-value fields and escapes control characters, and have the viewer render data as text. The OWASP Logging Cheat Sheet also recommends validating and length-limiting what goes in.
Detection rules come in three broad kinds, and the difference between them is mostly about what the attacker has to change to slip past.
Indicator matching looks for exact things already known to be bad: the hash of a malware file, an IP address, a domain name. These rules are precise and cheap, and they are brittle. Change one byte of the file and its hash is new; rent a new server and the address is new. Indicators are worth matching because they are nearly free, but they describe yesterday's attack.
Behavioral rules look for what the attacker has to do, whatever tools they use: one source failing against many accounts, a document editor starting a command shell, a machine contacting the same name at clockwork intervals. Changing behavior is expensive for the attacker, so these rules last longer. They are harder to write because legitimate software does odd things too.
Anomaly detection learns what normal looks like and flags departures from it: a user who has never logged in at 3 a.m., a server that suddenly sends ten times its usual traffic. It can catch things nobody wrote a rule for, and it is noisy, because most unusual things are harmless. New is not the same as bad.
David Bianco drew this idea as the Pyramid of Pain: the higher up the kind of indicator you detect on, the more it costs the adversary to change, and so the more pain your detection causes them. Hashes at the bottom are trivial to change; their tactics, techniques and procedures at the top are the hardest.
| Kind | Example | Strength | Weakness |
|---|---|---|---|
| Indicator | Alert on any lookup of a domain on a threat-intelligence list | Almost no false alarms; trivial to run | Misses anything not already on the list; goes stale in days |
| Behavioral | Alert when a document editor starts a command interpreter | Survives new tools and new infrastructure | Needs tuning: some legitimate software behaves the same way |
| Anomaly | Alert when an account's login country has never been seen before | Catches the unanticipated | Unusual is not malicious; high volume without context |
Suppose you have a detector that catches 99% of attacks and is wrong about only one benign event in a thousand. That sounds excellent. Now point it at a network that produces a million events a day, ten of which are malicious.
It catches 9.9 of the 10 bad events on average. It also flags 0.1% of the 999,990 good ones: about 1,000 false alarms. So of roughly 1,010 alerts a day, about 10 are real. An analyst who opens an alert from this detector should expect it to be false 99 times out of 100. At five minutes per alert, that is 84 hours of triage a day, every day, to find ten things.
This is the base-rate problem: when the thing you are looking for is rare, the false-alarm rate matters far more than the detection rate, because it is multiplied by the huge number of innocent events. It is the same arithmetic that makes screening a whole population for a rare disease produce mostly false positives. Bayes' theorem puts it in one line:
The way out is not a slightly better detector. It is shrinking f by orders of magnitude, or raising p by looking only where attacks are concentrated: score a narrower slice of events, combine several weak signals into one alert, or plant tripwires where innocent events essentially never happen. Try it below; the presets reproduce the example above and three ways out of it.
Two numbers describe a rule. Recall (sensitivity) is the share of real attacks it catches. Precision is the share of its alerts that are real. They pull against each other: loosen a rule to catch more and you usually flag more innocent events too. Accuracy, the share of all events classified correctly, is nearly useless here, because a rule that never fires at all is 99.999% accurate on a network where one event in 100,000 is malicious.
In practice, teams track alert volume per rule, the share of each rule's alerts that analysts marked real, and the time from event to alert to decision. A rule whose alerts are almost never real is not neutral: it trains people to click "close" without reading, which is how the one real alert gets closed too. That erosion is called alert fatigue.
Three standard ways to raise precision without losing much recall: narrow the population (run the rule only on servers, or only on admin accounts); require two signals (many failures and then a success; a rare parent process and then an outbound connection); and aggregate low-confidence findings into a risk score per user or host, alerting only when the total crosses a line.
MITRE ATT&CK is a public knowledge base of how real intrusions have unfolded, organized as a matrix. The columns are tactics: the adversary's goal at that moment, the why. "Credential Access" is a tactic. The cells are techniques: the how. "Brute Force" is a technique, and "Password Spraying", trying one common password against many accounts, is a sub-technique of it, written T1110.003. Each technique page lists the procedures seen in the wild, the data sources that can reveal it, and mitigations.
The Enterprise matrix currently has fifteen tactic columns:
Its best use for a defender is as a map of coverage. List your detection rules, tag each with the techniques it is meant to catch, and color the matrix. Gaps appear at a glance: perhaps you have six rules for Initial Access and nothing at all for Command and Control. That picture is how a team decides what to build next, and it is a common language for talking to other teams, vendors and auditors.
Its worst use is as a checklist to "complete". A colored cell says a rule exists. It does not say the rule fires for every procedure under that technique, that its log source is actually being collected on every host, or that anyone triages its alerts. Many techniques, such as Valid Accounts, look exactly like normal work and will never be fully covered by any rule. Treat the matrix as a question generator ("could we see this here?") and test the answers, rather than as a score.
A companion project, MITRE D3FEND, catalogs the defensive side: countermeasure techniques such as Authentication Event Thresholding, Process Lineage Analysis, DNS Traffic Analysis and Decoy User Credential, each linked to the offensive techniques it addresses. ATT&CK tells you what to look for; D3FEND names the ways of looking. Choosing which techniques matter most to you in the first place is the work of threat modeling.
Every SIEM has its own query language, which used to mean every good detection had to be rewritten for every product. Sigma is an open, vendor-neutral format for detection rules, written in YAML, that converters translate into each product's query language. A rule has three main parts: logsource says which kind of log it reads; detection holds named selections of field values and a condition that combines them; and metadata (title, status, level, ATT&CK tags) explains what it is for. Lists inside a selection mean OR; fields side by side mean AND.
Counting and ordering across events is handled by a separate correlation rule that references base rules by name. Here is a harmless, illustrative pair that looks for one source address failing against many different accounts in ten minutes, the signature behavior of password spraying:
# illustrative Sigma rules (two documents in one file)
title: Failed logon
name: failed_logon
status: experimental
logsource:
product: windows
service: security
detection:
selection:
EventID: 4625 # "an account failed to log on"
condition: selection
level: low
---
title: One source failing against many accounts (possible password spraying)
status: experimental
correlation:
type: value_count # count DISTINCT values, not events
rules:
- failed_logon
group-by:
- IpAddress
timespan: 10m
condition:
field: TargetUserName
gte: 8
tags:
- attack.credential-access
- attack.t1110.003
level: high
Notice the choice of value_count on the account name. A rule that counts failures per account catches someone hammering one password, and misses spraying entirely, because spraying tries each account only once or twice to stay under lockout limits. Sigma's temporal_ordered correlation type can then require that the spraying be followed by a successful logon from the same address, which is the version worth waking someone up for. The log hunt below lets you feel that difference.
Treating rules as code, kept in version control, reviewed and tested, is called detection engineering. Each rule goes around a loop:
Two steps are easy to skip and expensive to skip. Check the data means confirming the log source actually reaches the SIEM from every host the rule assumes; a perfect rule over a log nobody collects never fires. Retire means deleting rules whose alerts are never real or whose threat has moved on. A rule set that only grows ends up as the 99%-false detector from the calculator above.
Below is a morning of made-up logs from a small office: 480 lines from 08:00 to 12:00, covering authentication on the domain controller (dc-01), process launches on workstations, DNS lookups, and the edge firewall. All hosts are under corp.example.com; addresses come from the documentation ranges. Four incidents are planted in it, each a real ATT&CK technique. So is plenty of innocent weirdness: people who mistype passwords, a backup account with a stale password, an administrator who uses PowerShell, an update checker that phones home every five minutes, and a colleague who flew to Boston.
Write rules, run them, and see what you catch and what you drown in. The Naive rules are loaded to begin with: each one is a reasonable first idea, and between them they raise nine false alarms. Look at the purple lines, edit the rules, and press Run. Tuned rules shows one answer.
Filters use a tiny language: space-separated conditions, all of which must hold. field=value, field!=value, and field~text (contains). Commas mean "any of": image=powershell.exe,cmd.exe. Fields: type (auth, proc, dns, fw), host user src geo outcome parent image query dst dport action.
| Rule | Alerts | True | False | Caught (ATT&CK) |
|---|---|---|---|---|
| Run the rules to see results. | ||||
Coverage: tactics where at least one planted technique was caught by your rules.
Green bar and ✓: line is part of a true alert, with the technique. Purple bar and ✗: line only took part in false alarms. A log line can belong to several alerts.
Password spraying (T1110.003, Credential Access). One outside address tries one password against many accounts, one attempt each, then gets in as one of them. A per-account failure count never fires; a count of distinct accounts per source does. In the naive set the spraying is still found, by accident: the travel rule notices the sprayed account logging in from an unknown location. Two independent weak signals pointing at the same account are worth more than either alone. Defenses beyond detection live in Chapter 4: MFA and passkeys make the guessed password useless on its own.
Impossible travel (T1078, Valid Accounts). The same account logs in from New York and, thirty minutes later, from Singapore. The rule is "two successes, same user, different location, closer together than any flight". Set the window too wide and the colleague who really did go to Boston becomes an alert. Real systems compute distance and speed; VPNs and mobile networks make this noisier than it sounds.
A rare parent and child (T1204.002 and T1059, Execution). A word processor starting PowerShell is rare in normal work and common when a malicious document runs. Alerting on PowerShell alone flags every administrator; alerting on the parent alone flags printing. The pair is the signal.
Beaconing (T1071.004, Command and Control). Implanted software checks in with its controller on a timer, here by looking up a name every two minutes. Humans are irregular; timers are not, so a low spread in the gaps between lookups is the tell. The rule measures jitter as the standard deviation of the gaps between lookups, in seconds. The legitimate update checker is just as regular, and the tuned rule excludes a known, inventoried name rather than tightening the jitter. Real implants often add random jitter, which is why production rules also look at how rare a name is across the whole network.
The calculator's last preset points at the cheapest way out of the base-rate trap: put something where no legitimate person or program has any reason to go, and alert when anything touches it. Because innocent events at that spot are close to zero, the false-positive rate is close to zero, and nearly every alert is real. These are called honeytokens or canary tokens, after the canaries miners carried to warn of bad air.
Canaries do not replace rules; they only catch an attacker who wanders past them. But they are cheap, they work against tools and techniques nobody has seen before, and they convert the base-rate problem into its opposite. Three practical cautions: make them believable, make sure the alert reaches a person (a canary that emails an unread mailbox is decoration), and inventory them, so your own backup software, antivirus scanner or auditor reading every file does not trip them. A tripwire that fires every night on the backup job has become a normal rule, with a normal rule's noise.
What happens after a real alert, containing it, investigating, recovering and learning, is the subject of the next chapter, Incident response & recovery.
event_count, value_count, temporal_ordered), and the Sigma specification.