$ cat ~/field-notes/linux-signals-explained.md
Linux Signals: What SIGTERM, SIGKILL, and SIGHUP Actually Do
kill is badly named. The command does not necessarily kill anything—it sends a signal. What happens next depends on the signal and the receiving process.
Understanding that distinction explains graceful shutdown, terminal shortcuts, daemon reloads, zombie processes, and a surprising number of broken Docker entrypoints.
A Signal Is a Notification
A signal interrupts a process to report an event. Each signal has a number, a name, and a default action.
kill -l # list signals on this system
kill -TERM 1234 # send SIGTERM to PID 1234
kill -HUP 1234 # send SIGHUP
kill -0 1234 # send nothing; check existence and permission
Applications can install handlers for most signals. They may clean up, reload configuration, log a diagnostic, or ignore the signal.
Two signals are special: SIGKILL and SIGSTOP cannot be caught, blocked, or ignored. The kernel enforces them directly.
The Signals You Actually Use
SIGTERM polite shutdown request; default action is termination
SIGINT interactive interrupt, usually Ctrl-C
SIGKILL immediate forced termination; cannot be handled
SIGHUP terminal disconnected; commonly repurposed as config reload
SIGQUIT terminal quit, often produces a core dump
SIGUSR1 application-defined signal
SIGUSR2 application-defined signal
SIGCHLD a child process stopped or exited
SIGPIPE wrote to a pipe/socket with no reader
SIGSTOP pause immediately; cannot be handled
SIGCONT resume a stopped process
Signal meanings are conventions layered on kernel defaults. nginx uses SIGHUP to reload configuration. Another program may ignore it or terminate.
Read the application's documentation before sending signals in automation.
SIGTERM vs SIGKILL
SIGTERM starts a conversation:
orchestrator → SIGTERM → application
application:
stop accepting work
finish in-flight requests
flush buffered data
close connections
exit
SIGKILL ends the conversation:
kernel stops scheduling the process immediately
no finally blocks
no deferred cleanup
no buffer flush
no application log saying why it stopped
This is why kill -9 should be the last step, not the first. Files are not automatically corrupted by SIGKILL, but any application-level invariants or buffered writes in progress lose their chance to finish.
A sensible shutdown sequence is:
kill -TERM "$pid"
# Wait up to 15 seconds.
for _ in $(seq 1 150); do
kill -0 "$pid" 2>/dev/null || exit 0
sleep 0.1
done
kill -KILL "$pid"
Supervisors and container orchestrators implement the same pattern with a configurable grace period.
Handling Signals in a Shell Script
#!/usr/bin/env bash
set -euo pipefail
child_pid=''
shutdown() {
echo 'forwarding shutdown to child'
if [[ -n "$child_pid" ]]; then
kill -TERM "$child_pid" 2>/dev/null || true
wait "$child_pid"
fi
}
trap shutdown TERM INT
./server &
child_pid=$!
wait "$child_pid"
Without the trap, the shell can receive SIGTERM while the actual server keeps running. Signal forwarding is one reason wrapper scripts deserve more care than a one-line server & wait.
If the script does not need setup or teardown, replace the shell entirely:
exec ./server
exec makes the server inherit the script's PID, so signals go directly to it.
PID 1 Is Different
Inside a container, the configured command usually becomes PID 1. PID 1 has two extra responsibilities:
- Reap orphaned child processes so they do not remain zombies
- Handle or forward signals to the process tree
This Dockerfile form starts the program directly:
# Good: exec form, server receives signals
CMD ["node", "server.js"]
This form inserts a shell:
# Risky: /bin/sh is PID 1 and may not forward the signal
CMD node server.js
Use exec-form commands. If the application creates child processes and does not reap them, use a tiny init process such as tini or the runtime's init option.
Process Groups Matter
Sending a signal to one PID does not automatically signal its children.
# Signal one process.
kill -TERM 1234
# Signal the entire process group whose ID is 1234.
kill -TERM -- -1234
Build tools, test runners, and shell pipelines often create process trees. Killing only the parent can leave workers holding ports and temporary files. Supervisors normally manage a process group or cgroup for this reason.
SIGHUP and Configuration Reloads
Traditional daemons were attached to a terminal. When that terminal disappeared, the kernel sent SIGHUP—hangup. Daemons later adopted the signal as a reload convention.
A safe reload looks like this:
receive SIGHUP
read new configuration
validate the entire configuration
if valid: atomically replace current config
if invalid: log error and keep old config
Do not partially apply a broken configuration. And remember that environment variables cannot be changed inside an already running process by editing the parent's environment; a reload must read a file or another external source.
SIGCHLD and Zombie Processes
When a child exits, the kernel keeps a small status record until the parent calls wait(). That finished-but-uncollected entry is a zombie.
ps -eo pid,ppid,state,command
# State Z means zombie.
SIGCHLD tells the parent that child state changed. A server that forks children must reap them. Killing the zombie does nothing—it is already dead. Fix or restart the parent that failed to call wait().
Debugging Signals
# Watch signal-related system calls for an existing process.
strace -f -e trace=signal -p 1234
# Show signal masks and handlers exposed by the kernel.
cat /proc/1234/status | grep '^Sig'
# Ask a process to dump state if the application defines SIGUSR1 for that.
kill -USR1 1234
Never assume SIGUSR1 is harmless. "User-defined" means the application decides what it does.
The Bottom Line
Signals are a tiny control plane for Unix processes.
The rules:
- Send SIGTERM first and allow a bounded grace period
- Use SIGKILL only when the process will not cooperate
- Use
execin simple entrypoint scripts - Forward signals when a wrapper owns child processes
- Use exec-form container commands
- Reap children when running as PID 1
- Treat SIGHUP and SIGUSR signals as application-specific APIs
- Signal process groups when you intend to stop a whole tree
Once you see kill as "send a process event," the commands and failure modes make much more sense.