Job Control
Running, suspending, and switching between multiple commands within a single shell session - foreground and background jobs, and what happens to them when you log out.
Job control is what lets one terminal session run several commands at once without opening several terminals - starting things in the background, pausing a foreground command, and bringing things back to the foreground as needed.
Foreground vs. background
A command runs in the foreground by default, occupying the shell
until it finishes. Append & to start it in the background instead,
returning control of the shell immediately:
long-task.sh &
[1] 4821
The [1] is a job number (specific to this shell session); 4821 is the
process's actual PID (see Process Lifecycle).
Suspending and resuming
Ctrl+Z suspends whatever's running in the foreground, handing control
back to the shell without killing it - the process is paused, not
terminated:
some-command
# press Ctrl+Z
[1]+ Stopped some-command
From there:
jobs # list jobs in this shell, with their state
fg # resume the most recent job in the foreground
fg %1 # resume job number 1 specifically, in the foreground
bg %1 # resume job number 1, but in the background
kill %1 # send SIGTERM to job number 1
See Signals for what's actually happening
underneath Ctrl+Z (SIGTSTP), fg/bg (SIGCONT), and kill.
Jobs die when the shell does - unless you say otherwise
By default, a background job is tied to the shell session that started
it: closing the terminal (or losing an SSH connection) sends SIGHUP to
every job still attached to it, which terminates them. Two ways around
this:
nohup - makes a command ignore SIGHUP specifically, so it survives
the shell exiting:
nohup long-task.sh &
disown - detaches an already-running background job from the
shell's job table, after the fact:
long-task.sh &
disown
Neither of these keeps a process usable interactively after you log out - for that, you want a session multiplexer.
For anything you need to reattach to: tmux or screen
nohup and disown solve "don't kill my job when I disconnect," but
they don't let you reconnect to see its output later. A terminal
multiplexer does: it runs an entire persistent terminal session on the
server, independent of any one SSH connection - you can detach from it,
log out, log back in later, and reattach to find everything exactly as
you left it. See Terminal Multiplexers
for the full tmux/screen reference - the standard tool for
long-running interactive work (a build, a database migration, a
training job) on a remote machine, and generally preferable to nohup
for anything you might need to check back in on.