Linux and the terminal for AI work
The terminal is where you create environments, install packages, run your app, use Git, start Docker, check logs, connect to servers and run coding agents like Claude Code. You don't need to be a Linux administrator, but you do need about 40 commands for files, processes, ports, logs, networking and long-running jobs. Almost every server your AI application runs on is Linux, so learning these on your own machine pays off the first time you deploy.
Why the terminal matters for AI engineers
There are two ways to work with a computer. One is a graphical user interface, where you click buttons, open windows and drag files. The other is a command line interface (CLI), where you type commands into a terminal.
Developers use the command line because it is faster, there are no graphics to load, it can be automated in scripts, and it works the same way on your laptop and on a server that has no screen at all. When you deploy an AI application, you will be on a Linux server, inside a Docker container, or in a CI pipeline, and the terminal is the only interface you get.
It is also where the newest tools live. Coding agents like Claude Code run in the terminal, and so do Git, Docker, pip, uv and most cloud CLIs.
Which terminal to use
- macOS: the built-in Terminal app or iTerm2. macOS is Unix-based, so most Linux commands work the same way.
- Linux: whatever terminal comes with your distribution.
- Windows: install WSL (Windows Subsystem for Linux), which gives you a real Ubuntu environment inside Windows. Most AI tooling, tutorials and servers assume Linux, and WSL saves you from a lot of Windows-only problems. Git Bash is a lighter option if you only need Git.
VS Code has a built-in terminal (Terminal, then New Terminal), and you can open several side by side, for example one running your API server and another for Git commands.
Moving around and working with files
pwd
ls
ls -la
cd projects/rag-demo
cd ..
cd ~
mkdir -p data/raw
touch notes.md
cp config.json config.backup.json
mv old_name.py new_name.py
rm notes.md
rm -r build/
pwd prints where you are, ls lists what is there (-la includes hidden files like .env and .gitignore, with details), and cd changes the directory.
~ is your home folder and .. is the folder above the current one.
Be careful with rm, especially rm -r.
There is no recycle bin in the terminal, and a deleted file is gone.
Reading files and logs
cat requirements.txt
less app.log
head -n 20 data.jsonl
tail -n 50 app.log
tail -f app.log
wc -l eval_set.jsonl
grep -i "error" app.log
grep -rn "OPENAI_API_KEY" .
tail -f follows a log as new lines are written, which is how you watch a running server.
grep searches inside files, and grep -rn searches a whole folder with line numbers, which is also a quick way to check that no key is hard-coded anywhere in your project before you push.
wc -l counts lines, which for a JSONL file is the number of records.
Environments and packages
python3 --version
which python
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
pip list
deactivate
which python tells you which Python the terminal is using, and it is the first command to run when a package is "installed" but your code can't find it, because usually it was installed into a different environment.
Your system also has a package manager for tools outside Python: apt on Ubuntu, brew (Homebrew) on macOS.
Environment variables
Environment variables are settings that live in the terminal session, outside your code. API keys, model names and configuration are usually passed this way.
export MODEL_NAME="gemini-2.5-flash-lite"
echo $MODEL_NAME
env | grep MODEL
A variable set with export exists only in that terminal session.
For project settings, use a .env file instead, as the secrets page explains.
Processes and ports
When your API server is running, it is a process listening on a port, like 8000. Two common problems are a server that keeps running after you close something, and a port that is already in use when you start a new server.
ps aux | grep uvicorn
lsof -i :8000
kill 12345
kill -9 12345
top
lsof -i :8000 shows which process is using port 8000, and kill followed by its process ID stops it (-9 forces it, when a normal kill doesn't work).
top, or htop if you install it, shows what is using your CPU and memory right now.
To stop the program running in the foreground of your terminal, press Ctrl+C.
Testing APIs with curl
curl sends HTTP requests from the terminal, which is the quickest way to test your own API or check that a provider's API responds.
curl http://127.0.0.1:8000/health
curl -X POST http://127.0.0.1:8000/ask \
-H "Content-Type: application/json" \
-d '{"text": "How many leave days do I get?"}'
Install jq to format JSON responses so you can read them: add | jq at the end of a curl command.
Disk space
Models, datasets, Docker images and vector indexes fill disks quickly, and a full disk causes strange errors.
df -h
du -sh *
du -sh ~/.cache/huggingface
docker system df
df -h shows free space on each disk, and du -sh * shows the size of each item in the current folder.
Hugging Face downloads models into a cache folder in your home directory, and that folder alone can grow to tens of gigabytes.
Connecting to servers with SSH
When you deploy to a virtual machine, you connect to it with SSH (secure shell), which gives you its terminal on your machine.
ssh-keygen -t ed25519 -C "you@example.com"
ssh -i ~/.ssh/my-key.pem ubuntu@203.0.113.10
scp model.joblib ubuntu@203.0.113.10:/home/ubuntu/app/
ssh-keygen creates a key pair, you add the public key to the server or your cloud account, and you keep the private key on your machine.
Never share or commit the private key.
scp copies files to and from the server over the same connection.
Checking the GPU
If you work with local models or fine-tuning on a machine with an NVIDIA GPU, nvidia-smi shows the GPU, its memory usage, and the processes using it.
nvidia-smi
watch -n 1 nvidia-smi
When a training run crashes with an out-of-memory error, this is where you see how much memory was in use and what else was using the GPU.
Running long jobs safely
If you start a 3 hour embedding job over SSH and your connection drops, the job dies with it. There are two common ways to prevent that.
tmux creates terminal sessions that keep running on the server after you disconnect.
tmux new -s embeddings
python build_index.py
Press Ctrl+B, then D to detach, close your laptop, and later run tmux attach -t embeddings to come back to it.
nohup runs a command that ignores the disconnect, and sends its output to a file.
nohup python build_index.py > index.log 2>&1 &
tail -f index.log
For anything that should run permanently, like your API, don't use either of these. Use Docker with a restart policy, a process manager, or a managed platform that restarts the app for you.
Permissions
chmod +x run.sh
./run.sh
chmod 600 ~/.ssh/my-key.pem
chmod +x makes a script executable.
SSH refuses to use a private key that other users can read, so chmod 600 on the key file is often the fix when SSH complains about permissions.
Shell scripts for repeated work
When you type the same 5 commands every day, put them in a script.
#!/usr/bin/env bash
set -e
source venv/bin/activate
pip install -r requirements.txt
uvicorn app.main:app --reload --port 8000
set -e stops the script at the first command that fails, so you don't end up running the server with half the dependencies installed.
From the floor
In the first session of AI Engineer HQ cohort 2, before any AI topic, I showed the command line interface, why developers prefer it over clicking through an interface, and how to open several terminals inside VS Code.
Then we installed Claude Code from the terminal, created the session folders with cd and ls, and set up a virtual environment, because everything we built after that started from the terminal.