Deep Learning in Practice

By the end of this guide you will be able to build a neural network in PyTorch, feed it data correctly, train it with the recipes researchers actually use, figure out what's wrong when it won't learn, and adapt a pretrained vision, language, or graph model to your own research question. This is the third guide in the Vector Research machine learning track. It assumes you've worked through Python for Machine Learning and Machine Learning Foundations, or know what they teach.

What you'll be able to do when you finish

• Decide whether a project needs deep learning at all, and explain to a judge why you chose it.
• Work with tensors, shapes, devices, and automatic differentiation, and explain backpropagation with a working example.
• Write custom PyTorch models, data pipelines, and a complete training loop with validation, checkpointing, and early stopping.
• Apply the standard training recipes: learning-rate schedules, normalization, regularization, augmentation, mixed precision, and gradient accumulation.
• Debug a network that won't train, using a systematic checklist instead of guesswork.
• Fine-tune pretrained CNNs, vision transformers, and language models, including with LoRA, and build graph neural networks and generative models.
• Train detectors and segmenters, classify audio, label every word in a text, combine images with other data, and shrink a model to run on a phone or small computer.
• Run experiments on free cloud GPUs that someone else can reproduce.

How this guide fits with the others

Machine Learning Foundations, Chapters 13 and 14, introduced neural networks, activation functions, backpropagation, CNNs, RNNs, transformers, autoencoders, and GANs at the level of "what they are." This guide is about how to build and train them yourself, so it skips the basic definitions and goes straight to code, math, and practice. When an idea from Foundations is needed, you'll see a pointer back to the chapter.

The fourth guide, Advanced Machine Learning for Research, covers what happens around the model: gradient-boosted trees for tables, honest evaluation and statistical comparison of models, hyperparameter search, calibration and uncertainty, interpretability, fairness, working with language models through prompting and retrieval, and full project playbooks. When a topic belongs there, this guide says so and stops.

The "How to use this guide" page of Advanced Machine Learning for Research has a map of what all four guides cover together, and its Chapter 17 lists project types (images, audio, text, molecules, survival times, satellite imagery, and more) with the chapters of each guide you need for them. If you already have a project in mind, look it up there first.

Why PyTorch

Foundations used Keras because it's the gentlest way to start. This guide uses PyTorch, an open-source deep learning library maintained by the PyTorch Foundation, because most research code you'll find attached to papers is written in it, and so are the Hugging Face and PyTorch Geometric libraries used in Part III. PyTorch makes you write the training loop yourself. That's more typing, but it means you can see and change every step, which is exactly what you need when something breaks or when your project needs something unusual. Everything you learned in Keras still applies. Since Keras 3 can even run on top of PyTorch, nothing you built before is wasted.

The newest PyTorch release as of late October 2026 is 2.15. The code in this guide sticks to long-stable parts of the library, so it also works on the slightly older version that Google Colab installs.

How each chapter is built

Each chapter explains the idea in plain words, shows the math where it helps, then gives code. Two kinds of code appear:

• NumPy code that shows a mechanism (backpropagation, attention, graph convolution) step by step. Every one of these blocks was run, and the printed output shown beneath it is real.
• PyTorch code that does the job the way you would in a real project. These blocks need a GPU-equipped environment such as Colab, and they print results that depend on your hardware and random seed, so their output isn't shown. Where it helps, the text tells you roughly what to expect and why.

Each chapter ends with a short Check yourself list. If you can answer those without looking back, move on.

Running the code

Use Google Colab. PyTorch, torchvision, Hugging Face transformers, datasets, peft, and timm all come preinstalled. To get a GPU, open the Runtime menu, choose Change runtime type, and pick a GPU (on the free tier this has typically been an NVIDIA T4, though Colab doesn't guarantee a type). Then check it worked:

import torch
print(torch.__version__)
print(torch.cuda.is_available())          # True means PyTorch can see a GPU
if torch.cuda.is_available():
    print(torch.cuda.get_device_name(0))

Free GPU time on Colab is limited and not guaranteed. Colab's FAQ says free notebooks can run for at most 12 hours, that idle sessions time out, and that free usage limits fluctuate and aren't published. Chapter 14 covers how to work within those limits, including saving checkpoints to Google Drive so a disconnect doesn't cost you your progress. Code blocks within a chapter build on each other, so run them in order in the same notebook.

A suggested pace

Go at whatever pace works for you. If your project already has a clear direction, your mentor can help you jump to the architecture chapter you need after Part II. Don't skip Part II, though. Almost every failed student deep learning project fails for a reason covered in Chapters 4 to 7.

Deep learning is powerful, expensive, and easy to misuse. This chapter helps you decide whether your project needs it, what it will cost in data and computing time, and what people mean when they say a network "learns representations."

What deep learning adds

Every model in Machine Learning Foundations, from logistic regression to gradient boosting, works on features you hand it: columns of numbers someone chose to measure. A deep network can start from the raw data itself, such as pixels, audio samples, characters, or atoms and bonds, and learn its own features along the way. That's the main thing deep learning adds.

The features a network learns are called representations. In an image classifier, the first layers end up responding to edges and color blobs, middle layers respond to textures and simple shapes, and late layers respond to object parts like eyes or wheels. Nobody programs those detectors. They appear because they help reduce the loss. A useful picture is that each layer re-describes the input in a new coordinate system, and training adjusts those coordinate systems until the classes are easy to separate with a simple linear layer at the end.

Representations are also reusable. A network trained on millions of photos has layers that describe "photo-like things" well, and those layers help with X-rays, leaf images, and satellite tiles too. That reuse, called transfer learning, is why a student with a few thousand labeled images can build a strong model (Chapters 8 and 11).

Deep learning versus classical machine learning

In short, deep learning wins when the input is raw and structured (pixels next to pixels, words after words, atoms bonded to atoms) and when there's either a lot of labeled data or a pretrained model to start from. On ordinary tables of measurements, tree ensembles usually match or beat neural networks with far less effort. A judge will often ask why you used a neural network. "Because the input is raw images and a pretrained CNN beat my best feature-based model by 12 points" is a good answer. "Because deep learning is the most advanced method" is not.

What it costs

Data. Training a network from scratch for image classification typically needs thousands of labeled examples per class. With transfer learning, a few hundred per class often works, and sometimes fewer. Your test set also has to be large enough to measure the difference you care about; Advanced Machine Learning for Research covers how to check that.

Compute. Fine-tuning a ResNet on a few thousand images takes minutes on a free GPU. Fine-tuning a small language model on a few thousand sentences takes tens of minutes. Training anything large from scratch is beyond free resources, which is one more reason to start from pretrained models. Chapter 14 shows how to estimate memory and time before you start.

Your time. Neural networks have many more settings than a random forest, and they fail silently, since a buggy network still produces numbers. Budget time for the debugging process in Chapter 7.

Always have a baseline

Before you train a network, build the simplest reasonable alternative and record its score:

• A trivial baseline: always predict the most common class, or the training mean for regression.
• A classical baseline: logistic regression or gradient boosting on simple features (for images, even color histograms or the features from a pretrained network fed to logistic regression).
• A published baseline, if others have worked on the same dataset.

If your network doesn't clearly beat these on validation data, you haven't shown that deep learning helps, and that is itself a finding worth reporting.