This note explains transfer learning as reusing what a model already learned for a new but related task. It defines the idea with an everyday comparison between learning to drive a manual car and then an automatic. The how it works section walks through three stages. You pick a source model that is public and licensed for reuse. You fine-tune it by loading the pre-trained weights from hubs such as TensorFlow Hub, Keras Applications, or PyTorch Hub. You then adapt it to the new task by replacing the top layers. A short subsection explains why the feature extractor transfers so well, and the closing section gives three reasons the approach pays off.
User prompt: rewrite the content below concisely as a bullet list and save it to the appropriate file
What Is Transfer Learning?
- Reusing knowledge learned from a previous domain or task to serve a new domain or task
- Like a person who learns to drive a manual car and then learns an automatic: no need to start from scratch, only the differences have to be learned
How It Works
Choose a Source Model
- The source model is one built by someone else and trained to solve a similar problem
- Often it comes from tech giants or well-known research groups
- Trained on very large datasets such as ImageNet or the Wikipedia Corpus
- The chosen source model must be publicly available and licensed for reuse
Customize the Model (Fine-tuning)
- Use the knowledge the model has already learned: layers, features, weights, free parameters
- Load the source model into your own environment, where it becomes a file/folder holding the relevant information
- Prefer deep learning hubs that host many pre-trained models, such as:
- TensorFlow Hub
- Keras Applications
- PyTorch Hub
Apply to the New Task (Inference)
- In a neural network:
- The bottom and middle layers: represent general features
- The top layers: represent features specific to the problem that model was trained on
- The new problem will differ from the source model's original problem
- Add layers for the new task and remove the top layers to get higher accuracy on the new problem
- The model can be configured with a specialized optimizer
Why Does Transfer Learning Mostly Reuse the "Feature Extractor"?
- Basic features such as edges, corners, and lines are universal and can be reused across many tasks
- Only the head needs to be replaced or fine-tuned for the new task
Why Is Transfer Learning Effective?
- Neural networks learn high-level, generalizable information → it can be reapplied in other settings
- Training a model from scratch is expensive and prone to overfitting when data is scarce → Starting from a well-trained model saves a great deal
- Better generalization → Even with little new data, the model is already "used to learning"
See also: Fine Tuning Techniques · Vision Transformers