Learn Before
A Big Gap Between Training and Development Error Suggests Overfitting
In this document-classification example, training error is 3% and development error is 14%, an 11-percentage-point generalization gap. If the training and development sets come from the same distribution, this large gap is evidence of high variance, or overfitting: the model performs much better on its training examples than on unseen examples. If the development set has a different distribution, these two error rates alone cannot distinguish variance from data mismatch; that diagnosis requires comparison with a training-dev set drawn from the training distribution.
0
1
Tags
Machine Learning
Deep Learning
Machine Learning Strategy
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Yearning @ DeepLearning.AI
Related
What does variance mean in this informal comparison?
A larger performance gap between training data and validation data is an informal sign of variance.
Variance compares test performance to the ____ performance.
Match each term to its role in the informal variance idea.
Order the steps for judging whether a model has high variance
Why does the informal idea of variance require a comparison between training and held-out results?
Interpret a gap between training and validation performance.
What does “worse” mean in an informal variance comparison?
What information is needed to judge variance informally?
An informal variance score is defined as training-set error minus test-set error.
A Big Gap Between Training and Development Error Suggests Overfitting
Learn After
Estimate the Variance from Training and Dev Errors
Training Success Does Not Guarantee Generalization
When a model fits the training set very closely but performs poorly on new data, that is called _____.
Match each concept to the description that fits a binary classifier’s error analysis.
Order the steps used to diagnose high variance in the traffic-sign classifier example.
Which training and development error pattern is most consistent with overfitting?
A model with high variance usually shows high training error and low development-set error.
Variance is computed from _____ minus training error.
Match each term to its role in diagnosing high variance in a document classifier.
Order the reasoning steps to decide whether a model is overfitting from its error numbers.
Interpreting a Classifier with Low Bias and High Variance
Estimating Bias and Variance in a Crop Disease Classifier
Term for Poor Generalization Despite Strong Training Performance