Interpreting 10% Training, 11% Training-Dev, and 12% Development Error
A model with 10% training error, 11% training-dev error, and 12% development error has only a one-percentage-point increase at each comparison. The small training-to-training-dev gap suggests low variance on the training distribution, while the small training-dev-to-development gap suggests little data mismatch. Whether the 10% training error represents high avoidable bias depends on the best achievable, Bayes, or human-level error: high avoidable bias is supported only if that benchmark is substantially lower than 10%.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Variance, Informally
Mean Squared Error and the Bias-Variance Decomposition
Why Estimate the Lowest Plausible Error?
When Increasing Capacity Helps One Error and Hurts the Other
Validation-Error Learning Curve
Choosing Between Bias, Variance, and Dataset Mismatch Fixes
A Single Model Can Show Several Error Sources at Once
Reading High Bias from Training and Dev Error
Strong Classification Performance Comes from Low Bias and Low Variance
What are the two main sources of error in machine learning?
Why can it be useful to tell whether a model's main problem is bias or variance before deciding how to improve it?
In the usual pair of major supervised-learning error sources, bias and ____ go together.
Which pair names the two broad error categories used to guide model improvement?
When a model's errors are due to bias or variance, that information helps you decide whether collecting more labels is likely to help.
Major error sources in supervised learning
Match Each Error-Analysis Idea to Its Role
Use bias and variance to choose an improvement strategy.
How can knowing the bias and variance pattern of your model help you make better decisions?
Sources of Error in Machine Learning Models
Knowing bias and variance helps you decide whether _____ is worth the effort.
Match each child concept to the bias-variance idea it supports.
Order the steps for deciding whether more labeled data is the right fix for a classifier.
Using Bias and Variance to Choose the Next Improvement
Choosing an Error Check Before Expanding the Dataset
Why Diagnose Bias and Variance Before Choosing an Improvement Strategy?
Interpreting 10% Training, 11% Training-Dev, and 12% Development Error
Informal Bias from Large-Sample Training Error
Troubleshooting a deep learning model
Data Mismatch Problem in Deep Learning
Avoidable bias
Interpreting 10% Training, 11% Training-Dev, and 12% Development Error
Example: Diagnosing High Variance from a Training-to-Training-Dev Gap
Learn After
What does a pattern of 10% training error, 11% training-dev error, and 12% dev error most strongly suggest?
High training error signals poor fit on the training set
What type of bias is suggested when training error stays high?
Match each error metric in this classifier-debugging scenario to its correct value or diagnosis.
Diagnose High Bias from Three Error Rates
What does a tiny gap between training error and training-dev error indicate?
A model with 10% error on training data, 11% error on a training-like holdout set, and 12% error on the development set is mainly overfitting.
If training error is far worse than training-dev error, the model is doing poorly on the _____ set.
Diagnosing Error Sources from Training and Development Gaps
Diagnosing a High-Bias Model from Three Similar Error Rates
What a 10% train error, 11% train-dev error, and 12% dev error reveal
Diagnosing a model with 14% training error, 15% train-dev error, and 16% dev error
Recognizing the Bias Pattern in the Error Rates