Sizing a Training Development Set
A training-development set should be large enough to provide a dependable measure of progress and distinguish meaningful differences between training runs. Most development-set sizing guidance applies to this choice, although not every rule necessarily transfers unchanged.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Using Training, Training-Dev, Dev, and Test Sets for Different Questions
Sizing a Training Development Set
What is the key difference between a training dev set and the training set?
Should the development set reflect the training data distribution?
A validation set should not _____ on the data used to fit the model.
Match each split component to its purpose in model development.
Order the steps for setting up and using a training-dev split.
Why keep a development subset separate when it comes from the same data source?
Find the mistake in a team’s proposed holdout evaluation plan.
Choosing the Size of a Monitoring Dev Set
Which split best creates a training dev set?
Does a training-dev set need to be the same size as the training set?
Typical Development Set Sizes for Tiny Accuracy Gains
High-Impact Business Systems May Justify a Larger Development Set
Formal significance tests for validation-set changes
What dev set size is most suitable for spotting a 0.1 percentage-point gain in accuracy?
A development set should always be expanded to the maximum possible size, even after it is already large enough to reveal meaningful performance changes.
Validation set size for noticing a tiny accuracy change
Match each evaluation target with the dev set size it suggests.
Order the steps for deciding whether a dev set is large enough to detect a useful accuracy gain.
Match dev-set size to the smallest gain you care about.
Choose a dev set size that can detect a tiny but important gain.
Why is a 150-example dev set not enough to tell 83.0% from 83.4% accuracy?
When is a validation set much larger than 10,000 examples most justified?
If a validation set is already large enough to tell whether one model is meaningfully better than another, it does not need to be made much larger.
Sizing a Training Development Set
Accuracy Resolution on a 100-Example Development Set
Learn After
How should guidance for a general dev set be used when selecting a training dev set?
Rules for choosing a development set size usually also guide the training-dev set.
Most advice for selecting a dev set size also applies to the _____.
Match each dataset to its sizing role.
How to choose a training dev set size
How should earlier validation-set sizing guidance inform a training-dev set?
Choose how to size a new model-tuning set.
Which sizing principles should be reused for a training dev set?
What does the word “most” imply in applying earlier sizing guidance to a training dev set?
Reducing the validation set size is the standard way to fix validation-set overfitting.