Training–Dev Data Mismatch May Not Have a Clear Fix
When training and development data come from different distributions, obtaining or creating training examples that resemble the difficult development cases may reduce the mismatch. If suitable examples cannot be obtained, the mismatch may remain and there may be no obvious route to further improve development-set performance; success is therefore not guaranteed.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Synthetic Data That Approximates the Dev Distribution
Are Synthetic Training Examples Representative?
What should you do when training results are strong but validation results drop because the data sources differ?
A data mismatch problem happens when a model does well on training data but performs poorly on a dev set that comes from a different distribution.
Matching the Dev Set Environment
Match each data-mismatch idea to its description.
Order the steps for diagnosing and fixing a data mismatch in a customer-feedback classifier.
What most likely explains the model’s weak performance on the development set in this speech project?
Does training on examples that look more like the dev set always eliminate data mismatch?
When a speech recognizer performs poorly on noisy clips in the dev set, one remedy is to collect more training data that better _____ those difficult examples.
Match each part of a traffic-sign recognition scenario to its role in a data mismatch diagnosis.
Order the steps for deciding whether to collect training data that better matches difficult dev examples.
Using Targeted Data to Reduce Distribution Mismatch
Fix a training-dev distribution gap
What training data change helps with a data mismatch problem?
Training–Dev Data Mismatch May Not Have a Clear Fix
Learn After
What may happen if you cannot obtain training data that better matches your validation set?
A Mismatch Always Has a Clear Fix
If no additional data can be gathered that better matches the dev set, the route to better performance may not be _____ at first.
Match each data-mismatch idea with the statement that best describes it.
Order the reasoning steps when hard dev examples appear to come from data mismatch.
Why a Lack of Matching Training Data Can Block Improvement
Case Study: Model Improvement Hits a Dead End in Retail Forecasting
Is Success Guaranteed When Seeking Better-Matched Training Data?
What is the most accurate description of handling a training–target data mismatch?
True or False: If you cannot collect training data that better reflects the real-world setting, improving performance may become difficult or stall.