Choosing the Weight for Auxiliary Data
When an objective sums per-example losses from a main dataset and a larger auxiliary dataset, a coefficient can scale the auxiliary losses. To give both datasets equal aggregate weight, set . Thus, for 6,000 labeled storefront images and 144,000 auxiliary web images, . Treat this ratio as an initial choice, then compare candidate values using development-set performance.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
When is it sensible to give auxiliary training examples less influence during training?
Reducing the weight of secondary data can make model training less demanding when that data is not the main target distribution.
Use re-weighting only when the extra data comes from a _____ distribution than the dev/test set.
When to Reduce the Weight of Auxiliary Data
Order the steps for deciding whether to reduce the weight of auxiliary training data.
Why Equal Weighting Can Be Expensive
True or False: Reducing the weight of auxiliary training data is the same as deleting those examples from training.
When mixing a small auxiliary dataset with a main dataset, reducing its _____ can keep the model balanced.
Match each term to its best description when auxiliary data is used for training.
Order the reasoning that shows why reducing the influence of a large auxiliary dataset can make a compact network sufficient.
When to Reduce the Weight of Auxiliary Data
Weighting Target and Auxiliary Data Under Compute Limits
Effect of Lower Weight on Model Capacity
Choosing the Weight for Auxiliary Data
Learn After
Which beta value gives the two image collections equal total weight?
Does beta set how much the auxiliary image set influences the total loss?
Setting beta to _____ makes the two data collections contribute the same total weight.
Match each part of the weighted loss to its role.
Order the steps for setting the auxiliary-data weight.
Explain what beta = 1/40 does when mixing two training pools
Interpreting beta when two data sources differ in size
Why can beta = 1/25 balance collections of very different sizes?
How should you choose a different mixing coefficient?
A tuning constant for auxiliary data must always be set to 1/50.