Impact of Down-Weighting on Neural Network Size
Question: How does assigning a lower weight to a massive, different-distribution auxiliary dataset affect the required size of your neural network?
Sample answer: Assigning a lower weight to the auxiliary dataset means you do not have to build as massive a neural network. It reduces the computational burden of forcing the model to perform equally well on both the auxiliary and target-distribution tasks.
Key points:
- Reduces the needed size of the neural network.
- Lowers the overall computational burden.
Rubric: Acceptable answers will state that the neural network can be smaller or that the overall computational resources and burden are reduced.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Tuning Auxiliary Data Weight with the Dev Set
Which condition makes it appropriate to down-weight auxiliary training data according to Andrew Ng?
Down-weighting auxiliary data can reduce the computational burden of training a model on both auxiliary and target-distribution examples.
Re-weighting is needed only when you suspect the additional data has a very _____ distribution than the dev/test set.
Match each training scenario to its key implication for handling auxiliary data.
Order the reasoning steps a practitioner should follow when deciding whether to down-weight large-scale auxiliary training data.
In the 40x internet vs. mobile image scenario, what is the main cost of training equally on both sources without down-weighting?
Giving auxiliary data a lower weight in training is equivalent to removing it from the training set entirely.
By giving internet images a much lower _____, you don't have to build as massive a neural network.
Match each concept to its correct description in the context of auxiliary data down-weighting.
Order the causal steps explaining why down-weighting auxiliary data allows a smaller neural network to be used.
Explain the dual conditions and purpose of down-weighting auxiliary data.
Optimizing Resource Allocation for Mobile Image Classification
Impact of Down-Weighting on Neural Network Size