Case context: You are building a mobile image classification application. Your dev/test set consists of 5,000 images taken from mobile phones. To improve performance, you have acquired a supplemental dataset of 200,000 images scraped from the internet. However, you have limited computational resources and cannot afford to train a massive neural network.

Question: Given your limited computational budget, what strategy should you apply to the internet images during training, and why will this strategy help you avoid building an overly large neural network?

Sample answer: You should apply a much lower weight to the 200,000 internet images compared to the 5,000 mobile images during training. Because the internet images are both much larger in quantity and likely from a different distribution, treating them equally would require a massive neural network to model both distributions effectively. Down-weighting the internet images acts as a compromise, reducing the computational burden and allowing a smaller network to focus primarily on the target mobile distribution.

Key points:

• Assign a lower weight to the auxiliary internet images.
• The internet images are much larger in number and have a different distribution.
• Equal weighting requires a massive neural network to learn both distributions.
• Down-weighting focuses the model on the target distribution, reducing resource needs.

Rubric: The answer must propose down-weighting the internet images and explain that it prevents the model from expending excessive capacity on learning the auxiliary distribution, thus saving computational resources.