Google

Very large neural networks tend to perform best when they are trained on a very large amount of data.

Large Neural Networks Need Plenty of Data

Two dependable ways to improve an algorithm today are to make the model larger and to train it with more data.

Increasing Model Capacity and Training Data as Dependable Improvements

What setup is most likely to achieve the strongest neural network performance?

Larger models can sometimes achieve better results

Two ingredients behind _____ performance

Matching model scale to performance outcomes

How to pursue better performance with a large model

Question:
Explain why a very large neural network by itself is not enough to achieve top performance. What else is needed, and how do the two factors work together?

Sample answer:
A very large neural network provides the capacity to represent complicated patterns, but capacity alone does not guarantee strong performance. To use that capacity well, the model also needs a very large amount of training data. The data supplies enough varied examples for the network to learn reliable patterns instead of merely memorizing the training set. In practice, high performance comes from combining a high-capacity model with enough data to train it effectively.

Key points:
- A very large neural network gives high capacity.
- A very large amount of data is also required.
- Strong performance depends on using both together.

Rubric:
The response must explicitly mention the need for a very large amount of data together with a large neural network. It should explain that both are needed to achieve the best performance.

Why model capacity must be matched with data scale

Case context:
A team has built a customer support intent classifier with a modest neural network and 8,000 labeled chat transcripts. Its performance has stalled. They can either greatly enlarge the network or gather 60,000 additional labeled transcripts, but they cannot do both right away.

Question:
If the best result will eventually come from having both a much larger network and much more data, what should the team conclude about its long-term plan, even if it must act in stages?

Sample answer:
The team should conclude that reaching the strongest possible performance will likely require both changes: a larger network and a much larger dataset. Improving only one of them may help for a while, but each has a limit on its own. The long-term plan should therefore include a path to scale both model capacity and training data.

Key points:
- Improving only one factor has limits.
- The long-term target is scaling both dimensions.
- Best performance requires both a much larger network and much more data.

Rubric:
Evaluates whether the learner understands that the highest performance usually depends on increasing both model size and data volume over time.

Expanding a text classification system

Question:
What two conditions must be satisfied to get the strongest performance from a neural network?

Sample answer:
You need a very large neural network and a very large amount of training data.

Key points:
- Use a very large neural network.
- Use a very large amount of training data.

Rubric:
The answer must name both conditions: a very large neural network and a very large amount of training data.

What two ingredients are needed to reach peak neural network performance?

What happens when a neural network is made larger?

Model size and data volume

When a training set contains on the order of a million labeled examples, a neural network is often a good candidate because it can use the extra data to learn more complex patterns.

Learn Before

Related