Diagnosing a Translation Scoring Error
Case context: You are building a Chinese-to-English machine translation system. Your system outputs an incorrect translation Eout instead of the correct translation E*. You run the Optimization Verification test and find that the score computed for E* is actually lower than the score computed for Eout.
Question: Based on this finding, what component of your machine translation system should you diagnose and improve, and what is the underlying reasoning?
Sample answer: I should diagnose and improve the score computation component. Because the score for the correct translation E* was lower than the incorrect translation Eout, the scoring function failed to recognize E* as the superior output. The search algorithm cannot be blamed because even if it had found E*, the scoring function would still have preferred Eout.
Key points:
- Identifies the score computation as the component to fix.
- Explains that the scoring function ranked the incorrect translation higher.
- Exonerates the search algorithm since it relies on the flawed scoring function.
Rubric: The response must correctly identify the score computation as the source of the error and provide the reasoning that the scoring function incorrectly ranked the bad output above the true translation.
0
1
References
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Tags
Data Science
Foundations of Large Language Models Course
Computing Sciences
D2L
Dive into Deep Learning @ D2L
Machine Learning
Deep Learning
Supervised Learning
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Attributing Errors in MT Search
Exhaustive vs. Heuristic Search in MT
Relying on _____ Search
MT Optimization Verification Outcomes
Steps of the MT Verification Test
Analyzing MT Optimization Verification
Diagnosing a Translation Scoring Error
Purpose of Score Computation
Identifying the Source of Scoring Errors
Search Algorithm Blame Condition