In an experiment evaluating automatically generated metaphorical expressions, workers rated 1,360 expressions (derived from 40 nouns and 34 verbs) based on their metaphoricity, comprehensibility, and novelty using a 1-to-5 scale. The study found that factoring the abstractness or concreteness of the nouns into the overall evaluation was effective in ranking preferred expressions more highly.

Metaphorical Expression Evaluation Experiment

The overall evaluation score for a metaphorical expression is the unweighted sum of its metaphoricity, comprehensibility, and novelty ratings: $$\text{overall score}=\text{metaphoricity}+\text{comprehensibility}+\text{novelty}.$$

Google

University of Michigan - Ann Arbor

Metaphoricity: We collect metaphoricity scores by asking simple questions of crowdworkers. After that, we check if the results are reliable by using MIPVU.

Metaphoricity

Evaluating the novelty of an expression is often easier than assessing its creativity or originality. In manual evaluation, a high score is assigned if the evaluator has never encountered the expression, while a low score is given to conventional, widely used expressions. This process can be automated by measuring novelty based on the frequency of the expression's occurrences in a corpus.

Learn Before

Related

Learn After