neural_compressor.experimental.metric.bleu
¶
Script for BLEU metric.
Module Contents¶
Classes¶
Ad-hoc hack to recognize all punctuation and symbols. |
|
Computes the BLEU (Bilingual Evaluation Understudy) score. |
Functions¶
|
Tokenize a string following the official BLEU implementation. |
- class neural_compressor.experimental.metric.bleu.UnicodeRegex¶
Bases:
object
Ad-hoc hack to recognize all punctuation and symbols.
- nondigit_punct_re¶
The compiled regular expressions to recognize punctuation preceded with a digit.
- punct_nondigit_re¶
The compiled regular expressions to recognize punctuation followed by a digit.
- symbol_re¶
The compiled regular expressions to recognize symbols.
- property_chars(prefix: str) str ¶
Collect all Unicode strings starting with a specific prefix.
- Parameters:
prefix – The specific prefix.
- Returns:
- The join result of all Unicode strings starting
with a specific prefix.
- Return type:
punctuation
- neural_compressor.experimental.metric.bleu.bleu_tokenize(string: str) List[str] ¶
Tokenize a string following the official BLEU implementation.
- See https://github.com/moses-smt/mosesdecoder/”
“blob/master/scripts/generic/mteval-v14.pl#L954-L983
- Parameters:
string – The string to be tokenized.
- Returns:
A list of tokens.
- Return type:
tokens
- class neural_compressor.experimental.metric.bleu.BLEU¶
Bases:
object
Computes the BLEU (Bilingual Evaluation Understudy) score.
BLEU is an algorithm for evaluating the quality of text which has been machine-translated from one natural language to another. This implementent approximate the BLEU score since we do not glue word pieces or decode the ids and tokenize the output. By default, we use ngram order of 4 and use brevity penalty. Also, this does not have beam search.
- predictions¶
List of translations to score.
- labels¶
List of the reference corresponding to the prediction result.
- reset() None ¶
Clear the predictions and labels in the cache.
- update(prediction: Sequence[str], label: Sequence[str]) None ¶
Add the prediction and label.
- Parameters:
prediction – The prediction result.
label – The reference corresponding to the prediction result.
- Raises:
ValueError – An error occurred when the length of the prediction
and label are different. –
- result() float ¶
Compute the BLEU score.
- Returns:
The approximate BLEU score.
- Return type:
bleu_score