Abstract

TORRES-MORENO, Juan-Manuel et al. Summary Evaluation with and without References. Polibits [online]. 2010, n.42, pp.13-20. ISSN 1870-9044.

We study a new content-based method for the evaluation of text summarization systems without human models which is used to produce system rankings. The research is carried out using a new content-based evaluation framework called Fresa to compute a variety of divergences among probability distributions. We apply our comparison framework to various well-established content-based evaluation measures in text summarization such as COVERAGE, RESPONSIVENESS, PYRAMIDS and ROUGE studying their associations in various text summarization tasks including generic multi-document summarization in English and French, focus-based multi-document summarization in English and generic single-document summarization in French and Spanish.

Keywords : Text summarization evaluation; content-based evaluation measures; divergences.