Pairwise Ranking for Summarization or Readability Assessment

• @kryscinski2020 model-based evaluation method (wording taken from slide 6) for summarization model outputs

  • why
    • guard against hallucinations
    • factual consistency → basis for fact-checking

• Pairwise Ranking

  • calculating similarity between two sentences
    • e.g. Cosine Similarity
      • Tutorial: How to Calculate Cosine Similarity sklearn_cosine_similarity
      • Cosine Similarity Search for new documents using Scikit-Learn
      • 6.8. Pairwise metrics, Affinities and Kernels — scikit-learn 1.2.2 documentation
        • API Reference — scikit-learn 1.2.2 documentation
    • (1) 🤗 Tasks: Sentence Similarity - YouTube
    • Pairwise comparisons for document similarity

• datasets

  • scientific summarization
    • scientific_papers from long-summarization
    • Multi-XScience (multidocument summ) 2010.14235.pdf
    • COMPLING ScisummNet: A Large Annotated Corpus and Content-Impact Models for Scientific Paper Summarization with Citation Networks
      • scisumm-corpus
      • scisumm_net
      • 1909.01716.pdf
      • slides: CL-SciSumm2020_slidedeck.pdf
    • SciTLDR 2004.15011.pdf
    • semantic scholar network SSN ssn