CIT 01-LLMs4TextSummarization

Parameter Efficient Knowledge Injection approaches for domain-specific Text Summarization

CIT - Electrical and Computer Engineering

Chair of Software Engineering for Business Information Systems

Natural Language Processing

Computer ScienceComputer Science/ Informatics

Boltzmannstraße 385748 Garching bei München

Abstractive Text Summarization has been an active research area in the past years, and while state-of-the-art models can produce human competitive summaries, they are more suitable for general-purpose text. The performance of these models deteriorates when tested on a domain-specific text summarization task. One common explanation is the shift in the dataset distribution as most of the large language models (LLMs) are pre-trained on general-purpose corpora such as C4 (Raffel et al., 2020a), and hence do not fully comprehend the fine-grained linguistic details and concepts of a niche area such as the medical, scientific, or legal domain.
While there is no denying the LLMs’ abilities on various NLP tasks, their performance on domain-specific data and in general their domain-adaptation capabilities could be improved. The purpose of this project is to evaluate the LLMs on their domain-specific summarization capabilities by leveraging various domain-adaptation / parameter efficient fine-tuning capabilities and compare them to zero-shot or in-context learning approaches (Brown et al., 2020). The score of this project is on both open-source and closed-source LLMs.
Tasks:
• Literature Review on the state-of-the-art LLMs and domain-adaptation approaches.
• Evaluate the performance by employing prompting & domain-adaptation and benchmarking on domain-specific text summarization
• Exploring NLG evaluation approaches such as automatic evaluation scores and human evaluation
• Submission of well-documented code that can be used to later reproduce the results.
• A written report on the Literature Review findings.

Expected Outcome:
• General understanding of the domain-adaptation and prompting approaches for Large Language Models.
• Practical experiencing in terms of benchmark LLMs against the defined open-source dataset on a Text Summarization task.
• Well-structured experiments and analysis of the results to identify the best domain-adaptation techniques
• Understanding of NLG evaluation approaches and experience with surveys for human evaluation
A written report explaining all the findings.

35

3 years of bachelor studies completed

Machine Learning, Deep Learning, Natural Language Processing, Python Programming