hh.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Enhancing Retrieval - Augmented Generation Through Smart Embedding
Halmstad University, School of Information Technology.
Halmstad University, School of Information Technology.
2024 (English)Independent thesis Advanced level (degree of Master (Two Years)), 20 credits / 30 HE creditsStudent thesis
Abstract [en]

Retrieval-augmented generation (RAG) systems have emerged as a powerful approach to enhancing the performance of Large Language Models (LLMs) by integrating them with external knowledge sources. However, existing RAG systems face challenges in cost efficiency, retrieval efficacy, and the effective utilization of retrieved information. This report proposes an innovative methodology for optimizing the preprocessing pipeline of RAG systems, focusing on parsing, content refinement, and dynamic content-aware chunking. The proposed techniques aim to reduce token usage, improve attention score distribution, and enhance the semantic coherence of the knowledge base while preserving its informational integrity. The results demonstrate a significant reduction in token count while retaining a high semantic similarity between the original and refined knowledge base and enhanced retrieval efficacy on the PubMedQA dataset. The resulting chunks also displayed denser and more evenly distributed attention scores, indicating increased generative capability for large-context applications. These results are based on extensive evaluation metrics that assess different RAG processes. This research could serve as a basis for a comprehensive review of cross-domain applicability or impact analysis of similar preprocessing methods to develop more efficient and reliable RAG systems, promoting widespread adoption in various domains.

Place, publisher, year, edition, pages
2024. , p. 96
National Category
Computer and Information Sciences
Identifiers
URN: urn:nbn:se:hh:diva-55041OAI: oai:DiVA.org:hh-55041DiVA, id: diva2:1919188
Educational program
Computer Science and Engineering, 300 credits
Supervisors
Examiners
Available from: 2024-12-02 Created: 2024-12-07 Last updated: 2025-10-01Bibliographically approved

Open Access in DiVA

fulltext(10244 kB)196 downloads
File information
File name FULLTEXT02.pdfFile size 10244 kBChecksum SHA-512
ead92cf1ad5ee8a483683f7a84468ec62c02effd1288b5adeb3fc062a39c8d77b3060b6611f7b6338b51b03db51a316c8b3579626e9bc1dd4f81543a838645aa
Type fulltextMimetype application/pdf

By organisation
School of Information Technology
Computer and Information Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 196 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 1048 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf