Which change could the Generative Al Engineer perform to mitigate this issue?

A Generative Al Engineer is creating an LLM system that will retrieve news articles from the year 1918 and related to a user’s query and summarize them. The engineer has noticed that the summaries are generated well but often also include an explanation of how the summary was generated, which is undesirable.

Which change could the Generative Al Engineer perform to mitigate this issue?
A . Split the LLM output by newline characters to truncate away the summarization explanation.
B . Tune the chunk size of news articles or experiment with different embedding models.
C . Revisit their document ingestion logic, ensuring that the news articles are being ingested properly.
D . Provide few shot examples of desired output format to the system and/or user prompt.

View Answer

Answer: D

Explanation:

To mitigate the issue of the LLM including explanations of how summaries are generated in its output, the best approach is to adjust the training or prompt structure.

Here’s why Option D is effective:

Few-shot Learning: By providing specific examples of how the desired output should look (i.e., just the summary without explanation), the model learns the preferred format. This few-shot learning approach helps the model understand not only what content to generate but also how to format its responses.

Prompt Engineering: Adjusting the user prompt to specify the desired output format clearly can guide the LLM to produce summaries without additional explanatory text. Effective prompt design is crucial in controlling the behavior of generative models.

Why Other Options Are Less Suitable:

A: While technically feasible, splitting the output by newline and truncating could lead to loss of important content or create awkward breaks in the summary.

B: Tuning chunk sizes or changing embedding models does not directly address the issue of the model’s tendency to generate explanations along with summaries.

C: Revisiting document ingestion logic ensures accurate source data but does not influence how the model formats its output.

By using few-shot examples and refining the prompt, the engineer directly influences the output

format, making this approach the most targeted and effective solution.