提升大语言模型短篇小说生成的多样性
Improving Diversity in LLM Short Story Generation
Zahra Solati Dehkordi, Vasileios Lampos
AI 速览：提出 DivLM 两阶段后训练框架，通过继续预训练与复合奖励强化学习提升短篇小说在体裁、语气、风格和命名实体上的多样性。
Large language models (LLMs) can generate accurate responses, but these are void of diversity. We attempt to address this for the task of creative short story generation. Drawing on established writing conventions and known LLM limitations, we target variation in genre, tone, style, and named entities. To promote diversity across these dimensions, we introduce DivLM, an LLM post-training framework consisting of two phases. First, we perform continued pre-training on a creative writing corpus and restore instruction-following capabilities using weight residuals. We then apply reinforcement learning with a custom, composite reward function that jointly maximizes diversity across the targeted narrative dimensions while maintaining response quality. Our empirical results on two LLM families show that DivLM increases diversity metrics by more than 9% on average compared to alternative approaches, while preserving instruction following, overall response quality, and similarity to human outputs.