Evaluating LLM-Based Temporal Subject Assignment in Fiction: The Impact of Evaluation Frameworks on Metadata Interpretation

作者
Fereshta Westin and Johan Eklund
出版日期
14 Aug 2026
內容

This study examines how automatically generated temporal subject metadata for fiction can be evaluated when time is represented as year intervals. Using 45 Swedish novels with manually assigned chronological headings as the basis for comparison with model-generated intervals, the study contrasts two evaluation perspectives: overlap-based classification (precision, recall, F1) and boundary-based deviation (RMSE). The analysis shows how different evaluation frameworks highlight different aspects of temporal alignment between predicted and manually assigned intervals. By combining these perspectives, the study makes visible different types of temporal discrepancies and shows how evaluation design influences what is counted as alignment in interval-based subject metadata.

刊名
Cataloging & Classification Quarterly
卷期
Published online
頁數
26
關鍵字
Temporal subject assignment, large language model, evaluation framework, subject analysis, temporal metadata
網址連結
發布日期:2026年08月19日 最後更新:2026年08月26日