Evaluating LLM-Based Temporal Subject Assignment in Fiction: The Impact of Evaluation Frameworks on Metadata Interpretation
| 作者 | |
|---|---|
| 出版日期 | 14 Aug 2026 |
| 內容 | This study examines how automatically generated temporal subject metadata for fiction can be evaluated when time is represented as year intervals. Using 45 Swedish novels with manually assigned chronological headings as the basis for comparison with model-generated intervals, the study contrasts two evaluation perspectives: overlap-based classification (precision, recall, F1) and boundary-based deviation (RMSE). The analysis shows how different evaluation frameworks highlight different aspects of temporal alignment between predicted and manually assigned intervals. By combining these perspectives, the study makes visible different types of temporal discrepancies and shows how evaluation design influences what is counted as alignment in interval-based subject metadata. |
| 刊名 | Cataloging & Classification Quarterly |
| 卷期 | Published online |
| 頁數 | 26 |
| 關鍵字 | Temporal subject assignment, large language model, evaluation framework, subject analysis, temporal metadata |
| 網址連結 |
