The Brittle Compass: Navigating LLM Prompt Sensitivity in Slovak Migration Media Discourse
Samuel Harvan, Jaroslav Kopčan, Marek Šuppa, Andrej Findor
Abstract
In this work, we present a case study that explores various tasks centered around the topic of migration in Slovak, a low-resource language, such as topic relevance and geographical relevance classification, and migration source/destination location term extraction. Our results demonstrate that native (Slovak) prompts yield a modest, task-dependent gains, while large models show significant robustness to prompt variations compared to their smaller counterparts. Analysis reveals that instructions (system or task) emerge as the most critical prompt component, more so than the examples sections, with task-specific performance benefits being more pronounced than overall language effects.