GuideX: Guided Synthetic Data Generation for Zero-Shot Information Extraction

Information Extraction (IE) systems are traditionally domain-specific, requiring costly adaptation that involves expert schema design, data annotation, and model training. While Large Language Models have shown promise in zero-shot IE, performance degrades significantly in unseen domains where label definitions differ. This paper introduces GUIDEX, a novel method that automatically defines domain-specific schemas, infers guidelines, and generates synthetically labeled instances, allowing for better out-of-domain generalization. Fine-tuning Llama 3.1 with GUIDEX sets a new state-of-the-art across seven zeroshot Named Entity Recognition benchmarks. Models trained with GUIDEX gain up to 7 F1 points over previous methods without humanlabeled data, and nearly 2 F1 points higher when combined with it. Models trained on GUIDEX demonstrate enhanced comprehension of complex, domain-specific annotation schemas. Code, models, and synthetic datasets are available atthis http URL

@article{fuente2025_2506.00649,
  title={ GuideX: Guided Synthetic Data Generation for Zero-Shot Information Extraction },
  author={ Neil De La Fuente and Oscar Sainz and Iker García-Ferrero and Eneko Agirre },
  journal={arXiv preprint arXiv:2506.00649},
  year={ 2025 }
}