Distilling Multilingual Models for Few-Shot Gender-Bias Detection in Spanish Judicial Texts: A Systematic Literature Review

Abstract

This systematic mapping study examines 65 primary studies at the intersection of gender-bias detection, efficient natural language processing, and adaptation to legal-domain data. It focuses on knowledge distillation, domain-specific pretraining, and counterfactual evaluation for resource-constrained Spanish judicial-text settings. The review suggests that compact models such as DistilBERT and TinyLlama can retain a substantial share of the predictive performance of larger models while reducing computational requirements, and that domain adaptation can improve detection of implicit bias. It also highlights challenges involving counterfactual validity, evaluation consistency, and efficiency–fairness trade-offs.

Publication
Proceedings of the Argentine Symposium on Artificial Intelligence and Data Science (ASAID), 55th JAIIO 2026, La Plata, Argentina