Unravelling Job Advertisements Through Fine-Tuned LLM

Client/Partner
Partners: Future of Work Institute | Curtin University

Timeline
Jan 2022 – March 2022

Overview
In today’s competitive job market, the way a job advertisement is crafted can significantly influence the types of candidates it attracts. Subtle choices in language, framing and emphasis may shape how potential applicants perceive roles, organisations and workplace culture. Traditional methods of analysing job advertisements have relied on manual annotation, which is time consuming, subjective and not scalable. This created a gap in understanding how personality traits, values and job related attributes embedded in advertisements affect recruitment outcomes.

To address this challenge, the Curtin Institute for Data Science developed advanced natural language processing (NLP) techniques to automatically analyse job advertisements. Using 15,000 job ads and application data from 260,000 candidates, CIDS fine-tuned transformer based language models, including BERT and RoBERTa, on a dataset annotated for multiple abstract categories such as personality traits, work abilities, job characteristics, employment branding and work activities.

CIDS tested both hierarchical and flat multi-label classification approaches, ensuring nuanced features of recruitment messaging were preserved. The project applied cutting-edge machine learning infrastructure and advanced evaluation metrics, optimising for data imbalances that are typical in recruitment datasets.

The fine-tuned RoBERTa model delivered strong performance, achieving F1 scores between 73% and 86% across classification categories, with particularly high accuracy in identifying job abilities and characteristics. This demonstrates that large language models can provide scalable, objective insights into the implicit messaging within job advertisements.

By automating this analysis, the project reduces reliance on manual annotation, accelerates recruitment research and enables organisations to better understand how the wording of job ads may appeal to or deter different candidate groups. Looking ahead, the study highlights the potential for even greater performance using next-generation models such as LLaMa and GPT-4, laying the foundation for advanced tools that can support fairer and more effective recruitment practices.