How does artificial intelligence improve the quality of HR data?

In a context where companies handle a multitude of HR systems (Core HR, payroll, recruitment…), consolidating this data for a 360° view of the employee remains a major challenge. We recently met this challenge for one of our customers by developing a machine learning solution, capable of linking heterogeneous sources – even when they don’t share the same granularity.

Problem: incompatible granularities

Let’s take a concrete example:

• The Core HR system stores data with a fine granularity: person_number + assignment_number (representing the position held by the employee).
• The payroll system, on the other hand, only identifies employees via the person_number, without reference to the assignment_number.

The same person_number can therefore correspond to several assignments in CoreHR. How do you know which line reflects the actual payroll situation for a given month? And what do you do when you have several payroll lines with no assignment_number to differentiate them?

Solution: a supervised classification model

To address this issue, we have developed a supervised machine learning model, capable of automatically identifying the correct assignment_number line from among the various possibilities.

Methodology :

1. Data exploration and cleaning to harmonize key fields: date of hire, job title, division, department… as many explanatory variables as possible.

2. Naive source join: each payroll line is associated with the assignment status at the end of the month, for the same person_number.

3. Creation of a labelled dataset to serve as a training base:

• If the hiring date in payroll corresponds to the assignment start date, the line is identified as correct → label = 1
• Otherwise, the line is considered incorrect → label = 0 4.

4. Separation of data into training and test sets.

5. Multiple model training via 5-layer, 10-iteration cross-validation.

6. Selection of the best performing model based on metrics (accuracy, F1-score) and business validation by HR teams.

Model selected :

• SVM (Support Vector Machine) with radial kernel (RBF)
• Performance :
  • Accuracy: 99.48
  • Balanced Accuracy: 99.10
  • F1-score: 0.99

Solution architecture

The entire solution is deployed and automated on a Microsoft Azure environment, enabling regular updates as new data becomes available.

1. Data integration from payroll and HR databases (YES CoreHR)
2. Transformation: cleaning, enrichment and generation of explanatory variables
3. Modeling and prediction of the correct assignment_number line using the ML model
4. Feeding a datamodel into Power BI for reliable, enriched HR dashboards

A central repository is also updated, validated by HR teams, to correct any anomalies detected.

Observed benefits

• Easy cross-system analysis
• Accurate, consolidated HR vision for decision-making teams
• Automation of a previously manual and error-prone process
• Significant time savings for HR teams

Find out more: Case studies

The case described here can be generalized to other HR issues:

• Enriched Payroll/HR analysis: understanding pay discrepancies based on actual assignments
• Performance & remuneration: identifying the links between individual performance and pay
• Well-being at work: combining absenteeism, performance and HR data to detect weak signals
• Recruitment strategy: analyzing the candidate’s career path through to stabilization in the company

This method can be adapted to any combination of HR systems with a common identifier, such as the person_number.

Conclusion

This project is a perfect example of how AI can meet concrete challenges in HR data quality and integration. Thanks to this machine learning model, we are now able to cross-reference previously unreconcilable sources – providing our customers with a solid foundation for reliable, scalable and actionable HR data.