Entrago: User Deduplication Solution

by | Aug 14, 2021 | Applications, News, ServiceNow

Entrago recently developed a solution to help a large organisation manage and consolidate duplicate users in their ServiceNow systems, while ensuring that any duplicate accounts identified were merged so that the individuals did not lose access to any data or work items. This solution is configurable and audits any changes made for future reference.

From multiple sources of truth to name changes to human error to corporate mergers, there are many reasons why an organisation’s ServiceNow instance might contain duplicate user accounts. Having multiple accounts for a single individual is a problem for the individual (because their work items could be spread across multiple accounts and therefore not accessible to their current login) as well as for the organisation (because they could be paying additional licensing costs due to the additional accounts, as well as reduced productivity). The client provided a list of approximately 16,000 ‘primary’ user accounts which could be reliably linked to given individuals. This list was used as the basis for determining which accounts should be kept active in the merge; any duplicate user accounts found were merged into one of those user accounts. The solution was heavily scripted, making use of the Pro-Code functionality available in the ServiceNow platform. Two custom tables were used to help store information about the duplicates cleanly. Consolidating accounts requires first identifying which accounts belong to the same individual. This sounds straightforward at first, but quickly becomes complex once issues such as nicknames, common names used by multiple individuals, and poor data quality on some user accounts, are considered.

Merge Context

Entrago created a “Merge Context” table to be used as a form of ‘container’, allowing a user to start an identification process, merge recommendations, and configure parameters used for identification. Duplicates, in the form of merge recommendations, are displayed in a related list.


No alt text provided for this image

Identification process

Identifying duplicate accounts was a multi-step process. First, accounts with similar field values to the ‘primary’ accounts were identified, and the list of fields that matched was recorded. Subsequently, matches were scored, using a combination of factors:
  • Which fields were matched
  • Some fields, such as email address, are more likely to positively identify a match than others. Therefore, the specific fields matched in the search will contribute to the reliability of the match.
  • How many fields were matched
  • The more fields that are identical across both records, the more certain we can be that the user records are referring to the same person.
  • Correlating confidence boosters or penalties
  • Some fields (for example, manager) are not designed to be unique between users and therefore cannot be used as search criteria, however if these fields are present and identical on both records, we can be a little more confident that the records are referring to the same person.
  • In cases where there is only a partial match on particular fields, we should treat the confidence level as lower than we would if it was a full match. This will be implemented as a penalty for the score.

Scoring Formula

The following components are used in the scoring formula to determine the score for an individual match recommendation:

No alt text provided for this image
The following formula was used to determine the overall score of a match: Confidence Score = Base score + booster score + attribute count modifier This approach allows us to rank matches in order of quality. The Merge Context also allows us to specify different values for each field value or modifier value, increasing flexibility and allowing us to prioritise different types of user if required. Merge recommendations can be automatically approved if they are above a user-configurable threshold, or this functionality can be easily disabled through a system property to allow for manual review.    

Merging Accounts

Once a duplicate account is identified, the merge recommendation created as part of the identification process is applied. This means that the ServiceNow data associated with the “Merge From” account will be migrated to the “Merge To” account for that user’s tasks, cost centres, expense lines, and assigned configuration items. A summary of changes made as part of the process is added to the merge recommendation record, as a point of reference for historical data. Entrago also made a catalog item in the Service Portal, allowing the Service Desk to perform ad-hoc merges where requested. The catalog item uses the same merge process as the merge recommendation, however is not related to a merge recommendation and therefore will rely on the input of the Service Desk submitter to select the correct accounts to merge rather than automatically attempting to identify them.

Written by Steph Zylstra, Senior Technical Consultant at entrago

Steph’s decade of IT experience has given her a wealth of knowledge and a passion for automating as many things as possible in a variety of different languages. Her natural curiosity leads to understanding the root cause of any issues, and results in a solution that takes those variables into account. Steph has delivered projects across ServiceNow HR, Service Portal, ITSM implementations and more.  

No alt text provided for this image

Our team call on years of industry experience to create end to end improvements that achieve real efficiencies where it counts most. Entrago is dedicated to delivering solutions that drive positive outcomes across the enterprise. Contact Us about how we can improve your business processes with ServiceNow, AWS or UiPath.

Managed Services

IoT and Analytics


Compliance and Risk

Strategic Advisory

Document Understanding

entrago services

entrago is dedicated to delivering solutions that provide automation across the enterprise.

Our team call on years of industry experience to create end to end automations that achieve real efficiencies where it counts most.

Business Continuity Planning

Hospital on Duty

Vaccine Management

Cost Transparency

Request for Expenditure

BillView Connect

apps by entrago

entrago has a dedicated development team committed to providing solutions to solve business problems built on NOW.