Guiding Software Platform Analysis for the IRS

In late 2012, the IRS was faced with choosing between two separate software platforms designed by two industry-leading vendors, both armed with inflated sales pitches. The platforms promised to catch cases of fraud that were slipping past IRS models.

IRS staff, quickly determined that they needed a trusted third-party partner to help make a choice between the two. 

Vendors sell all types of software, especially when it comes to fraud. In the private sector, these platforms are frequently used in the financial space or even in health care and law enforcement.

When the two software vendors in question approached the IRS, they claimed that their commercial platforms could simply be dropped into a complex, public sector environment and work wonders.

Thanks to an extensive knowledge of the IRS’ current fraud-detection models and processes, the IRS chose Booz Allen as that partner. Our team was tasked with developing an unbiased, apples-to-apples evaluation process for the pair of software platforms.

The IRS’ Office of Compliance Analytics looked to Booz Allen to help evaluate the platforms. Together, IRS staff and the Booz Allen team analyzed the platforms’ abilities to link incidents of fraud among different actors.

They also set out to determine if the platforms would identify new cases of ID theft that were slipping past the IRS’ homegrown systems. But before they could start the evaluation process, Booz Allen had to ensure that the vendors understood the IRS’ wealth of data and process

That meant educating the vendors on what happens when a tax return comes through the door: How is it processed? How is it evaluated for fraud? What do the individual fields mean?

The team confirmed that the platforms could be implemented in the IRS environment. Once the vendor teams were up to speed, their models were tested individually and then, ultimately, the results were measured on a larger scale before a decision was reached.

The final step in the evaluation process involved looking at the platforms’ results on a holistic level, comparing three crucial aspects:

1. How many cases of fraud were caught outside of the IRS’s existing models.
2. How many of those cases were false positives.
3. What was the total cost of implementing the platform.

Ultimately, one of the platforms proved to be the standout solution.