Extrapolation: RAT-STATs versus Monte Carlo

Whenever there is an extrapolated audit, we tend to attack it on two fronts. We hire an expert statistician to debunk the extrapolation, and a clinical expert will testify to the compliance clinically.

This two-fold attack has been successful throughout the years, mostly, I believe, because on the government side, their work is – how can I say it? Sloppy. I remember back in 2013, when I had one of many New Mexico clients. One of the clients was accused of owing over $12 million. The amount was extrapolated. On the stand during cross-examination, on behalf of the State of New Mexico, the assistant attorney general testified that he had conducted the extrapolation and that he was qualified, in his opinion, because he took a statistics course in college. We won.

Most extrapolations use the RAT-STATs model. I am no statistician. But I do know that most of the Centers for Medicare & Medicaid Services (CMS) provider extrapolations use RAT-STATs, which is a computer simulation that determines extrapolations. I actually assumed that RAT-STATs was a legally required simulation. I mean, if you are going to implement financial penalties against providers, shouldn’t the method be identical in all cases? These extrapolations declare a 90 or 95-percent confidence level, not 100 percent.

Two new extrapolation cases came in earlier this year. Both extrapolations used the Monte Carlo simulation, which I had not encountered previously. Obviously, I did some digging. A simple Google search will offer you a free two-week trial.

I turned to the regulations. It appears that no particular model is required for calculating overpayment extrapolations, but RAT-STATS is recommended by OIG.

Through my research, I discovered that our friend and frequent guest panelist of Monitor Mondays, Frank Cohen, had discussed the Monte Carlo simulation on RACmonitor back in 2021. He said that a small subset of Medicare Administrative Contractors (MACs) and Qualified Independent Contractors (QICs) were using the Monte Carlo simulation. But he also reported that $41.5 million of the $42 million involved in the Monte Carlo simulations were reversed due to its use. That tells me that the Monte Carlo method may not have a 90-percent confidence rate.

I phoned a friend – Sean Weiss, an expert in Medicare and Medicaid audit and extrapolations. He informed me that the Monte Carlo simulation method is basically a joke. Any auditor worth its salt would use RAT-STATs. So, why did these two recent audits of mine use Monte Carlo? I do not know yet, but will find out and report back.

As I said, OIG recommends RAT-STATs. If you visit OIG’s website, it says this of RAT-STATs:

“RAT-STATS is a free statistical software package that providers can download to assist in a claims review. The package created by OIG, in the late 1970s, is also the primary statistical tool for OIG’s Office of Audit Services.” (Did you know RAT-STATs was created by OIG? News to me!)

It goes on to note that among other tasks, the software “assists the user in selecting random samples and estimating improper payments. We have attempted to make RAT-STATS as user-friendly as possible, keeping in mind the program uses technical statistical terms. OIG does not provide technical support for RAT-STATS.”

Then, most importantly, it notes that although OIG does not require the use of RAT-STATS, many providers download the software in their efforts to fulfill the claims review requirements for corporate integrity agreements or provider self-disclosure protocols.

RAT-STATs versus Monte Carlo. I do not know the winner yet.

Programming note: Listen to healthcare attorney Knicole Emanuel’s “RAC Report” segment every Monday on Monitor Mondays with Chuck Buck.