Subsidyscope is turning its attention to the transportation sector, and as we've dug into government spending data we've found everything from an extraordinarily expensive ham to a shipment of tigers bound for Iraq. But the real story is what the data doesn't show. Last month saw a flurry of stories about the poor quality of Recovery Act data, many of them prompted by the infamous million-dollar ham — a transaction record on Recovery.gov that made it look like the government had paid upward of a million dollars for two pounds of ham. In fact, the money was spent on a large quantity of ham in two-pound packages. The confusion caused by this ambiguous record led many to wonder about the quality of Recovery.gov's reporting.
As the Subsidyscope team continues its investigation into federal subsidies, we are becoming familiar with the Federal Assistance Award Data System (FAADS) and the Federal Procurement Data System (FPDS), the primary sources for Recovery.gov's data. Based on what we've learned, the million-dollar ham is only one cut of a very large data problem.
In truth, records like the one regarding the ham purchase are among the better ones available. Such records may be ambiguous or confusing, but they can be corrected with some additional investigation. We've found many cases where the data simply doesn't exist.
We've been comparing spending estimates from the government's Catalog of Federal Domestic Assistance (CFDA) with spending totals calculated from FAADS. These figures should, at least approximately, match. In many cases they do; in others it's clear that something is amiss. For instance, for fiscal year 2008 half of Amtrak's spending, or $660 million, is omitted. Some programs are entirely absent: according to FAADS, the Merchant Marine Academy (total FY08 budget: $62.7 million) doesn't exist. Our analysis uses the CFDA as a quality check on FAADS, but even the CFDA can be bafflingly inconsistent. For instance, a war risk insurance program for ships is included in the CFDA. A nearly-identical one for airplanes is not. It's difficult to know exactly how much data is missing, but the most recent analysis of FAADS by the Government Accountability Office found that 25 percent of the examined programs were under- or over-reporting their spending; another 22 percent weren't reporting anything at all.
USASpending.gov serves as the interface between the public and FAADS, but it does little to acknowledge the problem of stray spending data. A link to a discussion of "Data Quality" is displayed prominently on every page of the site, but the actual mechanism by which USASpending.gov tracks data quality seems woefully inadequate. The site appears to simply measure the completeness of each record that is reported to it. Identifying missing parts of a record is useful, but it does nothing to identify when whole records are missing. The system makes sure we've got a price on the ham, but can't figure out that the Maritime Administration, including the Merchant Marine Academy, has been mislaid. Currently there is no means by which USASpending.gov can ensure that it is receiving all of the records it should be.
These data quality problems are not new. Systems like FAADS and FPDS have suffered from inconsistent and inaccurate data since they were created. In 2006 Congress recognized that the existing systems had failed and passed legislation to create USASpending.gov. Unfortunately, the new system did little more than repackage existing data. The past three years have seen few improvements.
Under the new system, reporting remains incomplete and erroneous, and the resulting data remains unreliable. Both the current and former administrations have asked agencies to address the quality and timeliness of reporting, but as of yet, no true mandate for improvement has been issued and no measurements of success from these efforts have been reported.
At Subsidyscope we'll ensure that our analysis acknowledges the shortcomings of FAADS and FPDS by showing users where those datasets diverge from other spending data. Sadly, FAADS and FPDS remain the best available sources for transactional data about federal spending. But their limitations are real, and it's past time that they were improved.
Taxpayers have a right to know where their money is going. Good information on the costs of federal programs will be necessary if the nation is going to make informed decisions between and among federal programs. The problem of insufficiently descriptive spending data pales in comparison to the problem of data that is missing — and worse, not known to be missing. Until this is fixed, a million-dollar ham will be the least of our worries.