Bringing U.S. data to the masses

Government pooh-bahs live by the credo Information is Power. Here are a few of the guerrillas working to overthrow the resulting dictatorship.

* Carl Malamud has posted 20 million pages of federal court filings online for free, undermining the government’s fee-based PACER system and leveling the playing field between rich litigants and poor.

Malamud’s Sebastopol, Calif.-based nonprofit, public.resource.org, has also posted administrative and building safety codes from all 50 states, defying their copyright claims (and their steep per-copy fees). He’s also working to make court decisions widely available, challenging the commercial licenses granted to private companies like WestLaw and Lexis.

“We don’t think any state has the right to assert copyright over the law or give a corporation exclusive right to that law,” he told me.

* Ellen Miller is co-founder and executive director of the Washington-based Sunlight Foundation, which makes grants to nonprofits that improve public access to government data. Sunlight’s annual “Apps for America” contest recognizes the freelance software developers who have concocted the cleverest programs to slice and dice raw government data into digestible and edifying form.

This year’s winner was Datamasher.org, which enables users to juxtapose, or mash up, state-level statistics to illuminate policy issues. For example, which states have the highest graduation rates measured against their low income? (Best: Arkansas. Worst: Nevada. California’s rank: 47th.)

* Josh Tauberer started Govtrack.us as a college undergraduate. It’s now a one-stop shop for mashed-up data about Congress, its members and its bills.

“It was clear that Congress had lots of information but was horribly underutilizing it,” he says.

Fun fact: Which California member is the most successful at introducing bills later enacted? Sen. Dianne Feinstein, with 31. (She’s also the highest-ranking Democrat.) Who has the largest number of bills that never got out of committee? Among Californians, the prolific Feinstein again, with 488 dead ducks.

The work of Malamud, Miller and Tauberer points to a sea change taking place in the transparency of government: Technology is making it harder than ever for elected representatives to conceal how they’re spending our money (and making money -- the website Maplight.org tracks campaign donations and can match them, almost to the day, to votes by the recipient.)

The birth date of the transparency movement is hard to pinpoint. Some mark it as October 2006, when the nonprofit group OMB Watch, which monitors the White House Office of Management and Budget, launched FedSpending.org with a $334,000 grant from the Sunlight Foundation. The site allows users to search for information on federal contracts and grants drawn largely from two government databases reaching back to fiscal 2000.

But there had been plenty of action before then. Malamud, a pioneering insurgent, won a grant in 1993 to place the Securities and Exchange Commission’s EDGAR database of corporate filings online, breaking the near-monopoly then held by the commercial Lexis/Nexis service.

A few years later, he abruptly announced he was shutting down his website and invited its devoted clientele to direct their complaints to the SEC. Bowing to the inevitable, the agency made EDGAR available free of charge via www.sec.gov.

Malamud is now engaged in a similar effort to free Americans from the tyranny of PACER, which charges up to eight cents a page to download legal documents filed in federal courts.

Among his tools -- and the neatest transparency application I’ve seen -- is an add-on for the Firefox Web browser called RECAP (yes, PACER spelled backward), developed at Princeton University.

If you run a PACER search, RECAP will tell you via an on-screen icon whether a given document can be downloaded for free from its own database. If you download something not yet available, RECAP automatically adds it to the free database, in a sort of digital Kumbaya.

This cuts into the courts’ PACER revenue of $10 million a year, which may be why PACER has responded with a rather prickly warning about RECAP on its site.

Malamud contends, quite rightly, that “the whole point is that the courts should be available to the public. When they don’t do that to make money, they’re subverting their own purpose.”

An intriguing aspect of this work is its bipartisan appeal. “Transparency is one of the few issues that unites people across the board,” says Gary Chapman, director of the 21st Century Project at the University of Texas, which assesses the social implications of new technologies.

Malamud, a former staff member at the liberal Center for American Progress, is well aware that his work crosses the ideological divide.

“The Cato Institute loves me,” he jokes, referring to Washington’s leading libertarian think tank.

Another salient point is its bottom-up nature: The most important applications come from outside the government. Indeed, government agencies, confined by a web of overlapping information regulations, are terrible at distributing their own data in user-friendly formats. Try figuring out how to find a document on the Federal Communications Commission’s website, and you’ll know why some regulatory lawyers call it the “Federal Incommunicado Commission.”

Transparency advocates say the government should simply make all its information databases available to the public in raw, machine-readable form and let outsiders develop the tools to use them.

“There’s no way that agencies can single-handedly provide everything the public wants, so you need third parties to come in,” Tauberer says. It’s not in the mind-set of government webmasters to develop really penetrating tools, he says, nor should it be. “You wouldn’t want government to do what Maplight does -- you can’t expect government to be its own watchdog.”

While unofficial outsiders are indispensable for finding the best ways to force transparency on government, the value of support from the top can’t be overstated. The Obama administration has broken new ground in opening the federal data storehouse. One of Obama’s first acts as president was an openness initiative for all executive-branch agencies, and his White House chief information officer, Vivek Kundra, is likely to become a fixture at open-government conferences.

“People at the White House are returning my phone calls,” says Malamud, “which was not the case in the previous administration.”

The instinct of those in power to monopolize information to protect their position won’t die overnight, and if Obama discovers the political value of holding his cards close to his vest, or is succeeded by a president who does, could the progress of recent years be reversed?

Miller maintains, perhaps wishfully, that “the genie is out of the bottle.”

“There’s a cultural change taking place,” she told me. “It will be very difficult to stop putting information on the Web. Transparency online for government is an idea whose time has come, and it will not go away lightly. It’s hard to make an argument against it.”

--

Michael Hiltzik’s column appears Mondays and Thursdays. Reach him at [email protected], read him at www.latimes.com/hiltzik, and follow @latimeshiltzik on Twitter.