Finding the Data Before You Need It

Most data stories start the same way. Someone at a meeting says a number, and you wonder if it is true. The reporter who already knows where the numbers live can check it before the meeting ends. The reporter who doesn't spends two days searching, then quotes the official because the deadline came first.

So learn the map now, while nothing is due.

The federal shelf

A handful of federal agencies publish most of the numbers local stories need. They are free, documented, and go down to the county level for most topics. Bookmark these.

Census data lives at data.census.gov. BLS has a series search and a CPI inflation calculator. NASS puts everything into Quick Stats, which lets you filter by commodity, year and county and download a CSV. EIA's electricity data browser draws charts you can check against the raw tables.

Decennial census versus ACS

Beginners mix these up constantly. The decennial census tries to count every person. It asks few questions, and its numbers are counts, with no margin of error printed beside them. The ACS asks a sample of households a long list of questions every year, so its numbers are estimates that come with a margin of error. If you want how many people lived in a county on April 1, 2020, use the decennial count. If you want median household income, there is no decennial number at all. That is an ACS question.

The ACS comes in flavors, and the Census Bureau states the rules plainly:

• 1-year estimates: areas with 65,000 or more people. Most current, smallest sample, least reliable.
• 1-year supplemental estimates: areas with 20,000 or more people, a shorter list of tables.
• 5-year estimates: all areas, down to block groups. Built from 60 months of responses. Most reliable, least current.

For a rural county of 18,000 people, the 5-year estimate is the only one that exists. Say so in your copy: "the Census Bureau's 2020 to 2024 American Community Survey estimate," for example. A 5-year estimate describes a period, and writing it as if it were one year is a small, common error.

Reading an ACS margin of error

Every ACS estimate comes with a margin of error. The Census Bureau publishes these at the 90 percent confidence level. In plain words: if the survey were repeated many times, the range of estimate plus or minus MOE would catch the true value about 90 times in 100.

Two more tools from the same Census guidance come up often.

Adding estimates. Say you combine two tracts to describe a neighborhood: 1,240 renters (MOE 310) and 860 renters (MOE 270). The total is 2,100. Its MOE is approximately the square root of 310 squared plus 270 squared, which is about 411. The MOEs do not simply add, and the combined MOE is smaller than 310 plus 270 = 580.

Changing confidence levels. If you are comparing against a poll that reports at 95 percent, convert: divide the ACS MOE by 1.645 to get the standard error, then multiply by 1.96. The $4,950 MOE becomes about $5,898 at 95 percent.

State and local shelves

Federal data tells you about people. State and local data tells you about government, which is usually the story. Places to look:

• State open data portals and transparency sites. Many states post vendor payments, salaries and contracts. Search your state name plus "open data" or "transparency."
• Agency budgets and audits. Adopted budgets, the audited annual financial report, and outside audit findings. These are PDFs more often than spreadsheets, which is why the cleaning lesson covers PDF tables.
• State agencies for licensing, inspections, crashes, school enrollment and test scores, and well permits.
• Meeting packets. Staff reports often contain the exact spreadsheet a board saw, pasted as a table.

Asking for a database

Most government records worth having sit in a database: permits, inspections, jail bookings, payments, 911 calls. You do not want a printout. You want the export.

The federal Freedom of Information Act is direct on this. 5 U.S.C. 552(a)(3)(B) requires an agency to provide a record "in any form or format requested" if it is readily reproducible in that form, and 552(a)(3)(C) requires reasonable efforts to search electronic records. State public records laws differ in wording, so read yours. For Idaho requests, Jeff's free Idaho FOIA / Public Records Letter Generator builds the letter, and the Laws and Rules module in this course covers records law in more depth.

Ask for the layout every time. A column named STAT_CD with values 1 through 7 means nothing until someone tells you 4 means "expired" and 6 means "withdrawn by applicant."