Bob Herbert Doesn't Understand Sampling

With the media's obsession over campaign polls, you'd think that a political commentator like Bob Herbert would have a basic understanding of sampling. But he does not.

Sen. Bernie Sanders (I-VT) showed Herbert a batch of heartbreaking letters from constituents describing the harsh economic realities of being middle-class in a bum economy. So Herbert spent his entire column today describing the letters and lamenting the disconnect between the "polling data and alleged trends," and "ordinary Americans."

But are the letters really from "ordinary Americans"—that is, do they accurately represent the economic circumstances of the typical middle-class citizen? No. As Herbert notes:

Senator Sanders asked his constituents to write to him about their experiences in a difficult economy.

So Sen. Sanders only solicited letters from constituents who are having a difficult time in a worsening economy? What a surprise that the overwhelming majority of the letters were from people who are struggling! Herbert, though, doesn't consider that the letters might be skewed (can't let those pompous statistics get in the way of a good story!):

The letters to Senator Sanders offer a glimpse into the real lives of ordinary people in an economic environment that was sculpted to favor the very rich.

Reporting anecdotal evidence can be very useful—it's important to recognize the realities underlying macroeconomic statistics. And Herbert is absolutely right that the middle-class is—and has been—under incredible economic strain. But reporting anecdotal evidence from a highly skewed sample is misleading. I'd say that Herbert knows better, but I really don't think he does.