The Futility of Representing Religion With a Bar Chart

[rug]

It’s hard to understand faith without polling and data. But a lot can be lost in relying too much on statistics.

Emma Green | 7:01 AM ET

There is no sphere of life that polling hasn’t touched. The language of data is ubiquitous: X percent of Americans think this, Y percent identify as that. Even when they don’t actually say much, numbers carry a sheen of authority, which is part of why polling has become the unquestioned tool of choice for establishing Truth in the public sphere.

Not all statistics are created equal, of course. Many rely on tiny samples or skewed audiences or biased responses, or are produced by firms with a vested interest in reaching a certain conclusion. These biases can be hard to detect, or too much trouble to decode; and when statistics are reported in the media, they are often embraced with a misguided deference to factiness.

Bad stats are easy targets, though. Setting these aside, it’s much more difficult to wage a sustained critique of polling. Enter Robert Wuthnow, a Princeton professor whose new book, Inventing American Religion, takes on the entire industry with the kind of telegraphed crankiness only academics can achieve. He argues that even gold-standard contemporary polling relies on flawed methodologies and biased questions. Polls about religion claim to show what Americans believe as a society, but actually, Wuthnow says, they say very little.

In its worst form, this kind of critique can be self-indulgent, overly academic, and boring. It’s a micro-polemic: a big, philosophical attack on a topic that’s fairly narrow and small. Wuthnow’s critique, though, is ultimately concerned with how people derive knowledge about themselves, which is important. It speaks to the most basic project of public life: people collectively trying to figure themselves out, trading observations about the nature of existence as they all march steadily toward death.

http://www.theatlantic.com/politics/archive/2015/11/the-meaning-of-life-in-one-amazing-chart/412318/

[Jim__]

1. Sounds like an interesting book.

From Amazon:

Today, a billion-dollar-a-year polling industry floods the media with information. Pollsters tell us not only which political candidates will win, but how we are practicing our faith. How many Americans went to church last week? Have they been born again? Is Jesus as popular as Harry Potter? Polls tell us that 40 percent of Americans attend religious services each week. They show that African Americans are no more religious than white Americans, and that Jews are abandoning their religion in record numbers. According to leading sociologist Robert Wuthnow, none of that is correct. Pollsters say that attendance at religious services has been constant for decades. But during that time response rates in polls have plummeted, robotic "push poll" calls have proliferated, and sampling has become more difficult. ...

Inventing American Religion offers a provocative new argument about the influence of polls in contemporary American society. Wuthnow contends that polls and surveys have shaped-and distorted-how religion is understood and portrayed in the media and also by religious leaders, practitioners, and scholars. He calls for a robust public discussion about American religion that extends well beyond the information provided by polls and surveys, and suggests practical steps to facilitate such a discussion, including changes in how the results of polls and surveys are presented.

[rug]

2. It looks like he excoriates the pollsters.

http://ukcatalogue.oup.com/product/9780190258900.do

I wonder if his critique can spill over to political polling. This summer has raised some serious questions.

[Jim__]

3. In the Amamzon review, it says the accuracy of political polling can be known - I left that out.

... The accuracy of political polling can be known because elections actually happen. ...

Of course, that doesn't apply to any political polling done this summer. I do wonder how many Trump supporters are going along with the crowd as determined by the polls.

[MisterP]

4. it's a double illusion: that they're measuring someTHING (instead of providing a reassurance about

what categories to use, or just cherrypicking the respondents to get the results exactly right), and the illusion of numerization, of turning an issue into a set from 0.0 to 1.0