Last week, Univision released a survey of twelve thousand Catholics in a dozen countries across five continents. The idea occurred to them after the Vatican asked the world's bishops conferences to find out what their people think about a range of social issues and report back. But, as the Univision survey's executive summary notes, "the papal questionnaire is not an opinion-gathering instrument." True, it's not exactly reader-friendly (several dioceses chose to adapt it in order to make it more intelligible to the people whose views it was designed to gather). Nor were its results easy to compile. So Univision sponsored a large-scale survey that would adhere to contemporary standards of data collection, and allow us to say with a measure of confidence: This what the world's Catholics think now.

The results won't shock you. (The German and Swiss bishops certainly weren't surprised.) They represent "an alarming trend for the Vatican," because the "majority of Catholics worldwide disagree with Catholic doctrine on divorce, abortion, and contraceptives," according to Bendixen and Amandi International--the communications firm that conducted the study. (It's been published a few ways: as an interactive feature, a slideshow, and an executive summary--which explains the survey's methodology.)

The country-by-country breakdown also holds few surprises. Generally speaking, the more developed a country is, the less likely its Catholics are to fully agree with certain church teachings. So, while a significant majority of U.S. Catholics (59 percent) say that women should be ordained priests, 81 percent of Ugandan Catholics disagree (the breakdown is similar on the question of married priests). Of course huge majorities of American Catholics (88 percent) have no problem with the use of artificial contraception. Ninety-four percent of French Catholics support the use of contraceptives--edging out Brazil's 93 percent to take the top spot in that category. And when it comes to divorce, the percentages line up similarly: 60 percent of U.S. Catholics believe that being divorced and remarried outside the church should not bar one from receiving Communion, while 72 percent of Catholics in the Democratic Republic of the Congo agree with that church teaching. On gay marriage, most Catholics agree with their bishops: about 40 percent of U.S. Catholics oppose it, compared with 99 percent of Catholic Africans.

The abortion results are more interesting.

Twenty-one percent of U.S. respondents agree that it should not be allowed at all. Sixty-six percent say abortion should be allowed in some cases. And just 10 percent say it should be allowed in all cases. Pollsters have been asking American Catholics about this for years, and have consistenly found much stronger support for keeping abortion legal in all cases--usually it hovers in the 20 to 30 percent range. Have U.S. Catholics changed their minds? Or was there a glitch in the survey?

For his part, Bill Donohue is not impressed. "The media," he says, are "choosing not to discuss" the survey's findings on abortion and gay marriage. He's not wrong that those aspects of the data have not been emphasized (at least not in accounts I've seen), but it's not as though major media outlets have totally ignored them. They've just tended to look at the global figures (this is what Univision itself has accentuated--and, given the scale of the survey, it's hard to blame them). For example, the Washington Post reported the broad opposition to gay marriage among the world's Catholics, and noted that 8 percent of respondents said abortion should be allowed in all cases.

But Donohue goes further, attempting to undermine the survey's methodology:

The survey of 12 nations yields some interesting results, but first a note on its methodology. Asking people to identify themselves as Catholic is not a sufficient condition for drawing conclusions: we need to know whether they regularly attend to the sacraments, or not. The survey made no effort to distinguish between practicing and non-practicing Catholics.

True, a survey of Catholics that failed to take account of their frequency of worship would hold less value than--wait, what's that? The poll did ask respondents about their Mass attendance? And it's mentioned in the report? On page three? Roll tape.

"Of the more than twelve thousand Catholics surveyed, roughly 30 percent described themselves as infrequent attendees, defined for the purposes of this study as those who attend services only a few times a year...as well as those who never attend services." The survey instrument defines "frequently" as "every week / a few times a month." The report then notes that the data show "a clear divide" between those who go to Mass "on a regular basis" and those who don't. Granted the report could be clearer about how that divide shows up in the data, but, for example, look page eight of the executive summary. There you'll see that of all the Catholics surveyed, 72 percent of frequent churchgoers said that they support the use of contraceptives (90 percent of infrequent attenders agree). And on page twenty-one, you'll read that 60 percent of frequent churchgoers oppose ordaining women (32 percent of infrequent attenders have no problem with women priests).

That doesn't mean that reporting about the survey has been flawless. Much of the coverage has highlighted the aggregated margin of error for all respondents: .9. But "this is rather meaningless," according to Mark M. Gray, director of polls at Georgetown University's Center for Applied Research in the Apostolate. Rather, "it is the country-level margins of error that are more important," he told me. "These are about +/- 3 percentage points in each country."

What really concerns Gray about the Univision survey is not its margins of error, but the fact that the survey has so little to say about response rates and weighting. Often pollsters find that their respondents aren't sufficiently representative of the area they're surveying. Weighting allows them adjust answers to account for over- and under-represented groups. Let's say you're surveying a country whose population is 52-percent male, but nearly three-quarters of your respondents are men. You'd have to add weight to the women's answers and subtract it from men's. It's difficult to assess the accuracy of a poll without knowing how its responses were weighted.

But even if the report had been more transparent about weighting, its executive summary makes possibly unwarranted claims about the wider regions surveyed--Europe, Latin America, North America, Asia-Pacific, Africa. "But one simply cannot make generalizations when you have interviews in just a few countries in a region," Gray explained. "National culture really, really matters. The poll is only relevant to the countries where interviews were conducted."

What's more, Gray pointed out, the Mass attendance figures are "far too high," which is probably the "result of social-desirability" bias. This is always an issue with polling--and Mass attendance is often overreported, but "usually not this high," according to Gray. Still, he concluded, apart from that, Univision's "results are not too out of line with existing, recent data."

Perhaps an enterprising bishop might sneak a copy in when he travels to Rome in October to share his own results with Pope Francis.