assessing quality in scientific research and reporting

Below is an abridged version of a paper I submitted on 15 Sept. 2012 as part of my LIBR 501 class at UBC.

Problem! “Almighty Echo Chamber for Lies and Falsehoods”

Dr. Richard Cox observes that “For many, the computer and the resultant Information Age heralds a time when every person, with a modicum of cost, effort, and education, can harness more information in practical ways than ever before.” Nevertheless, such a wealth of information can lead to problems. Science journalist Jim Giles argues that “as well as acting as an almighty echo chamber for lies and falsehoods, the internet has given a more powerful voice to those who wish to sow confusion and conspiracy” (44). And even when there is no intent to mislead, poor reasoning, flawed research methods, and sloppy journalism contribute to the confusion.

Examples abound. In debates on important subjects such as autism and vaccines, climate change, the efficacy of homeopathic medicine, or, in an example from Giles, whether Obama’s healthcare reform will bring in so‐called “death panels” (46), it is difficult, in this internet echo chamber, to know which side to believe. Faced with conflicting data, internet users can become frustrated and misinformed at best, and risk actual physical danger at worst. Particularly in the area of scientific journalism, misinformation can be a serious problem.

Information professionals must be aware of the colliding perspectives on the internet and the media’s inclination to proliferate inaccuracies. At the same time, we should be aware of the grassroots efforts by bloggers and the like to address these issues, and how various new software tools are making this work easier and more inclusive.

Solution? “The Internet: Peer Reviewed”

Some promising software tools are starting to appear, and their arrival is of great interest not only to information professionals but also, indeed, to any consumer of content on the internet. “There’s a way to cut through the piles of nonsense on the internet,” says Giles in a recent article in The New Scientist (44). In “Truth Goggles,” he reports on various products that are, or soon will be, available to help readers separate truth from falsehoods. Among these is Hypothes.is, open‐source software that will allow users to annotate anything found online without fear that the content owner can revise or remove the comment (46). Hypothes.is inventor Dan Whaley calls it “the Internet, peer reviewed” (qtd. in Giles 46).

A key piece of the software is its ranking system, whereby users can rate each other’s contributions and, therefore, their credibility (Giles, 46). Just as you can assess the reputation of an eBay seller by checking how other customers have assessed her, read praise for or complaints about a pseudonymous Wikipedia editor by checking his talk page, and easily find the top‐ranked stories on Reddit, you will be able to see how peers rate a given Hypothes.is contributor. This rating, in conjunction with other factors such as her productivity and how she has been rated by randomly‐selected moderators, helps rank the quality of her contributions (Giles, 46).

Bad Science Bad

A big reason why we need tools like Hypothes.is is that the media are not sufficiently interested in bringing us the truth. This is not just an issue for gossip columns, which most of us already read with a healthy dose of scepticism; even so‐called science journalists are failing us. Science news with a sensationalistic spin gets attention, regardless of whether the journalist has properly read and understood the research, and regardless of the quality of the research itself. Ben Goldacre, who maintains the blog Bad Science, wrote a piece for The Guardian on the subject of the media’s role in spreading misinformation about the supposed link between the MMR vaccine and autism in children. He examined the take‐up of researcher Andrew Wakefield’s controversial findings:

Wakefield was at the centre of a media storm about the MMR vaccine and is now being blamed by journalists as if he were the only one at fault. In reality, the media are equally guilty. Even if it had been immaculately well conducted—and it certainly wasn’t—Wakefield’s “case series report” of 12 children’s clinical anecdotes would never have justified the conclusion that MMR causes autism, despite what journalists claimed: it simply didn’t have big enough numbers to do so.

But the media repeatedly reported the concerns of this one man, generally without giving methodological details of the research, either because they found it too complicated, inexplicably, or because to do so would have undermined their story. (“The Media Are Equally Guilty” 17)

In another article, this one for Significance, Goldacre describes how a study that found no statistically significant increase in cocaine use in children was misinterpreted and misrepresented in the popular press to such an extent that by the time it got to The New York Times, the headline read, “Cocaine Floods the Playground” (“When the Facts Get in the Way of a Story” 84).

No wonder people are confused.

Open Science Good

Hypothes.is and similar tools arrive at a time when we are already seeing an increased interest in openness, bottom‐up investigation, and the groomed assessments of the populace, thanks in large part to the wide adoption of social media. In the same way that open‐source software makes computer code available for anyone to review, use, and build upon for non‐commercial purposes, open science advocates want to make scientific research data available for examination by anyone, for the sake of the public good: “The more data is made openly available in a useful manner, the greater the level of transparency and reproducibility and hence the more efficient the scientific process becomes, to the benefit of society” (Molloy). The Hypothes.is software certainly sounds like it will support this goal. As one openscience.com writer puts it:

In terms of Open Science, my guess is that we’ll start to see authors publishing their articles straight onto a homepage or library repository, allowing for their work to be peer reviewed [through Hypothes.is] almost instantaneously. Meanwhile, journals will likely operate in a post‐peer review niche, whereby they collect the most valuable articles and publish them in a context where they increase said article’s reputation value. (Winters)

Rosie Redfield and #arseniclife

We have already seen how an open peer‐reviewing model might work. One of the best‐known recent examples is in the work of the University of British Columbia’s Rosie Redfield. In 2010, when NASA‐funded scientists reported that they had found arsenic‐based life in California, Redfield, a microbiologist, took to her blog to express concerns about the data and the quality of the findings. Thanks to social media, her assessment spread rapidly among her scientific colleagues and in the press, casting doubt on NASA’s results (Zimmer).

What became known as the #arseniclife affair “is one of the first cases in which the scientific community openly vetted a high‐profile paper, and influenced how the public at large thought about it” (Zimmer). In 2011, Nature magazine named Redfield among the top 10 “people who mattered” for that year (Hayden).

Conclusion

Tools that allow internet users to more accurately assess information quality are a positive addition to our online lives. In the realm of scientific research and reporting, Hypothes.is and similar products will inspire scientists to be forthcoming about their data and research methods, encourage journalists to take more care when describing and disseminating scientific findings, and give information professionals and consumers in general more confidence that what we are reading has been through some sort of transparent review and assessment process. I look forward to seeing this in practice.

[PS: Sign-up now for your Hypothes.is username!]

—–

Works Cited
Cox, R. “The Information Age and History: Looking Backward to See Us.” Ubiquity Sept. 2000. Web. 9 Sept. 2012. http://d-scholarship.pitt.edu/2698/1/r_cox_1.html

Giles, J. “Truth Goggles.” New Scientist 15 Sept. 2012: 44‐47. Print. http://www.newscientist.com/article/mg21528821.700-reality-checker-how-to-cut-nonsense-from-the-net.html

Goldacre, B. “The Media Are Equally Guilty.” Guardian 20 Jan. 2010: 4. Web. 24 Sept. 2012. http://www.guardian.co.uk/science/2010/jan/28/mmr-vaccine-ben-goldacre

‐‐‐. “When the Facts Get in the Way of a Story.” Significance 4.2 (7 June 2007): 84‐85. Web. 24 Sept. 2012. http://www.significancemagazine.org/details/magazine/868989/When-the-facts-get-in-the-way-of-a-story.html

Hayden, E. C. “365 Days: Nature’s 10. Ten People Who Mattered this Year. Rosie Redfield, Critical Enquirer.” Nature 480 (22 Dec. 2011): 437–445. Web. 29 Sept. 2012. http://www.nature.com/news/365-days-nature-s-10-1.9678

Molloy, J.C. “The Open Knowledge Foundation: Open Data Means Better Science.” PLOS Biology Dec. 2011. Web. 24 Sept. 2012. http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001195

Winters, J. “Hypothes.is: The Future of the Internet and Peer Review.” 11 July 2012. Web. 1 Oct. 2012. http://openscience.com/category/open-access/peer-review/

Zimmer, C. “The Discovery of Arsenic‐Based Twitter: How #arseniclife Changed Science.” Slate 27 May 2011. Web. 29 Sept. 2012. http://www.slate.com/articles/health_and_science/science/2011/05/the_discovery_of_arsenicbased_twitter.html

Advertisements

lululemon says “zeitgeist be damned!”

Today, the Globe has informed me that Lululemon, Canadian-born vendor of stretchy pants for soccer moms, ladies who lunch, and I guess maybe the odd well-off yoga-doing person, has a new slogan on its shopping bags: “Who is John Galt?”

This, of course, is from Ayn Rand’s sophomoric and interminable Altas Shrugged, one of several books Rand wrote devoted the values of individualism and consumption. It should come as no surprise that any for-profit corporation has this mentality; most just don’t say it so bluntly. I am not going to spend time here arguing with Ayn Rand. There are plenty of sites on the Internet that do so very nicely. And I am not even going to point out how strange it is to conflate libertarian values with those of yoga. For some fantastic comments on this odd confluence, check those that have been left on the Lululemon blog.

What is surprising, however, is that Lululemon executives have chosen this as a slogan now. At a time when we have seen Occupy movements spring up around the world, when there is a growing awareness of the divide between the 99 percent and the 1 percent, when the public is fed up by massive bailouts for the banks that have contributed to the economic mess we are seeing all throughout the West, when we’ve had enough of the bonuses received by the very people that have driven the US and other economies into the ground, when the people are speaking out and rising up against the privatization of infrastructure and institutions, Lululemon has chosen to go against the Zeitgeist. Dramatically. Which is pretty interesting.

Are they paying attention? Maybe they are, and there’s something I’ve missed, something that promises today’s public will love this sentiment. But I don’t think so. I think, at least in Canada, that the reaction will be more negative than positive, and that it won’t be long before the Lululemon bag returns to its previous vanilla feel-good sentiments.

“Let’s get sweaty, bendy, toasty, happy!”

what’s next, severed heads? stop extorting bc’s teachers

If I were to tell you that I’d just read about a group of teachers who were being forced to turn over 15% of their pay to a bunch of heavyweights, I know what you’d say.

“Yeah, in Mexico. To the drug lords. Yikes… Glad we don’t live *there*!”

Well, yes, terrible things are happening to teachers in Acapulco. Indeed, last month, five severed heads were left outside a school as a threat: an attempt to force teachers to hand over half their salaries to the thugs that largely control the city. I remember reading this, four or five weeks ago, and feeling shock that gangs would choose teachers — teachers, of all people! — to pick on. Teachers, who are already famously underpaid, overworked, and charged with one of the most important jobs of all: helping to raise their nation’s children to be informed and productive members of society.

Well, today we have the following headline in The Globe and Mail (print edition) “BC teachers asked to turn over 15% of pay.” (The online version now has a slightly different headline.) Forgive me for immediately seeing a parallel.

This is happening at a time when teachers in BC are stretched thinner than ever. In the last 10 years, in BC, we’ve seen deep cuts to our schools, affecting library services, counselling, special education, and ESL specialists. At the same time, class size has increased to the point where we now have the second-highest student-teacher ratio in the country.* Our teachers — working in these shameful conditions — are among the lowest paid in the country. And let’s not forget that they live and work in the province with the highest cost of living. At this rate, it won’t be long before our teachers are in the same position as were those in the Bay Area during the dotcom boom: they couldn’t afford to live in the communities where they worked, and so special low-cost teacher ghettos were built just for them. Can’t really call that “living in your community” either.

So our teachers are objecting to these conditions — can you blame them? — by taking very limited job action. So limited, in fact, that I (mother of three) have barely noticed it. This year, so far, my kids have had plenty of homework, plenty of projects, extra-curricular sports, a field trip or two, and teachers willing to talk to me about what’s going on in class. I’m not sure what they’re *not* doing, exactly, although I hear there’s now some question about report cards. So in return, the BC Liberals are planning to extort 15% of the teachers’ pay to apply sufficient pressure to end the job action.

If that’s not criminal, I don’t know what is.

And I believe that Christy Clark and the BC Liberals are simply wrong about the public’s attitudes about this. I don’t think that the voters of BC are behind this latest move from the government. Many of us are parents, after all! And even those of us who aren’t know how important it is to invest in our young people. When Carrie Gelson, teacher at Admiral Seymour Elementary in one of the poorest neighborhoods in Vancouver, wrote a heartfelt letter about her working conditions and those of her needy students, she was inundated with donations from a sympathetic and equally frustrated public. And Gelson’s not alone: her letter was followed by a tearful, public call for help from another BC teacher, this time from Duncan. And there are others. It infuriates me that we’re putting our teachers in this position: forcing them to make emotional pleas for help for the sake of the children they teach. It’s monstrous. But at the same time, I am encouraged by the responses that these teachers have received from a fed-up public. People seem to get it.

So call an election, Christy. We’ll show you what we think.

* Statistics from the BCTF website.