[Editor’s note: Laura Laing is a freelance writer and the author of Math for Writers: Tell a Better Story, Get Published, Make More Money. This post is the first stop in a virtual book tour she’s conducting to promote. Congratulations Laura! — Michelle]
It’s the age of Big Data, and numbers are everywhere. With a little Googling, you can find the number of gallbladder surgeries each year or where the U.S. ranks in childhood poverty or the amount of money shoppers spend each hour at Wal-Mart.
If you’re really paying attention, you’ll notice all statistics are not equal. The math of statistical analysis is pretty cut and dried, but not everyone plays by the rules. If you’re using numbers in a story, whitepaper or blog post, be sure you can trust where they came from.
Use Primary Sources
It’s so easy to misinterpret statistics that even the best journalists and writers goof at least once in their careers. Some outfits intentionally mislead with statistics. For those reasons, when using numbers in articles, it’s important to go to the primary source, rather than pulling data from another story or a third party’s review of the research. That means digging up the study or visiting the website of the organization that provided the number in the first place.
Reading a study can be tough at first, but you’ve got this! Start with the abstract or summary. Jot down questions that come to mind as you go along. Take a look at the tables and charts. If you really want to be certain you’ve got the details right, speak to the research authors directly.
Question Everything
Time to be cynical — very cynical. Even if you’ve read the study and talked to the authors, be wary of research funded by organizations that might benefit from the results.
Be careful how to report calculations and rankings. Rankings are particularly difficult to assess for accuracy, especially if the survey methodology is not clear. Washington Post reporter Max Fisher does a good job explaining this phenomenon in a post about Reporters Without Borders’ most recent rankings. Check how graphs and charts are presented visually to be sure the information is not skewed.
If you notice any problems with a study, skip it. Bad data leads to bad reporting.
Build a List of Go-To Sources
Your inventory of reliable statistics sources will vary from beat to beat, but a few tried and true resources are useful for most anyone:
- Government agencies, including the U.S. Census, Congressional Budget Office, Bureau of Labor Statistics and Centers for Disease Control and Prevention.
- Research-oriented universities and colleges such as Johns Hopkins University and University of Michigan – Ann Arbor.
- Journals with peer-reviewed research, including American Behavioral Scientist and New England Journal of Medicine. It’s important to note that not all research published in well-respected journals is peer-reviewed.
- Non-partisan think tanks such as Public Agenda and the Pew Research Center.
- Research-driven companies that are not funded by the industries they serve, including Nielsen and Harris Interactive.
This is not to say that you can’t trust some self-serving entities, such as the Democratic National Committee or the National Rifle Association. These types of groups could be the best places for certain data, such as accurate counts of Democrats, or research on the opinions of NRA members. Common sense rules.
Whether you’re using numbers to drive a story or simply adding interesting details, it’s critical to keep an eye on where statistics come from. Careful reporting and a little statistical know-how go a long way to ensuring that your readers get the facts – without the spin.
Visit Laura Laing’s website to see a full virtual book tour and sign up for a free, live teleseminar just for writers who need math: Math for Writers virtual book tour.
Thanks so much for having me, Michelle. It’s an honor to be here!
Great info! Laura, when you’re done with your book tour, maybe you can get to work on a book/series of articles to help regular ordinary citizens objectively assess and analyze the numbers they routinely encounter in news reports and articles.
Great idea Jennifer!
Michelle
That is a great idea! Thanks, Jenny.
Laura, how do you feel about “big picture” stats without context? Say I’m writing a story about credit card processing fees and I find a stat about the billions of dollars merchants spend on swipe fees each year but I can’t find a stat from that same source (a different source might be helpful but it wouldn’t be an apples to apples comparison) about the number of merchants that includes or the average each one spends on swipe fees. Is it still worth including that big stat to give a feel for the scope of spending on fees or is that stat worthless without more context? This is something I wrestled with recently, so I’d love your thoughts (and perhaps yours too, Michelle) on the topic!
Great question, Susan. You are right to be cautious about using data/results from two different sources in one context. One question I have though is this: If the source finds that $X billions are spent on swipe fees each year, why doesn’t it also report the number of merchants or cost of fees? Not to get too mathy here, but the total ($X billions) is the dependent variable; it depends on the number of merchants and swipe fees themselves.
In other words, that $X billions could mean that the fees are high or that there are lots and lots of merchants or both!
So, you can certainly use that big number as a reference, but you should be very careful about drawing conclusions based on that number. I’ve seen tons and tons of stories like this: “Merchants going broke on credit card swipe fees; $X billion spent each year.” The implication is that the $X billion is due to high fees not that there are lots and lots of merchants paying these fees.
Hopefully that helps. If not, shoot me another question. 🙂
Laura
Laura did a better job that I could of explaining how to parse a number. The other part of your question was what to do when you see a statistic in am article or report that you’d like to use but the source or other relevant details of the statistic aren’t given. That happens to me a lot because of the type of writing I do. I use a couple different methods to try to get to the source of the statistics, including:
(1) Doing a Google search on the statistic. In the example you provided, I’d use the keywords “$XX billions in swipe fees” to see if I could find the original source material or other reports on the numbers, eithe of which could include the data I was looking for.
(2) Contacting the source to ask for the numbers, either by email or phone.
(3) Going to other tried-and-true sources to find out if they have equivalent data that I could use instead. It might not be the same exact information, but if it’s close enough and I have a source I can name, I’ll use it.
Michelle
Laura…
What a GREAT post! Congratulations on beginning your Virtual Book Tour.
When I think about math, I usually think about solving mathematical problems. But you are so right, we need to broaden our thoughts about math. The statistical sources you mentioned in this article is FANTASTIC!!
I must confess, I added the Bureau of Labor Statistics to Laura’s list. I write about employment, jobs and workplace issues, and the BLS is my go-to source for data on unemployment, fast growing or declining jobs, and other work related statistics. Even better, the BLS website as a Java-based database tool that you can use to create your own mini-programs for looking up specific demographic data. For example, when I covered workplace issues for people over 40 for SecondAct.com, I used the tool once a month when the BLS published the previous month’s unemployment numbers to look up the current unemployment rate for men and women over 45, 55 and 65. I used the numbers as the basis for the stories that I wrote.
Here’s the main BLS website: Bureau of Labor Statistics
Here’s a link to the BLS’ data tools page: Data Tools
Here’s a link to another helpful BLS resource, the Occupational Outlook Handbook, which is published every other year and includes details about hundreds of job classifications: Occupational Outlook Handbook
Thanks for adding BLS to this list. There are so many great government-sponsored statistics resources, and that one is definitely at the top of the list!
Thanks to the three classes in statistics I had to take in journalism school, I know that stats are very easily misinterpreted—especially when it comes to the question of what is actually being measured.
So I’ve often found it useful to call the source of the statistic and discuss the numbers I plan to use and to ask if they mean what I think they mean…. Usually the people involved are very grateful that someone is taking the time to dig deeper.
Great idea, thanks for sharing.