How much care is taken with stats in digital?


How many times have you been in a presentation with a digital guru and heard a whole lot of stats that on the surface sound too good to be true.

“xyz.com is growing at 15,000% per minute. It already has half the worlds population using it” or “if zzz.com was a country it’d be really big and bigger than many medium sized countries.”

Ugh.

You very rarely here a stat that isn’t one known in the biz as a “power stat” … the only figures we want to quote are ones that show massive growth, or lots of zeroes or big impactful statements around ‘the future’. I worked at one place and they had a collection of these on a folder on the network. “xxxx.com’s monthly users could fill the Telstra Dome 17 times” … erm, what does that even mean?

So I saw this article slapped across the front page of the SMH.com.au yesterday and it caught my attention as it had a big dramatic headline that grabbed me.

“Myspace loses market share as social networks soar”

“New figures released by Nielsen this week revealed that Australians spent 1.6 million hours on social media sites in June this year, up from 800,000 hours a year earlier.

“The two major beneficiaries of the social media rush have been Facebook and Twitter, which, according to Nielsen, now have 8 million and 1.5 million unique Australian users, respectively.”

1.6m hours. Sounds like a really long time.

Article is here – http://www.smh.com.au/technology/technology-news/myspace-tanks-as-social-networks-soar-20091014-gwxj.html

Then you have a closer look. Facebook has 7.9m users. Divide 1,600,000 by 7,900,000 and it works out at just over 12 minutes per person. Hardly earth shattering and not really front page news.

Unfortunately, the data is wrong. Very wrong. Actually ‘very wrong’ is an understatement And what makes it worse is ‘research experts’ are quoted in the article.

Here’s the issue. Facebook has 7.9m users … each user on average spends 374 minutes per month on the site.

7.9m multiplied by 374 equals 2,954,600,000 minutes … or 2.95 billion minutes. Work this back to an hourly figure and it’s 49.2m hours.

I’m quoting Nielsen numbers here too. Now this data is from August and the article refers to June data – but the increase between June and August isn’t enough to account for what is a massive difference.

49.2m hours per month for Facebook versus 1.6m hours per month quoted by the SMH for the entire ‘social media category’. That’s a pretty remarkable difference.

Now, I don’t think this is any more than a lazy journo looking for an easy story and not checking them against a source … but it does show as an industry we perhaps have a pretty lax attitude to how robust the data we claim as valid is.

How could this have slipped through? 1.6m hours across 8m+ people over a month is hardly news, is it? Or is it?

Advertisements

10 responses to “How much care is taken with stats in digital?

  1. its pretty much par for the course isnt it?

    look at how many of the stories (dare i call them articles, they do seem to make them up as they go along) on fairfax and news ltd news sites get comments about how bad their data is, or how poorly it has been interpreted, not to mention they used data as a prop for stories with no substance or worth to start with.

    its no wonder they dont enable commenting on every story they publish, they’d be drowning in the flames.

  2. Ben, I agree 100% with your sentiment.

    Numbers are published and taken as gospel and rehashed time and time again.

    I read the PR stories in AdNews and B&T on Tuesday and thought they looked weirdly low, but had a few audience measurement committee meetings that too precedence.

    I’ve had a think about it, and after about a half a hour it dawned on me. I’m pretty sure the explanation is quite simple.

    The reports in the trade press clearly say “1.6 million hours a month” in both AdNews and B&T. That is less than the Hey Hey fiasco racked up … 2 and a bit million people for a couple of hours is around 4-5 million, so 1.6m in a months is just simply way too low.

    While I don’t have access to the NetView data, the 374 minutes you quoted for FB in a month “feels right”. Unsurprisingly, your 49.2m hours is correct.

    Then it dawned on me. Divide the 49.2m hours by the 31 days in August and you get … 1.6m hours!

    Yep … it looks pretty certain that the data reported is an average of 1.6m hours PER DAY and not per month as reported.

    So, it looks like either (i) two different journos have made exactly the same mistake, (ii) someone in Nielsen Online did the conversion to “daily hours” but didn’t write the report properly or (iii) the PR writer stuffed it up.

    It’s pretty amazing that it takes a blog like this to pick it up!

  3. A follow-up post to provide some perspective.

    During the month of August the average daily TV viewing was 3 hrs 6 minutes a day.

    With 22 million people in Australia that’s about 67 million minutes of TV viewing a day, around 42 times higher.

    Mind you, at the rate that social media is growing … if it can sustain that growth (and we all know that growth must taper at some point of inflection), I predict that by the end of 2010 that ratio will be down to under 20 to 1.

    How big IS big?

  4. John, never leave this business.

    Ben and I were just talking about how big a TV country would be and true to form you arrive and ‘out stat’ everyone

    Gold and not a media wank phrase in sight

  5. Thanks DG.

    But sorry to disappoint, I think I am still off the mark. I wasn’t happy with the TV to social media ratio so I’ve dug a bit deeper.

    I’ve heard from Stuart Pike from Nielsen that while the actual numerals (the 800 and the 1.6) numbers are correct meaning that social media usage has doubled in a year (no surprise there), that the correct number for 2009 is 1.6 billion minutes per month and the correct number for 2008 is 800 million minutes per month.

    The articles in AdNews and B&T said they were “hours per month” and it is just a coincidence that Ben’s monthly Facebook data converts to 1.6m minutes per day – which was my late night attempt to work out what all these numbers meant. Memo to self: wait until morning and check things out … coincidences DO happen.

    Now we can confidently rework the numbers.
    1.6b minutes per month is 51.6m minutes per day compared to TV’s 67m minutes per day – MUCH closer to parity which makes much more sense, and makes my prediction look like shite! Revised prediction … more social media minutes than TV in the first half of 2010.

    But we still have to consider the potential to “make money” off this usage via advertising. TV’s ‘ad-density’ is much higher than on-line’s. The FTAs’ run 13 minutes an hour of ads and make up around two-thirds of that daily usage. The ABC make up around one-eighth of the viewing and run no ads. STV make up the rest and probably run half the ads of the FTAs (remember some channels take no ads like Disney and some of the movie channels).

    Putting this all together, my guess is around 15% of all “time” on TV is an ad (21% in FTA commercials, 12% in STV and 0% on the ABC). This means that every 6th or 7th viewing minute is an ad minute.

    Do we think that if we foisted a minute of ads every six to seven minutes on Facebook the users would tolerate this? I suspect not – they are there for the social experience and not the ads.

    The moral of this addendum is that usage minutes do not equate to ad dollars (yes they are correlated but they do not equate), so I don’t want to see any predictions that during 2010 the social media will be worth $3.7b of ad revenue !

  6. talkingdigital

    2 things stand out to me here

    1/ Is the incorrect number being reported an unfortunate coincidence or a Press Release error?

    2/ I think the ‘hours spent on media’ thing is sometimes over played. I think for TV it is valid … but it’s like saying – Australian’s spend 2.9b hours in office environments per month so as a result media budgets should shift to advertising within lifts and in office buildings. or that people use their mobile x hours a day so we should be investing more money in mobile advertising. JG is right – people are using a lot of these tools for the social experience and it’s ambitious and perhaps misguided to assume you can add advertising to the mix here and get away with it. They’re there to discuss inane shit and post self portrait photography in flattering light and not talk about brands and corporations surely …

  7. Great post, Sheps!

    Just one point, you criticise journalist and the industry for being a little lax with their numbers but then you come out with a broad brush stroke of a statement like ‘They’re there to discuss inane shit and post self portrait photography in flattering light and not talk about brands and corporations surely.’

    A bit of a simpleton example of the behaviour that is actually occurring on social networks.

    With the changes to Facebook Pages, brands now get to have a seat at the table within social networks. I can give you multiple examples proving this too.

  8. talkingdigital

    true jules – it’s a broad point i make and i am sure some people are discussing less inane stuff than i am.

    i have been reading your posts re the facebook pages etc and we’re using them too – and i agree, as a brand tool it’s improving and has a lot of potential.

  9. Hi Ben,

    Just thought I should jump in and say that upon investigation this definitely appears to be our fault as, due to some internal miscommunication, the relevant press release went out with incorrect units.

    It’s not something I am happy about or delighted to admit to but in the interests of owning up and demonstrating the sort of clear accountability we expect from others in this industry you can put this one squarely on Nielsen.

    I’d like to think this is a rare occassion and that we nearly always get it right but this time we didn’t … plain and simple.

    Regards,

    Stuart Pike

  10. Kudos Stuart and Nielsen.

    I could go on to say “it takes a big man to admit a mistake” … but you’d only hit me next time we meet …. chuckle, chuckle.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s