What Comscore's Traffic Numbers Really Mean
Posted 10/06/2006 12:54:00 PM |
As you can clearly see, the statistic being reported is unique visitors, a common web analytics statistic indicating that an IP address has made one visit to a website. Unfortunately, the statistic being reported in the press is user, which is distinctly different from a unique visitor.
Let's consider the following case. A parent knows that their child has a Myspace page. That parent visits Myspace.com, attempting to learn about the service. In Comscore's index, they would be validly counted as a unique visitor. However, in conflating unique visitor with user, the press and blogosphere is inaccurately assessing the age range of the site's userbase.
If we can, lets step back and think about this objectively. Do we reasonably believe that 33% of Facebook's userbase is between the age of 35 and 54? In browsing graduate student use of the Facebook, we can clearly see that even grad student age range skews very young. Applying this logic to Xanga, a youth social network - do we reasonably believe that 34% of Xanga's userbase is between 35 and 54? Of course not.
Comscore's methodology is sound here, but the breathless conflation of user and visitor (guilt: Blogosphere, Media and Myspace's press spokesperson [1]) is terribly misleading. So how can we explain this phenomenon? We can elaborate, and use the other age ranges as a check. Let's first state a concept - that a majority of the 35-54 users are actually parents of young people, who are making visits to social websites to learn about the sites their children use. We'll then state a hypothesis that these users are not actually joining these social networking sites in any significant trend. To support this hypothesis, we will examine the relationship between visit trends and age-appropriateness for each age range. If we determine that a positive relationship between visit trends and age appropriate social network sites exist for our age ranges, we will declare that a relationship exists. We will then see if any similar relationships exist for our 35-54 age range.
First, lets look at our other age ranges. A positive relationship will be defined as significant derivation from the mean for our age-appropriate SNS inside the age range. For our youngest users, Xanga is the appropriate SNS. For our college-age users, Facebook is the appropriate SNS. For our 25-34's, Myspace and Facebook are our appropriate SNS [2]. For each of our age ranges, we can see that visit trends derivate significantly from the mean for our context-appropriate SNS. This allows us to conclude that the existence of a significant positive relationship between "using" an SNS and deviation from the mean. Therefore, if no derivations from the mean exist in our 35-54, we will declare that no relationship exists - meaning that those age 35-54 do not actually "use" social networks "more" -i.e., the notion is meaningless.
As we look at our 35-54 age range, we see very little derivation from the mean across our four social networking services. This lack of variance indicates that parents demonstrate non-selectivity in social networking sites. Apparently, this age range simply "visits" all sites, and is non-preferential. Since social networking sites clearly target age demographics, doesn't this seem a little odd? Of course it does - because our visitors actually aren't becoming members of the site, they are simply visiting them for more information. The above 35 "members" of the sites are simply coming out in the wash.
The point I'm getting at is this: we see a clear positive relationship between unique visitor variation from the mean for context-appropriate SNS. More young people visit Xanga. More college students visit Facebook. More twenty- and thirty-somethings visit Facebook. However, no 35-54 year olds demonstrate any preference (no significant positive relationship) toward any social networking site. These adults simply visit these sites, for many good reasons, but they aren't joining them in a significant trend. This is a rejection of a hypothesis that "more" adults are joining social networks. If anything, we can say that adults 35-34 aren't joining sites like Facebook, Friendster and Xanga.
While I think it is obviously the case that social networking is penetrating a wider demographic, the Comscore statistics can only tell us so much. They simply tell us who is visiting a site - for any reason - and they clearly don't show a relationship between adult use of SNS. The media has this quite wrong, unfortunately.
[1] In danah boyd's talk, she explains the non-reliability of self-reported ages in Myspace. Young folks frequently self-identify as being the maximum age possible, a non-symmetrical relationship.
[2]For the purpose of this comparison, Myspace will be considered an outlier due to its size.
Permalink |
|
to this post
View blog reactions | Post to
7 Comments: (Post a Comment)
-
At October 06, 2006 4:02 PM,
jkd said...
-
Just listened to a report on "Day-to-Day" that also embraced the conventional wisdom unquestioningly. Amazing how that brain-concrete sets in.
-
At October 07, 2006 2:36 AM,
Joe Suh said...
-
Great analysis. My question is who sponsored Comscore to do this study? I'm betting its the company that will benefit from it. How many more advertisers will be turning to Myspace now because of this study?
-
At October 11, 2006 1:41 PM,
JoeDuck said...
-
Fred -
No! It seems to me that you and danah are both pulling a new definition of "user" out of your hats unless their has been a change in definition. Maybe that change is needed, but it's not common in any metrics parlance with which I'm familiar.
The distinction you and Danah seem to suggest - that a user is not the same as a unique visitor - is unfamiliar to me. Where are you getting this definition? -
At October 11, 2006 1:45 PM,
Fred Stutzman said...
-
Joe, the definition of user we are operationalizing is a credentialed account-holder. Of course, a "unique visitor" is also a "user" of the site, but in this case we are thinking of user as an account holder. So - for a common example, a person visiting Facebook.com would be a unique visitor, and yes, a user of the site - but if that person does not hold an account, they are not considered a user in our interpretation.
I hope this makes sense! -
At October 11, 2006 7:20 PM,
Fred Stutzman said...
-
Michael Rubin of Comscore has posted the following boyd's blog. It is very illustrative:
We (i.e. comScore) would like to clarify some of the issues and answer the questions being raised in this conversation.
First and foremost, we don't rely on the age that individuals submit when they register for a MySpace account. Our demographic data are based on the ages of the individuals in a household that we record when they join the comScore panel. That means we do not need to use the age the individual provides when they register at MySpace. Any accuracies inherent in that are not reflected in our data.
Regarding the issue of "users" vs. "visitors":
* We use the terms interchangeably and do not mean to imply that a "user" of the site is necessarily a "registered user".
* As you rightly point out, our press release was talking about unique visitors. We anticipated there might be some confusion, so we made sure the headline clearly indicated visitors ("More than Half of MySpace Visitors are Now Age 35 or Older, as the Site's Demographic Composition Continues to Shift").
* The data we highlighted in the release does not speak to engagement or intensity of usage -- just visitation.
Let's put this whole story in context.
More than anything, an aging visitor base speaks to the fact that MySpace has filtered into the mainstream. While older visitors may be less likely to be registered users, it's still worth noting that they are being directed to the site one way or another:
* In some cases, they are linked to people's blogs at MySpace (especially from search results).
* In other cases, they are being linked to videos. Our Video Metrix data shows MySpace is #1 in videos streamed in the U.S.
* Or perhaps they are just curious to see what the buzz is all about and what their kids or grandkids or the media are talking about.
We hope this clarifies some of the issues being raised here. If you have any further questions, please don't hesitate to email us at press@comscore.com. -
At October 12, 2006 10:27 PM,
Paul DiPerna said...
-
Fred.. I really enjoyed this post and insightful analysis. I think you have been ahead of the rest of us on this controversy.
Great work.
- Paul -
At October 21, 2006 1:50 AM,
JoeDuck said...
-
Thanks Fred - by that definition I'm now understanding why you and Danah were so skeptical. As we see from Comscore's comments they are defining "user" with a marketing metrics definition.
I'm strongly guessing that a deeper analysis would find that youth spend much more time at these sites, so that at any given moment there are a lot more young people. Comscore can thus be correct and somewhat misleading at the same time.
