Still in the Dark Ages?

If you want to learn about online media in a certain market, one of the first steps is to get the correct numbers. Starting from the general demographics, going on with overall usage patterns and more detailed data about specific market segments, down to the hardcore access data of single websites.

Demographics

In Germany there are some pretty reliable resources for this kind of information. For the demographics we have two main sources: the annual ARD/ZDF Online Survey, published by the research unit of the big public broadcasters, and the (N)Onliner Atlas by D-21, a public-private initiative. Both are based on large-scale surveys with sufficiently big samples to provide for significant results. Whereas the ARD/ZDF study focuses more on the specific usage of online media, often relating it to the use of traditional electronic media like radio and TV, the Atlas provides for some interesting region-specific demographics, aiming at infrastructural development needs.

For China, this type of bird’s eye view is provided by the semi-annual surveys of the government-backed China Internet Network Information Center (CNNIC). They are roughly comparable to the ARD/ZDF Online Survey, even though due to their organizational background they focus a little more on technical details like domain name space, the number of IP addresses or international outlet bandwidth. Also the surveys, even though they inquire into questions like preferred applications etc, comparatively lack in detail, at least in the abridged english version that I could easily access.

Compared to the (N)Onliner Atlas, one is struck by the CNNIC surveys’ lack of geophysical differentiation. In an emerging market like China with its huge socio-economic differences between the big developed metropoles and the rural hinterland, and even between the few most advanced cities and the majority of other urban environments, one would expect some significant information about the whereabouts of the unavoidable digital divide. But I’ve found no province-specific data or information about the urban/rural divide. Maybe this is too sensitive information.

In a blog entry on the last CNNIC report, the Internet marketing expert Paul Denlinger has set up a wish list for future reports, including a more transparent methodology, more geo-specific differentiation and free access to the raw data. But apart from these critical observations, the CNNIC report is an important and always fascinating resource.

Site Measurement

I will skip over market research for certain media segments, which is a broad and interesting topic but not really pertinent to my immediate concerns. Instead I will go directly to the dirty details of site-specific measurement. This is how the success or failure of single media brands is evaluated, and the prevalent type of measurement reflects the relevant success criteria defining the market. Or so it should…

When it comes to this kind of usage statistics, Germany has for many years provided a highly standardized environment. Very early on a publishers’ organization called IVW defined a more or less transparent procedure for measuring page impressions (PI) and visits (V), and the monthly statistics of this organization, presenting the numbers for all important commercial media websites, are still very much a basis for measuring success in online publishing.

Of course the obvious shortcomings of the PI/V currencies have forced the publishers to inquire into alternative ways of measurement. Not only does the PI/V system give the wrong incentives – content providers hype their numbers by using a variety of tricks like offering overblown picture galleries – , it also doesn’t give any interesting data about audience, neither about the number nor about the socio-demographic profile of the users.

So another organization of web marketers was created, the AGOF, which set up a rather sophisticated three-tier measurement system, based on offline sampling data, online surveys and server statistics. The quarterly AGOF Internet Facts provide much more valuable information, not only about online media usage, but also about the users. The establishment of a reliable and transparent way to count unique users (UU) has been hailed as major breakthrough by the industry.

The downside of this is that the results are restricted to members of the organization, and that the measurement procedure still favors big content providers, not taking into account that small audiences can also be very attractive to advertisers if they open up the possibility of accurately targeting the intended socio-demographic profiles.

When it comes to China there still seems to be no such thing as the IVW or AGOF standards. According to the CEO of Youku, one China’s biggest video portal providers, internet measurement in China is still in the “Dark Ages”. In a widely read contribution to the business website Venture Beat, Victor Koo decries the substandard state of web metrics in China.

Koo, like most other Chinese, wouldn’t look to Germany for inspiration. The dominant role model is the US, where you have companies like Nielsen/NetRatings offering site census measurement to paying customers, or big marketing research companies doing industry-wide panel-based surveys comparing different content providers.

The problem with reliable third-party single-site census data, argues Koo, is that not everybody is participating, so no real comparability is achieved. You have to add the open secret that at present, lacking established standards and procedures, all chinese content providers significantly ‘beautify’ their numbers. So stepping out of the game and being among the first to be honest about these numbers is something that might be more damaging than helpful.

Panel-based industry-wide surveys on the other hand, like they are conducted by the highly-reputed Shanghai-based company iResearch, suffer according to Koo (and others) from the shortcoming that they don’t take into account the huge percentage of China’s netizens who access the Net in Internet cafés and schools, a problem partly due to governmental restrictions about the software to be used on these computers.

Let me add that panel-based data, like sample data, have the problem that they might not give too reliable information on small content providers like bloggers. By being fuzzy about the ‘long tail’ they distort the picture in a way not adequate to the Google-age of highly-distributed but still marketable web content. (Fairness demands to say that this is not so much a problem in China, where content provision is mostly restricted to a few big players, and nearly all of the many million blogs are published and marketed under big-media brands.)

Most other complaints of Victor Koo sound rather familiar: That the growing portion of dynamic, AJAX-based pages and the immense significance of streaming media demands a fundamental revision of the old, page-impression- or visit-based success criteria, asking for unique user- and usage-time-based paradigms. That the deplorable predominance of the old criteria results in most of the advertisement budgets going to the old-style text-and-picture portals. Koo also notes that advertisement in China is still mostly sold not per CPM, but per time unit.

In conclusion, I’d like to say that a standardization (like Victor Koo calls for) might be a blessing in many, if not most respects. But the german experience shows that there are also negative consequences. With the institutionalization of measurement standards (like the IVW and AGOF measurement procedures) the system gets very inflexible with respect to any kind of innovation. Newcomers with new business paradigms would have huge problems getting the necessary consensus to get their angle incorporated into the system. Another example is the standardization of advertising formats in Germany, which has led to a ridiculous and tedious monotony in german online media design.

So you better be careful what you ask for.