Sunday 09 February 2003

Stuck(?) in the middle, again

A TrackBack ping from diveintomark.org alerted me to Clay Shirky’s latest essay: Power Laws, Weblogs, and Inequality. Checking Phil Ringnalda’s site while writing my previous entry, I noted that he, like Mark Pilgrim, had commented on the Shirky piece, which seeks to explain why a relatively few bloggers garner most of the hits:

In systems where many people are free to choose between many options, a small subset of the whole will get a disproportionate amount of traffic (or attention, or income), even if no members of the system actively work towards such an outcome. This has nothing to do with moral weakness, selling out, or any other psychological explanation. The very act of choosing, spread widely enough and freely enough, creates a power law distribution.

Shirky uses as an example of the power law distribution economist Vilfredo Pareto’s observation that “wealth follows a ‘predictable imbalance’, with 20% of the population holding 80% of the wealth.” I was immediately reminded of a remark made 25 years ago by John Singleton who, in addition to being Australia’s most provocative and influential adman, had at that time a morning radio talkback program then a late night TV chat show.

I was working in the darkroom, listening to Singo on the wireless, when a young man called in to complain about the unequal distribution of wealth in capitalist societies. Singleton heard him out then said: “Mate, if we pooled all the wealth in Australia and distributed an equal share into every bank account in the country on New Year’s Day, within a fortnight the balances of those accounts would be pretty much the same as they were on New Year’s Eve.”

Though I was shocked by Singleton’s unsentimental statement, I was also struck by its essential accuracy, concluding that it was the role of government to attempt to hold the ratio at 20:80.

Yet, even though Clay Shirky’s explanation of the uneven distribution of traffic amongst weblogs came as no surprise, I can’t help feeling that his conclusion misses both the point and the real potential of blogging:

At the head will be webloggers who join the mainstream media (a phrase which seems to mean “media we’ve gotten used to.”) The transformation here is simple - as a blogger’s audience grows large, more people read her work than she can possibly read, she can’t link to everyone who wants her attention, and she can’t answer all her incoming mail or follow up to the comments on her site. The result of these pressures is that she becomes a broadcast outlet, distributing material without participating in conversations about it.

Meanwhile, the long tail of weblogs with few readers will become conversational. In a world where most bloggers get below average traffic, audience size can’t be the only metric for success. LiveJournal had this figured out years ago, by assuming that people would be writing for their friends, rather than some impersonal audience. Publishing an essay and having 3 random people read it is a recipe for disappointment, but publishing an account of your Saturday night and having your 3 closest friends read it feels like a conversation, especially if they follow up with their own accounts. LiveJournal has an edge on most other blogging platforms because it can keep far better track of friend and group relationships, but the rise of general blog tools like Trackback may enable this conversational mode for most blogs.

In between blogs-as-mainstream-media and blogs-as-dinner-conversation will be Blogging Classic, blogs published by one or a few people, for a moderately-sized audience, with whom the authors have a relatively engaged relationship. Because of the continuing growth of the weblog world, more blogs in the future will follow this pattern than today. However, these blogs will be in the minority for both traffic (dwarfed by the mainstream media blogs) and overall number of blogs (outnumbered by the conversational blogs.)

Interestingly, Mark Pilgrim appraises the situation in similar terms—as a dichotomy between high-traffic, mainstream-media blogging and low-traffic, dinner-conversation blogging—though he resolves the dilemma by valuing the quality of human feeling above the quantity of inbound links:

All the friendships I’ve developed in the past 2 years—starting long before I was in the Technorati Top 10—grew out of connections I made through writing this weblog and reading others. This month I’ll get 1 million hits on my weblog, and have lunch with 10 friends I met through my weblog. Guess which I care about more.

I suggest that the real action will occur in the Blogging Classic arena, which Shirky seems to dismiss as a kind of limbo—neither the “heaven” inhabited by Glenn Reynolds and Andrew Sullivan nor the “hell” where millions of Live Journal users are doomed to dwell.

Neither high-traffic (mainstream?) blogging (“a broadcast outlet, distributing material without participating in conversations about it”) nor low-traffic (conversational?) blogging (“publishing an account of your Saturday night and having your 3 closest friends read it”) hold out much appeal to me. (I’m ambitious enough to want an audience larger than 3 yet sufficiently realistic to admit that my interests are too esoteric for the mainstream.)

My instinct is that the real innovations in blogging will be made by those of us in limbo: without the pressures of producing for mainstream tastes but with the ambition to do more than chat amongst a tiny number of friends. The conversations we’re already having—about politics, relationships, geek stuff, and a lot more—go infinitely deeper than Shirkey’s “account of a Saturday night” and yet we haven’t even begun to scratch the surface of what’s possible. How that will transform the ways individuals and organizations interrelate remains to be seen but I strongly believe that this is one of those rare occasions where the excitement and potential exists in the middle rather than at the edges.

Permalink | TrackBack (9)

Comments

One question that came to my mind when reading Clay Shirky's essay was: 'And who will these main-stream bloggers get their info from?'

Well, if they can't anylonger read news and answer their email due to mainstream overload, I guess they in turn will be relying on inpiut from other, smaller 'blogs'. Jon Udell is a very positive example of that. He has a very prolific blog, but also read a great many smaller ones. Ans although he mostly links his source, I doubt very many of his reader will visit those very often. (I know I don't).

So there we hacve a mainstream blog that gathers all the traffic, but itself relies in part on the smaller ones that get very little traffic themselves.

And then, do we really _want_ that much traffic? Some of us pay for their bandwidth...

Posted by: Martin on 10 February 2003 at 12:05 AM

I think the key phrase here is "esoteric interests."

In my own case, I don't think literature, and particularly poetry, is popular enough to draw a large audience, but that's what I'm most interested in.

What's more, it turns out that I'm not even too interested in poetry as discussed at the college level. Instead, I'm merely interested in it as a tool for seeing life more clearly, an even smaller subset.

The blogs I read regularly all seem to have a similar focus. Though like yours and Shelley's many have a strong secondary (technical) interest, they mainly focus on leading an intelligent, informed life.

Strangely enough, judging from the media in general, that seems to be considered an "esoteric interest."

Posted by: Loren on 10 February 2003 at 04:09 AM

I posted on this. Clay's article is flawed from start to finish, beginning with flawed data, and continuing to extrapolations based on that data that are not born out by empirical observations, but do support Clay's myopic view of static social patterns.

Statistics -- boys with toys.


Posted by: Burningbird on 10 February 2003 at 05:35 AM

Well put.

I'd add that the "limbo" you describe is precisely what the web is fashioned to carry:

Low-cost + wide-distribution = a medium devised for niche publications.

High-traffic wide-circulation? That was taken care of by commercial broadcast and print media. Geographically dispersed conversations with your closest friends? That was taken care of by bulletin boards and email.

What the web UNIQUELY provides is easy access -- for both readers and writers -- to the material that's traditionally been been self-published or micro-published or circulated in manuscript: difficult to finance, difficult to find.

Since that sort of material makes up the majority of what's been worth reading through history, I've been infatuated with the possibilities of the web since 1992. But (possibly due to the early over-commercialization) it's really only with blogging that I've seen the potentiality coming to fruition, despite blogging's somewhat self-destructive privileging of updates over archives.

For myself, I hope to enjoy the fruits instead of complaining about the inevitable brush and bugs that surround them.

Posted by: Ray on 10 February 2003 at 05:43 AM

Martin, at a guess, I think the "mainstream" bloggers will have underlings -- in the form of producers and researchers -- just like other mainstream media professionals do. It will be the role of these staff members to keep up with what John Udell and other "smaller" bloggers are saying.

"interested in it as a tool for seeing life more clearly"
You clearly express my purpose too, Loren. From a mainstream perspective, that is (always has been and always will be) an esoteric interest. Absolutely worth pursuing, though.

Bb, though I read your post with interest, I'm not convinced that because Shirky's data is "tainted" it automatically follows that his observations are invalid *on a macro level*. It's just that, in stratifying weblogging into three "styles" (based on "static social patterns", as you put it), he neither addresses the possibility of innovation nor suggests where it is likely to occur.

Alex Halavais has some very useful observations to make on the subject:
http://alex.halavais.net/news/archives/2003_02.html#000268

Posted by: Jonathon on 10 February 2003 at 09:20 AM

Wow, guess I should posting then. Not mentioning ontology and all...

Posted by: Burningbird on 10 February 2003 at 09:54 AM

Shirky's observations are not tainted, they are circular: Where the data fits the curve, he claims an effect, where the data does not, he goes looking for another anecdote. Isn't it a circular argument?

Where I find my best disproof is in the dynamic nature of pop stardom. How many here lived in a house where you had a copy of Saturday Night Fever soundtrack? In Canada, by 1979, sales were such that every home in the country had a copy, yet when was the last time anyone here listened to it? Popularity fades, fashions, fads, whatever, the "in-crowd" is a fleeting thing, whereas Shirky's model would predict stasis, and rock-solid stasis at that. Instead, fashion and pop culture move ever more quickly in the networked world; where pop stars once reigned for years on the charts, today they last a few months.

And that's the problem: If you look at the distribution for the top Billboard artist, you might find Shirky's graph, but two weeks later the recommenders are pointing elsewhere --- so what happened at the mid-point of the transition? At that point, there is confusion, fuzz, noise whatever, at that moment all bets are off.

Shirky is applying a static analysis to a dynamic system, and applying a random choice to a situation where people and their pet likes and dislikes form the basis of the link qualities. how often do you really like the top site in the Google results? More to the point, in all other realms of life do you react to the "top rated" by avoiding it because it will be "lowest common denominator" and be most likely base and boring?

What Shirky says is probably true instantaneously and in a statistical sense that is useful for network capacity planning, but it is useless for this supposed issue of the "A-list" -- they are the A-list by the grace of Fate, and nothing more; what's more, we can deconstruct the current blog hysteria over this story by looking at the pro vs con counts on technorati. Do we see Power Law shadows again because, for those who's power is justified by the Shirky assumption, ie those with the most links, there is the most gain and knee-jerk support for saying blindly that this story is therefore ipso facto true?

By contrast, those off the A-list, those of us who are only the sublime rarities of dinner conversation, could we, with nothing to lose by it, be the only one's who'd dare say the Emperor is naked?

Mark Pilgrim, though, probably said it best: Like, who cares? Do we write to be read by a few who will understand and take it to heart, or do we write for the numbers game and all aspire to be Fox News? If the former, of what use is it to know the Shirky results even if it was true?

Posted by: mrG on 10 February 2003 at 12:50 PM

"What Shirky says is probably true instantaneously and in a statistical sense..."

Good of you to say so.

"...that is useful for network capacity planning, but it is useless for this supposed issue of the "A-list" -- they are the A-list by the grace of Fate, and nothing more"

Well, no. There is no real A-list, because there is no real discontinuity, but there are weblogs that get (and send) consistently high traffic. And I don't predict stasis, rock-solid or otherwise. Since the system is dynamic, I didn't pin my analysis on any given set of webloggers. There is always a power law, no matter who is up and who is down at any given point.

So the question becomes "Is there a group of webloggers who tend to stay at the top?", to which the answer is obviously yes, and the follow-on is "How long will they stay there?", to which the answer is "We don't know yet, but it has already been longer than songs stay at the top of the pop charts."

Posted by: Clay Shirky on 10 February 2003 at 01:37 PM

Clay mentions LiveJournal, and I really see no one paying much attention to that particular phenomenon around the traps today. Last I heard, there were more people writing 'blogs' with LiveJournal than with any other tool, and last I noticed, the overwhelming majority of those were of the "publishing an account of your Saturday night and having your 3 closest friends read it" variety.

Ignore them (or to use more emotionally charged language, ghettoize them) and you get an incomplete picture of the whole.

It amuses me, and is predictable, that people would respond with 'Who cares?' Obviously, we do, or we wouldn't spend so damn much time talking about it!

If I have a problem with what Clay was saying (well, I have a few, but) it would be to, by implication or explicitly, attach qualitative criteria to what he's describing, and thus create a hierarchy, where none exists in reality. That, I'm guessing, is in part why some people seem to have their back up over this.

Posted by: wonderchicken on 10 February 2003 at 02:40 PM

..er "it would be his attachment, by implication or explicitly, of qualitative criteria"

~Must learn to read comment before hitting post~

Posted by: wonderchicken on 10 February 2003 at 02:43 PM

Clay writes: "How long will they stay there?" and that's dead on -- they stay there longer than a pop idol /because/ there is a small population. Back when pop music had a comparable niche audience, say, circa 1953, idols lasted years.

Then, as the record-buying/radio-listener market expanded and matured, the power law dominance structure fell apart, or perhaps fractured or I don't know what but what is obvious is the length of a stay at the top went way down as the subscribing population increased, aged and diversified.

There are, as with any generalization, loads of anecdotes. I can point to Nana Mouskouri as one who's stayed pretty much at the top for decades, and Sinatra too, but if Billboard Magazine, or any of the fan-zines, if these are any indication, then the only reason you can /spot/ an a-list today is because, like it or not, bloggers are a fringe in-group, a microscopic subset of the half-billion people online. They are also constrained to particular ethno-demographic in (I will wager) exact proportion to the power law subscriber observations.

There was one day when, I don't know what went 'wrong', but 3/4ths of Blogdex's Top Sites were Brazillian. It was a wonderful (and refreshing) tour of a completely different world.

Unfortunately it wasn't the herald of the world finally in order, it was a glitch. The next day, we were back to the usual familiar list of famous geeks again, not one of which saying anything the least bit interesting to anyone I might meet in my local grocery store or Chamber of Commerce meeting (_I_ find them interesting, but I am talking about the everyday people I meet in and around my local village and neighbouring towns).

Posted by: mrG on 11 February 2003 at 08:23 AM

If you look at the distribution of book sales by author, you'd see Tom Clancy way at the left side and Paul Valéry and Charles Lamb way, way over to the right. (You'd need a very wide monitor.) If you like Tom Clancy, that's fine, but I don't see how that takes anything away from Charles Lamb.

Content--for lack of a better word--is not fungible. The only thing Paul Valéry and Tom Clancy have in common is that their labors resulted in artifacts printed on paper and bound in covers. (I don't mean that to be snide.) A category that is broad enough to contain them both is so broad as to be not worth worrying about.

The power-law distribution may be interesting from an abstract, mathematical, systems-theory viewpoint, but working backwards from the abstraction to reach conclusions about real readers would be catastrophically misleading. And for any self-web-publisher to do anything differently in light of the shocking revelation of the existence of a self-perpetuating, self-reinforcing "A-list" (Wasn't Mr. T on that show?) would be just sad.

Posted by: Pete on 11 February 2003 at 12:08 PM

Nice commentary. You write that the most interesting action happens in the middle of the traffic/link count distribution.

And I believe it is indeed so because *that's where the new and valuable interpersonal connections are being made*, every day. Big bloggers can't track their readers and commenters; diary-like bloggers don't care to.

Posted by: Seb on 12 February 2003 at 09:52 AM

Jonathon,

You said:

"I suggest that the real action will occur in the Blogging Classic arena, which Shirky seems to dismiss as a kind of limbo—neither the "heaven" inhabited by Glenn Reynolds and Andrew Sullivan nor the "hell" where millions of Live Journal users are doomed to dwell."

After reading Clay's essay, I got to thinking - there's a heck of a lot of great blog
content that comes from the masses who are not in the "Technorati top 100" (http://www.technorati.com/cosmos/top100.html ), but who have enough draw that they are good bets, and who have just posted something that people find interesting.

So, I came up with the Technorati Interesting Newcomers list (http://www.technorati.com/cosmos/interestingnewcomers.html ). Basically, I set the ranking algorithm to give more weight to people with a moderate link cosmos (but at least 40 bloggers are linking to them, so they have a reputation) who have said something that has caused a proportionately large number of new inbound links to come their way. Bloggers with a larger draw can still get on the list, but they have to add proportionately more links to their inbound link cosmos in order to do so.

It isn't a perfect system, but hopefully a random click on any of the blogs listed in the top 100 will lead to interesting reading, and perhaps, a new addition to your blogroll.

Dave

Posted by: Dave Sifry on 12 February 2003 at 12:28 PM

Dave, I note that as well as "Interesting Newcomers" (http://www.technorati.com/cosmos/interestingnewcomers.html) you have an "Interesting Recent Blogs" list (http://www.technorati.com/cosmos/interestingblogs.html).

I can't help thinking that both of these lists would instantly become far more *interesting* if you filtered out all of the blogs on the Top 100 list.

I can understand that you may be reluctant to do this for the "Interesting Recent Blogs" list and I realize you're still fine tuning the search, but at the moment rather than bringing up "relatively less-known blogs that have new, interesting content posted on them," this list mainly comprises Top 100 members who've posted new, interesting content.

Posted by: Jonathon on 12 February 2003 at 02:37 PM

Hello,

After everybody and my mother have posted their thoughs on Clay's paper, here's mines.

http://blog.scriptdigital.com/index.php?entry=/Review/review_shirky_powerlaw.html

Cheers

-Emmanuel

Posted by: Emmanuel on 18 February 2003 at 02:58 PM

This discussion is now closed. My thanks to everyone who contributed.

© Copyright 2002-2003 Jonathon Delacour