On Friday I wrote how Rupert Murdoch and other newspaper execs are pretending to be upset that search engines like Google link to news stories on their Web sites.
Murdoch claims Google is "stealing" his content, but he knows the truth
is that Google sends a ton of traffic his way that he wouldn't
otherwise get, and the last thing he really wants is for Google to stop
doing that. Friday's posting pointed out that anyone can easily block
Google. All you have to do is go to your Web site's robots.txt file and
type in two short lines. After that, your site will no longer appear in
Google's search results. It's so simple, Murdoch could do it himself.
Google will even help him—there's a page on its Web site
explaining how. Of course, Murdoch is no fool. He knows he can do this,
but he doesn't because it would cost him readers, and money.
Now comes the next beat: the folks over at Mike Masnick's must-read blog Techdirt.com picked up on my piece, and a smart reader in the comments section advanced the story. He went to Murdoch's FoxNews.com
and took a look at the site's robots.txt file, to see just how far
Murdoch goes in his attempts to stop Google from raiding his cupboards.
This is what he found:
User-agent: gsa-crawler
Allow: /printer_friendly_story
Allow: /google_search_index.xml
Allow: /google_news_index.xml
Allow: /*.xml.gz
#
Sitemap: http://www.foxnews.com/google_search_index.xml [foxnews.com]
Sitemap: http://www.foxnews.com/google_news_index.xml [foxnews.com]
You
don't need to be a code monkey to understand what's going on here. As
the commenter points out, not only is FoxNews.com not doing anything at
all to block Google, the site explicitly points Google to its news
stories. To borrow Murdoch's robbery metaphor, it's as if he left the
front door of his house wide open, and planted a huge sign on the front
lawn that says, "Please come in and take anything you want. The valuables are in the bedroom." So much for Murdoch's faux outrage.
What
seems to get Murdoch and other legacy news execs so crazy is that on
the Internet, they no longer have total control over their stuff.
Murdoch made his pile when newspapers were virtually monopolies and he
could extract large sums from readers and advertisers. Now innovation
and competition have entered the marketplace, and readers and
advertisers have a wealth of new choices—and bargaining power—that didn't exist before. Murdoch is, understandably, not happy about it.
But
Murdoch's desire for old-style control is not only futile on the Web,
it's bad business. The way to profit online isn't by trying to hold
back the chaos of the Internet by hiding your stories or putting up
paywalls—another thing he is threatening to do—but by embracing the
chaos and letting it work for you.
Take
the chain of events that led to the blog post you're reading right now.
I wrote up that first item smacking Murdoch on Friday afternoon, and
sent it into the ether. Many other Web sites and blogs picked up on the
item, because that's how, you know, the Internet works.
Now,
a lot of the bloggers who took note of my piece cut and pasted huge
chunks of it onto their sites, sometimes with extra commentary of their
own, but not always. A few of them just took the whole thing and dumped
it onto their page. All of the blogs that picked up the story included
a link back to the item here at Newsweek.com, but I'm sure many readers
didn't bother to click on the link and read my original post, since
they got the gist of my story from reading the "stolen" version on the
blog where they found it. On many of the sites, my work was surrounded
by their ads.
Should
I be angry about this? The Murdoch way of looking at it is that
NEWSWEEK lost thousands of readers because these other sites
piggybacked on NEWSWEEK's content, and even placed ads against it. They
robbed us!
That
would be bad, if it were true. But the better—and I think more
accurate—way of looking at it is that those "thieves" made it possible
for thousands and thousands more people to read my piece than otherwise
would have.
If
Murdoch's Web world view prevailed, I'd have posted my piece and hoped
that people would come to NEWSWEEK to read it. That's the way it was
when newspapers ruled. You had to buy the paper to see what was inside.
But that makes no sense now. Instead, I had hundreds, if not thousands,
of strangers working to promote my story for me, for free. And even
though many readers who discovered my piece on some other site didn't
click through to NEWSWEEK to see the original story, thousands of them
did. The overwhelming majority of readers who did read my piece on
Newsweek.com did so because they learned about it on a site that linked
to it—including Slashdot, Techmeme, Fark, and, yes, Google.
None
of those people would have read—or even known about—my story if NEWSWEEK made it hard for aggregators and search engines to find it and
promote it on their sites. Do I deserve a portion of the ad dollars
those other sites might have made off my story? Not unless they deserve
a portion of my ad revenue as a finder's fee for sending me readers I
wouldn't have had without them.
And inviting the
"theft" of my work had other benefits. It led to that Techdirt
commenter improving on my original story by adding interesting new
information.
So
let's review: I can either have 100 percent control over a small,
closed market—which means everyone reads my stories on my site, but
there are fewer readers coming to my site overall—or I can have far
less control over a much larger market, which means many people will
wind up reading my work on other people's sites, but there will be many
more readers coming to my site overall. Oh, and by keeping the doors
open, I have the benefit of a continuous stream of new readers coming
to check out my stuff, which they learned about elsewhere—something I
won't have with a locked-up site that not as many readers will find.
One
last point: In this light, Murdoch's complaints seem especially
ill-considered because in the case of Google, we're not even talking
about other sites lifting his content. Google just links to his
stories. Readers who click on those links are sent to read them on his
Web sites, surrounded by his ads. So when it comes to Google, he gets
all the free traffic, and all the eyeballs on his ads. And again, if he
doesn't want all that rich delicious goodness, he can shut off the
spigot at any time. So just what is it again that he's complaining
about?