Some years ago, not long after Herb Caen’s death, I decided to make a series of pilgrimages to the San Francisco Public Library to dust my hands and wrangle microfilm. I had known Caen’s three dot columns for some time. Or, at least, I thought I had known. When Caen passed away, as others dwelt on his coinage of “beatnik” and “Baghdad by the bay,” I felt that it was my civic duty as a San Franciscan to begin at the beginning, which very few at the time had thought to do.

As it turned out, in the late 1930s, Caen had started off as a nightlife columnist, attending swank parties and banging out his observations. What’s rather amazing about this old school epoch is that the newspapers once hired about five or six guys to go around town like this. They’d drink a good deal at upscale hot spots and write columns about their social engagements late into the night as their heads crashed with the competing crassitude of too much gin. When scanning through the microfilm rolls for Caen’s words, I was stunned to see photographs of other dapper gentlemen next to other columns. And I suspect that, beyond the prohibitive cost of scanning and providing all this online, the newspapers may not want you to know that they once actually paid whole armies of columnists of this ilk. This was, in short, a newspaper in which plentiful voices were represented, even on a seemingly pedantic subject. Here was a cadre of niche-specific columnists gathered together under one umbrella. And with multiple newspapers in town, there was a healthy competitive spirit that encouraged the columnists to do better.

You might say that these columnists were the bloggers of their time. And Caen, with his little snippets, certainly reflected the compact summation that Izzy Stone would later offer by mail and bloggers would later present through the roundup format (which has subsequently gravitated to Twitter, where the act of reader engagement becomes more explicit). But these columnists were different because there was an odd journalistic quality attached to these activities. You’d think that columns about running into dilettantes and drinking martinis would be somewhat superficial. But despite this emphasis on swank social tableaux, Caen always had a good eye for observation. He noted odd conversations and paid attention to the details around him. And he did this without belittling what could easily be belittled. (To compare this with the present epoch, we’re now expected to see a report of a party or an event from some snarky Gawker type. Easy targets are eyed and assessed. But what do we really learn about how this world works? Does Gawker really have the longer view in mind? Would it not be better if it dared to detail or if it dared to establish an off-the-record trust with which to convey the scene?) Because Caen was able to establish a trust with the social scene he was documenting, he was able to acquire details and, decades later, his columns remain immensely helpful. For instance, I learned from these old columns that there had been a chain of stores called the Martha Washington Candy Shop. (This was essentially the See’s Candies of its day.) The chain had inexplicably folded and there simply wasn’t any information about it on the Internet. So I began jotting down all of these details, compressing them into months and putting them all into a short-lived blog that I called Raising Caen.

Herb Caen, as we all know, became indelibly associated with the San Francisco Chronicle. He was a revered figure (and many attempted to cajole or influence him) because of his details, and because of his voice. There hasn’t really been a Chronicle columnist on that level since. Unless you count Mark Morford (Steve Outing draws the line), who provides an often frenetic metrosexual voice to the Chron. Hiring Violet Blue was a step in the right direction. The vanilla newspaper simply had to come to terms with the fact that they were circulating in a sex-friendly metropolis. But here’s the thing about Morford and Blue. Neither of them are particularly good at using their voices to get at those important details about a location or an event. Blue does interview people from time to time, but opts for a predictable Q&A format. What if her editors pushed her to give us multiple sources or a description of a scene? What if an editor demanded that Blue provided those vital details that made Caen a draw? As for Morford, his problem is that he is so caught up with wild conceptual approaches and stunts that we often don’t get a sense of Morford either (a) in the thick of things or (b) engaging directly with the community. (The alternatives to this, of course, are the dutiful Matier and Ross, the bland and voiceless Debra J. Saunders, and dependable cultural columnists like Tim Goodman. But what has caused this schism between voice and journalist? Why must it be an either-or proposition?) The newspaper columnist, who once served as a vital chronicler and detailer, is now viewed as an apparent draw only in so much as she can present a perspective. The columnist, in turn, deals with the public through letters and emails.

But perspective, as important as it is, simply isn’t enough. What made Caen such a local household name was his ability to include his readership within his columns. If he found a particular morsel, he would always attribute the reader who included it. His readers therefore felt a level of engagement.

One must therefore ask why Roger Ebert, aside from his television work and his Pulitzer Prize, remains such a household name with the Chicago Sun-Times. It is because he also engages directly with his readers. Consider his blog. Read through the comments and you will find Ebert personally responding to comments in bold. Ebert, like Caen, knows that a columnist’s responsibility involves engaging with his readers. What has changed, however, is the manner in which that engagement is presented to the public. What was once a series of private exchanges now becomes open to public scrutiny and dissection. But by including the readers in the manner that he does, Ebert offers his readership a place for their own ideas. His site remains a draw. Trolls are discouraged and a spirit of civil disagreement is maintained because the readers know that Ebert may respond to their comments.

In the past several days, many have fawned over Clay Shirky’s “Newspapers and Thinking the Unthinkable,” as if Shirky’s obvious and belabored points about newspapers failing to seize the possibilities of the Internet were new. What Shirky fails to observe in his section on micropayments is that Paul Krugman was, in fact, a big draw for the New York Times. When Krugman was behind a paywall, there were ways of obtaining his column. An informed perspective seemed to matter. And this wasn’t all that dissimilar to the rampant Dave Barry piracy with which Shirky initiates his essay. For that matter, we must ask whether those who clipped out columns (and there were many who did this in the pre-Internet days) were any less piratical than those who pass along a link to an article by email or Twitter. The information, I suspect, has always wanted to be free, even before this notion became a hip catchphrase. It’s wanted to be free whether a second-hand newspaper swiped from a cafe or a printout of a microfilm decades later. The real question is whether the columnist is fulfilling a public need. And by “public need,” I am not necessarily referring to a mass market. (A recent Minnesota Post article pointed to small local papers still doing well. The number of adults reading small community newspapers actually increased from 81% in 2005 to 86% in 2008.) The real question is why newspapers have failed to provide an atmosphere in which tomorrow’s Dave Barry or Herb Caen might be allowed a voice.

Small wonder then that readers have turned to blogs as a substitute for this. Indeed, since expanding the word count of these posts, I have seen readers refer to my posts as “columns,” as if I am fulfilling some journalistic duty that I did not anticipate. I leave the comments open to everyone and permit anyone to take me to task, if they must. But some of the more heavily trafficked blogs have not, contrary to Caen or Ebert, respected the readership like this. Love or hate Boing Boing, one of its key appeals involves massive strings of comments attached to each post. But Teresa Nielsen Hayden’s egregious disemvoweling strikes me as anti-communal and disrespectful of the readership. This autocratic arrogance is not advancing the case for trust between columnist and reader. And it’s just as bad on other sites. There was a time when, if you want to leave a comment at one of the Gawker sites, you were expected to “audition” for it. (Thankfully, this control has been relaxed.) There is, in these sites, a fundamentally antidemocratic act of disengagement. The commenter must humble herself to the blogger, and not vice versa. All of this fails to acknowledge the fundamental democratic ripple floating from from the undulations spawned by any newspaper columnist.

Shirky is right to point out how the exclusive informational terrain of newspapers has transformed. A specific journalistic item can be disseminated in a 140 character tweet, and it’s no longer new news. CNN’s scrolling news ticker has likewise suggested that audiences want their news in capsule form. But the successful journalism at Talking Points Memo works because the investigative process is now a part of the relationship between journalist and reader. This approach now permits a journalist to carry out his work and to obtain helpful tips with which to pursue a story. The reader, again, is engaged with the process. And instead of print people and bloggers seeing this dramatic shift in the presentation of information as an opportunity to do better and to attract a greater readership, they have instead declared war on each other. The Washington Post‘s Kathleen Parker writes a vitriolic column bemoaning the “drive-by pundits” who are pointing to the deficiencies of present journalism. A South by Southwest panel labeled “New Think for Old Publishers” sees publishers who aren’t providing new information to a paying crowd, but demanding this information from the audience. Instead of the print people listening to the criticisms and learning from these developments, they ignore them and refuse to listen. And the bloggers, in turn, don’t always consider that there are virtues in long-form journalism. In many cases, they wish to tap-dance on the hospital bed of the dead tree patient succumbing to a terminal cancer. (Jeff Jarvis is by far the worst offender in this regard.)

And when Shirky declares

Round and round this goes, with the people committed to saving newspapers demanding to know “If the old model is broken, what will work in its place?” To which the answer is: Nothing. Nothing will work. There is no general model for newspapers to replace the one the internet just broke.

the idea-slinging optimist in me wants to muzzle the man. Nothing will work? Really? Is it possible that the medium itself doesn’t matter? Will the Seattle Post-Intelligencer‘s investigative work be any lesser because the newspaper is now only available online? (Indeed, the big question is whether or not the Post-Intelligencer becomes self-sustaining if the costs of print production are reduced. As Nicholas Carlson recently suggested, it would cost the New York Times twice as much to print and deliver the newspaper in one year than it would to send every subscriber a Kindle.) If the local papers in Minnesota are attracting more readers, might it not have something to do with this broken covenant between the reader and the journalist? Might it not have to do with the information itself? Have newspapers seen their subscription base dropped because they have failed to respect the readers? And have bloggers been hindered from teaming up along the lines of the 1930s nightlife columnists because this has become a zero sum game predicated on one’s authority and rank on Technorati? Are bloggers and newspapers guilty in not respecting the old covenant?

The New York Times‘s dreadful practice of referring to a “well-known consumerist blog” without citing the URL that first established the connection runs counter to this spirit of connectivity, and the demands of the covenant. Technology chipped away at the verdigrised armor that we all begrudgingly accepted before the Internet spawned what Parker refers to as “drive-by pundits.” And I suppose this is the fruit of Shirky’s “unthinkable” proposition: the idea that print and online journalists might join forces and a more effective economic model will emerge. Because a fusion of voice, the journalist-reader covenant, and investigative journalism will become a must-read central point for all concerned parties.

When Maureen Dowd fixates on Michelle Obama’s biceps, she is breaking the covenant. When Lee Siegel impersonates a reader and leaves a comment in a desperate effort to feed his own hubris, he is breaking the covenant (indeed, so much so that he should not be invited to be part of the process). When Jeff Jarvis or a clueless publisher lets ego get in the way of listening to what somebody else has to say, they are breaking the covenant. The readers are intelligent and they want to be engaged. They want others to synthesize the information so that they, in turn, can synthesize it. They look to any columnist or journalist or blogger and they want to be engaged and challenged. They want voice and they want to be a part of the process.

The nice thing about the covenant is that it doesn’t necessarily mean that the journalist has to capitulate to the readership. The journalist can be as subjective or as wild as she needs to be. The only part of the deal is this: The journalist must listen. Particularly to the points of view that seem unseemly.