Washington Post Has It Wrong: Streaming Video Files Are Not That Big A "Search Challenge"

The other day, the Washington Post ran a piece entitled Video Files Present A Search Challenge. The piece by Leslie Walker, as well as another article on our own site, both describe the obstacles that video indexing services such as the newly launched BlinkxTV and Yahoo! Video Search face.

I don't know Perl script from Pearl Jam, but if you are talking about searching the Web for streaming video files and then indexing them (as opposed to extracting searchable text from the video, which is difficult) I don't see what the big deal is.

Folks, this ain't hard. This isn't rocket science. It's barely computer science.

Several years ago, I worked on a video Web search project for the now-defunct Streamsearch.com, a $20 million-in-VC-backed guide to streaming video and audio events. They flew me out to St. Louis to build a database of hundreds of sites that regularly Webcast streaming video and audio content. The database of sites was added to Streamsearch's crawlers. Steered by a troupe of caffeine-addled programmers and creative GUI jocks, we harnessed the database into a site where visitors could search for live and archived Internet video by keyword, day, time, bit rate, and topic.

The business model failed in part because streaming event producers only wanted links to their spawning pages, not to the streaming URLs themselves. Also, the proprietary streaming media player guides wanted all the affiliate referral income for themselves.

Now, the situation is different. The encoding Balkanization problem has largely receded. Rather than requiring a specific streaming video player, many streaming video formats are mutually intelligible to media player software heretofore encoded for only one format. In fact, RealNetworks' Helix Producer is capable of authoring video streams for several other formats besides Real's.

Plus, thanks in large part to Google, robots have improved greatly since then. Google, plus the blessed arrival of RSS, enables content that once took crawlers weeks to find to be retrieved in seconds. Plus, with so many more broadband subscribers around then four or five years ago, streaming video is an established fact with better demographics, production values and usability.

I'm sure we could do better now.

So, as the temporarily "smartest man in the world" Lt. Barclay once said about an hypothetical advanced computer on "Star Trek: The Next Generation," "here's how you build it:"

Most video files have an extension that identifies them as having video content: .wmv, for example, means Windows Media Video, and .qt stands for QuickTime. Much in the way that Google news crawls 4,500 news sites in real time, a significant number of sites that offer streaming video could be crawled in the same real time for newly posted video file extensions.

Additionally, the websites for the three leading streaming video media players (RealPlayer, Windows Media Player and QuickTime) have timely indexes of live and on-demand streaming video content. With the right licenses, as well as descriptive, indexable phrases provided by event programmers - a video search service could crawl these indexes and insert cited programs and content into their own listings.

There are three other ways such an index could be augmented. Bots could be set up to crawl the Web for keyphrases such as "video," "live video," "on-demand video," and so forth. RSS functionality could be added to streaming video launch pages, thrusting their links out front and center. Finally, a streaming video index could add a form for Webcasters to submit their own listings for inclusion.

Memo to Sand Hill Road: If you fund a truly turbo-powered video search engine, it will be built. If its built, visitors will click. If they click, sponsors will pay. If the sponsors pay, you'll be paid.

Posted by Russell Shaw, December 19, 2004 07:41 PM | | TrackBack