Top Ten Real-Time Search Engines

With Google and Facebook racing furiously to enter into the real-time search market, Twitter remains dominant on the world stage as to how news is obtained and consumed by the "wisdom of the crowds". From the immediacy of an airplane losing control over the Hudson River to the horrific moment-by-moment cries of revolt on the streets of Tehran, real-time search has dynamically changed news dissemination.

As a result, a new generation of search engines are emerging and competing with the big boys to tap into the torrent of tweets, blog postings and online photos that can be captured by cell phones. Check out the Top Ten Real-time search engines as they Beta their way into this soon-to-be very crowded terrain.

1- Scoopler
Scoopler aims to be the single online channel from which you can watch, read and listen to real world events as they happen. It aggregates and organizes content shared on the Internet instantaneously, similar to eye-witness reports of breaking news, with photos and videos from the events, and links to the hottest memes of the day. Indexing live updates from Twitter, Flickr, Digg and Delicious, they are able to surface some of the most relevant results, updated in real-time. Providing balance, its results page is divided into two columns, one with unfiltered live content and the other with results sorted by popularity..  Created by AJ Asver & Dilan Jayawardane, and launched in Private Beta in April 2009, the company has offices in San Francisco and London.

2- Collecta
Collecta streams real-time information of news sites, social media, popular blogs and Flickr. Launched just recently ,the service claims to be the fastest of the real-time search contenders. Its use of the XMPP instant messaging protocol (the same technology powering Google’s communication and collaboration tool Wave) allows it to show information that’s truly in real-time, rather than items that are simply recent. Collecta’s interface is user-friendly, as users can opt to review stories, comments, updates, photos, and video, or a combination of all options. Plus, should the flux of results become overwhelming, you can opt to pause the stream. Gerry Campbell joins the company as CEO with founders Jack Moffitt, Brian Zisk and Derek Powazek. Under its original name of Stanziq, the company raised a little less than $2 million in 2008.

3- Topsy
Topsy's search approach differs than the others in that it weighs a source’s authority and how many times specific content has been shared. It ranks the influence of certain individual Twitter accounts by measuring the tweets that attract responses and re-tweets from followers, while giving more weight to “influential” members of the Twitterati. Topsy sees the Internet as a stream of conversations all taking place at the same time. When you search for something on Topsy,it finds snippets of conversations that match what you’re looking for. It also has “trackback” pages for everything in its index, showing what everyone is saying about that particular topic. According to the angel investor and Listbot founder Scott Banister, "Topsy is so cool because it’s actually run by 42 hyper-intelligent pan-dimensional white mice at Topsy Labs, Inc., although our investors think it’s run by humans."

4- CrowdEye
CrowdEye describes itself as an ecosystem that has the ability to "tap into the stream of bits and bytes and use it to draw conclusions and make informed decisions. "As a new generation of search engine which looks at the worldwide web in a new way, CrowdEye has created an innovative technology to scan through tweets, retweets and links. They then provide users with a powerful yet easy method to slice, dice, summarize and categorize the data to answer questions. However according to a Mashable source, one of its deficiencies is its lack of access to Twitter's “firehose” of data and not having real-time access to the Twitter stream puts CrowdEye at a disadvantage. However, popular links being tweeted about one's specific search term can be highlighted, making CrowdEye a competitor to link-centric real-time search engines like Tweetmeme. CrowdEye is privately funded and was co-founded by Ken Moss, who previously founded and ran the Bing search technical team for Microsoft.

5- TweetMeme
Tweetmeme is a service which aggregates all the popular links on Twitter to determine which links are popular. Tweetmeme is able to categorize these links into categories and subcategories, making it easy to filter out the noise to find what your interested in. It actually crawls the page you link to, indexing it, and adding it to its search database. This enables some pretty impressive real-time search results, without the clutter of a thousand retweets and increasingly Twitter spam.  Perhaps the most impressive differential between Tweetmeme and the others is its 'retweet' widget similar to the other social bookmarking widgets like Digg. The button provides a fully integrated URL shortener and lists the headline or page title in its automated retweet. Each link which is posted on twitter adds one link to the link count on that story. Once the story has reached a critical mass of votes it can be ranked as one of the Top 10 on the front page (again similar to Digg). Tweetmeme is run by news aggregator Favorit and founder Nick Halstead.

6- OneRiot
OneRiot focuses on the actual content users are sharing via Twitter and Digg instead of their tweets and commentary. The company says it factors in 26 different criteria in its search engine's algorithm, including a link’s freshness, its domain authority or how reputable the Web site is, and velocity or the speed at which a link is shared through the community. OneRiot crawls the links people share on social bookmarking services, then indexes the content on those pages in seconds. Tobias Peggs, the company’s general manager, says OneRiot’s advantage is the level of spam guards the company has built into the engine and its scalability. The company has had two rounds of funding totaling $20 million from firms including Spark Capital, Appian Ventures and Commonwealth Partners.

7- Yauba
Yauba is a new, experimental, Indian search engine that was launched in "early beta/late alpha" mode on March 2009. In development for over 25 years, it promotes itself as offering "cutting edge research" from the Indian Institutes of Technology, the University of Delhi, the Massachusetts Institute of Technology, Harvard University, and the University of California at Berkeley. At Yauba, the approach to search rejects the view that search engines need to data mine their own users. Instead, they pride themselves on taking the opposite approach of protecting the privacy of their users. This rings true with the most recent election protests in Iran. Ahmed Hossain, CIO of Yauba, tells Pandia (a search engine news site): “Our traffic from Iran has jumped 300% over the past several days, as many of them are using the Yauba Search Engine and the anonymity proxy filter to access blocked sites and get news from foreign sources.”

8- itpints
Developer Javier Arias, describes itpints as a real-time search engine that weighs relevancy and immediacy as two major components of its effectiveness. Airas feels his point of differentiation from Yauba, Scoopler and OneRiot is their heavy focus on Twitter. With itpints, its functionality scours the entire Internet and then filters the results you want excluded. Results are displayed based on freshness, and come from multiple online resources such as Twitter, FriendFeed, FaceBook, and YouTube Another unique feature is its RSS Feed capability. If users want to be updated regularly, they can subscribe to the RSS feed and receive results directly into ones' preferred reader without the need to visit the site.

9- dailyRT
At its surface, dailyRT is a tweet aggregator that gathers the most popular tweets on Twitter and displays them using their own scoring algorithm and filters. Like many of its competitors, including Tweetmeme, dailyRT displays retweets in a real-time feed on its homepage. It also appears somewhat similar to Tweetmeme and OneRiot in that its search is more reliant on retweets and sharing.The key feature, however is one the others lack: the ability to search and find top retweeted topics from the past. "Hot tweets' displays tweets in sequential order based on their dailyRT rank. It also gives you the ability to search tweets based on keywords, and tweets containing links, videos, and/or images. You can also view tweets from people with a certain maximum and/or minimum number of followers.

10- is distinctive from the others, based on its 'event' feature. It provides a live stream of tweets and shared links and photos from any event that has been tagged. Once and event is established, its users who are actually on location can report back to the larger community. It does this by having users nominate Twitter accounts that appear to be tweeting live from the event for others to follow. Developer David Cann points out that his search engine allows event organizers to embed widgets so that group followers who can’t physically attend an event can receive a stream of content updates. The site also offers premium accounts for users who want to track terms outside the three or four events follows each day.

Real-time search requires a delicate balance of weighing relevance with immediacy and popularity. As Kim-Mai Cutler noted in a recent VentureBeat article without the proper amount of filtering you end up with "irrelevant chatter," and when you distill the information too much your search could could become watered down or lacking in its real-time delivery. It will be interesting to see how this new crop of search engines survive and thrive in the marketplace or if they will follow suit with the motley crew of search engines that were start-ups less than a decade ago. Anyone still remember AltaVista, Infoseek, Webcrawler and Lycos?

If you are familiar with any of the TOP TEN listed here, please take the POLL below and let us know the one you foresee becoming the dominant player. If you are still learning about them, please stop back and vote when you have had a chance to kick their tires and have given them a test run. Or if you know of one that I have overlooked, please leave us a comment with the company's name and URL.