Do It Yourself Vertical Search Engine

There are a number of companies, big and small, trying to build a better search engine.  Whether it be vertical search engines, or more general search engines with better algorithms than Yahoo & Google, they all face a big hurdle to success.  Simply stated, the hurdle is the near zero marginal cost of doing a search.   And the more companies I see the less comfortable I get that any will cross that hurdle.

The promise of vertical search is a good one.  If the search engine focused on one sector, such as shopping, blogs, electronics or health, they don't have to crawl the entire web, and focus on much fewer sites and hone their algorithms to deliver more relevant results faster.  As a result, the company will attract more relevant ads, therefore both cost per click and click through rates will be higher than others.  Eventually, the company will be runaway success, reach out and conquer other verticals, and then eventually take over from Google.

Almost all of these companies, when fundraising, take a keyword, like "digital cameras"or "san francisco" and compare their highly relevant search results with the generic results you get from Google.  It looks great.  However, what's sorely missed is that when comparing search engines, one should not compare results for a given keyword, but compare the overall cost of finding the information that's desired.  If you type the wrong keyword, you may have to refine your search, lengthen the phrase, or reach out to page 2-3-4 of the search results to find what you need.  Eventually you find what you think is what you want.  It may take 1-2 minutes longer than in vertical search engine, but I claim that that cost is not a big problem for any consumer.  Let's test that hypothesis.  How many times have you searched something in Google and got frustrated concluding "I just can't find what I need!".  I would guess that that is never.

When a user tries a number of keywords or lengthens his/her phrase, what they are effectively doing is adding relevance manually.  That is what the vertical search engines try to do for you, but when the cost for the consumer doing it themselves is nearly zero what do they bring to the table?  If "soccer" is too generic, try "weekend soccer" or "weekend soccer san francisco", eventually you bring the relevance you are looking for and you do it happily without complaining.  So what's the real value proposition for the added relevance from a vertical search engine and how does that compare to the cost of acquiring the users and getting them to your web page routinely?  That's the hard question I have not yet seen the answer for.

As long as the cost of doing another search on Google is zero, I see a difficult time for vertical search engines to keep the relevance edge over time.  I would love to be proven wrong.

Comments

[Balaji]

Baris,

Nice post. Can you please shed some light on image search or media file search ? Do you consider them to be a vertical search or different ?

Coming back to other vertical search engines that you'd mentioned, I guess those companies are not just focussed on grouping all relevant information in one place. They are grouping it properly. For example, if I am planning for a trip to Lake tahoe, I would really want to goto a site where I can get everything that is required for the trip. I think that's what is focussed on all vertical search engines.

If I have to do that all by myself by searching on Google, then it's very likely that I'll miss something or ponder around for information.

Same is true with Health search. WebMD is still widely used for health related search and I heard Kosmix is getting traction too in that area.

[zheng]

Yahoo has launched the Search Engine Builder a few weeks ago. http://builder.search.yahoo.com/.

You build a vertical search engine limiting search to a set of sites of your choices. They even allow your ads on it.

[baris]

Balaji,
I don't consider image searh or media file search vertical search yet, because they are very primitive and not much more than text search on file names or tags.

I think Cem got it right here (http://www.typepad.com/t/trackback/5923224)
Vertial search has to do more than better search, be a mashup or bring in new content. Then it's not just search.

[Nitink]

Baris:

This is a great post - you make some good points. I agree
with you that if the only advantage offered by
Vertical Search Engines (VSEs) is to provide quicker and
more relevant results by limiting the scope of their
crawl/search to only a subset of the web, then that is
not enough to draw users away from Google, Yahoo, MSN et al.

But vertical search engines, by virtue of focusing on a
specific domain, can use their deep understanding of that
domain to help users solve important problems in
that domain - find a job, buy an iPod, research and book
a vacation. They can encourage communities in their area of
interest, integrate crawled data with proprietary content
and offer domain-specific comparison and configuration tools
based on structured metadata. I discuss this in detail in
this blog post on my blog. Examples of companies that are
already doing this are Shopping.com, SimplyHired and
Healthline.

What do you think? Is the addition of customized,
integrated content and services into crawled search results
enough to draw users away from the major search engines
for specific tasks?

[SolutionPoint]

I suppose more than search technology it the understanding of consumer search behavior that is more important to create a successful engine like Google. Technolgy that can appeal to masses has to be simple and marketed that way. Google excels in it plus it became a global brand synonymous with search + it has the first-mover advantage also. What you guys think? If you get time do visit my SEO Blog and put your comments.

[SEO Services Provider India]

Search Engine Optimization optimizes the content of website and build link popularity so that the website can achieve high ranking and visibility in several search engines.Seo Services India can not only provide SEO Services but also guide for maintaining the top position in Leading Search Engines for keywords.
Site : http://ww.ecodesolution.com

[Searchboth]

Searchboth.net is the first site to place google and yahoo side by side on one split screen. The web site takes the user's query and creates a browser window with two frames, with the results from Yahoo! on one side and those from google on the other. It has completely end up the hassles involved while searching different search engines at the same time.

[Search Engine Optimization Tips and Tutorial]

Good article; I never heard of vertical search engine before.

[auctions]

i will try vertical search engine

[Invierta en proyectos]

I look forward to reading more on the topic in the future. Keep up the good work! This blog is going to be great resource. Love reading it...

[generic viagra]

Help with symbols, maybe an image match search engine?
I have a necklace that an Aunt gave to me. She has a touch of dementia in her age and has no idea where this necklace came from, but I still love it. I could draw/ scan the necklace then upload and search it, but I don't know of any search engines that look for similar pictures rather than words. If you know of one that would be great!

[SEOP]

Using longtail keywords adds up your chance for higher rankings because you target more keywords, at the same time target more specific market.

[Tiffany jewelry]

Helping select the right care option for an aging relative or friend is a daunting task and a major decision. With one out of every two people in the United States 65 and older expected to ultimately need long-term care, CarePilot enables individuals to search for, compare, and select from more than 8,000 Medicare-certified home health care agencies.

[SEO Company]

Thanks a lot for this comprehensive article. Kudos to you!

[SEO reviews]

"When a user tries a number of keywords or lengthens his/her phrase, what they are effectively doing is adding relevance manually" - i had no idea of that before. Now this is one useful information i read here. Thanks

[Nike Vandal]

Our destiny offers not the cup of despair , but the chalice of opportunity .

[Seo Link Building]

This an interesting approach. It is common to take care of SEO from the individuality of ones' blog or web while leaving the incredible opportunity social media offers aside. This post clears up the way to better understand the way both activities interact and the steps to be taken to get the best results in your way to success.

[Kamagra]

What you mentioned before sounds very logic, but I can not understand why the company where I work didn't think this stuff before. I'm gonna share the idea with the owners, and maybe by this way I could help the company to grow more in the industry.

[Rolex Submariner]

Hello sir
how are you
i hope you are fine
sir this post is very nice.

[Rolex Sweep]

Good post again.Thank you for sharing, I hope you happy and wish you good luck! this helpful information.

[Rolex Watches]

This is really fantastic advice, thank you so much .

This post is one of the great post that i have seen...

[Sterling Silver Bracelet]

Great Article, I really like the way that you have presented with the images and make things really simple to follow

[Used laptops in karachi]

Nice post. This post is different from what I read on most blog. And it have so many valuable things to learn.
Thank you for your sharing!

[link building]

There are several companies and firms that provide link building services to clients based on their requirements.

[Windows 7 Activator verified torrent]

My pal sent me an email suggesting i read this. I love it, i bookmarked your site. Keep the updates coming.