Posted on 2009-03-17 17:19
Robert Su 阅读(224)
评论(0) 编辑 收藏 所属分类:
工程相关
http://www.freepint.com/gary/direct.htm#top
大多数搜索引擎存在着非常大的问题，很多人已经意识到这个问题了。
现在的问题是，海量的网络有一些通用搜索引擎——谷歌、百度抓取不到的“看不见的网页”
这部分网页比例是比较高的；
特别由于AJAX 以及RIA的大量应用，crawler面临挑战不小……
待续
There's a big problem with most search engines, and it's one many
people aren't even aware of. The problem is that vast expanses of the
Web are completely invisible to general purpose search engines like
AltaVista, HotBot and Google. Even worse, this "Invisible Web" is in
all likelihood growing significantly faster than the visible Web
you're familiar with.
So what is this Invisible Web and why aren't search engines indexing
it? To answer this question, it's important to first define the
"visible" Web, and describe how search engines compile their indexes.