Normalized web distance (NWD) is a similarity or normalized semantic distance based on the World Wide Web or another large electronic database, for instance Wikipedia, and a search engine that returns reliable aggregate page counts. For sets of search terms the NWD gives a common similarity (common semantics) on a scale from 0 (identical) to 1 (completely different). The NWD approximates the similarity of members of a set according to all (upper semi)computable properties. We develop the theory and give applications of classifying using Amazon, Wikipedia, and the NCBI website from the National Institutes of Health. The last gives new correlations between health hazards. A restriction of the NWD to a set of two yields the earlier normalized google distance (NGD) but no combination of the NGD's of pairs in a set can extract the information the NWD extracts from the set. The NWD enables a new contextual (different databases) learning approachbased on Kolmogorov complexity theory that incorporates knowledge from these databases.
MiLQ: Benchmarking IR Models for Bilingual Web Search with Mixed Language Queries
Graduate School of Artificial Intelligence, POSTECH, Republic of Korea（POSTECH人工智能研究生院）；Department of Computer Science and Engineering, POSTECH, Republic of Korea（POSTECH计算机科学与工程系）；Lilt, Inc.（Lilt公司）
本文提出首个真实且较受偏好的混合语言查询公开基准 MiLQ，评估多语言 IR 模型并揭示其表现不一致，同时验证英语混用可通过增强 token 匹配改善英文文档检索。
Using Exclusive Web Crawlers to Store Better Results in Search Engines' Database
基于爬虫的搜索引擎是Web和Internet用户最常用的搜索引擎，它涉及到Web爬虫、数据库存储、排序、索引和向用户显示。但值得注意的是，由于网站的变化越来越大，搜索引擎遭受了很高的时间和传输成本，这些成本消耗在搜索时调查数据库中每个页面的存在，更新数据库，甚至在任何搜索操作中调查其存在。“Exclusive Web Crawler”提出了抓取特征、链接、媒体和其他元素的指导方针，并将抓取结果存储在其网络数据库中的某个表中。通过这样做，搜索引擎将每个站点的表存储在其数据库中，并在其上实现其排名结果。因此，确保了每个表中数据的准确性（以及它的最新性），并且在搜索结果中不显示404结果，因为事实上，该数据爬虫程序抓取网站管理员输入的数据，并且数据库存储他想要显示的任何内容。
