Search results

The literature of the evaluation of Internet search engines is reviewed. Although there have been many studies, there has been little consistency in the way such studies have been carried out. This problem is exacerbated by the fact that recall is virtually impossible to calculate in the fast changing Internet environment, and therefore the traditional Cranfield type of evaluation is not usually possible. A variety of alternative evaluation methods has been suggested to overcome this difficulty. The authors recommend that a standardised set of tools is developed for the evaluation of web search engines so that, in future, comparisons can be made between search engines more effectively, and that variations in performance of any given search engine over time can be tracked. The paper itself does not provide such a standard set of tools, but it investigates the issues and makes preliminary recommendations of the types of tools needed.

Present and possible future developments in the techniques of document management are reviewed, the major ones being text retrieval and scanning and OCR. Acquisition, indexing and thesauri, publishing and dissemination and the document management industry are also addressed. The emerging standards are reviewed and the impact of the Internet is analysed.

The Internet gives access for blind and visually impaired users to previously unobtainable information via Braille or speech synthesis interpretation. This paper looks at how three search engines, AltaVista, Yahoo! and Infoseek presented their information to a small group of visually impaired and blind users and how accessible individual Internet pages are. Two participants had varying levels of partial sight and two Subjects were blind and solely reliant on speech synthesis output. Subjects were asked for feedback on interface design at various stages of their search and any problems they encountered were noted. The barriers to access that were found appear to come about by lack of knowledge and thought by the page designers themselves. An accessible page does not have to be dull. By adhering to simple guidelines, visually impaired users would be able to access information more effectively than would otherwise be possible. Visually disabled people would also have the same opportunity to access knowledge as their sighted colleagues.

To describe a small‐scale quantitative evaluation of the scholarly information search engine, Google Scholar.

Google Scholar's ability to retrieve scholarly information was compared to that of three popular search engines: Ask.com, Google and Yahoo! Test queries were presented to all four search engines and the following measures were used to compare them: precision; Vaughan's Quality of Result Ranking; relative recall; and Vaughan's Ability to Retrieve Top Ranked Pages.

Significant differences were found in the ability to retrieve top ranked pages between Ask.com and Google and between Ask.com and Google Scholar for scientific queries. No other significant differences were found between the search engines. This may be due to the relatively small sample size of eight queries. Results suggest that, for scientific queries, Google Scholar has the highest precision, relative recall and Ability to Retrieve Top Ranked Pages. However, it achieved the lowest score for these three measures for non‐scientific queries. The best overall score for all four measures was achieved by Google. Vaughan's Quality of Result Ranking found a significant correlation between Google and scientific queries.

As with any search engine evaluation, the results pertain only to performance at the time of the study and must be considered in light of any subsequent changes in the search engine's configuration or functioning. Also, the relatively small sample size limits the scope of the study's findings.

These results suggest that, although Google Scholar may prove useful to those in scientific disciplines, further development is necessary if it is to be useful to the scholarly community in general.

This is a preliminary study in applying the accepted performance measures of precision and recall to Google Scholar. It provides information specialists and users with an objective evaluation of Google Scholar's abilities across both scientific and non‐scientific disciplines and paves the way for a larger study.

The purpose of this paper is to evaluate Google question-answering (QA) quality.

Given the large variety and complexity of Google answer boxes in search result pages, existing evaluation criteria for both search engines and QA systems seemed unsuitable. This study developed an evaluation criteria system for the evaluation of Google QA quality by coding and analyzing search results of questions from a representative question set. The study then evaluated Google’s overall QA quality as well as QA quality across four target types and across six question types, using the newly developed criteria system. ANOVA and Tukey tests were used to compare QA quality among different target types and question types.

It was found that Google provided significantly higher-quality answers to person-related questions than to thing-related, event-related and organization-related questions. Google also provided significantly higher-quality answers to where- questions than to who-, what- and how-questions. The more specific a question is, the higher the QA quality would be.

Suggestions for both search engine users and designers are presented to help enhance user experience and QA quality.

Particularly suitable for search engine QA quality analysis, the newly developed evaluation criteria system expanded and enriched assessment metrics of both search engines and QA systems.

This paper explores resource discovery issues relating to New Zealand/Aotearoa information on the WWW in the twenty‐first century. Questions addressed are: How do New Zealand search engines compare with global search engines for finding information relating to New Zealand? Can search engines find everything that is available on the web? What are effective strategies for finding information relating to New Zealand on the web? What is the quality of NZ information on the web? What can librarians do to make NZ information more accessible on the web? Based on a study, it concludes that neither local nor global search engines are by themselves sufficient, and that to maximize retrieval a variety of engines is necessary. The NZ librarian can play a role in ensuring that NZ information is made both available and accessible. Although the paper discusses the situation in New Zealand, the results and conclusions are applicable to other countries.

The purpose of this paper is to present a review on the use of the recall metric for evaluating information retrieval systems, especially search engines.

This paper investigates different researchers’ views about recall metrics.

Five different definitions for recall were identified. For the first group, recall refers to completeness, but it does not specify where all the relevant documents are located. For the second group, recall refers to retrieving all the relevant documents from the collection. However, it seems that the term “collection” is ambiguous. For the third group (first approach), collection means the index of search engines and, for the fourth group (second approach), collection refers to the Web. For the fifth group (third approach), ranking of the retrieved documents should also be accounted for in calculating recall.

It can be said that in the first, second and third approaches, the components of the retrieval algorithm, the retrieval algorithm and crawler, and the retrieval algorithm and crawler and ranker, respectively, are evaluated. To determine the effectiveness of search engines for the use of users, it is better to use the third approach in recall measurement.

The value of this paper is to collect, identify and analyse literature that is used in recall. In addition, different views of researchers about recall are identified.

The purpose was to evaluate the effectiveness of Google (as an international search engine) as well as of Parsijoo, Rismoon, and Yooz (as Persian search engines).

In this research, Google search engine as an international search engine, and three local ones, Parsijoo, Rismoon, and Yooz, were selected for evaluation. Likewise, 32 subject headings were selected from the Persian Subject Headings List, and then simulated work tasks were assigned based on them. A total of 192 students from Ferdowsi University of Mashhad were asked to search for the information needed for simulated work tasks in the selected search engines, and then to copy the relevant website URLs in the search form.

The findings indicated that Google, Parsijoo, Rismoon, and Yooz had a significant difference in the precision, recall, and normalized discounted cumulative gain. There was also a significant difference in the effectiveness (average of precision, recall, and NDCG) of these four search engines in the retrieval of the Persian resources.

Users using an efficient search engine will attain more relevant documents, and Google search engine was more efficient in retrieving the Persian resources. It is recommended to use Google as it has a more efficient search.

In this research, for the first time, Google has been compared with local Persian search engines considering the new approach (simulated work tasks).

This study evaluates the retrieval of New Zealand information using three local New Zealand search engines, four major global search engines and three metasearch engines. Searches for NZ topics were carried out on all the search engines, and the relative recall calculated. The local search engines did not achieve higher recall than the global search engines or metasearch engines, but no search engine achieved more than 45 percent recall. Despite the theoretical advantage of searching the databases of several individual search engines, metasearch engines did not achieve higher recall. Of relevant pages for the queries, 36 percent were outside the .nz domain. Implications for searching for geographically specific information, and for evaluation of search engines, are discussed.

The purpose of this study is to explore the retrieval effectiveness of three image search engines (ISE) – Google Images, Yahoo Image Search and Picsearch in terms of their image retrieval capability. It is an effort to carry out a Cranfield experiment to know how efficient the commercial giants in the image search are and how efficient an image specific search engine is.

The keyword search feature of three ISEs – Google images, Yahoo Image Search and Picsearch – was exploited to make search with keyword captions of photos as query terms. Selected top ten images were used to act as a testbed for the study, as images were searched in accordance with features of the test bed. Features to be looked for included size (1200 × 800), format of images (JPEG/JPG) and the rank of the original image retrieved by ISEs under study. To gauge the overall retrieval effectiveness in terms of set standards, only first 50 result hits were checked. Retrieval efficiency of select ISEs were examined with respect to their precision and relative recall.

Yahoo Image Search outscores Google Images and Picsearch both in terms of precision and relative recall. Regarding other criteria – image size, image format and image rank in search results, Google Images is ahead of others.

The study only takes into consideration basic image search feature, i.e. text-based search.

The study implies that image search engines should focus on relevant descriptions. The study evaluated text-based image retrieval facilities and thereby offers a choice to users to select best among the available ISEs for their use.

The study provides an insight into the effectiveness of the three ISEs. The study is one of the few studies to gauge retrieval effectiveness of ISEs. Study also produced key findings that are important for all ISE users and researchers and the Web image search industry. Findings of the study will also prove useful for search engine companies to improve their services.