Search Smart suggests the best databases for your purpose based on a comprehensive comparison of most of the popular English academic databases. Search Smart tests the critical functionalities databases offer. Thereby, we uncover the capabilities and limitations of search systems that are not reported anywhere else. Search Smart aims to provide the best – i.e., most accurate, up-to-date, and comprehensive – information possible on search systems’ functionalities.
Researchers use Search Smart as a decision tool to select the system/database that fits best.
Librarians use Search Smart for giving search advice and for procurement decisions.
Search providers use Search Smart for benchmarking and improvement of their offerings.
We defined a generic testing procedure that works across a diverse set of academic search systems - all with distinct coverages, functionalities, and features. Thus, while other testing methods would be available, we chose the best common denominator across a heterogenic landscape of databases. This way, we can test a substantially greater number of databases compared to already existing database overviews.
We test the functionalities of specific capabilities search systems have or claim to have. Here we follow a routine that is called “metamorphic testing”. It is a way of testing hard-to-test systems such as artificial intelligence, or databases. A group of researchers titled their 2020 IEEE article “Metamorphic Testing: Testing the Untestable”. Using this logic, we test databases and systems that do not provide access to their systems.
Metamorphic testing is always done from the perspective of the user. It investigates how well a system performs, not at some theoretical level, but in practice - how well can the user search with a system? Do the results add up? What are the limitations of certain functionalities?
Goldenfein, J., & Griffin, D. (2022). Google scholar – platforming the scholarly economy. Internet Policy Review, 11(3), 117. https://doi.org/10.14763/2022.3.1671 [goldenfein2022platforming]
Gusenbauer, M., & Haddaway, N. R. (2019). Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of google scholar, pubmed and 26 other resources. Research Synthesis Methods. https://doi.org/10.1002/jrsm.1378 [gusenbauer2019academic]
Segura, S., Towey, D., Zhou, Z. Q., & Chen, T. Y. (2020). Metamorphic testing: Testing the untestable. IEEE Software, 37(3), 46–53. https://doi.org/10.1109/MS.2018.2875968 [segura2020metamorphic]