AI chatbots in research practice: possibilities for searching and analysing scientific sources

In modern research practice, AI chatbots are gaining recognition as effective digital assistants that facilitate the search and analysis of scientific literature. In this context, the article presents a comprehensive analysis of the features of using AI chatbots in research practice, particularly for searching and analysing scientific sources. The research methodology is based on a mixed approach that combines a desk review of scientific sources with a comparative assessment of results from 5 AI chatbots (ChatGPT, Microsoft Copilot, Gemini, Claude, Perplexity AI) across 8 criteria. The analysis used quantitative and qualitative metrics to assess AI chatbot responses to the same set of research queries by experts from four fields of knowledge (Engineering, Natural Sciences, Humanities, Social Sciences). A comparative analysis of AI chatbots showed their different capabilities and limitations in working with scientific sources. ChatGPT is the most balanced tool, with high accuracy and objective responses, but it does not cover many sources and is prone to interpretation. MS Copilot is distinguished by the speed of response generation and the sufficient relevance of its sources but is inferior in the objectivity of its analysis. Gemini provides sufficient correspondence with the topic and objectivity in response, but the responses are often excessively long and less accurate in analysis. Claude demonstrates the highest indicators of relevance and objectivity by working with a small number of sources, yet makes almost no generalisations. Perplexity AI is the leader in terms of the volume and quality of the source base, forms generalisations well, but is characterised by the lowest relevance of sources and objectivity of response. The results showed that all the studied AI chatbots demonstrate a high level of formal reliability of sources, but differ significantly in the relevance of sources, analytical depth, and the ability to interpret the scientific content of articles adequately. AI chatbots are effective tools for initial understanding of a research topic, rapid collection of sources for analysis and preliminary structuring of the material. However, they cannot fully carry out a deep and critical analysis of scientific sources on a specific research question.