Google’s conversational search feature is already causing accuracy problems
If you are looking at YouTube today and you enter how to fix a leaky faucet into the search box, you will see the typical grid of video thumbnail results. However, for some U.S.-based YouTube Premium subscribers using this browser version, the same query will produce something new – an AI-created text summary describing how to replace washers in a faucet, along with video clip selections (timestamped) and other videos related to the subject. This is part of an experiment called Ask YouTube, which represents the platform’s most aggressive attempt so far to move away from being primarily a video repository and toward becoming a question-answering tool.
Ask YouTube uses AI to pull from short-form and long-form videos, Shorts, and text-based sources to generate responses to queries entered into the search bar. In the experiments conducted by Android Authority, Ask YouTube was able to process additional questions based on the first question entered and maintain the context of each question throughout a multi-question conversation. Although the ability to engage in conversations is a useful aspect of Ask YouTube, the fact that the AI often creates details that don’t actually exist in the source material means that there are fundamental accuracy issues.
In Mobile Syrup’s experiments, the AI created specifications for discontinued hardware with confidence and inaccurately stated the processor details and release date for the items searched. The AI generated plausible sounding but completely fictional technical details about specific products queried. Both of these were simple searches regarding consumer electronics; they weren’t difficult-to-answer edge case or “trick” prompts.
These accuracy concerns mirror the problems that all major conversational AI systems have experienced — including ChatGPT and Bard. While each major conversational AI system has struggled with creating accurate and detailed responses to questions entered into their platforms, Ask YouTube poses unique risks because it may potentially create authoritative-sounding responses containing inaccurate information. For example, when searching for medical symptoms or seeking financial advice, users require accurate information; not invented text. Since YouTube already experiences challenges regarding the spread of misinformation via its search results, adding an authoritative-looking AI layer generating factual errors may exacerbate these problems.
According to NDTV Profit, Google understands the limitations of Ask YouTube. The company’s documentation detailing its testing conditions includes language warning users to verify any factual details provided by the AI. Although it seems that this would be sufficient language to caution users about relying solely on AI-generated content presented in the same visually prominent manner as featured snippets in Google Search.
Get the latest model rankings, product launches, and evaluation insights delivered to your inbox.
This development also appears to be driven by competitive factors. Microsoft has integrated its Copilot technology across multiple aspects of its Office and Windows platforms, prompting many other major tech companies to deploy AI-related tools rapidly. Meta has begun integrating AI-assistants within its messaging apps — including WhatsApp and Instagram. And Apple reportedly plans to announce similar developments soon. With over 2.7 billion monthly visitors, YouTube can’t afford to seem like it’s lagging behind the competition.
However, Google’s rush to deliver Ask YouTube highlights a key trade-off. YouTube initially became successful largely due to its reliance upon human-created videos. Now, however, the company is developing a product that could allow users to obtain answers from AI summaries without even viewing the associated videos. As noted by PCMag, although Ask YouTube integrates text summaries with video recommendations, if the text summary itself contains the answer to the user’s query — then what incentive does the user have to continue and view a video?
Additionally, while Google has developed the initial version of Ask YouTube — no solutions to monetizing the feature have been developed. The majority of revenue generated by YouTube comes directly from ad views and watch time. Therefore, if Ask YouTube allows users to quickly find answers to their queries via AI summaries — without requiring them to view videos — then how will creators be compensated? At this point in time, Google has made no statements as to how sponsored content will be incorporated into Ask YouTube.
At present, the feature is still limited to a very small test group. Users who meet the criteria can access Ask YouTube via youtube.com/labs. From here, users can interact with Ask YouTube in a format similar to a ChatGPT interface — where users submit prompts as text entries and receive formatted responses (including embedded video clips).
Premium subscribers in the United States can opt-in to test Ask YouTube by accessing YouTube Labs. Ask YouTube produces both text summaries of responses along with selected video clips and timestamps. Testing indicates significant accuracy issues with Ask YouTube — producing specifications and details that do not exist in its source materials. Additionally, Google has made no comments as to how creator compensation models will function if users are obtaining answers via AI summaries without having to view videos. Currently, Ask YouTube is only accessible by users aged 18+ with active Premium subscriptions.
It’s expected that Google will make iterative improvements to improve accuracy before expanding availability beyond its current scope. Nevertheless, a fundamental concern exists: Will YouTube seek to become a platform where viewers watch videos — or one where AI watches videos for viewers? How creators ultimately respond to this choice will impact whether they will be partners in YouTube’s future development — or merely serve as historical data for its previous evolution.