Artificial Intelligence

Chinese AI DeepSeek Fails 83% of News Prompts, Raising Reliability Concerns

Chinese AI startup DeepSeek’s chatbot has been evaluated by NewsGuard, a trustworthiness rating service, and found to have achieved only 17% accuracy in delivering news and information. This performance placed it tenth out of eleven in a comparison with Western competitors, including OpenAI’s ChatGPT and Google’s Gemini. 

The audit discovered that the DeepSeek chatbot provided incorrect information 30% of the time and gave unclear, useless responses 53% of the time for news-related questions, resulting in an 83% failure rate. The failure rate among Western competitors averaged 62%, however DeepSeek’s performance fell short of this criterion, with an 83% failure rate.

Such an evaluation calls DeepSeek’s AI system into question, despite the fact that the business touted its technology as being equivalent to or superior to Microsoft-backed OpenAI while being less expensive to develop. According to performance criteria, the chatbot lags below its Western competitors in terms of providing factual news information.

The initial deployment of DeepSeek’s chatbot resulted in its position as the App Store’s top downloaded program, leading American technology stock prices to fall by $1 trillion and raising concerns about the United States’ supremacy in AI technology.

NewsGuard employed a series of 300 queries, which DeepSeek incorporated, similar to how it did for Western news outlets, which used 30 questions based on ten bogus claims circulating on the internet. The claims investigated included the assassination of UnitedHealthcare executive Brian Thompson last month, as well as the Azerbaijan Airlines flight 8243 plane disaster incident.

The audit also revealed that DeepSeek reiterated the Chinese government’s stance on the subject in three of the ten queries, without being prompted to provide any information regarding China. DeepSeek responded with Beijing’s stance on the Azerbaijan Airlines accident in response to inquiries that were not related to China.

D.A. Davidson analyst Gil Luria commented, “The importance of the DeepSeek breakthrough is not in answering Chinese news-related questions accurately, it is in the fact that it can answer any question at 1/30th of the cost of comparable AI models.”

Like other AI models, DeepSeek was most vulnerable to repeating false claims when responding to prompts used by people seeking to use AI models to create and spread false claims, NewsGuard added.

Sponsored
Tehniyat Zafar

Share
Published by
Tehniyat Zafar
    Sponsored

Recent Posts

Global Fashion Icon Jimmy Choo Inspires Ambitious Vision for Pakistan’s Fashion Future

ISLAMABAD: As the globally renowned fashion icon Professor Jimmy Choo visits Pakistan. A wide belief…

10 hours ago

Bybit Suffers Historic $1.5 Billion Crypto Heist in Largest Exchange Hack

Bybit, one of the leading cryptocurrency exchanges, has fallen victim to a massive security breach,…

11 hours ago

Pakistani Networks Face Cybersecurity Threats Due to VPN Vulnerabilities, Warns PKCERT

The National Cyber Emergency Response Team of Pakistan (PKCERT) has issued an urgent advisory highlighting…

11 hours ago

Pakistan Integrates into Africa-1 Submarine Cable Network, Improving Internet Connectivity

Karachi: Pakistan has taken a major leap in digital connectivity with the successful landing of…

13 hours ago

Meta’s AI Models Trained on Copyrighted Books? Court Filings Say Yes

Recently unsealed court documents have disclosed internal discussions at Meta regarding the use of copyrighted…

13 hours ago

IOC and TCL announce long-term global TOP Partnership through to 2032

Lahore: TCL, Pakistan’s LED TV brand, and the International Olympic Committee (IOC) today announced a…

13 hours ago