Skip to content
report
Data quality: Verified
Global
Academic institution
AI & agent security
AI security
Consumer / individual
Data & ML teams
Data security & privacy
English
Human-AI interaction & UX
Identity & access security
Language (LLM / NLP)
MLOps & model tooling
Model developer
Model evaluation
Research organization
Researchers
Responsible AI, ethics & safety
Security teams
Software & technology

Google’s HelpBench tests AI advice on everyday cybersecurity

Google reports that 18 AI models averaged 82% on its cybersecurity advice benchmark, but roughly one in ten responses fell below a threshold associated with risky guidance.

GlobalGraph Editorial17 November 20261 min readSource: Google
Google says research conducted with University College London produced HelpBench, a benchmark for assessing AI responses to everyday cybersecurity questions. Its 450 curated questions cover concerns including scams, account access and privacy settings. Experts developed scoring criteria addressing factual usefulness, potentially harmful guidance and how clearly advice is delivered; an automated evaluator then graded model responses. According to Google, testing of 18 models across seven companies yielded an average score of 82%. Roughly one in ten responses scored below 65%, a level the researchers associate with inaccurate or potentially harmful advice. Repeating identical questions five times also produced score standard deviations reaching 12 percentage points, highlighting inconsistency. Some responses recommended unnecessarily disruptive measures, such as replacing a device or switching banks. To understand users’ needs, the researchers used a fine-tuned Gemini model to identify four million relevant questions within two billion community posts spanning five years. Scam-related questions were most common. Google says it has made HelpBench public and incorporated it into its Gemini 4 Carbon safety evaluation. These results are publisher-reported benchmark findings, rather than evidence that every model response is dependable in real-world situations.
Source
Google
Author
GlobalGraph Editorial
Published
17 November 2026
Region
Global
Format
report
Categories
Academic institution, AI & agent security, AI security, Consumer / individual, Data & ML teams, Data security & privacy, English, Human-AI interaction & UX, Identity & access security, Language (LLM / NLP), MLOps & model tooling, Model developer, Model evaluation, Research organization, Researchers, Responsible AI, ethics & safety, Security teams, Software & technology
Reading time
1 min read
Last updated
17 November 2026
Read the original source

Data sources & provenance

Where this information came from, how it was checked and when it was last looked at. “Last updated” records a change to the record — it does not mean the record was verified.

Data quality
Data quality: Verified
Sources cited
0
Last reviewed
17 November 2026
Last updated
17 November 2026

Editorial records cite their original reporting source. This story lists its source and publication date above.

Spotted something out of date?

Anyone can suggest a correction — you don't need an account. An editor checks every suggestion against the sources before the record changes.

Related coverage