Stanford and Georgetown have a new report on the security risks of AI—particularly adversarial machine learning—based on a workshop they held on the topic.
Jim Dempsey, one of the workshop organizers, wrote a blog post on the report:
As a first step, our report recommends the inclusion of AI security concerns within the cybersecurity programs of developers and users. The understanding of how to secure AI systems, we concluded, lags far behind their widespread adoption. Many AI products are deployed without institutions fully understanding the security risks they pose. Organizations building or deploying AI models should incorporate AI concerns into their cybersecurity functions using a risk management framework that addresses security throughout the AI system life cycle. It will be necessary to grapple with the ways in which AI vulnerabilities are different from traditional cybersecurity bugs, but the starting point is to assume that AI security is a subset of cybersecurity and to begin applying vulnerability management practices to AI-based features. (Andy Grotto and I have vigorously argued against siloing AI security in its own governance and policy vertical.)
Our report also recommends more collaboration between cybersecurity practitioners, machine learning engineers, and adversarial machine learning researchers. Assessing AI vulnerabilities requires technical expertise that is distinct from the skill set of cybersecurity practitioners, and organizations should be cautioned against repurposing existing security teams without additional training and resources. We also note that AI security researchers and practitioners should consult with those addressing AI bias. AI fairness researchers have extensively studied how poor data, design choices, and risk decisions can produce biased outcomes. Since AI vulnerabilities may be more analogous to algorithmic bias than they are to traditional software vulnerabilities, it is important to cultivate greater engagement between the two communities.
Another major recommendation calls for establishing some form of information sharing among AI developers and users. Right now, even if vulnerabilities are identified or malicious attacks are observed, this information is rarely transmitted to others, whether peer organizations, other companies in the supply chain, end users, or government or civil society observers. Bureaucratic, policy, and cultural barriers currently inhibit such sharing. This means that a compromise will likely remain mostly unnoticed until long after attackers have successfully exploited vulnerabilities. To avoid this outcome, we recommend that organizations developing AI models monitor for potential attacks on AI systems, create—formally or informally—a trusted forum for incident information sharing on a protected basis, and improve transparency.
More Stories
Live Video of Promachoteuthis Squid
The first live video of the Promachoteuthis squid, filmed at a newly discovered seamount off the coast of Chile. Blog...
YubiKey Side-Channel Attack
There is a side-channel attack against YubiKey access tokens that allows someone to clone a device. It’s a complicated attack,...
Spyware Vendors’ Nebulous Ecosystem Helps Them Evade Sanctions
The secret web of at least 435 entities across 42 countries making up the spyware landscape facilitates unpunished security and...
US and Allies Accuse Russian Military of Destructive Cyber-Attacks
The joint government advisory highlighted the cyber activities of Unit 29155, which has launched destructive cyber-attacks against critical infrastructure globally...
PyPI Revival Hijack Puts Thousands of Applications at Risk
Revival Hijack Python Package Index supply chain attack threatens 22,000 packages through malicious downloads Read More
Security Budgets Come Under Pressure as “Hypergrowth” Ends
Despite rising threats researchers find a third of firms see flat or falling security budgets and hiring slows Read More