In a net positive for researchers testing the safety and security of AI systems and models, the US Library of Congress has ruled that certain types of offensive activity – such as rapid injection and circumvention of rate limits – do not violate the Digital Millennium Copyright Act (DMCA). , a law used in the past by software companies to combat unwanted security research.
The Library of Congress, however, declined to create an exemption for security researchers under the provisions of the Fair Use Act, arguing that an exemption would not be sufficient to provide a safe haven for security researchers.
Overall, the three-year update to the legal framework around digital copyright works in security researchers’ favor, as does having clearer guidelines on what’s allowed, says founder Casey Ellis and advisor to the crowdsourced penetration testing service BugCrowd.
“Clarifying things like this – and just making sure that security researchers are operating in as supportive and clear an environment as possible – is an important thing to maintain, regardless of the technology,” he says. “Otherwise you end up in a position where the people who own the (large language models) or the people who deploy them, they’re the ones who end up having all the power to control whether security research is happening or not .in the first place, and this results in a poor security outcome for the user.
Security researchers increasingly benefit from hard-won protections against lawsuits and lawsuits for conducting legitimate research. In 2022, for example, the US Department of Justice stated that his prosecutors would not charge security researchers with violation of the Computer Fraud and Abuse Act (CFAA) if they did not cause harm and continued the research in good faith. Companies that sue researchers are regularly shamed, and groups such as the Security Legal Research Fund and the Hacking Policy Council providing additional resources and defenses to security researchers under pressure from large companies.
In an article published on its site, the Center for Cybersecurity Policy and Law called the clarifications from the US Copyright Office “a partial victory” for security researchers – offering more clarity but no refuge. The Copyright Office is organized under the Library of Congress.
“The lack of legal protection for AI research has been confirmed by law enforcement and regulatory agencies such as the Copyright Office and the Department of Justice, but good faith research on AI continues to lack a clear legal safe harbor.” the group declared. “Other AI trustworthiness research techniques may still result in liability under Section 1201 of the DMCA, as well as other anti-piracy laws such as the Computer Fraud and Abuse Act.”
The brave new legal world
The rapid adoption of generative AI systems and algorithms based on big data has become a major disruption in the information technology industry. Since many large language models (LLMs) rely on the massive ingestion of copyrighted information, the legal framework for AI systems started on a weak foundation.
For researchers, past experience provides frightening examples of what could go wrong, says BugCrowd’s Ellis.
“Given that this is a very new space – and some of the boundaries are much fuzzier than they are in traditional computing – a lack of clarity always turns into a deterrent,” says -he. “For people who are aware of this, and many security researchers are very concerned about making sure that they don’t break the law in doing their work, it has sparked a bunch of questions from the community.”
The Center for Cybersecurity Policy and Law and the Hacking Policy Council have proposed that red teams and penetration tests intended to test AI security and safety be exempt from the DMCA, but the Librarian of Congress recommended refusing the proposed exemption.
The Copyright Office “recognizes the importance of AI trustworthiness research as a policy issue and notes that Congress and other agencies may be best positioned to act on this emerging issue.” the registry entry indicatedadding that “the negative effects identified by advocates arise from third-party control of online platforms rather than the operation of Section 1201, so an exemption would not alleviate their concerns.”
No going back
As major companies invest massive sums in training the next AI models, security researchers could find themselves targeted by some pretty deep pockets. Fortunately, the security community has established fairly well-defined practices for handling vulnerabilities, says BugCrowd’s Ellis.
“The idea that security research is a good thing is now quite common…so the first instinct of people deploying new technology is not to experience a massive explosion like we have done in the past,” he said. “Cease and desist letters and (other communications) have been exchanged much more quietly, and the volume has been quite low.”
In many ways, penetration testers and red teams focus on the wrong problems. The biggest challenge right now is overcoming hype and misinformation about AI’s capabilities and security, says Gary McGraw, founder of the Berryville Institute of Machine Learning (BIML) and software security expert . Red teaming is about detecting problems, not taking a proactive approach to security, he says.
“As they are designed today, ML systems have flaws that can be exposed by hacking, but cannot be fixed by hacking,” he explains.
Companies should strive to find ways to produce LLMs that do not lack factual presentation—that is, “hallucinate”—or are vulnerable to rapid injection, McGraw says.
“We’re not going to red team or pen test to ensure AI is reliable: the real way to make ML secure is at the design level, with a strong focus on training data, representation and evaluation,” he says. “Penetration testing has high sex appeal but limited effectiveness.”