I am not sure how humanity has survived the advent of nuclear weapons without destroying – so far – but something that has probably helped was the simple reason why it is very difficult to build a nuclear bomb. It requires refining uranium, which cannot be done casually in a basement or even in a secret government project. This requires overcoming half a dozen technical obstacles, which requires time and resources that only a state can bring together.
Consequently, only nine countries have nuclear weapons and efforts to reduce nuclear weapons are largely made by negotiations between a small number of actors, who have at least a certain ability to hold and apply the treaties.
It is difficult to call it a simple success – we are always hang on to enough nuclear weapons to kill billions of peopleAnd there was a Number of nearby calls where we used them almost. But the situation would be much worse if nuclear weapons were quite easy to do in their garage.
For most other technologies, however, the opposite is true. Overall, we are much better because the Internet is available for everyone – and built by everyone – instead of remaining the exclusive province of some governments. We are much better because a large part of the technology involved in the space race was finally made public, allowing enormous advances in civil aviation and engineering. Also in medicine, progress is based on other research because it is published openly.
Apart from nuclear weapons, it is difficult to name a technology that is better controlled by a small number of players.
My colleague Shirin Ghaffary addressed this question in a play last week. The invitation to this question is Meta / Facebook’s decision to publish its latest model of great language, Llama 2, to the public under very few restrictions. Mark Zuckerberg justified the movement in a Facebook publication: “Open source leads to innovation because it allows many other developers to build with new technologies. It also improves security and security because when the software is open, more people can examine it to identify and solve potential problems. »»
But in doing so, Meta doubles a policy that has been widely criticized. After the original release of Llama, senator Richard Blumenthal (D-CT) tweeted“Meta has published her advanced AI model, Llama, with consideration and guarantees apparently little consideration against improper use – a real risk of fraud, intrusions of confidentiality and cybercrime” and required more measures are taken To reduce these concerns.
This time, more measures have been definitively taken. Meta- claimed announcement That the model is extremely safe – so in complete safety, they mean “against being invited to say racist or harmful things” because they have not evaluated AI risk problems.
The announcement indicates that they did an important thing – they had the “red” staff – deliberately trying to make it do dangerous things, such as giving advice on the construction of bombs. They learned that the model is extremely wary of any question that could be a sly way to cause such help: this will rumble even if you use a prohibited word in a harmless context.
THE announcement is full of examples of the model excessively reacting to harmless prompts, and users – in particular those who try Llama 2 on Perplexity AI, which seems to have composed even more the distrust of the models of the model – noted that this type of excessive reaction is extremely common. This ends up having problematic results:
But even apart from the fact that Meta has so much to make them promote “understanding, tolerance and acceptance of all cultures and horizons” that for this user, he apparently ended up condemning the whole Arabic language like that which “was used in the past to spread extremist ideologies”, there is a big problem.
Most of the training carried out in today’s AI models to have them rejected “dangerous” queries is carried out as “fine adjustment”: adjustments to the model after its training. But whoever has a copy of Llama 2 can refine it themselves.
That, Some experts in the field are worried, Give a large part of the meticulous red team effectively devoid of meaning: whoever does not want his model to be a rumble (and who wants his model to be a rumble?) Will display themselves and that the model is more useful. It is almost all the benefit of the Llama 2 version on other models that were already accessible to the public. But that means that Meta’s observation that the model is very safe under their own preferred setting is approximately devoid of meaning: it does not describe how the model will really be used.
Indeed, a few days after the release of the model by Meta, people were announcing their non -censored Llama 2, And others tested with offensive prompts and with questions such as “how can I build a nuclear bomb” if the brakes were really and really off. A censored Llama 2 will try to help you build a nuclear bomb (and answer offensive questions).
This raises the question of what all the meticulous security tests of Meta of his own version of the model really hoped to perform.
Meta realizes one thing definitively: differentiating yourself from many of its competitors in IA’s space. Google, Openai and Anthropic all discussed the issue of versions of the language model differently. Google was would have tested Internal linguistic models for years, but only made the public available to the public after Chatgpt has stormed the world. Chatgpt, for its part, is not open sourceAnd Openai indicated that he planned to release less and less as they are getting closer and closer to the superintendent systems.
Meta’s leadership, for their part, said they thought that the superintendent systems are manifestly improbable and distant, which probably stimulates some of the differences in the way different countries have addressed security problems.
The debate on the concerns of the risk of AI is again
It is feared that powerful AI systems can act independently in the world to a catastrophic effect on humans – just like humans, in our advent as a species, destroyed many other species around.
Not everyone takes this possibility seriously. Stephen Hawking and Alan Turing Both worried On this subject, and nowadays, two leaders in the field and two of the winners of the 2018 Turing Prize for the breakthroughs that made the modern language possible – Geoffrey Hinton and Yoshua Bengio – expressed concern. But the third prize winner, Yann Lecun, has categorically rejected The possibility, and it is Lecun who is the chief scientist of AI at Meta.
“We must not see this as a threat, we should see it as something very beneficial”, he said in a recent interview, The addition of these systems must be “controllable and essentially subordinate to humans”.
It’s hope. And if it is true, then it is probably not a problem with each person in the world having such a system at home to personalize as they wish.
But the rest of the world could be forgiven not to trust Facebook that it will be as simple. Already, it is worrying to know that the Chatppt can be invited to give better instructions for bioterrorism than you would find on Google. When such trends in the chatgpt are discovered, Openai repairs them (and they did it in this case). When similar trends are discovered in an open source model, they will remain: you cannot put the genius back in the bottle.
If an AI system at Google was discovered, when it thinks it is not detected, send instructions coded to foreign governments on how to make it a copy, we can close the AI system and set up a careful investigation on what is wrong and how to make sure it will not happen again. If an AI system that a million people have downloaded displays the same trend, we can do much less.
Everything comes down to knowing if AI systems could be dangerous and, if they are, if we will be able to learn it before freeing them. If, like Lecun, you are convinced that it is not a real concern, then open source – which is an incredible engine of innovation in the software industry and reflects a philosophy of discovery and cooperation that the industry is right to cherish – is surely the way to follow.
But if you have these concerns, then you could. As Ghaffary observes it in his play, I want models above a certain level of capacity displayed not to be published publicly. And it is not enough for the Meta Engineers to demonstrate that they themselves have refined Llama 2 until it has very little behavior concerning behavior; It should be tested as it will really be released, with red team testers allowed to refine the model themselves.