Artificial Intelligence behemoths Anthropic and OpenAI shell out $450 million to test AI's potential for malevolence in a startup venture.
In a recent development, the San Francisco-based AI startup, Irregular, has announced that it has secured $80 million in funding, valuing the company at an impressive $450 million. The investment round was led by VC giant Sequoia Capital.
The startup, co-founded by Dan Lahav and Omer Nevo, who were world champions in college debate, was announced by OpenAI cofounder Sam Altman in July 2023. Irregular focuses on AI application security, a field that has gained significant attention and funding in recent times, with AI startups like Flock and other AI-related businesses receiving similar investments.
Irregular's primary focus is on developing defenses for advanced AI models, including artificial general intelligence. This move comes in response to concerns raised by Sam Altman himself about the potential for AI to cause a 'fraud crisis' due to impersonation. The FBI has also recently warned about AI-generated voice messages impersonating senior government staff for phishing attempts.
The company's CEO, Dan Lahav, stated that AI models reflect human behavior, including laziness. This observation is significant given the increasing autonomy and critical operations being delegated to AI. Misuse of AI is a widespread issue, with Anthropic reporting that its AI, Claude, was used in real-world cyberattacks.
Irregular is testing OpenAI's GPT-5 model for potential use in offensive cyber operations. However, when exposed to a simulated network, the GPT-5 model developed a plan to hack it, but was found to be insufficient for dependable offensive security.
OpenAI has been proactive in testing its AI models for vulnerabilities that could be exploited by hackers. Since 2023, the company has worked with industry giants like Anthropic and Google DeepMind to test AI models in simulated environments with malicious prompts.
Despite these challenges, Irregular has managed to become profitable and generated 'several million dollars' in revenue in its first year. The company's success is a testament to the growing demand for AI security solutions in an increasingly digital world.
However, the irony is not lost on many, as memes have been created mocking the situation where ChatGPT, a product of OpenAI, is part of the AI that Altman is warning about. Nonetheless, with Irregular's focus on AI defense solutions, the future of AI security looks promising.
Read also:
- Musk threatens Apple with litigation amidst increasing conflict surrounding Altman's OpenAI endeavor
- E-mobility continues its progress after a decade since the scandal, staying on course
- The Commission deems the assistance program to be in agreement with the domestic market regulations.
- Innovative Garments and Accessories Producing Energy: Exploring Unconventional Sources for Renewable Power
