Armilla wants to give companies a warranty for AI

There’s a lot that can go wrong with GenAI — especially third-party GenAI. It makes stuff up. It’s biased and toxic. And it can run afoul of copyright rules. According to a recent survey from MIT Sloan Management Review and Boston Consulting Group, third-party AI tools are responsible for over 55% of AI-related failures in organizations.

So it’s not surprising, exactly, that some companies are wary of adopting the tech just yet.

But what if GenAI came with a warranty?

That’s the business idea Karthik Ramakrishnan, an entrepreneur and electrical engineer, came up with several years ago while working at Deloitte as a senior manager. He’d co-founded two “AI-first” companies, Gallop Labs and Blu Trumpet, and eventually came to realize that trust — and being able to quantify risk — was holding back the adoption of AI.

“Right now, nearly every enterprise is looking for ways to implement AI to increase efficiency and keep up with the market,” Ramakrishnan told TechCrunch in an email interview. “To do this, many are turning to third-party vendors and implementing their AI models without a complete understanding of the quality of the products … AI is advancing at such a rapid pace that the risks and harms are always evolving.”

So Ramakrishnan teamed up with Dan Adamson, an expert in search algorithms and two-time startup founder, to start Armilla AI, which provides warranties on AI models to corporate customers.

How, you might be wondering, can Armilla do this given that most models are black boxes or otherwise gated behind licenses, subscriptions and APIs? I had the same question. Through benchmarking, was Ramakrishnan’s answer — and a careful approach to customer acquisition.

Armilla takes a model — whether open source or proprietary — and conducts assessments to “verify its quality,” informed by the global AI regulatory landscape. The company tests for things like hallucinations, racial and gender bias and fairness, general robustness and security across an array of theoretical applications and use cases, leveraging its in-house assessment technology.

If the model passes muster, Armilla backs it with its warranty, which reimburses the model’s buyer for any fee they paid to use the model.

“What we really offer enterprises is confidence in the technology they’re procuring from third-party AI vendors,” Ramakrishnan said. “Enterprises can come to us and have us run assessments on the vendors they’re looking to use. Just like the penetration testing they would do for new technology, we perform penetration testing for AI.”

I asked Ramakrishnan, by the way, whether there were any models Armilla wouldn’t test for ethical reasons — say a facial recognition algorithm from a vendor known to do business with questionable actors. He said:

“It would not only be against our ethics, but against our business model, which is predicated on trust, to produce assessments and reports that provide false confidence in AI models that are problematic for a client and society. From a legal standpoint, we aren’t going to take on clients for models that are prohibited by the EU, that have been banned, which is the case for some facial recognition and biometric categorization systems, for example — but applications that fall into the ‘higher risk’ category as defined by the EU AI Act, yes.”

Now, the concept of warranties and policy coverage for AI isn’t new — a fact that surprised this writer, frankly. In 2018, Munich Re debuted an insurance product, aiSure, designed to protect against losses from potentially unreliable AI models by running the models through benchmarks similar to Armilla’s. Outside of warranties, a growing number of vendors, including OpenAI, Microsoft and AWS, offer protections pertaining to copyright violations that might arise from deployment of their AI tools.

But Ramakrishnan claims that Armilla’s approach is unique.

“Our assessment touches on a wide range of areas, including KPIs, processes, performance, data quality, and qualitative and quantitative criteria, and we do it at a fraction of the cost and time,” he added. “We assess AI models based on requirements set out in legislation such as EU AI Act or the AI hiring bias law in NYC — NYC Local Law 144 — and other state regulations, such as Colorado’s proposed AI quantitative testing regulation or New York’s insurance circular on the use of AI in underwriting or pricing. We’re also ready to conduct assessments required by other emerging regulations as they come into play, such as Canada’s AI and Data Act.”

Armilla, which launched coverage in late 2023, backed by carriers Swiss Re, Greenlight Re and Chaucer, claims to have 10 or so customers, including a healthcare company applying GenAI to process medical records. Ramakrishnan tells me that Armilla’s client base has been growing 2x month over month since Q4 2023.

“We’re serving two main audiences: enterprises and third-party AI vendors,” Ramakrishnan said. “Enterprises use our warranty to establish protection for the third-party AI vendors they’re procuring. Third-party vendors use our warranty as a stamp of approval that their product is trustworthy, which helps to shorten their sales cycles.”

Warranties for AI make intuitive sense. But part of me wonders whether Armilla will be able to keep up with fast-shifting AI policy (e.g. New York City’s hiring algorithm bias law, the EU AI Act, etc.), which could put it on the hook for sizeable payouts if its assessments — and contracts — aren’t bulletproof.

Ramakrishnan brushed aside this concern.

“Regulation is rapidly developing in many jurisdictions independently,” he said, “and it’ll be critical to understand the nuances of legislation around the world. There’s no ‘one-size-fits-all’ that we can apply as a global standard, so we need to stitch it all together. This is challenging — but has the benefit of creating a ‘moat’ for us.”

Armilla — based in Toronto, with 13 employees — recently raised $4.5 million in a seed round led by Mistral (not to be confused with the AI startup of the same name) with participation from Greycroft, Differential Venture Capital, Mozilla Ventures, Betaworks Ventures, MS&AD Ventures, 630 Ventures, Morgan Creek Digital, Y Combinator, Greenlight Re and Chaucer. Bringing its total raised to $7 million, Ramakrishnan said that the proceeds will be put toward expanding Armilla’s existing warranty offering as well as introducing new products.

“Insurance will play the biggest role in addressing AI risk, and Armilla is at the forefront of developing insurance products that will allow companies to deploy AI solutions safely,” Ramakrishnan said.