Jimmy Spears


5 min read

Share on:

Putting tech to the test: How do we make sure new solutions work?

It’s not easy being an insurance company in 2022. The business environment is shifting quickly, both from the impact of COVID and as new technologies enter the market – introduced both by rival incumbents, and new tech-enabled start-ups. As a result, it’s vital for established companies to test the new solutions out there to see how well they work, and whether they can improve existing processes. Otherwise, they risk getting left behind.

However, that evaluation process itself is tricky, not least as with so many providers out there, it’s hard to parse through the noise. Solution vendors come armed with lofty claims about the potential of their technology and promise the earth, but in practice the results may not pan out – wasting time, resources and energy. 

How, then, can insurers ensure they are evaluating solutions quickly, and optimally? At Tractable, we’ve successfully moved over 30 of the world’s top insurers through from initial testing phases to becoming full-fledged customers, using our AI on real claims and helping real policyholders recover faster when they’ve had an accident.

As a result, we’ve seen a successful trial or two – as well as ones that haven’t gone to plan. Here’s our advice on how to get the most out of a test, with a couple of examples of how we’ve seen this play out in practice. 

Put the solution under pressure: test ‘as live’

At Tractable, we’re confident that our AI solution is the market leader, with the aim that it’s as accurate as a human assessor, and used successfully by leading companies in the US and elsewhere, including Tokio Marine, Ageas, MS&AD, and many more. We process claims worth over $2bn a year, accelerating claims from, in some instances, taking weeks into minutes – being processed on a single phone call…

But then, we would say that, wouldn’t we? 

And that’s the problem – it’s all too easy for us, and other companies that offer tech solutions, to make claims about the efficacy of what they offer. And it’s not easy to tell the truth from what might be well-written hyperbole. 

That’s why testing is crucial – but this step too can fail if a test isn’t precise enough. For example, there have been times when an insurer has asked Tractable to show its results against other providers that claim to have a working AI – but said that every party could return the results in 48 hours, or even longer.

That timeframe isn’t useful, as it doesn’t put any pressure on the AI. You need to create a situation where the vendor has to rely on its technology to do the work, not provide a get-out clause whereby it can manually check the computer’s math and make changes accordingly. 

So our suggestion: propose a live test. At Tractable, we want realistic – and even difficult – testing conditions that weed out solutions that can’t help, and highlight those that can. Only that way can you really be sure that a technology can do what its vendors claim it does – while also ensuring that the vendor itself is a good potential fit. 

Let’s take a look at how this might work in practice.

Ensure the solution can handle scale

One of the challenges for an insurer is knowing whether a solution can perform as well as the provider claims outside of a controlled environment. 

It’s relatively easy for a vendor to ‘prove’ that a technology performs at a promised standard if you’re measuring it on a few claims that have been carefully pre-selected to include no surprises. However, applying that solution at scale, in an environment where every variable (from weather, to vehicle make and model, to accident type, to image quality) is a potential unknown, is another question entirely. 

Introducing more realistic testing conditions means the buyer can have more confidence in what they are buying, even before a purchase is made. For example, earlier in 2021 one of the world’s largest insurers asked us to carry out a live test of our AI solution to see how well it would solve their problems. The test itself was challenging – it required us to process 1,000 claims in just a few hours, and to analyze the results in a workshop with senior leadership. 

That’s a big ask: normally, an insurer invites you to process a much smaller number of claims – say 100 or so – but this company really wanted to put our technology through its paces. Their leaders made it clear that they wanted to test the AI’s ability to work at speed, at a scale that made potential human intervention impossible – ensuring they established as accurate an impression as possible of its capacity and any limitations.  

We wanted that too – but there was a catch. Normally, you only upload so many claims for a company that’s actually a customer – so you’ve already built all the integrations you need for that to happen. But this was a test, which meant none of that infrastructure was in place yet. So, we had to construct, from scratch, an automation system that would take data from the images and ingest it into our assessment system – allowing us to run the ‘real life’ test for the client as per their specifications, and enabling us to show them the results in the desired time frame. 

By doing so, we were able to prove to the client that we could meet their expectations on accuracy and timing (as the results from the AI exceeded their targets). However, we were also able to show that we can work with a very important customer in a way that proves we will be a reliable and supportive partner to them, which is key when partnering with one of the world’s largest and most credible insurers. They need to be sure they can trust both your technology, and you. 

Check for adaptability: how does a solution handle workflow issues?

Another significant challenge for insurers is less to do with the technical brilliance of a solution – it’s whether their employees will be able to use it effectively. 

That might sound counterintuitive – after all, if a solution saves time and effort, it stands to reason that it should be of use. But that’s not necessarily the case – as if it disrupts existing workflows too much or fails to integrate with existing back office systems, then it may create more problems than it solves. 

So, our other testing tip is to make sure a solution solves the challenges and fits into the workflows of the people who’ll actually be using it. In the case of Tractable, that is often an insurer’s claims handlers. 

Why do they matter? Well, in this case, they are the party who presents our technology to the policyholder and says: ‘I’d like you to try this new solution’. And that means if they don’t trust the technology or – worse – don’t know how to use it, they won’t present it well – making it less likely you generate the results you need.

So how do you set yourself up for the best chance of success? Again, in 2021, we carried out a major test of our AI Estimating solution with a European insurer where we processed over 1,000 claims. 

We carried out a huge amount of initial work to scope the claims handlers’ needs and what would help them most in their daily operations, and ensured the solution fit into those requirements – helping them understand that it was designed to help them do their jobs better, not create problems. 

We also carried out regular check-ins to understand any workflow issues. Through this communication, it became clear that while the AI was performing above the standards anticipated, some claims couldn’t be processed because of internet connectivity issues and outages. 

As a result, we quickly built an offline mode for the app, which enabled claims to be put through even if the customer couldn’t get on line immediately. That helped us maintain the levels of processed cases we needed for the insurer to fairly evaluate the technology – again, a result that benefited both parties. 

Getting up to speed

It’s great to be able to meet challenging requests, especially when there’s a need to prove the worth of the technology in a market where it hasn’t been tested yet. 

However, in areas where the use of Tractable’s AI is now relatively well established, we’re now also seeing a separate trend come into play: companies are seeing the results their rivals are having in market. Consequently, we’re seeing some sizable insurers approach us for an initial small-scale test, be impressed at what they’ve seen, and seek to go straight to full production as quickly as both sides can move forward. 

That certainly eases the pressure in terms of creating innovative solutions for intractable problems. However, we’re always happy to let companies test our technology – whether that’s in our well-established auto insurance sector, where we can help insurers accelerate claims at any point of the repair journey, or in our new growth areas – auto inspection, and property claims. 

If you think our solutions can help you accelerate accident and disaster recovery, we’d love you to give them a thorough going over. Please reach out at information@tractable.ai 

Images: https://unsplash.com/photos/DbLlKd8u2Rw, https://unsplash.com/photos/j2c7yf223Mk

Discover more related content