Seven Myths About the True Costs of AI Systems

While there is much excitement today around implementing AI at the enterprise level, the financial costs of this process are often unexpected and underappreciated. These seven myths are crucial lessons learned that executives should know before heading down the road to AI.

By Praful Krishna, Coseer.

Artificial Intelligence is the rage these days. Every enterprise executive is thinking about using AI for their team. However, most are surprised by how expensive AI truly is, and despite that, how little does it eventually deliver.

People who have implemented multiple AI systems will tell you that the dollar cost you set aside for the AI vendor or the budget you allocate for an AI team is only the tip of the iceberg. Costs stack up very quickly when you think of the data and the training. They also seem to be ongoing, because unlike the well-behaved systems from the age of traditional IT, AI systems are quirky, and keep changing their behavior! They will also tell you that the true cost includes not merely the dollars you spend, but there is a direct time cost as well as the political cost you or your team has to personally pay.

Here are seven myths about the true costs of AI systems that every executive should know:


Myth 1. AI Works as Advertised

At the current state-of-art, the balance of probability is that it doesn’t. Sometimes vendors are outrightly lying, but mostly they are either too hopeful, or have made the stuff work in a limited way on some limited data. It’s not worth your time to engage with vendors who have not already solved the problem for someone else of your size and complexity. Sure, it does make sense to try new technologies, but in that case you need to manage the political costs well, because it may turn out to be vaporware.


Myth 2. The World is Moving to Public Cloud

It is, but not your world. One of the key problems with implementing AI solutions is that of managing data. You want it completely secured, which a lot of credible vendors like IBM promise. However, these vendors often use your data to train their AI models and then sell solutions based on those models to your competitors. CIOs need to be very careful about this, and they are.

In any case, due to training complexities of an AI system, a lot of benefits of public cloud, like multi-tenant architecture, are not applicable. You still want to be able to scale your hardware on demand, and not rely on traditional IT constructs (see below). It seems private cloud, where your data stays within your firewall, is your best bet. It appears to be an expensive option but is far cheaper once you factor in all the true costs of enterprise AI systems.


Myth 3. Corporate IT Should Manage AI Projects

Corporate IT already has a lot on its plate. While there are exceptions to this, corporate IT teams are typically under-resourced and overtaxed. Their primary responsibility is to make the trains run and to minimize risks to the business. AI projects, on the other hand, are meant to disrupt the status quo in unknown ways. To saddle corporate IT with AI when it is completely experimental is a political folly.

To control costs here, you are perhaps better off setting a small center of excellence within your team or at least hire a system architect who has worked with AI before. Using a private cloud service like AWS or Azure will also simplify things.

Once things are stable, it will be a great time to involve corporate IT and let them manage and nurture the AI systems.


Myth 4. AI is a Product

If an AI solution claims to be a plug-and-play canned product, you need to be highly suspicious. The AI must learn from your data to be able to give highly relevant answers. To decide between various algorithmic approaches to your problem, you need to understand how exactly the training takes place under the hood, and what can be done to improve the results. If this is not clear up-front, you may end up in a situation that after spending millions, the system is not performing, and nobody can explain why.

There are two acceptable answers for “plug-and-play” constructs. Either the AI has been trained on public data, and results are what they are. In this case you are not buying AI, but an AI-enabled product, which could work out to be great. Or, the AI gets trained on its own with no human input. In this case you still need to understand how it all works.


Myth 5. There is Sufficient Data to Train an AI System

Another big reason why #4 above is so important is that executives routinely underestimate how much data does it take to train an AI system. Even if that is understood correctly, you may make the mistake of assuming that this amount of data is available. It may be available to your company, but is it available to you? Have the data owners signed off? Have you thought through how the data will migrate to and from the AI system? How will data be kept updated in real-time – sometimes even a day-old data makes the system obsolete. These costs seem insignificant at the first brush, but quickly stack up on all dimensions – money, time, and your career prospects!


Myth 6. You can Train the AI

This should perhaps be number one. In all likelihood, you are deploying an AI system that must be trained on your data to be effective. I am surprised at how many executives underestimate this part.

The true power of AI is that it can understand the nuances of your domain. The source of that AI, the vendor, or the engineering team will rely on your team to provide the domain expertise. In other words, you need to plan for time of people who can interact with this AI in its infancy and coach it step-by-step. If you are using a solution that needs tagging/ annotation, then somebody in your team needs to devote time to that.

Consider this: MD Anderson had to spend three years and $62m on training a Watson based system. Almost all of it was the time of subject matter experts. (It still did not work, but that is a separate issue altogether.)


Myth 7. It’s a One-Time Cost

In the world of traditional IT, once a complex system is built and deployed, things calm down. There are maintenance and security costs, for sure, but the system itself is predictable. AI, on the other hand, keeps learning as it is used more and more. A good AI system learns for the better, but sometimes metrics of effectiveness, e.g.,  accuracy starts to dip over time. In the data science world this gradual phenomenon is termed as ‘drift’.

While thinking through the true cost of an AI system, it is also important to understand what costs could be applicable over time, and what strategies should you use to control them. For example, can you negotiate with your vendor upfront about per-use pricing?

Once implemented correctly, AI can do wonders, but it is best to start on things with eyes open and after considering every possible pitfall. Perhaps you can learn from others’ mistakes!

Bio: Praful Krishna is an AI expert who understands the technical and business aspects of AI. He has successfully executed high ROI AI projects at Fortune 500s by identifying the correct asks and bringing all stakeholders together. He is well versed in quirks of AI re organization, product management, technology development, IT architecture, and data security. This strength is built upon numerous roles in leadership, product, and strategy.