To discover more about the unspoken risks associated with the AI model assembly line, read this exclusive article from AITech.
Artificial intelligence (AI) is transforming the way financial services institutions operate and engage with customers. From automating routine processes to offering personalised financial advice, AI’s potential to drive efficiency, innovation, and customer satisfaction is substantial. However, realising this potential is another matter. Gartner research shows that while 80% of executives believe that automation can be applied to any business decision, just 54% of AI models make it from pilot to production. This statistic underscores the gap between the promise of AI and its practical implementation, raising the question of what are the pitfalls in the AI model assembly line that stand in the way of success?
Pitfall One: protracted and disjointed processes dampen momentum
While financial institutions ambitions for AI projects are rapidly growing, most do not yet have a mature AI development infrastructure that is ready to meet this demand. Research from Deloitte shows that the number of AI-related activities undertaken by companies (across several sectors) is increasing, but that productionalising projects is a key issue, with organisations relying on manual, ad hoc processes to bring projects to life.
In financial institutions the AI model development process lasts 5.5 months on average, involves a significant number of specialised employees, and comprises multiple stages. Each of these stages has its own pain points, which act as a brake on momentum and significantly add to the cost of model development. If, as Gartner’s data shows, 46% of models don’t make it to production, this represents a huge sunk cost to financial institutions and a major dampener on staff morale.
- Defining the use and business case: Often consists of weeks of circular meetings between business leaders, business analysts, product managers and data scientists.
- Data provisioning: Locating and verifying accurate data can take weeks to months due to organisational silos.
- Development: Prioritisation dilemmas are commonplace, with long queues for resources and multiple bureaucratic hurdles. Often this situation is aggravated by the fact that a single data science team has to serve many competing business units, leading to potential conflicts of interest.
- Quality Assurance: Models can run into more queues to pass the QA testing stage
- DevOps: High demand and resource constraints can mean yet more queues for the final checks to ensure a smooth transition into the production environment
Pitfall Two: data management challenges elongate the model development process
Good data management is paramount for transitioning AI models into production efficiently and effectively. It provides the bedrock upon which models are built and trained, with the quality, volume, and diversity of data directly impacting the accuracy and generalisation capability of the models. Furthermore, it facilitates reproducibility, allowing models to be reliably retrained as new data becomes available, when model drift is detected, or models need to be upgraded and updated.
The reality is that for many financial institutions data management is a major challenge. The volumes of data are substantial, a lot of it is in legacy formats and it is located in multiple systems. In fact, it is often not clear exactly which data sets the financial institution has and who owns them. As a result, model scoping is rendered more challenging and more time is added to development or the model is pushed to the back of the development queue, while work is undertaken to get the underlying data in order.
Pitfall Three: lack of strategic planning, change management and business user buy-in reduces uptake
For the 54% of AI models that eventually make it into production, not all of them go on to enjoy strong business uptake. Often, new ideas for AI models are initiated by data science teams, which, while technically proficient, may not be fully attuned to the strategic needs of the business. Or a model may be developed off the back of a passing conversation with a divisional lead, but the buy-in of the rest of the business unit has not been established. Essentially, the ideas are not rooted in a solid understanding of the business’s strategic objectives and therefore risk being misaligned with the organisation’s broader goals. This misalignment can result in models that fail to gain traction, thus rendering all the development time, cost and effort a waste. Often, the lack of strategic planning means multiple models are developed and it is challenging to achieve a return on investment (ROI) on them all.
Combined these pitfalls can have a significant impact on financial institutions looking to develop their AI capabilities and senior management should be cognisant of the pitfalls that exist. Protracted development processes, data management challenges, and lack of strategic planning with the business are all key contributors to the low success rate in transitioning AI models to production and driving business uptake. Developing an awareness of these common dangers, communicating this effectively across the IT and business organisation, and assigning accountability for prioritisation and decision-making, as well as simplifying processes can all contribute to a more efficient data assembly line, empowering the microcosm of AI enablers and the business users who actualise their value.
Visit AITechPark for cutting-edge Tech Trends around AI, ML, Cybersecurity, along with AITech News, and timely updates from industry professionals!
