Releasing Gulliver

Overcoming regulatory risk avoidance: Enterprise AI puts technology and human oversight into perspective

Wiebe Draijer, the chair of the board of a major Dutch bank, recently observed that, in the regulatory risk domain, he at times felt like Jonathan Swift’s Gulliver being tied down by the Lilliputs. Draijer was referring to the many rules emanating from EU, IMF and domestic regulations like GDPR, Anti-Money Laundering and KYC compliance that financial institutions are dealing with on a daily basis. He pointed at the upcoming trend them avoiding risks in fear of non-compliance to stay clear of major fines.

This implied paralysis is more serious than it appears. Minimal or even partially reduced risk appetite will not only dent operational results, but also hurt entrepreneurship which already comes with its own uncertainties. On top of that, and crucially, it can stifle the development and effectiveness of technical innovations in the industry.

No good news

This trend will not only limit new applications of AI, which is the focus of this article, but also affect technologies like blockchain, big data and cloud computing, not only in Fintech but also in traditional financial sectors. And there’s no good news. Expectations are that the financial sector in future will have to comply with even more handed down rules, with the EU AI Act as the centerpiece, expected to come into effect in 2024 or 2025. The current draft of this Act brands the financial sector as high-risk.

Some point at certain positive outlooks in all this. Strictly audited technical innovations may eventually result in more robust technological capabilities, with built-in compliance and security. But there’s a catch: to achieve it, more time is needed, and this slower development may undermine the competitiveness of EU financial institutions compared with their counterparts elsewhere.

This conundrum can be solved by conscientiously bringing AI expertise and business culture together in a clear working structure on enterprise level, called Enterprise AI. EAI unifies and coordinates existing technical, architectural and business capabilities, aligning them with requirements in terms of audit and compliance. Effectively, it offers management a controlled AI environment, with proper reporting and intervention options when needed.

AI accountability outside IT

As a relatively new technology, AI is still considered a strictly technical topic and its development and governance an isolated IT matter, branded ModelOps (the combination of MLOps, DataOps and DevOps). In such environments the mere observation of coding standards suffices to define model quality, and regular functional testing is used to demonstrate a model’s purposefulness. In many organizations AI is still regarded as a black box with modest participation, let alone governance by non-IT people like model consumers or senior managers.

But, similar to other data operations, AI mainly informs non-technical decisions, like generating new business insights, identifying new market opportunities, finding logistic solutions, establishing credit risks, or hardening business processes through an AI-supported QMS. For that reason and in their own interest, non-IT departments should steer AI operations much more than is the case today. By business taking initiative and ownership, next-gen AI can render more meaningful, business-colored results. Strategic governance and risk management will limit the threat of non-compliance. In short, the mainly technical AI domain needs to add more people, process and managerial involvement.

Human oversight

Emboldening structural human oversight will improve the effectiveness of AI technology and create a robust and reliable AI governance. Consistent value monitoring and timely intervention can pre-empt failures and substantially increase the yield of investments in tech innovation, while observing regulatory guardrails. By beefing up the human factor, Financials will be better equipped to overcome obstacles in densely regulated markets and even benefit, as coherent oversight implies business risk mitigation.

By empowering non-IT employees to also accept responsibility for AI, the EAI governance framework can branch out to all areas where AI plays a role. This will substantially reduce effort duplication, security breaches, undesired model results, and unforeseen costs. Most importantly, this consolidated approach can signal and pre-empt non-compliance. Where trial-and-error may generally be the fastest way to achieve technical progress, for the regulated domain of Financial Services, EAI scenario-thinking and prognostics are preferred, avoiding compliance overlooks that can have dire consequences. The availability, sourcing and quality of technical resources should be constantly tested, like alignment of business purpose and model execution. All these activities belong to the EAI framework.

EAI: strategy and integration

In essence, EAI is a defined long term strategic and integrated approach to AI development and governance. It offers an operational structure for the productive collaboration of IT and non-IT to not only ensure technical development, but also business value and RoI. From an architectural perspective, EAI offers a technology-agnostic and cost-efficient environment like leveraged hybrid cloud computing, able to scale and sufficiently resilient to respond to sudden business challenges. Also, and not unimportantly, EAI helps to define and emphasize the ethical boundaries of an AI practice by monitoring the trustworthiness and fairness of algorithms and to act on undesirable events.

As EAI mainly re-organizes existing roles and functions, implementing it is relatively straightforward, especially when introduced in an Agile or sprint-like fashion. EAI defines not only the AI ambitions of an organization and the fit-for-purpose technical and business architectures, but it also monitors ModelOps maturity and mobilizes people through the training, process and setup of BusDevOps or cross-functional teams. EAI offers a better leverage of resources and management options to keep AI development and execution within regulatory guardrails. The EAI playbook centers on the following five general rules-of-thumb:

1. Create awareness and consensus

The operationalization of AI, its significance for business operations, its contribution to value creation and the requirements for reliable oversight are mostly underdeveloped in organizations. To fully benefit from AI, a long term strategy is indispensable. But before that, AI awareness and consensus needs to germ in the organization, especially with the designated AI consumers in non-IT departments. This starts with business adoption efforts to train non-IT employees and communicate the AI objectives throughout the organization.

2. Ensure clear ownership

The rapid coming about of intelligent automation over the past years has more than once led to haphazard organizational implementations. Professional ownership of algorithms, ModelOps process management, selection of data sources and construction of data-sets is often still rudimentary. To amend this, all AI efforts should be overseen by both IT and non-IT. Absence of ownership hinders internal and external audit, which in its turn may constitute compliance risks. As a rule, ownership rests with the sponsor of the use case and should be assigned and communicated before work starts.

3. Involve non-IT

In model creation and maintenance processes most focus is on algorithm development, machine learning and model inference. Broad participation in AI by model consumers, product owners, and auditors is less prominent. Still, non-IT involvement and guidance is essential for AI to ensure added value. Apart from AI ownership, EAI governance assigns an across-the-board responsibility to business representatives during essential process phases like use case intake, quality assessment and model sign-off.

4. Track quality

Meaningful AI ownership requires regular assessments of model purpose, execution risks, and source data. These subjective quality checks should be part of standard DataOps procedures, in both the development and monitoring phases. Upfront use case intakes (e.g. what results will be trustworthy?) and post-test business value appraisals (are the results within the trust bandwidth?) help interpreting and valuating model outcomes against the original use case objectives.

In addition, automated AI behavior measurements can gauge model effectiveness, behavior, data quality, re-use ratios, bias-reduction, and stability. These outcomes can be compared with predefined thresholds to obtain the calculated quality level.

5. Intervene decisively

These assessments facilitate running ModelOps in a fluid environment, able to source and scale on demand. In terms of model development, fluidity means that, while not being consumed, models should be considered concepts instead of end-products in progress. As soon as the outcome of a model raises doubts, work on it should be halted immediately. Instead of spending effort on correcting code or adapting use cases in hindsight, the team should go back to the drawing board, applying lessons learned. By learning from failures and intervening decisively, only trustworthy developments will receive investment, turning ModelOps increasingly cost-efficient and its results more compliant.

These five EAI pillars strengthen the non-tech participation in ModelOps, and offer a governing structure to ensure the effectiveness of people and process going forward. From a more technical angle, the EAI architectural framework can redress innovative AI technology to fit the business-inspired principles of security and compliance by design. In this context, the re-use of existing reference architectures, and standardized and tested functional components can speed up auditing and accelerate implementation of compliant models. Also, technologies like Federated Learning, where ModelOps are dispersed over various sites and data is not shared between them, and Data Twinning, which creates and employs synthetic data sources, may turn out to be promising innovations. With EAI striking the right balance between technology and human oversight, such developments will not only strengthen compliance and risk-appetite, but also help releasing the Gullivers of this world.