Growing number of businesses recognize need for ethical and trusted AI-powered systems but progress is still patchy
Capgemini report finds that 70% of customers expect organizations to provide AI interactions that are transparent and fair
Artificial Intelligence (AI) has the power to positively transform society and the environment, and to harness that power to its full potential, organizations need to focus on addressing the ethical challenges. That’s according to a new report from the Capgemini Research Institute, “AI and the ethical conundrum: How organizations can build ethically robust AI systems and gain trust,” which found that the share of customers who believe organizations are being fully transparent about how they are using their personal data has fallen from 76% in 2019 to 62% today. The report also finds that only 53% of organizations have a leader who is responsible for ethics of AI systems. Additionally, governance and accountability for AI, and deploying pragmatic tools and frameworks for AI systems to be developed and used, is coming at a high cost for organizations.
The report notes that while organizations are more ethically aware, progress in implementing ethical AI has been inconsistent. For example, nearly the same share of executives in 2019 said that they have taken steps on building “fairness” (65% in 2020 vs. 66% in 2019) and “auditability” (45% in 2020 vs. 46% in 2019) dimensions of their AI systems; also, transparency has dropped from 73% to 59%, despite the fact that 58% of businesses say they have been building awareness amongst employees about issues that can result from the use of AI. The research also reveals that 70% of customers want a clear explanation of results and expect organizations to provide AI interactions that are transparent and fair.
Discriminatory bias of AI systems and negative AI customer experiences come at a high cost for organizations
As public and private organizations increasingly continue to deploy a range of AI technologies, it is critical for them to uphold customer trust by furthering positive relationships between AI and consumers. However, reports show that datasets collected for healthcare and the public sector are subjected to social and cultural bias1.
This is not limited to just the public sector. Capgemini’s research found that 65% of executives said they were aware of the issue of discriminatory bias with AI systems. Further, close to 60% of organizations have attracted legal scrutiny and 22% have faced a customer backlash in the last two to three years because of decisions reached by AI systems. In fact, 45% of customers noted they will share their negative experiences with family and friends and urge them not to engage with an organization, 39% will raise their concerns with the organization and demand an explanation, and 39% will switch from the AI channel to a higher-cost human interaction. Over a quarter of consumers (27%) say they would cease dealing with the organization altogether.
Establish ownership of ethical issues – leaders must be accountable
Only 53% of organizations have a leader who is responsible for the ethics of AI systems at their organization, such as a Chief Ethics Officer. It is crucial to establish leadership at the top to ensure these issues receive due priority from top management and to create ethically robust AI systems.
In addition, leaders in both business and technology functions must be fully accountable for the ethical outcomes of AI applications. Our research shows that only half said they had a confidential hotline or ombudsman to enable customers and employees to raise ethical issues with AI systems.
Ethical governance has become a prerequisite
The need for organizations to implement an ethical charter is also driven by increased regulatory frameworks. For example, the European Commission has issued guidelines on the key ethical principles that should be used for designing AI applications2. Meanwhile, guidelines issued by the US Federal Trade Commission (FTC) in early 2020 call for transparent AI3, stating that when an AI-enabled system makes an adverse decision (such as declining credit for a customer), then the organization should show the affected consumer the key data points used in arriving at the decision and give them the right to change any incorrect information. However, while globally 73% of organizations informed users about the ways in which AI decisions might affect them in 2019, today, this has dropped to 59%.
According to the report, this is indicative of current circumstances brought about by COVID-19, as well as societal and environmental needs, growing complexity of AI models, and a change in consumer behavior, which have all disrupted the functionalities of the existing AI models. New factors, including a preference of safety, bulk buying, and a lack of training data for similar situations from the past, has meant that organizations are redesigning their systems to suit a new normal; however, this has led to less transparency.
The report highlights seven key actions for organizations to build an ethically robust AI system, with each being underpinned by a strong foundation of leadership, governance, and internal practices4:
- Clearly outline the intended purpose of AI systems and assess its overall potential impact
- Proactively deploy AI for the benefit of society and environment
- Embed diversity and inclusion principles proactively throughout the lifecycle of AI systems
- Enhance transparency with the help of technology tools
- Humanize the AI experience and ensure human oversight of AI systems
- Ensure technological robustness of AI systems
- Protect people’s individual privacy by empowering them and putting them in charge of AI interactions
“Given its potential, the ethical use of AI should of course ensure no harm to humans, and full human responsibility and accountability for when things go wrong. But beyond that there is a very real opportunity for a proactive pursuit of environmental good and social welfare,” comments Anne-Laure Thieullent, Artificial Intelligence and Analytics Group Offer Leader at Capgemini. “AI is a transformational technology with the power to bring about far-reaching developments across the business, as well as society and the environment. Instead of fearing the impacts of AI on humans and society, it is absolutely possible to direct AI towards actively fighting bias against minorities, even correcting human bias existing in our societies today. This means governmental and non-governmental organizations that possess the AI capabilities, wealth of data, and a purpose to work for the welfare of society and environment must take greater responsibility in tackling these issues to benefit societies now and in the future, all while respecting transparency and their own accountability in the process.”
To read a full copy of the report and its recommendations, click here.
Research methodology
For the 2020 report data points, Capgemini conducted a global consumer and executive survey during April – May 2020. The consumer survey polled 2,900 consumers in six countries, while the executive survey polled 884 executives (functions include: Information Technology, AI developers, data scientists, sales, marketing and customer services) from across ten countries. These results were further compared against 2019 surveys conducted during April 2019 and June 2019, involving 5,000 consumers, and 722 executives which included AI developers, data scientists, sales, and marketing teams. Capgemini also conducted in-depth interviews with a number of industry executives, academicians, and subject matter experts in the area of ethics in AI, during August-September 2020.
