Citibeats & Giskard: A Partnership to Understand the Behavior of NLP Models

Towards Transparent NLP

The success of Natural Language Processing (NLP) systems has become more rampant during recent years and the industry owes this boom to Large Natural-Language Models (LLMs). However, this success is impeded by the models’ inability to prove that their predictions are derived by objective, inclusive means. We know what the inputs and outputs are, and how likely it is for the outputs to be correct, but we still don’t have a clear understanding of what happens in between. 

This lack of transparency might deter users from using the algorithm because of their hesitation to completely trust it. Understandably, major risks can arise if such models are misunderstood or improperly applied to make business and governance decisions. 

What’s more, these models are often trained on massive volumes of data collected from people’s daily internet activities, the primary way that people today voice their opinions, wishes, and complaints. This has led researchers to warn against the dangers of such models being founded on certain prejudices and biases such as racism and gender discrimination. 

A commercial sentiment analysis model showing vulnerability to NER perturbations

Ribeiro et al., 2020 - Beyond Accuracy: Behavioral Testing of NLP Models with CheckList

Some Citibeats models rely on pre-trained LLMs to process human opinions and extract insights that policymakers and stakeholders can use when making the appropriate decisions for various social matters. We therefore aim to build and deploy Machine Learning systems that are accurate but also fair and representative of all relevant opinions.

Thanks to Giskard, our data scientists can quickly implement and automate a screening process to ensure the release of fair and trustworthy NLP models; models that not only abide by Citibeat’s Ethical AI values, but also show an accurate understanding of Natural Language.  Additionally, our data scientists can collaborate with business stakeholders to exchange feedback about the screening results in a centralized and streamlined manner.

This collaboration represents one of the first steps towards a Model Explainability open-source project under Citibeats’ open Data Science initiative.  

If you are concerned about bias in AI models, join Citibeats’ Ethical AI community and help decision-makers make better decisions based on unbiased data. 

About Giskard

Giskard is a collaborative and open-source quality assessment tool for AI models. It offers code presets that enable users to efficiently write and automate different types of Machine Learning (ML) tests. The tool enables users to customize tests and perform instant inspections to monitor the model’s output for a given set of feature values. This feature is extremely useful as it allows the tester to visualize feature contributions and also directly apply changes to their values and monitor the output. 

Giskard offers an intuitive model inspection interface. Besides investigating the model's behavior, this interface also empowers collaboration. It enables team members to leave notes and tag teammates directly using discussion threads that facilitate follow-up and decision making. Then, it enables automated testing of AI models to avoid the risk of biases and ensure reliability & robustness.

About Citibeats

Citibeats - a 2022 World Economic Forum Technology Pioneer - is a SaaS ethical artificial intelligence platform that searches and analyzes large amounts of text provided by citizens.  We structure this data using NLP and ML to identify social trends, leading indicators, and actionable insights for governments, organizations and private companies.

Contrary to marketing brand watching, Citibeats offers a much more complex analysis and interpretation of data. Instead of being purely focused on single keywords, our research method is culturally focused, language agnostic, easily adaptable, and monitors complex narratives that evolve  over time around a specific topic and that involve other related subtopics and multiple opinions expressed by the people.