February 2, 2023
In the next five minutes (about the amount of time it takes for you to read this article), approximately 2.25 million opinions on Twitter, 1.5 million thoughts on Facebook, and 200,000 feelings on Instagram will be shared — a literal explosion of data.
We now know that 90% of the world’s data has been generated in the last few years. 80% of this information is unstructured, heterogeneous, and expressed in different languages. In other words, that data is not “clear” or “immediate” for human comprehension.
What does this mean for decision-makers – those who formulate policies based on their understanding of the communities they serve – and how can they be helped to base reforms on a more nuanced understanding of development needs?
Enter modern techniques for understanding large, unstructured, datasets of communications
Humans need technology and algorithms that allow for unstructured data to be translated into visualizations and formats. These insights help decision-makers make sense of the data and then take appropriate actions needed to improve the lives of citizens. It is essential/important that the data must be sampled and acquired accurately to provide statistical significance for conveying sentiments that are widely shared in the population. Otherwise, the data may provide a misleading understanding of an unrepresentative group of society.
If data is representative, technology and algorithms can contribute to extracting meaningful information. This is possible thanks to the use of natural language processing (NLP) which allows for named entity recognition, and topic modeling, among other things.
A machine learning process takes place on top of such extraction and analysis. Meaning algorithms can be trained to automatically identify patterns and trends in data. This can be especially useful for large datasets where manual analysis may be infeasible.
Finally, the data output requires visualization to help identify patterns and trends that may not be immediately apparent when looking at the raw data. Tools like graphs, charts, and maps can be used to represent the data in a more intuitive way.
Citibeats: Leveraging AI & Data Analytics
One of the companies enabling these technologies is Citibeats. An ethical artificial intelligence technology platform that uses data to give voice to all citizens in real time — providing continually updated data about citizens’ needs and opinions — while reducing gender bias and respecting individual privacy.
This collaboration has been made possible through the financial support of the Danish-led Tech for Democracy initiative, which focuses on leveraging digital technologies to strengthen democracy and human rights, by promoting civic engagement, strengthening democratic institutions, and enhancing human progress. Through their support, UNDP Pakistan has been able to work with Citibeats – a platform that is language-agnostic and culturally adaptable which allows leaders to extract actionable insights from data collected by citizens. This inclusive data includes all dialects, genders, economic status and educational levels, thereby giving voices to people that may otherwise remain unheard.
This focus has proven essential for many global projects, like the IDB research project in the Amazon basin. In the short term, the IDB was able to leverage Citibeats to strengthen its data collection process by gathering information directly from over 500 representatives of indigenous people, Afro-descendant groups, peasants, local and regional organizations. As a result, the Green Climate Fund approved 279 million dollars financing for the Amazon Project, one of the largest ever issued by the Fund. In the longer term, this initiative will help design projects at a national level and allocate funds to those areas with the greatest needs. For example, promoting access to credit for women in the Amazon basin, improving the sanitary conditions of people and the environment, promoting recycling policies, and more.
The case of Pakistan: Understanding to uplift
Pakistan has similar potential. With a population of over 225 million people, the country has had to contend with a host of challenges. Major improvement projects are already under way, namely in urban planning. Pakistan currently has 82 million people living in urban areas, 30 million of whom are living in informal settlements. In Karachi, this is the case for half of its population.
This urban population growth presents a dire need for effective urban planning and development initiatives, even more so in the wake of the recent devastating floods that have affected 33 million people and left one-third of the country underwater. The flooding has also displaced eight million people and caused additional migration to already migrant-packed cities like Karachi.
UNDP in Pakistan works with the Government and people of Pakistan to develop local solutions to development challenges. In the context of the ‘data explosion’, UNDP is collaborating with Citibeats to collect, analyze, structure, and extract actionable insights from social data that will be collected directly from citizens. Powered by advanced AI technology, Citibeats is providing data analytics of social indicators which will help empower Pakistan’s leaders to develop actionable, data-driven strategies and inclusive policies.
Early lessons using Citibeats
While UNDP and its partners are making great strides in improving major grievances that Pakistani communities face, there is still much to be done, and the collaboration with Citibeats is already making an impact.
Citibeats has already collected more than 2 million data points in which Pakistani people express their daily concerns and problems. To be fully representative, the Citibeats platform is gathering insights into the entire population while placing an emphasis on citizens that are often excluded — like women living in poor conditions — and understanding their greatest needs and concerns.
So far, the data has brought three important points to the surface:
- The most immediate needs of citizens
Given the devastation and state of emergency caused by the flooding, plus other socio-economic hardships happening at the same time, it’s vital to understand what people need right now. Topics heard so far include water, sanitation, infrastructure, and schools. Knowing what these are in real time allows decision-makers to address their needs in the most timely way possible.
- The resilience of the Pakistani people
An impressive amount of data is uncovering how resilient and supportive Pakistani communities are in times of need. For example, for citizens that don’t have enough to eat, people are cooking together in the streets to make ingredients more affordable and helping families keep their children fed. In terms of infrastructure, people are coming together to help rebuild bridges that were destroyed during the floods and build shelters for families that need homes.
But while they’re resilient, they need help/support from civil society institutions and the government. The data gathered by Citibeats is helping leaders understand the areas where Pakistanis are already resilient so that they can scale up in those areas, use what already exists at a grassroot level and elevate it further. The platform is also identifying the areas where people are unable to cope so that they can get support in bridging these gaps.
- Pakistani citizens want to build for the future
Despite the flooding catastrophe and other hardships, Pakistani citizens show hope for recovery and rebuilding. Through the platform, Citibeats and UNDP have access to hearing their ambitions and dreams for the future of Pakistan. The data is not only about giving access to immediate needs but providing foresight into the country that the people want to see Pakistan become.
Many important issues/topics have already come to light despite the early stages of the collaboration between Citibeats and UNDP. It’s also clear that the project will be valuable in making a difference in the short term (identifying immediate needs) and long term (building for a future that is sustainable and aligned with citizens' aspirations and hopes).
While the road to recovery is long and far from immediate, every step taken to help improve the lives of people in Pakistan is a step in the right direction. Technology based on ethical AI can build bridges between citizens and both government and non-government institutions and organizations. Establishing these connections helps decision-makers find out about the needs of people and understand the changes in society in real time to make faster and better-informed decisions that will best serve the people.
Citibeats leverages ethical AI for social understanding. Gathering and analyzing unstructured data from social media comments, blog posts, forums, and more, our Sustainability and Social Risk Monitors provide insight into millions of unfolding conversations regarding inflation, protests, food shortages, and more—empowering world leaders to develop data-driven strategies and inclusive policies.
Schedule a demo today to learn more.
