In November 2022, the National Institute of Statistics of Uruguay (INE Uruguay) faced a growing challenge in data management in the information age. The amount of data generated daily continues to increase exponentially and will continue to do so, making the need to process and interpret it at speed a key priority.
In order to address INE's need for capacity development support, the Global Partnership organized and facilitated the training of INE Uruguay's data analysts. The training program, made possible thanks to the generous support provided by the Hewlett Foundation, focused on equipping the team with the necessary skills to effectively utilize Superset, a user-friendly and open-source data visualization software application. As a result, 21 data analysts trained gained the ability to utilize Superset for data analysis and visualization purposes.
Big improvements, achieved quickly
The three-day Superset training taught us the correct use of the tool and how to connect it to our existing databases. We can now process large data sets efficiently and generate relevant information in a timely manner. Previously, creating a dashboard would take more than a week and could only be done by data analysts with advanced coding skills. Now, the visualizations can be done by all data analysts regardless of their experience, and within a few days. This has doubled our efficiency and productivity while enabling us to generate high quality interactive products that are more user-friendly and easy to explore.
Superset has been particularly useful for working with microdata, which are individual-level data collected via administrative systems, censuses, and sample surveys. They offer details about the traits of certain individuals or entities, such as households, commercial buildings, farms, or even specific geographic areas like villages or towns. With older visualization tools, processing data of this nature was a laborious process that generated heavy products, making it time-consuming to share them with our users and for the users to view them. Now, thanks to Superset which is a lightweight software, we can process large amounts of data in much less time and still generate high-quality visualizations. The tool has an intuitive user interface that makes it easy to create interactive dashboards and data visualizations, resulting in visually appealing and easily understandable information for our users to access online.
Since completing the Superset training, we have also achieved a significant improvement in the security of our data. The tool has advanced security features that allow us to control access to data and keep it protected at all times. Additionally, we have been able to implement additional security measures to ensure the privacy and confidentiality of our users' data.
INE Uruguay’s Superset products
Thanks to the training we have been able to generate high quality products and publish them on our website. One of these products is the formal labor market viewer, built from administrative records, which are housed in the SIREE Data Warehouse (Integrated System of Statistical Records and Surveys). On this platform the general public, as well as policy and decision-makers, can access detailed information about Uruguay’s labor market. Formal workers are categorized according to their age, sex, place of residence, income, functional relationship, sector (ISIC Rev. 4), and associated metrics, presented in user-friendly visual formats. For example, the evolution of the average salary can be visualized and broken down by age group, sector of activity, and more. This makes it possible to alert and allocate public policies to specific sectors of the population. For example, by leveraging the data available through the labor market viewer, policymakers can identify significant disparities in average salaries across different sectors of the economy and implement targeted policies that aimed at improving the wage levels and working conditions in the affected sectors. This approach ensures that public resources are allocated efficiently and effectively, leading to more equitable outcomes and improvements in the targeted sectors of the population.
INE Uruguay launched its labor market dashboard in March 2023 after enhancing their data visualization capabilities using the Apache Superset BI tool. Photo: courtesy INE Uruguay.
The graph below shows the evolution of the number of jobs by sector of activity. The information presented in the graph can be disaggregated by sex, age group, contribution fund, place of residence or functional link with the company using the available filters.
A visualization from the labor market viewer showing the evolution of the number of jobs by sector between July 2021 and October 2022.
As an example, if we select only the evolution of the activity sector 'Accommodation and food services' which is mainly dedicated to tourism, it can be seen that jobs in the sector increased from November to January in summer (high season) and decreased towards and during winter (low season).
The ‘Accommodation and food services’ sector in the previous graph.
A great value in upskilling
Ultimately, the Superset training supported by the Global Partnership has been a great success for our team and we have significantly improved our efficiency and productivity. It has enabled us to generate high quality products that have been very well received by users, including decision makers such as the Ministry of Labor and Social Security, and the Planning and Budget Office. Superset has proven to be a valuable tool that allows us to process large amounts of data and generate relevant information in a timely manner.
We will continue to upskill the team and use Superset to further improve our work, providing valuable and timely information to the general population and decision makers. Currently, we are preparing further visualizations about population (demography) data, based on administrative records, and we are planning a workshop to promote awareness and use of the software in other departments of INE Uruguay. There are also plans ongoing to migrate INE Uruguay’s business demography visualizations to Superset.
“The use of new ways to communicate statistics generates greater incentives for its use. The Superset tool provides opportunities for the INE Uruguay to build visualizations on different topics that are useful to decision-makers.”
- Fredy Rodriguez Galvis, Senior Regional Manager LAC, The Global Partnership