Barcelona Metropolis

On 12 February, the new Barcelona Datastore portal went live, offering a tool to analyse and understand the city. It was launched against the backdrop of society’s rapid datafication, with a notable emphasis on statistical techniques aimed at harnessing new sources and utilising big data​.

Barcelona Datastore portal represents a commitment to improving access to and understanding of city data through significant technological advancements that affect both data generation and dissemination. The main objective of the Barcelona City Council’s new website is to integrate and disseminate city data promptly, effectively and efficiently, providing high-quality information about the recent evolution and current situation of Barcelona from demographic, economic, social, cultural and political perspectives.

The portal continues a long tradition of data production and dissemination: Barcelona published Spain’s first municipal statistical yearbook in 1902, the Anuario estadístico de la ciudad de Barcelona [Statistical Yearbook of the City of Barcelona], and in 2018, it was a pioneer in setting up Spain’s first Municipal Data Office. Over nearly 120 years, the city has maintained its mission to capture, store and disseminate information about the city and the municipal administration.

This extensive history includes the merging of the Department of Statistics with various departments specialised in data analysis, the execution of surveys and opinion studies, as well as data management and integration. The convergence of these departments was formalised with the approval, in February 2018, of the Measure for ethical and responsible data management governance: Barcelona Data Commons. This is based on the premise that data is a source of economic development, social progress and good democratic governance. Responsible data management throughout its lifecycle (planning, acquisition, management, analysis, protection, access, preservation and reuse) implies that data should be searchable, accessible, interoperable and reusable. This calls for the establishment of a governance strategy, architecture, processes and operational standards based on application programming interfaces (APIs) and the aggregation of various information from multiple sources into a common repository.

To meet this governance measure, the City Council established a Municipal Data Office (OMD), which is part of Barcelona City Council’s Municipal Management. This office brings together units responsible for data management, analysis and dissemination into a single structure. In this context, there is a need to modernise and operationally integrate data dissemination infrastructure.

An integrated open-source platform

The information provided by Barcelona City Council includes various types and formats: indicators, statistical tables, opinion studies, surveys, sectoral analyses, maps, georeferenced data, documents and images from archives, among others. Previously, this information was dispersed across several platforms, including the Department of Statistics and Data Dissemination website, the Barcelona Economia website, the Open Data BCN portal, or the Surveys and Opinion Studies Register.

Now, the new Barcelona Datastore portal serves as an integrated, open-source content management platform that is configurable and scalable. Notable features include an accessible and user-friendly experience: users can search, download, embed or share data in multiple formats. It also enables tabular, graphic and cartographic interaction according to user preferences and includes explanatory videos on API operation, export and connectivity. Accessible via the internet, tablets and mobile phones, it is also available in Catalan, Spanish and English, making it multi-channel and multi-lingual. Additionally, it offers a personalised section that users can customise to explore and utilise data according to their preferences.

Explaining the city through data

The key to this portal is Barcelona. In other words, it provides a means to understand the city through data. The portal offers highly detailed information, such as the precise locations of all trees in the city, as well as aggregated data at various territorial levels, including population figures by census section, basic statistical area, neighbourhood, district or the entire municipality. In addition, it provides insights such as the evolving public perception of the city over the last thirty years or analyses of the economic landscape. All of this can be accessed in data format, specific analysis documents or through interactive visualisations.

Alongside these well-established datasets, the Barcelona Datastore portal operates within a landscape evolving towards a highly data-driven society. Here, the Barcelona Datastore portal takes a step forward by integrating new data sources derived from the daily activities of city residents, workers and visitors, including private consumption data from credit card transactions and visitor data from mobile phone records.

The integration of these new sources calls for rethinking data acquisition and integration procedures, with a focus on process automation and the dissemination of high-frequency data. Custom-built software and processes are employed to aggregate, normalise and store data in a non-relational database (using open-source Solr technology) that integrates all the information, facilitates scalability and will enable the application of artificial intelligence, one of the challenges we will be up against in the immediate future.

Behind this new data model lies another strategic element, perhaps less visible but crucial for keeping the information organised: the implementation of data governance. This involves a set of procedures and policies aimed at controlling, through rules and standards, data management, including the allocation of different roles and functions.

We hope that this effort to build this new platform will make it easier to access and understand a system as interconnected and data-rich as Barcelona.