Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Crowdsourcing policies and solutions to wicked problems: case study of crowdsourcing on air quality in European cities

1 e-Governance Academy, Estonia ↗
2 Citizen Foundation, Iceland ↗
Publication year
Digital Object Identifier10.54941/ahfe1003887

Abstract

The current world is facing unprecedented crises and wicked problems that need solving. Even if some of them might affect our life more right now and in the short term (e.g., global security crisis, energy crisis, etc.), the biggest challenge for societies for years to come is climate change. Digital solutions have the potential to offer relief and solutions to various problems, including climate challenges, but the biggest concern is still the large gap between policymakers and people's concerns and expectations. Future developments should not only be guided by the technologically feasible but by what is socially desirable and acceptable. Therefore, the citizens’ engagement, particularly digital crowdsourcing, for the design of digital services, as well as policies that affect their lives, is crucial.To understand the usage of crowdsourcing for policy-design purposes, we have piloted a crowdsourcing activity in five European countries on the subject of air quality. To assess the results of the pilot, we use a novel e-participation assessment framework created by the joint effort of the e-Governance Academy and the project partners of the Co-Designing Europe Project . The assessment framework was created based on extensive analyses of e-participation cases from various countries and contexts using the Theory of Change (ToC) methodology. The framework also has a practical checklist which enables any initiator of e-participation - be it from a public authority or from the civil society, to design an e-participation/crowdsourcing activity and assess its impact after completing the case. The aim of this article is, based on theory and practice, to highlight and analyse the biggest challenges, obstacles, and success factors for impactful crowdsourcing, the potential to use the “wisdom of the crowd” to solve wicked problems in society, and the role technology plays in this process.

INTRODUCTION

We are living in a moment where it seems that previous or current socie- tal decision-making and solution-creation models are not working anymore. Even if some of the current crises (e.g., global security crisis, energy crisis, etc.)

1Co-Deciding Europe: Civic Tech for Good Governance and Active Citizenship! (CODE Europe). This project is supported by Iceland, Liechtenstein and Norway through the EEA and Norway Grants Fund for Regional Cooperation.

© 2023. Published by AHFE Open Access. All rights reserved. 107

108 Reinsalu and Bjarnason

might have rather short-term effects, there is a bigger crisis and challenge for societies for years to come, climate change.

Policies and solutions addressing climate change are only effective if citi- zens and any other target groups who should follow them and whose attitudes, behaviours and choices are decisive, understand the challenge, are motivated to contribute, and trust those who lead the transformation. Citi- zen engagement and trust are the cornerstones of a happier, wealthier, more innovative, and sustainable society, and even more so at times of crises or transformations.

Digital solutions have a big potential to amplify and support democratic processes. Moreover, they offer some relief and solutions to various problems, including climate challenges. However, the biggest concern is still a large gap between policy makers and people’s concerns and expectations. Future deve- lopments should not only be guided by the technologically feasible, but by what is socially desirable and acceptable. Therefore, the citizens’ engagement, particularly digital crowdsourcing, for the design of digital services, as well as policies that affect their lives, is crucial.

To understand the usage and impact of crowdsourcing for policy-design purposes, we have piloted a crowdsourcing activity on air quality in five European countries. To assess the results of the pilot, we use a novel e- Participation Assessment framework created as a joint effort between the e-Governance Academy and the project partners of the Co-Designing Europe Project2.

As the pilots have only been completed in January 2023, we cannot pre- sent in this paper the comparative results nor discuss how the same digital engagement methodology and digital platforms work in the different coun- tries’ or cities’ contexts. However, we are presenting the results of one of the pilots – the one implemented in Tallinn City.

The aim of this article is to highlight and analyse the biggest challenges, obstacles, and success factors for impactful crowdsourcing, the potential to use the “wisdom of the crowd” to solve wicked problems in society, and the role technology plays in this process.

The paper is structured as follows: First, we introduce the pilot on crowdsourcing, its aims, and its implemen- tation (different phases, activities, platforms).

Secondly, we introduce the theoretical framework for assessing e-Participation.

The final part of the paper summarizes and highlights the main conclusions of the pilot on crowdsourcing and discusses the theoretical conversations and arguments found in literature and evaluates the pilot in light of these.

CROWDSOURCING ON AIR QUALITY: PILOT IN TALLINN CITY, ESTONIA The general aim of the CODE Europe project was to empower citizens to co-create policies with decision-makers by piloting a crowdsourcing activity

2Co-Deciding Europe: Civic Tech for Good Governance and Active Citizenship! (CODE Europe). This project is supported by Iceland, Liechtenstein and Norway through the EEA and Norway Grants Fund for Regional Cooperation.

Crowdsourcing Policies and Solutions to Wicked Problems 109

in five European countries on the subject of air quality. Crowdsourcing is a participatory democracy mechanism that takes advantage of the availability of technological solutions to solicit and analyse “the wisdom of the crowd”. We want to empower citizens by giving them the opportunity to learn from each other, collaborate and participate in the decision-making.

One of the pilots was carried out by e-Governance Academy in Tallinn. In this chapter we introduce the pilot in Tallinn, its aims, and its imple- mentation (different phases, activities, platforms).

The crowdsourcing pilot activities in Tallinn took place in 2022, in four phases as shown in Figure 1 below; the content, and results are described in the following sections.

Phase 1 “Problem Identification” The purpose of the first phase was to gather information about what concerns people had regarding air pollution in Tallinn and how they rated their expertise in this area.

The phase focused on creating a network of civil society organizations who a) deal with climate issue topics, b) focus on community activism, enhancement of participatory democracy, etc.

The aim was to reach out and activate the residents to participate in the crowdsourcing pilot and signal their problems related to air quality in Tallinn on the platform3. The platform had a total of 2,500 unique visitors, and 296 responses were left on the platform.

The analysis of the responses indicated that people were most concer- ned about the negative impact of air pollution on the environment (36%) and health (35%). The open answer “Other” indicated additional sources of concern such as noise pollution, the city’s bad reputation, damage of buildings, impairment of real estate, restrictive effects on lifestyle and dete- rioration in the quality of life. 10% of the respondents had no personal experience with air pollution. Tallinners consider transport to be the biggest air polluter, followed by the industry and home heating.

Evaluating personal awareness of air pollution on a 10-point scale, 17% found it poor (1 - 3), 59% mediocre (4 - 7) and 24% good (8 - 10).

As a result of the first phase, the areas of air pollution, that the residents of the project cities consider as most important, were identified.

Figure 1: Pilot phases and timeframe (2022).
Original PDF Figure 1:

3The Platform https://crowdsourcing.ecas.org/et/air-quality-tallinn was functional in the period 9.02 - 31.03.2022.

110 Reinsalu and Bjarnason

Phase 2 ‘Problem Solving’ The purpose of the second phase was to invite the residents of Tallinn to pro- pose solutions on how to reduce air pollution and mitigate its consequences in the areas identified in Phase 1.

The second phase took place in the second quarter of 2022 using the other platform4. The promotion of the platform started in the mid of April with the usage deadline of July 15th. The platform received 61 proposals from 152 participants. There were nearly 1,800 unique visitors but only a small part of them contributed with ideas. In addition to the discussion on the platform, some ideas have also been lively commented on social media.

61% of the proposals submitted by the residents of Tallinn are related to reducing air pollution from transport. The proposals could be classified as follows:

  • Make the city bike-friendly
  • Improve traffic management
  • Limit the use of cars
  • More greenery in the city
  • Promote the use of public transport
  • Reduce pollution sources
  • Better urban planning Phase 3 ‘Selection of Ideas’

The purpose of the third stage was to identify the solutions that find the most support throughout all the cities participating in the project. For this purpose, platform users had the opportunity to rank the proposals presented on the platform according to their priority. The final list for rating consisted of 31 solutions identified by the project partner European Environmental Bureau (EEB).

The third phase took place mainly in the third quarter of 2022 using the third platform5. Due to low participation during the summer months, the period of rating collection was extended until October 21, 2022. As a result, 137 users rated the solutions on the Tallinn platform. There were more regi- stered users – 195, but not everyone contributed to the evaluation of the solutions.

The following proposals turned out to be the most important for the respondents of Tallinn:

1. Increased cycling and walking infrastructures and their quality 2. More green spaces in cities 3. Speed limit of 30km/h in cities 4. Strengthen public transport network in non-urban areas 5. Car-free cities 6. Support renewable energy infrastructures (e.g., Solar panels) 7. Tax big polluters (companies)

4https://tallinn-code-europe.yrpri.org/ 5https://code-europe.onesource.pt/et/tallinn

Crowdsourcing Policies and Solutions to Wicked Problems 111

All the proposals that proved popular with the respondents of Tallinn, were carried over to the 4th phase.

Phase 4 ‘Co-Creation’: The purpose of the fourth phase was to prepare policy proposals for submis- sion to the EU level that would help improve air quality in the European cities. Considering the priority solutions identified by crowdsourcing at the third stage, the project partner, the EEB, formulated policy proposals on the following topics:

  • Establish a speed limit of 30 km/h in cities
  • Electrifying bus fleets
  • Workshops and school programs to increase awareness about air pollution and environmental protection
  • Establish rules to get zero-emissions industries
  • Insulate buildings
  • Strengthen public transport network in non-urban areas
  • Car-free cities and more green space in cities
  • Increased cycling and walking infrastructures and improve their quality
  • Support renewable energy infrastructure (e.g., Solar panels) The proposals were made available to the public on the fourth platform6. There was an invitation to support the proposals and, if necessary, leave comments to improve the proposals. The platform became public in Decem- ber 2022. In the first order, we invited experts to visit the platform to get competent suggestions about the texts on the platform.

ASSESSING THE IMPACT OF E-PARTICIPATION Citizen participation – mostly e-participation in present world - and trust are the cornerstones of modern democracy, good governance, and civil soci- ety. Societies where the people can follow their interests, initiate, and discuss public issues and be engaged in the decision-making and policy-making that affect their lives, are happier, wealthier and more innovative. Even if this is a broadly accepted concept in theory, it is not often realized in practice.

The driving force to create such a framework (and a checklist for citizen engagement planners) was that despite hundreds of attempts to engage citi- zens in policymaking, their real impact hasn’t been assessed often and we still lack the knowledge and understanding of what works well and what not, and how much and which digital tools support this the most.

Therefore, the assessment framework for e-participation created within the Code Europe Project7 is aimed to support a result-oriented implementation of 6https://code.parvaipret.lv/ 7The development of the methodology was a joint effort of the experts of the Co-Deciding Europe project WP3 partner organizations: European Citizen Action Service (ECAS), Belgium; Institute for Electronic Participation, Slovenia; Centre for Public Policy PROVIDUS, Latvia; Centre for Social Sciences, Hungary; Citizens Foundation Iceland and e-Governance Academy, Estonia. The process was led and the assessment framework for e-participation was compiled by the e-Governance Academy expert team.

112 Reinsalu and Bjarnason

e-participation for democratic governance in digital society. The framework can be used as a theoretical platform for the elaboration of success criteria and success factors, for comparative assessments of e-participation initiatives, as well as for designing and monitoring e-participation projects. It could also serve as a tool for knowledge sharing and capacity building to enhance the effectiveness and impact of e-participation initiatives.

The assessment framework was created based on extensive analyses of e- participation cases (35) from various countries and contexts using the Theory of Change8(ToC) methodology, which makes it possible to assess whether the implemented projects are successful in terms of a strategic vision of the situation to be achieved using e-participation.

The condition for each of the 35 e-participation cases was the adva- ncement of effective participation of citizens in political decision-making processes through consultation, collaboration, and empowerment. The ini- tial analyses provided a preliminary selection of successful and failed digital engagement cases. Both categories were analysed in-depth (desktop study and interviews with key persons from governments and civil society) to elabo- rate the framework for assessment, list of success (and failure) factors, and recommendations.

The framework also has a practical checklist which enables any initiator of e-participation - be it from a public authority or from civil society, to design an e-participation/crowdsourcing activity and assess its impact after completing the case.

Success Factors Input includes the general conditions affecting the e-participation initiative and resources needed to accomplish the process. For the Input the following conditions (factors) are documented in different studies, and contribute to the success of the initiative:

  • public interest in the topic of the initiative
  • political importance of the initiative
  • legal framework supporting the initiative
  • link to the formal decision-making process
  • resources available to the initiator of the case
  • user friendliness of the platform
  • sustainability of the platform Activities are the actions taken on the e-platform and outside the platform to achieve the aim of the initiative. Activities represent the actual imple- mentation of the e-participation project. The principal difference from the traditional forms of citizen engagement is the use of e-platforms and chan- nels to carry out the project. Depending on the type of initiative, different e-platforms and activities could be envisaged. From the published analysis of success factors of e-participation, the following conditions have been found to have a positive effect:

8https://www.theoryofchange.org/what-is-theory-of-change/

Crowdsourcing Policies and Solutions to Wicked Problems 113

  • clarity on the participation process and its aim
  • possibility to interact with other participants
  • combination of online and offline activities
  • feedback to participants about what has been done with their contributi- ons/the results of the initiative
  • public outreach and engagement For a comparative analysis of the cases, appropriate questions need to be formulated to assess the presence or absence of success factors in a consi- stent manner. The shown sample questions could be reviewed considering the particularities of the analysed cases.

Success Criteria The success criteria, output, outcome, and impact, characterize the results of the e-participation initiative on different levels.

Output consists of immediate, direct, and tangible results of the e- participation initiative.

Outcome is the short- or medium-term effect of the e-participation initi- ative in terms of democracy (benefits for citizens, better policies, increased public awareness, etc.).

Impact is generally characterized as the long-term effects of the e- participation initiative on democratic processes in the society.

Assessing the outcome of the e-participation initiative is particularly important, as it is directly related to desired outcomes of the Theory of Change. The level of output could be excluded from the comparative analy- sis since it does not directly reflect the public value of the initiative. It is also difficult to assess the strategic impact of any individual initiative; in case the immediate impact is detected (for example, the direct effect on decision making), it could be considered as an outcome of the project.

Based on the success criteria discussed above, the desired outcome of e- participation initiatives could be characterized as follows:

  • participants are satisfied, they acquired new knowledge and skills
  • public authorities are satisfied, they gained valuable input for their work
  • the case has had a direct effect on political decision-making.

According to the chosen interpretation, any case could be assessed as successful if the result would be assessed as positive in least for one of the success criteria.

DISCUSSION AND LESSONS LEARNED In this section we present the results of the initial analysis of the pilot and the recommendations using the assessment framework introduced in the second chapter of the article.

This section also discusses the theoretical conversations and arguments found in the literature and evaluates the pilot in light of these.

114 Reinsalu and Bjarnason

However, it must be stated that at the moment of writing this article, we can mostly focus on success factors and less on success criteria as the crowd- sourcing pilot has just been completed; the survey of participants and the city administration for obtaining their evaluation of the process is still ongoing.

The main basic conclusions about success factors that we can draw are:

1) Both public interest in the topic of the initiative and political importance of the initiative are crucial success factors. The pilot proved how impor- tant the timing and context are in crowdsourcing. Unfortunately, the beginning of our pilot nearly coincided with a geopolitical catastrophe. The Russian invasion of Ukraine on February 24, 2022, made all other topics irrelevant for people. Estonia and Latvia (two of the five pilot coun- tries) were and still are very closely affected by the war, and it has been a very big effort for us to ensure relatively high numbers of participants (especially compared to other partners – from Greece, Belgium, etc.). The context equally matters when it comes to political interest and engage- ment. In this case, the regional security crisis, along with other crises, pushed climate issues off the real agenda. In our case, although no one in the city government denies the importance of climate challenges and the engagement of people in addressing them, the understanding of the value of crowdsourcing or e-participation is low, and the public authorities cannot give this value a practical output. 2) As we focus on e-participation, obviously one of the crucial elements are the digital platforms which enable participation. If the platform is not user-friendly, if its functionalities do not support the purpose and there are many technical barriers, as we experienced in different phases, then it is hard to reach the goals. In the pilot project, co-creation was carried out in four stages, each using different technical platforms. In retrospect, it must be recognized that it was certainly not justified. In addition to the user-friendliness of technical platforms, the sustainability of the plat- form and the whole process is very important. Firstly, it was so difficult to invite people to participate in the next stage because it was a different and new platform. Secondly, it made the whole e-participation process less transparent, and therefore also less attractive and trusted. Besides the clarity of the process, our pilot also clearly demonstrated that it is impor- tant to create a community around the topic and platform combining online and offline activities, to provide the possibility to interact with other participants and to be very concise and consistent in giving feed- back to participants about what has been done with their contributions, and what are the results of the initiative. 3) The lack of resources is a big limitation to those crowdsourcing pilots: minimal marketing budgets predictably resulted in a limited number of participating citizens. Marketing budgets are often a limitation of publicly funded projects, as reaching citizens on the platforms they spend their time on, like Facebook, Twitter, and TikTok requires significant payments to those platforms, which is not popular with funders. But as those were pilot projects, the results are still exciting. It is essential to remember for

Crowdsourcing Policies and Solutions to Wicked Problems 115

non-pilot projects that a significant marketing budget is needed to reach a substantial number of citizens to participate.

Le Blanc (2020) stated that while e-participation platforms using new tech- nologies have spread rapidly during the last 10 and more years, it is not clear that their multiplication has translated into broader or deeper citizen participation. One of the conclusions he makes is that the main reasons for that are the lack of understanding of citizens’ motivations to participate and the reluctance of public institutions to genuinely share agenda setting and decision-making power.

As mentioned above, it is too early to make any final conclusions on citizens’ or public institutions’ satisfaction with the process, or point out dif- ferences by countries, as the assessment is ongoing. However, our pilot proves that currently the “wisdom of the crowd” is not valued as much as it can by public institutions, nor it is used in practice.

As Le Blanc (2020) posits, there is always a gap between policy discourses, which may put citizen engagement and participation high on the political agenda, and the real appetite for increasing participation.

On one hand, this is understandable since the numbers of e-participants are marginal and the “crowd”is usually not there anyway. Our pilot confirms the statement many authors make (Le Blanc, 2020 Toots, 2019; Sæbø et al. 2011) that e-participation platforms still fail to sustain interest from citizens, even in countries very advanced in e-government, such as Estonia, register failure in e-participation.

On the other hand, it is possible that the reason why the big hope for democracy that was placed on e-participation a few decades ago has not been fulfilled is due to this too digital-centred approach to e-government in general, but also in e-participation.

As stated by many authors already years ago (Sæbø et al., 2008; Macintosh et al., 2009; Medaglia, 2012), setting up platforms for e-participation is not sufficient to stimulate participation. Our pilot clearly proved this argument is still very relevant. More broadly, technology, by itself, cannot be expected to increase civic engagement and participation.

One clear lesson that emerges from our pilot but is also supported by avai- lable data and earlier studies (Le Blanc 2020) is that successful e-participation projects combine online and offline activities and tools. We know, and this was also clear while implementing the pilot, that this is usually the latter which implies and requires more costs and resources than digital platforms. Although there were, and sometimes still are, great expectations put on social media for making it easier to organize public campaigns and attract citi- zens, and also bring them to e-participation platforms, our pilot proves the argument of Epstein et al., that social media on their own do does not provide a miraculous weapon for organizing meaningful consultations in complex areas such as rulemaking (Epstein et al., 2014).

To conclude, even though e-participation has been in the limelight of scholars and politicians for years, it is still a phenomenon that is occasio- nally overestimated and mystified, and at the same time probably still often underestimated.

116 Reinsalu and Bjarnason

In this context, it is clear that this dynamic phenomenon needs systema- tic further research - a good framework for how different factors affect the outcome and impact of the e-participation initiative and how to evaluate the result of the involvement to be more successful next time. Hopefully, the fra- mework we created in the CODE Europe project and introduced in the article will also contribute to this.

ACKNOWLEDGMENT

The authors would like to acknowledge all the project partners of the CODE Europe project Co-Deciding Europe: Civic Tech for Good Governance and Active Citizenship! (CODE Europe).

The work on the article and the presentation of it was also partly sup- ported by the CODE Europe project. The project is supported by Iceland, Liechtenstein and Norway through the EEA and Norway Grants Fund for Regional Cooperation.

REFERENCES

Epstein, D., Newhart, M., Vernon, R., (2014). Not by technology alone: The “analog” aspects of online public engagement in policymaking. Government Information Quarterly 31, 337–344. Le Blanc, D., (2020). E-participation: a quick overview of recent qualitative trends.

DESA Working Paper No. 163. Macintosh, A., Coleman, S., Schneeberger, A., (2009). eParticipation: The Research Gaps, in: Macintosh, A., Tambouris, E. (Eds.), Electronic Participation, Lecture Notes in Computer Science. Springer, Berlin, Heidelberg, pp. 1–11. Medaglia, R., (2012). eParticipation research: Moving characterization forward

(2006-2011). Government Information Quarterly 29, 346–360. Sæbø, Ø., Rose, J., Skiftenes Flak, L., (2008). The shape of eParticipation: Characterizing an emerging research area. Government Information Quarterly 25, 400–428. Toots, M., (2019). Why E-participation systems fail: The case of Estonia’s Osale.ee,

Government Information Quarterly, Volume 36, Issue 3.

Cover of the proceedings volume Human Factors, Business Management and Society

Explore the complete digital volume

Human Factors, Business Management and Society

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-73-5

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →