Main authors: Janja Rudolf, Špela Železnikar, Matjaž Glavan, Andrej Udovč, Sindre Langaas, Marina Pintar
Editor: Jane Brandt
Source document: »Rudolf, J. et al. (2021) Actor's feedback on practices for improvement of water quality in FAIRWAY case studies and interim project results. FAIRWAY Project Deliverable 7.2R 74 pp

 


Contents table
1. Questionnaire and survey methodology
2. Responses to the questions
3. Conclusions from the FAIRWAY case studies

1. Questionnaire and survey methodology

The questionnaire

The researchers assessed the responses of MAP leaders to a questionnaire sent by email. The duration of the survey was 40 days, starting with 10th April 2019 and finishing with 20th May 2019. In this period, two reminder emails were sent to obtain more results. 30 finished questionnaires arrived at that point, so the work group decision was to postpone the deadline until 1st of August 2019. MAPs responded and obtained more data, so at the end we exceeded our goal of 44 questionnaires, and we received 46 finished questionnaires which showed good cooperation between MAPs and the thematic research task.

The questionnaire included both closed questions with single choice answers and open-ended questions with predefined answers, offering respondents the possibility to grade on a Likert scale of agreement (from 1= do not agree to 7= very much agree) with the findings from »Barriers and issues in providing integrated scientific support for EU policy. For open-ended questions, additional questions were provided, to ask respondents if they have a different (not already presented) view on specific topics. This part is considered of great importance for the survey as it provides additional valuable material for recognising specific needs on the local level.

The questionnaire included four blocks. The first and second blocks focussed on nationality and stakeholder groups, respectively. The third block dealt with barriers and issues concerning integrated scientific support between the national and local level. The fourth block focussed on the improvement of the system; what are the possible solutions for integrated scientific support for policy on protecting drinking water resource against nitrates and pesticides pollution.

The full questionnaire is given in the Annex of this report:

Survey sample

The survey targeted all 13 MAPs represented in FAIRWAY. From each MAP at least four different fully finished paper questionnaires had to be supplied, which meant four different stakeholder group representatives per MAP, resulting in a survey sample of 52 questionnaires. In the event, 30 questionnaires were supplied from seven MAPs, from seven countries: United Kingdom, Slovenia, Portugal, Germany, Netherlands, Ireland and Romania (Table 1). Only Norway MAP did not supply any data.

Table 1: Origin of Multi-Actor Platforms that contributed to the results, number of returned questionnaires per Multi-Actor Platform, a stakeholder group that corresponded (farmers, advisory, policy makers, water policy implementation, retail, regional management and water company)

MAP origin Number of returned questionnaires  Stakeholder groups 
Farmers  Advisory  Policy makers Water policy implementation Retail Research and science Regional management Water company
England 5 1 2     1 1    
Slovenia 7 3 2   2        
Portugal 5 1   1     1 1 1
Germany 4   2   1   1    
Denmark 4 1     1   2    
France 3           3    
Netherlands 4  1     2   1    
Northern Ireland 4   2   2        
Romania 5 1 1 1     2    
Greece 5 2 1   1 1      
Total 46 9 11 2 9 2 11 1 1

MAPs are composed of different stakeholder groups and involve the FAIRWAY case studies and national authorities as shown in the FAIRWAY conceptual framework, Figure 1.

D72 fig01
Figure 1

The representatives of the different stakeholder groups that were asked to fill out the questionnaires are not necessarily experts in the field of protecting water resources. They are within their professional duties involved in the process of making decisions or otherwise influencing how to maintain drinking water resources protected on a daily or regular basis. Selected representatives gave as a survey sample of important actors involved in different fields of the protection of drinking water resources.

The responses received enabled a so-called incomplete DELPHI method to be conducted. The Delphi method (also known as Estimate-Talk-Estimate (ETE)) is a structured communication technique or method, originally developed as a systematic, interactive forecasting method which relies on a panel of experts (Dalkey and Helmer, 1963). In the complete DELPHI method researchers want to connect experts and structure communication about the idea so that consensus can be achieved. A selection of experts is a critical element. They are chosen based on professionalism and not randomly. Likewise, a selection of presenters of different characters involved in different fields of protection drinking water resource was made in this incomplete DELPHI method to get their feedback on the evidence-based practices for water quality improvement of the different FAIRWAY MAPs. MAPs in different EU countries enabled the observation and analysis of the difference between them in the context of their legal system, geographical position and in the historical context of connecting new and old EU members. Last but not least, the opinion/feedback of combined MAPs data was also analysed and commented.

For statistic analayse an average of Likert scale, standard error and coefficient of variation was calculated for every statement presented in figures. The coefficient of variation (CV) ranged between 6 and 75 %, in most statements it ranged between 15 and 45 %. Data sample with CV of up to 25 % goes in first quartile (Q1), 25 to 35 % goes in second quartile (Q2) and 35 to 45 % goes in third quartile (Q3). This means that statements that received CV in Q1 contain least fragmented data sample and therefore they could show a sufficiently high certainty in comparison to other statements taken into account in the claims. Statements that received CV in Q2 have more fragmented data sample. Statements that received CV in Q3 contain most fragmented data sample and should be taken with great caution in interpretation.

2. Responses to the questions

In »Barriers and issues in providing integrated scientific support for EU policy, EU representatives were asked to define some major issues and barriers for solving matters related to drinking water resource protection against diffuse pollution of nitrates and pesticides from agriculture in the EU. The researchers were interested if representatives of different stakeholder groups could agree with the opinion of EU representatives, concerning their national and local level and to what extent. Their answers were present in three forms as:

  1. a structure of responses in the Likert scale;
  2. an average of Likert scale for all MAPs in the sample. The coefficient of variation (from now on CV) and standard error was calculated as well. The CV is essential because it gave a percentage of variation between statements within MAPs. A decision was that average of Likert scale that has CV in Q1 (< 25 %) in figures is coloured with a yellow point.
  3. an average of Likert scale calculated for five individual MAPs that contributed at least five complete questionnaires, i.e., MAPs coming from the United Kingdom, Slovenia, Portugal, Romania and Greece.

Their differences in average of Likert scale are presented. A CV and standard error for every type of answer was calculated and discussed if necessary. Here the decision was that statements that receive CV in Q2 or Q3 (> 25 %) are presented in figures with labels in stripes, because we wanted clearly to see in which countries on what topics there was statistically non-uniform opinion between stakeholders.

Type of stakeholders represented by the respondents

The majority of answers came from the advisory sector (23 %), farmers (20%), research and science (25%) and water policy implementation (18%) (Figure 2).

D72 fig02
Figure 2

Do you agree with the opinion of EU representatives of issues concerning the protection of drinking water resource?

Figure 3 presents the structure of responses of the MAP representatives on the opinion of EU representatives. The structure of responses shows that more respondents slightly agree to strongly agree with all the issues presented.

D72 fig03
Figure 3
D72 fig04
Figure 4
D72 fig05
Figure 5

Next, the average of Likert scale showing differences in agreement with the statement was made (Figure 4). It shows which statements are more agreeable among respondents and which are less. The Financial question “who is paying, where the money goes?” has the highest average in the Likert scale and could be the most crucial issue for all MAPs. However, this statement has CV > 25 %, what shows more fragmented data and less certainty than the yellow coloured statement “Fragmented data of water quality and not easily available” (CV < 25 %). The other statements have CV > 25% which means they have more fragmented data and therefore are less reliable in the result. A more extensive survey is needed to obtain more reliable results for all statements. Nevertheless, the average Likert scale is always higher than, which means that the stakeholders recognise these issues with a slight agreement in their local environment. However, stakeholders do not recognise specific issues between them to the same degree. 

The results also show substantial differences between the four MAPs (United Kingdom, Slovenia, Portugal and Romania) (Figure 5). It shows how different member states of the EU have different individual issues for the protection of drinking water resources and respond differently with proposed issues that are of high importance at the EU level. Figure 5 presents responses with CV higher than 25 % and with points on vertical stripes. For the United Kingdom, the statement with the lowest average on the Likert scale was (CV 54 %): No coherent Policy implementation of EU policies transition to the local level. In the Slovenian MAP, the lowest average on the Likert scale was for Fragmented data of water quality and not readily available (CV 75 %). In the MAP of Portugal two statements: No coherent Policy implementation of EU policies transition to the local level and there is a low balance between targets and objectives, both had CV 60 %. Romanian MAP did not agree with statements: No coherent Policy implementation of EU policies transition to the local level (CV 58 %) and More harmonisation of legislation at EU level (CV 51%). Nevertheless, they all slightly to strongly agree that patience is needed to see results (change policy takes time). Development is already positive. This statement was also the only one with more certainty (CV < 25 %) for all four MAPs.

Do you agree with these barriers in solving the issues of EU representatives within your national and local regulations?

The structure of responses (Figure 6) shows us more strong agreement than in the previous question. Most of the respondents decided that for barriers presented in the questionnaire, they could agree and strongly agree that they are also present in their local environment. 23 respondents decided that they strongly agree with the barrier There is a time lag between action (measures) and results (water quality) and 20 that Financial means to apply certain measures are needed. More than 70 % of respondents agree and strongly agree with these barriers.

D72 fig06
Figure 6
D72 fig07
Figure 7
D72 fig08
Figure 8

The responses to this question (Figure 7) are showing a high average of Likert scale for the barriers related to the protection of drinking water resources on the local level. Most of the recognised barriers (6 of 7) have an average of Likert scale 5.2 or more, which means that these barriers are highly recognisable within all MAPs in the sample. Three statements have CV less than 25 % (coloured with yellow). Only barrier Site-specific aspect: target concentration for pesticides and nitrates are not achievable in some regions has CV 44 %, which means there are problems with a unified opinion of the importance of this barrier for all MAPs. Other barriers have CV 21 to 32 %, which makes data quite unified and trustable. A reasonable conclusion can be made that in most cases, the barriers that were recognised among EU representatives are also moderately to highly recognisable among all MAPs.

Figure 8 shows the difference in response between the five different MAPs. In contradiction to the other MAPs only in Portuguese MAP (CV of 53 %) stakeholders do not have unified opinion if this barrier: Site-specific aspect as target concentration for pesticides and nitrates are not achievable in some regions is indeed barrier also on their local level. The average of Likert scale is highest among MAPs for the barriers:

  • There is a time lag between action (measures) and results (water quality),
  • Farmers are not enough involved; raising awareness and communication is needed and
  • Limited financial means to apply measures by farmers, water sector, consumers.

Strong (Likert scale of 6 to 7) and unified opinion (CV is 0 to 10 %) had MAPs as follows:

Greece for barriers:

  • Financial means to apply certain measures are needed.
  • Limited financial means to apply measures by farmers, water sector, consumers.

Romania for barriers:

  • There is time lag between action (measures) and results (water quality).
  • Farmers are not enough involved; raising awareness and communication is needed.

Slovenia for barrier:

  • There is time lag between action (measures) and results (water quality).

Moreover, United Kingdom for barrier:

  • Site specific aspect: target concentration for pesticides and nitrates are not achievable in some regions.

Do you agree with the opinions of EU representatives about the relationship between experts and policy within your national and local regulations reflected in your legislation?

The EU representatives were also asked to define how the relationship between experts and policy in the EU regulations reflects in EU legislation and how the system at EU level can be improved. The question was asked: What are the possible solutions for integrated scientific support for EU policy, with particular attention to drinking water resource protection against diffuse pollution of nitrates and pesticides from agriculture. In this bloc of questions, stakeholders of MAPs marked how much they agree with the opinions of EU representatives.

The results show that most respondents slightly agree to strongly agree in all statements concerning the actor's issues of science integration into policy on the local level (Figure 9). The statements where respondents most agree on are:

  • Science - policy relationship could be improved; we can see both populistic and economically driven decisions (34 respondents slightly to strongly agree) and
  • It is good that member states have voice in solving problems on local level; Multi Actor Platforms (MAP) are good way to closely engage stakeholders (30 respondents slightly to strongly agree).

D72 fig09
Figure 9
D72 fig10
Figure 10
D72 fig11
Figure 11

Next, the results show a moderate average of Likert scale with all statements concerning the actor's issues of science integration into policy on the local level (Figure 10). The average Likert scale was between 67 and 81 %, which means that these issues are also moderately recognised at the local level and not exclusively at EU. The CV was for all statements higher than 25 % which means that there is ununified opinion between MAPs on the matter. This could suggest that there is a local significance of MAPs that effects different opinions. 

In contradiction to the previous questions, there were small differences between the five MAPs for almost every statement (Figure 11). United Kingdom had the highest CV (30 %). In Slovenia statement Legislation shows that certain policymakers lack knowledge, more education and communication is needed, scored a CV of 0 %, which means a unified opinion. However, Slovene actors show highly fragmented data with the statement that there are Not enough experts that can tackle the complexity of the problem; in comparison to other sectors agriculture sector has week financial support (CV of 28 %). For the Portugal case, three statements had CV more than 25 %. These statements are: In legislation it is seen that in certain policymakers lack knowledge, more education and communication is needed (CV 38 %); science - policy relationship could be improved; populistic and economically driven decisions are observed (CV 38 %); there is not enough emphasis on real practical work and experiences; and agriculture sector represents a small share of GDP (CV 27 %). In the MAP in Romania, the CV was 29 % for statement In legislation it is seen that in certain policymakers lack knowledge, more education and communication is needed, and 41 % for Links between science and policy are weak.

Unified opinion (CV is less than 25%) among all five MAPs is presented in two issues of integration science into policy:

  • More education of the general public is needed and
  • It is good that member states have a voice in solving problems on the local level, Multi-Actor Platforms (MAP) are the right way to closely engage stakeholders.

The first one is very general, and the second one being particular and gives a good sign that the conceptual framework of FAIRWAY is recognised as a right solution at all MAPs in the sample and needed for integration of science into policy.

Do you agree with these solutions of integrated scientific support within your national/local policy?

MAPs were asked to present their opinion on solutions for better integration of science into policy. Most respondents decided that with most solutions (5 of 6) slightly agree to strongly agree (Figure 12). However, a solution to Separate Pesticides and Nitrates in projects and policy communications has quite equally distributed responses between slightly disagree to disagree (16 in total) and agree to strongly agree (24 in total). This result can mean that this solution is not a solution for all MAPs in the sample and that further research should be done to investigate reasons beyond.

D72 fig12
Figure 12
D72 fig13
Figure 13
D72 fig14
Figure 14

The results (Figure 13) show a moderate to high average of Likert scale (between 63 and 82%) in the opinions of different stakeholders in different MAPs with the proposed solutions. The average of Likert scale was noticeably lower for the statement "Separate Pesticides and Nitrates in projects and policy communications" (only 63 %). This statement had a CV of 45 %, and an average of Likert scale of 63 %, suggesting this solution does not work for all stakeholders and MAPs. Results suggest that if EU representatives seriously think about separating Pesticides and Nitrates in projects and policy communications, they should invite different MAPs to share their opinion on the matter and listen to them. There should be a possibility to rethink of separating Pesticides and Nitrates only on national level if the local environment supports this.

In Romania, the proposed solution to Separate Pesticides and Nitrates in projects and policy communication scored the lowest average (CV was 16 %; Figure 14). Other solution scored high average in the Romania case and had a high agreeability among stakeholders (CV between 14 and 19 %) for all solutions except one: Strengthen trust among concerned actors, inter-alliance, thought non-concerned databases on various level (easily accessible) (CV 27 %). Also, in Portugal case, the solution to separate pesticides and nitrates scored low, but the CV was relatively high (43 %). This result points on different opinions of the stakeholders (very fragmented data) in the Portugal MAP. These results are essential because a solution for a particular issue has the highest perspective if it has the support of a broad group of stakeholders. In the United Kingdom case, the CV was less than 25 % for only one statement: Stronger involvement of actors in the science-policy interface (CV 16 %). For all the other solutions proposed stakeholders have different views, having a CV between 27 and 43 %. In the Slovenia case, all presented solutions have a high average of Likert scale and quite unified opinion among stakeholders (CV between 6 and 23 %).

The solution "Stronger involvement of actors in the science-policy interface" has a unified opinion (CV < 25 %) among all MAPs. This result makes sense according to previously mentioned results were MAPs recognised that It is good that member states have a voice in solving problems on the local level; Multi-Actor Platforms (MAP) are the right way to closely engage stakeholders.

3. Conclusions from the FAIRWAY case studies

The findings from »Barriers and issues in providing integrated scientific support for EU policy were distributed among all project’s MAP leaders in the form of paper questionnaire to evaluate possible correlations between the EU and local level, on barriers and issues in providing integrated scientific support for policy regulations related to drinking water resource protection against diffuse pollution of nitrates and pesticides from agriculture.

Different stakeholders that were asked to fill out the questionnaires were not necessarily experts; however, they are involved daily in the process of making decisions of how to maintain drinking water resources protected. The survey sample thus included the critical stakeholders involved in different fields of protection drinking water resource. The sample size (46 questionnaires) was satisfactory, and we were able to conduct an incomplete DELFI method (explained in the introduction), which provided higher result importance.

An average of Likert scale to statements for each MAP was calculated and presented in Figures. Also, a coefficient of variation (CV) and standard error was calculated. Results showed that CV ranges from 0 to 70 %; results with a CV less than 25 % are considered as a high agreement among all MAPs or among all stakeholders in the MAP.

MAPs were asked to decide how much could they agree or disagree on the Likert scale of 1 to 7 for different issues that EU representatives recognised as necessary for the protection of drinking water resources. The average Likert scale for these issues was always higher than 60 %. Stakeholders recognise that these issues are also at least moderately important in their local environment. However, specific issues are considered as not essential or not evenly important between stakeholders (fragmented data). The issue: "Patience is needed to see results (change policy takes time). Development is already positive" is highly agreed by all presented MAPs (CV < 25%).

The results of the assessments show a higher average of Likert scale with the statements on barriers that are present in solving the issues concerning the protection of drinking water resources on the local level. The agreeability among MAPs was highest for the barriers:

  • "There is a time lag between action (measures) and results (water quality)",
  • "Farmers are not enough involved; raising awareness and communication is needed" and
  • "Limited financial means to apply measures by farmers, water sector, consumers".

These results imply that EU representatives should work on solving these barriers because they are uniquely recognised among different MAPs and different stakeholder groups.

In the third part, results show a high average of Likert scale with statements concerning the actor's issues of science integration into policy on the local level. Average was between 71 and 86 %, which means that this is recognised issues also at the local level and not exclusively at EU. CV ranged between 0 and 30 %, which gave in most cases satisfactory unified opinion on the matter. Integration of science into policy is quite a challenge in all MAPs in the sample.

Two issues of integration science into policy have higher agreeability among all MAPs:

  • "More education of the general public is needed" and
  • "It is good that member states have a voice in solving problems on the local level; Multi-Actor Platforms (MAP) are the right way to engage stakeholders closely".

The first one is very general. However, the second one is particular and gives a good sign that FAIRWAYs conceptual framework is recognised as the right solution at all MAPs in the sample and needed for integration of science into policy.

MAPs were also asked of their opinion on solutions for better integration of science into policy. There was a high average of Likert scale for all statements, except for the solution "Separate Pesticides and Nitrates in projects and policy communications". This statement had a CV of 45 %, and a relatively low average (63 %), suggesting that this solution is not a solution for all stakeholders and MAPs. Results indicate that if EU representatives seriously think about separating Pesticides and Nitrates in projects and policy communications, they should invite different MAPs to share their opinion on the matter and listen to them.

The agreeability among all MAPs was highest for the solution: "Stronger involvement of actors in the science-policy interface". Stronger involvement corresponds with a reflection on science integration into policy, where MAPs recognise that "It is good that member states have a voice in solving problems on a local level; Multi-Actor Platforms (MAP) are the right way to engage stakeholders closely".

 


Note: For full references to papers quoted in this article see

» References

Go To Top