Report on the Eurostat 2019 User Satisfaction Survey

EUROPEAN COMMISSION EUROSTAT

Directorate A: Resources Legal affairs; Document management

REPORT ON THE EUROSTAT 2019 USER SATISFACTION SURVEY

Commission européenne, 2920 Luxembourg, LUXEMBOURG - Tel. +352 4301-1 Office: BECH - Tel. direct line +352 4301-37123 http://ec.europa.eu/eurostat [email protected]

Index 1. Background – about the survey ...... 5 2. Main outcomes ...... 8 3. Results of the USS 2019 ...... 14 3.1 General information...... 14 3.1.1 Who uses Eurostat's European statistics? ...... 14 3.1.2 To do what? ...... 15 3.2 Information on quality aspects ...... 20 3.2.1 What are the perceived quality and user friendliness of Eurostat products? ...... 20 3.2.2 Overall quality ...... 22 3.2.3 Timeliness ...... 25 3.2.4 Completeness ...... 27 3.2.5 Comparability ...... 29 3.3 Trust in European statistics ...... 31 3.4 Information on dissemination aspects ...... 34 3.4.1 Access to and understanding of European statistics on Eurostat Website ...... 35 3.4.2 Release calendar ...... 43 3.4.3 Metadata and methodological information ...... 45 3.4.5 User support ...... 49 3.5 Overall quality of data and services ...... 50 3.6 Comparison with previous survey...... 51 4. Messages from the users ...... 52 Annex 1 - Statistical areas ...... 53 Annex 2 - Breakdown of respondents by country of work place...... 54 Annex 3 - Example of calculations for the question on overall quality ...... 55

Index of Charts Chart 1. Number of survey respondents, 2011 - 2019 ...... 5 Chart 2. Distribution of respondents by user group, in % ...... 6 Chart 3. Assessment of overall data quality in 2017 and 2019 ...... 9 Chart 4. Assessment of overall timeliness in 2017 and 2019, in % ...... 10 Chart 5. Assessment of overall completeness in 2017 and 2019, in % ...... 10 Chart 6. Assessment of overall comparability in 2017 and 2019 ...... 11 Chart 7. User types, in % ...... 14 Chart 8. Use of European statistics by statistical domains and user types, in % ...... 15 Chart 9. Uses of European statistics by user types, in % ...... 16 Chart 10. Importance of statistics by user types, in % ...... 16 Chart 11. Importance of statistics for different uses, in % ...... 17 Chart 12. Importance of statistics 2011-2019, in % ...... 18 Chart 13. Frequency of use by user types, in % ...... 18 Chart 14. Frequency of use by statistical area, in % ...... 19 Chart 15. Frequency of use 2011-2019, in % ...... 20 Chart 16. Assessment of quality of Eurostat products, in % ...... 21 Chart 17. Assessment of quality of products, 2011-2019, in % ...... 21 Chart 18. Assessment of friendliness of Eurostat products, in % ...... 22 Chart 19. Assessment of overall quality per statistical area, in % ...... 23 Chart 20. Overall data quality 2011-2019, in % ...... 24 Chart 21. Comparison with other statistical data producers by user types, in % ...... 24 Chart 22. Assessment of timeliness per statistical area, in % ...... 26 Chart 23. Assessment of overall timeliness in 2011-2019, in % ...... 27 Chart 24. Assessment of completeness of European statistics per statistical area, in % ...... 28 Chart 25. Assessment of overall completeness in 2011-2019, in % ...... 29 Chart 26. Assessment of comparability of European statistics per statistical area, in % ...... 30 Chart 27. Assessment of overall comparability in 2011-2019, in % ...... 31 Chart 28. Trust in European statistics by user types, in % ...... 32 Chart 29. Trust in European statistics in 2012-2019, in % ...... 33 Chart 30. Trust in European statistics by importance, in % ...... 33 Chart 31. Assessment of overall quality of European statistics by trust, in % ...... 34 Chart 32. User satisfaction with the Eurostat website, in % ...... 35 Chart 33. Assessment of the access to and understanding of European statistics, in % (Is it easy to access and to understand European statistics?) ...... 35 Chart 34. Assessment of the Eurostat website content by user types, in % ...... 36 Chart 35. Eurostat’s website content 2011-2019, in % ...... 36 Chart 36. Assessment of technical characteristics of the Eurostat website, in % ...... 37 Chart 37. Assessment of technical characteristics of Eurostat’s website 2011-2019, in % ...... 38 Chart 38. Assessment of Eurostat’s digital publications and visualisation tools, in % ...... 39 Chart 39. Users of Eurostat’s digital publications and visualisation tools, in % ...... 40 Chart 40. Users of Eurostat’s experimental statistics, in % ...... 41 Chart 41. Usefulness of Eurostat’s experimental statistics, in % ...... 42 Chart 42. Assessment of the information on microdata access services on the Eurostat website, in %43

Chart 43. Assessment of the information on microdata access services on the Eurostat website in 2016 and 2019, in % ...... 43 Chart 44. Awareness of the release calendar among user types, in % ...... 44 Chart 45. Awareness of release calendar 2011-2019, in % ...... 44 Chart 46. Assessment of sufficiency and relevance of information in the release calendar by user types, in % ...... 45 Chart 47. Sufficiency and relevance of information in the release calendar 2011-2019 ...... 45 Chart 48. Use of metadata by user types, in % ...... 46 Chart 49. Usage of metadata 2011-2019 ...... 46 Chart 50. Metadata accessibility, in % ...... 47 Chart 51. Easiness of access to metadata 2011-2019, in % ...... 47 Chart 52. Assessment of sufficiency of metadata for the different types of users, in % ...... 48 Chart 53. Sufficiency of metadata 2011-2019, in % ...... 48 Chart 54. Satisfaction with user support, in % ...... 49 Chart 55. Satisfaction with user support 2011-2019 ...... 49 Chart 56. Overall satisfaction with the quality of the data and services, in % ...... 50 Chart 57. Overall quality of data and services 2012-2019, in % ...... 50 Chart 58. Changes in perception of Eurostat's data and services quality, in % ...... 51

1. Background – about the survey

Eurostat’s mission is to provide high quality statistics on Europe. In order to measure the degree to which it meets its obligations towards its users, Eurostat carried out a general User Satisfaction Survey (USS) over the period of April – July 2019. It was based on the agreed model questionnaire for the European Statistical System and was designed to obtain a better knowledge about users, their needs and satisfaction with the services provided by Eurostat. The first survey of this kind was held in 2007 and then repeated in 2009, 2011, 2012, 2013, 2014, 2015, 2016 and 2017. The USS 2019 is, therefore, the 10th of a general nature. The present survey covered four main aspects:  information on types of users and uses of European statistics,  quality aspects,  trust in European statistics,  dissemination of statistics. The survey was carried out online, with a link on Eurostat website. It was launched on 29 April and was open until 9 July. To guarantee a high participation Eurostat used to send an invitation to all users registered on the Eurostat website. However, due to the entry into force of the new Regulation 2018/1725 on personal data protection in the EU institutions1, Eurostat had to clean the list of users that were registered on the Eurostat website and to revise the way they were contacted. The list of users of Eurostat website now includes only about 24 000 users which could be contacted instead of 172 000 in 2017 and this had a big influence on the number of responses. More channels to advertise the survey were also used (Facebook, Collaboration in Research and Methodology for Official Statistics (CROS) portal) but this could not compensate for the huge reduction of the number of users who could be contacted directly. Moreover, the invitation was sent to 600 researchers, who are using Eurostat’s microdata. In the end Eurostat received 1009 replies, compared to the 3000 – 4800 of the previous editions of the survey. Chart 1. Number of survey respondents, 2011 - 2019 4839 5000 4447 4558 4500 4247 4000 3279 3500 3101 3038 3000 2500 2000 1500 1009 1000 500 0 2011 2012 2013 2014 2015 2016 2017 2019

Source: Eurostat 2011, 2012, 2013, 2014, 2015, 2016, 2017 and 2019 user satisfaction surveys

1 https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32018R1725

The questionnaire was similar to the one used until 2016, allowing for a comparative analysis over time. The questionnaire used in 2017 was a shorter one, focusing only on the quality of the data and services provided by Eurostat. Changes were made in the sections where the situation has evolved since 2016, as in the questions concerning the dissemination products. However, on top of having far less replies, the distribution of the respondents among different categories of users was also different from the past, due to the changes in the procedures to inform them about the survey. The most notable change was a larger share of respondents working in the European Commission or in other European and international institutions. This makes the results somewhat less comparable. Chart 2. Distribution of respondents by user group, in % 60

50 4646 45 434444

40 38

30 252525 2423 24 24 23 21 19191919 20 18 1817 13

10 8 8 7 7 6 6 6 6 6 5 6 6 6 2 1 0 Students, academic and Business Government EU and international Others private users organisations

2011 2012 2013 2014 2015 2016 2017 2019

Source: Eurostat 2011, 2012, 2013, 2014, 2015, 2016, 2017 and 2019 user satisfaction surveys When analysing the results, users were grouped differently than in the past, so results cannot be compared by user groups but only on an overall level. Such change was implemented to follow the outcome of the Digital communication, User analytics and Innovative products (DIGICOM) project2. Users were classified as “light”, “intermediate” or “advanced”. A similar classification of users was tried in 2017, by distributing the traditional groups of users in the three categories, but the results were not conclusive. This time users were asked to put themselves in each category by using the following definitions:

2 The project aims to modernize the communication and dissemination of European statistics, by developing innovative products and services, based on new technological opportunities, experiences in the European Statistical System and the concrete needs of users. An in-depth analysis of European statistics users was conducted in DIGICOM, concluding that it is meaningful to group users based on two predefined criteria – frequency and complexity of use – resulting in a new proposed grouping of European statistics users.

 Light user: e.g. use data visualisations, graphs and statistical articles which are easy to read to get interpreted data; use data to support opinions in discussions, share data on social media, use data in class or want to explore what is available out of curiosity; visit the Eurostat website on a weekly to less than monthly basis; medium to low statistical literacy and computer proficiency.  Intermediate user: e.g. look for raw data / predefined tables or work with existing data visualisations and ready-to-use interpretations in publications/reports to support work, for personal interest (e.g. to verify data in news articles) or to get a basic understanding of what is available for future reference; use Eurostat data on a weekly to monthly basis; have a medium statistical literacy and computer proficiency.  Advanced user: e.g. use the database to mainly obtain raw data and adjust table and data to their needs; draw their own conclusions based on specific data for their job; download data very frequently (even daily); have a high statistical literacy and computer proficiency. This new system worked as differences could be consistently noted among the three classes of users. A separate specific survey was carried out for press and media users. However, some media users might have nonetheless responded to the general user satisfaction survey. They might have identified themselves as belonging to “other users”. The results presented in this report constitute a summary of the most interesting and compelling findings, supported by graphs. The report also shows the main differences compared to the previous survey (2016 or 2017 depending on the question) and an evolution of the users' opinion since 2011, date of the first comparable survey. Even if comparisons of the results have to be taken with caution, for the reasons explained above, the majority of the results show a stability in the opinion of the respondents with small variations in the degree of satisfaction.

2. Main outcomes

General aspects  In 2019 the survey was open on line for about two months getting 1009 replies, compared to the 3000 – 4800 of the previous editions of the survey.  Looking at the distribution of responses by user groups, a larger share of respondents was working in the European Commission or in other European and international institutions, 23.1% compared to 6-8% of the previous surveys. All other groups were less represented than in the past. Looking at user types, most of the respondents identified themselves as advanced users (44.2%), followed by intermediate users (35.6%) and light users (20.2%).  Like in the past, respondents indicated that “Population and social conditions” and “Economy and finance” were the two areas they used most frequently. The former received from 16.3% to 18.4% of responses whereas the latter ranged from 15.1% to 17.2% across all user types.  Differently from the past, “monitoring or formulating policy” (31.9%) and “general background information” (23.5%) were the most common purposes for all users combined. However, the purposes of statistical data use varied by types of respondents reflecting different needs of each type.  More than three quarters of participants (77.3%) indicated European statistics to be either “essential” or “important” for their work, the highest share ever registered. Accounting for a breakdown by purposes, statistical data was most significant for “Econometric model building and forecasting”, where it was indicated to be “essential” by 56.6% of respondents and “important” by 35.1%. "Preparing legislation", “Monitoring or formulating policy”, “Research” and "Re-dissemination of statistical data" also got combined shares of "essential" and "important" exceeding 80%.  Around one third of respondents (33.6%) stated they used European statistics in their daily or weekly activities, 32.8% did so on a monthly basis and the remaining 33.6% at other intervals.  User assessment of the quality and user friendliness of Eurostat’s products was generally positive, approaching or exceeding the 60% of "very good/good" judgements for most products, with Eurostat press releases (65.9%), Digital publications (65.5%), and the Eurostat database (63.5%) receiving the best scores.  Eurostat was interested to check if users continue to trust European statistics in a period when European citizens sometimes persist to be sceptic about the role and functioning of the EU institutions. As in previous years and even more this time, responses were overwhelmingly positive, with 96.0% of users stating they trusted European statistics greatly or tended to trust them. Only 3.1% said they did not trust statistics and 0.9% had no opinion.

 Trust seems related to the importance and the perceived quality of statistics. Those respondents, for which the statistics are of value, trust more the statistics than those for whom statistics are not so important. The respondents who trust more European statistics are also more convinced of their overall good quality. Quality aspects Overall quality  The level of satisfaction with the overall quality of European statistics remained high, with 58.2% of all users considering the quality to be “very good” or “good” (1.4 percentage points less than in 2017) and 19.9% considering it as “adequate”. Chart 3. Assessment of overall data quality in 2017 and 2019

2019 58.2 19.9 16.2 5.7 Very good/Good Adequate Poor/Very poor 2017 59.6 20.7 15.2 4.6 No opinion

0 10 20 30 40 50 60 70 80 90 100 Source: Eurostat 2017 and 2019 user satisfaction surveys  At a more disaggregated level, “Population and social conditions” received the highest positive evaluation this time (59.9% of “very good/good” answers). “Economy and finance” and “Policy indicators” were also close to 60%, with shares of 59.7% and 59.5%, respectively.  On the other side of the spectrum, "Science, technology and innovation", “Environment statistics” and “Industry, trade and services" were among the ones with lowest share of positive views on overall quality, with 53.0%, 53.2% and 54.2%, respectively. Nevertheless, the differences between all statistical domains (excluding “other statistics”) were the smallest registered since the survey started.  When analysed by user types, intermediate and advanced users were more satisfied (60.4% and 58.0% of “very good” and “good” answers) compared to light users (53.7%).  The quality of Eurostat’s data fares very well compared with other statistical data producers. The majority of participants perceived the quality as better or same, resulting in a combined share of 66.8%. Among other positive sides of Eurostat, users highlighted better quality and reliability of the data provided; more complete, more timely and harmonised data; better coverage and comparability; better metadata; friendly and easier to use interface; and the independence from national politics.

Timeliness  On average 52.5% of users saw timeliness of European statistics as “very good” or “good”, 24.1% as “adequate” and 18.4% as “poor” or “very poor”, shares very close to those of 2017. Chart 4. Assessment of overall timeliness in 2017 and 2019, in % 0 10 20 30 40 50 60 70 80 90 100

2019 52.5 24.1 18.4 5.1 Very good/Good Adequate Poor/Very poor No opinion 2017 52.4 24.4 16.7 6.6

Source: Eurostat 2017 and 2019 user satisfaction surveys  From a statistical domain perspective, “Economy and finance” was again rated as having the best timeliness across all areas, followed this time by “Policy indicators” and “Fishery statistics”, accounting for 57.4%, 56.1% and 55.6% of “very good/good” responses, respectively.  Looking at the user types, as for all quality dimensions intermediate and advanced users are quite more satisfied than light users. The share of “very good” and “good” responses from intermediate users was 54.8%, from advanced users a similar 54.1% and from light users only 42.6 %. Completeness  On average for all areas, 50.1% of users saw data completeness as “very good” or “good”, 24.1% thought it was “adequate” and 18.9% perceived it as “poor” or “very poor”, values that are again close to those of 2017. Chart 5. Assessment of overall completeness in 2017 and 2019, in % 0 10 20 30 40 50 60 70 80 90 100

Very good/Good 2019 50.1 24.1 18.9 6.9 Adequate Poor/Very poor No opinion 2017 51.0 25.4 17.0 6.5

Source: Eurostat 2017 and 2019 user satisfaction surveys  “Economy and finance” once again stood out as the best rated domain, followed by “Policy indicators” and "Population and social conditions" (54.5%, 54.1% and 51.3% of “very good/good” replies, respectively). The least performing area is this time “Transport statistics” with 39.8% of “very good” or “good”.

 From the user type perspective, there is again a notable difference when comparing advanced and intermediate users with light users (51.5%, 51.0% and 44.1% of “very good/good” ratings, respectively). Comparability  Comparability was the single quality dimension with the best score this time. The average of “very good/good” responses across all areas was 53.1%, 18.9% saw comparability as “adequate” and 15.7% did not feel positive about it. Comparability was the quality dimension that saw the most notable variation compared to 2017, with an increase of 4.2 percentage points in the shares of “very good” and “good” responses. That makes the satisfaction share for 2019 the highest ever registered Chart 6. Assessment of overall comparability in 2017 and 2019 0 10 20 30 40 50 60 70 80 90 100

Very good/Good 2019 53.1 18.9 15.7 12.3 Adequate Poor/Very poor No opinion 2017 48.9 24.0 15.8 11.4

Source: Eurostat 2017 and 2019 user satisfaction surveys

 “Economy and finance”, “Fishery statistics” and "Policy indicators" were the three domains with more than 55% of the respondents being satisfied, getting shares of 55.8%, 55.6% and 55.2% of “very good” and “good”, respectively. For this quality dimension, the differences among the domains were smaller than for other dimensions, "Regional statistics" having still 47.2% of satisfied respondents.  Once more intermediate and advanced users were more positive than light users. 54.1% and 54.0% of respondents from the first two types judged comparability as “very good” or “good”, versus 48.4% of those identifying themselves as light users. Dissemination aspects  The overall satisfaction with the Eurostat website, not counting the respondents not giving an opinion, is 58.1% with another 34.9% being partly satisfied. The values are similar to those of 2016 but with 2.2 percentage points less of satisfied users. The share of those not giving an opinion is now of 5.5%.  On the Eurostat website, the respondents identifying themselves as advanced users are the least satisfied (57.0%) even if the shares are almost equal for the three groups, with 58.5% for the light users and 59.2% for the intermediate users.  More than half of the respondents (53.0%) found it easy to access and to understand the statistics on the Eurostat website and more than another third (35.4%) partly easy. 8.0% were not satisfied while the remaining 3.6% did not express an opinion. The results are practically identical to the ones of 2016.

 As in previous years, respondents were very positive about the content of Eurostat website. On average 67.7% of all respondents were satisfied with the content, which is 2.9 percentage points more than in 2016 and almost the highest value ever registered.  Intermediate and advanced users were more satisfied that light users when accessing and understanding the statistics and advanced users were quite more satisfied with the content of the website (71.5%) than those from the other two types (65.2% for the intermediate users and 63.7% for the light users).  Respondents were less positive on the website’s technical characteristics; results are slightly worse than in 2016 and uneven. Some technical characteristics, like clarity of information and performance and speed, get more than half of "very good/good" judgements (53.3% and 53.0% respectively) while for others like the help texts and facilities or the search facilities, the share of satisfied users does not reach 40% and even without taking into account those not giving an opinion, it would not reach 50%.  User assessment of Eurostat's digital publications and visualisation tools was rather positive, with well more than 60% of respondents, who expressed an opinion, judging the tools as “very good/good”, up to the 71.1% of the digital publication on “Air traffic in the European Union”. 30.1% to 58.9% of the survey respondents used the different publications and tools but the percentage of those who actually gave their opinion was about 9 - 12% points smaller.  On an overall level, once again respondents from advanced users giving an opinion are more satisfied with the publications and tools (71.6%), compared to intermediate users (65.2%) and light users (63.0%).  A new question was added this year, on the usefulness of Eurostat’s experimental statistics, a domain that Eurostat has recently started to develop and that did not exist in 2016. The shares of those who gave an opinion was around 10% - 20% of the respondents, from 11.6% (117 respondents) for “World heritage sites statistics” to 21.8% (220 respondents) for “Statistics on the joint distribution of income, consumption and wealth” and “Income and consumption: comparing social surveys and national accounts”. The majority of those who gave an opinion found experimental statistics useful. The shares of “very good/good” answers went from 55.8% for the Quality Adjusted Labour Input (QALI) to 66.4% for the Labour market transitions statistics.  On an overall level, also for this question respondents from advanced users giving an opinion are more satisfied (65.1% of “very good/good”), compared to intermediate users (56.9%) and light users (63.2%).  Users were asked for the second time this year to rate the information on microdata access services on the Eurostat website. Almost half of the respondents (47.0%) gave an opinion, so showing that they use the microdata. The share was higher, as it could be expected, for the respondents from advanced users (52.7%) than for intermediate users (44.8%) and light users (38.2%). However, advanced users seem to be also more

exigent for this kind of services, as their satisfaction (54.9% of “very good/good”) was in this case lower than for intermediate users (60.9%) and light users (59.0%). Compared to 2016, the share of those considering the information on microdata access services as least good registered an increase of 2.8 percentage points. In their comments respondents seemed to consider not only the information on microdata access but also the access procedure and the set of microdata available.  Users’ awareness of Eurostat’s release calendar, which provides information on the dates and times of Euro indicators’ releases and other news releases and publications, increased by 3.3 percentage points compared to 2016 to reach 34.6% of the respondents. Among user types, advanced and intermediate users, with respective shares of 38.8% and 36.2% were much more aware than light users (22.5%), which could be expected.  A large part of the users who are aware of the release calendar, are satisfied with its content (67.3%), and another 18.6% said the calendar satisfied their needs partly.  Metadata was used by more than half of the respondents (57.6%), well more than in 2016 (48.5%), with a big difference among the three types of users. Only 30.4% of light users declared to use metadata, versus 56.0% of intermediate users and 71.3% of advanced users.  The share of metadata users who find it easily accessible went slightly down to less than a half (48.4%) this year and 45.2% found metadata sufficient for their purposes. This is 8.9 percentage points less than in 2016 and the largest registered decrease of all questions. Another 40.8% of users found metadata partly sufficient and 7.2% stated that metadata was not sufficient.  Looking at user types, intermediate users were surprisingly a bit more satisfied with the metadata accessibility and sufficiency (51.2% and 45.7%) than advanced users (48.4% and 45.6%), while light users were the least happy (38.7% and 41.7%).  Leaving out those with no opinion or not aware of the user support function, the degree of satisfaction with it remains the highest of all services, with 75.0% of the respondents saying that they were either “very satisfied” or “satisfied” with the support service provided by Eurostat, the highest value ever reached. All types of users were very satisfied, advanced users (76.5%) a bit more than light users (74.0%) and intermediate users (73.5%).  The level of overall satisfaction with Eurostat’s data and services was very high with 70.1% of all respondents evaluating data and services as “very good” or “good”, 19.5% as “adequate” and only 3.3% as “poor” or “very poor”. The small difference with the highest value ever reached in 2017 is only due to a larger number of respondents not giving an opinion. Advanced users were again more satisfied (72.9%) than intermediate (68.5%) and light users (66.7%).

3. Results of the USS 2019

3.1 General information

3.1.1 Who uses Eurostat's European statistics? Looking at the distribution of responses by user types (Chart 7), a bit less than half of the respondents identified themselves as advanced users, a bit more than one third as intermediate users and only a fifth as light users. Chart 7. User types, in % 50 44.2 45

40 35.6 35

25 20.2 20

0 Light users Intermediate users Advanced users

Source: Eurostat 2019 user satisfaction survey As in previous years, geographical distribution of European statistics’ users remained strongly tilted towards the EU countries with 87.0% of respondents coming from the 28 Member States and remaining 13.0% from non-EU countries. On a country level, the biggest proportion came from Belgium (17.7%), which was followed by Italy (9.7%), Germany (8.1%) and Spain (7.1%). It is worth noting that the high percentage of users coming from Belgium can be explained by their relationship to the European institutions based in Brussels. Participants were also asked to specify which statistics they used most frequently and given an option to pick more than one answer. As seen from Chart 8, “Population and social conditions” and “Economy and finance” remained the two dominating areas across all user types. The former domain received from 16.3% to 18.4% of responses whereas the latter ranged from 15.1% to 17.2% across user types. The least utilised statistics were “Digital economy and society”, “Transport statistics” and “Fishery statistics”, with approximate average shares below 4%. When compared to the results of previous years, the order remained roughly the same.

Chart 8. Use of European statistics by statistical domains and user types, in %

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

All users 17.1 15.7 10.3 9.3 8.5 6.8 5.9 5.6 5.1 5.0 3.9 3.81.41.3

Light users 16.3 17.2 11.5 10.2 9.4 4.6 4.1 5.7 5.9 4.3 4.3 3.91.11.7

Intermediate users 16.0 15.1 10.0 8.8 8.9 7.5 7.0 6.1 4.5 5.0 3.9 3.91.51.8

Advanced users 18.4 15.6 10.2 9.5 7.8 7.1 5.8 5.2 5.4 5.3 3.8 3.81.40.8

Population and social conditions Economy and finance

Policy indicators Industry, trade and services International trade statistics Regional statistics

Environment statistics Energy statistics

Science, technology and innovation Agriculture statistics

Digital economy and society Transport statistics

Fishery statistics Other

Source: Eurostat 2019 user satisfaction survey

3.1.2 To do what? The users of European statistics were also asked to indicate the purpose of their interest in it. Multiple responses were available. As shown in Chart 9, “monitoring or formulating policy” (31.9%) and “general background information” (23.5%) were the most common purposes for all users combined. A bigger share than in the past of institutional users can justify the high share of use for monitoring or formulating policy. However, a closer look at the purposes reveals a different nature of statistical data use by types of respondents. For advanced users the second purpose is “econometric model building and forecasting” (22.0%) and “research” is also more mentioned (14.8%) while “general background information” less (16.4%) than for the other types of users. On the contrary, for light users “general background information” is the most common use (35.3%), which is normal.

Chart 9. Uses of European statistics by user types, in %

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

All users 31.9 23.5 14.7 12.3 4.5 3.8 3.81.71.41.41.2

Advanced 34.5 16.4 22.0 14.8 2.73.62.51.61.10.20.7 users

Intermediate 33.1 25.6 10.9 12.0 5.0 4.2 3.91.11.11.71.4 users

Light users 24.0 35.3 5.4 7.4 7.4 3.4 6.4 2.92.53.42.0

Monitoring or formulating policy General background information Econometric model building and forecasting Research Market analysis Re-dissemination of statistical data Other Negotiations Preparing legislation Decision-making in business Media use

Source: Eurostat 2019 user satisfaction survey 3.1.2.1 How important are the statistics? Looking at the importance of European statistics, more than three quarters of participants (77.3%) indicated them to be either “essential” or “important” for their work (Chart 10). As it could be expected, statistics are more important for advanced users, 91% of their respondents declaring them as essential or important, compared to the intermediate users (76.8%) and to the light users (47.5%). Chart 10. Importance of statistics by user types, in %

0% 20% 40% 60% 80% 100%

All users 38.7 38.6 22.8

Essential Advanced 54.5 36.5 9.0 users

Important Intermediate 33.1 43.7 23.1 users

less important Light users 13.7 33.8 52.5

Source: Eurostat 2019 user satisfaction survey Accounting for a breakdown by purposes, statistical data was most significant for “Econometric model building and forecasting”, where it was indicated to be “essential” by

56.6% of respondents and “important” by 35.1%. "Preparing legislation", “Monitoring or formulating policy”, “Research” and "Re-dissemination of statistical data" also got combined shares of "essential" and "important" exceeding 80%. European statistics were considered least essential for “market analysis” and “general background information” (34.6% and 33.4% share of responses, respectively). Chart 11. Importance of statistics for different uses, in % 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

All purposes 43.1 37.0 19.9

Econometric model building and forecasting 56.6 35.1 8.4

Preparing legislation 52.6 34.6 12.8

Monitoring or formulating policy 52.3 35.0 12.7

Research 44.9 37.7 17.4

Essential Re-dissemination of statistical data 44.1 41.7 14.2 Important Less important Decision-making in business 39.7 34.9 25.4

Negotiations 36.9 33.3 29.8

Media use 35.4 38.5 26.2

Market analysis 34.6 41.7 23.7

General background information 33.6 35.0 31.4

Other 31.7 38.1 30.2

Source: Eurostat 2019 user satisfaction survey Chart 12 below shows the importance of statistics over time, throughout the period between 2011 and 2019. The importance of statistics has increased this year, with more than three quarters of participants (77.3%) reporting them to be either “essential” or “important” for their work, reaching its highest value ever.

Chart 12. Importance of statistics 2011-2019, in % 100%

90%

80% 77.3% 69.5% 67.3% 67.5% 70% 62.9% 66.4% 66.5%

60%

50%

40%

30%

20%

10%

0% 2011 2012 2013 2014 2015 2016 2019

Source: Eurostat 2011, 2012, 2013, 2014, 2015, 2016 and 2019 user satisfaction surveys 3.1.2.2 How often are European statistics used?

Knowing the purpose of use and importance of statistical information, it is interesting to see how frequently statistics were used. As Chart 13 shows, almost one third of users (33.6%) stated they used European statistics in their daily or weekly activities, 32.8% did so on a monthly basis and the remaining 33.6% at other intervals. When compared to the results of the survey carried out for media users, statistical information was used more frequently by press and media representatives, with a percentage of daily and weekly usage exceeding 70%. Advanced users are the most frequent users of European statistics with 48.6% using them daily or weekly. Chart 13. Frequency of use by user types, in % 0% 20% 40% 60% 80% 100%

All users 9.6 24.0 32.8 33.6

Advanced Daily 17.0 31.6 25.3 26.0 users Weekly Monthly Intermediate At other intervals 5.0 21.7 42.3 30.9 users

Light users 1.5 11.3 32.4 54.9

Source: Eurostat 2019 user satisfaction survey The frequency also differed by statistical domains (Chart 14). Highest daily use was found in the areas of “Policy indicators” (16.8%), “Economy and finance” (16.2%) and “Regional statistics” (15.6%). On the opposite, least frequently utilised domains contained “Fishery

statistics”, “Agriculture statistics” and “Environment statistics”. The differences, however, were rather small. Chart 14. Frequency of use by statistical area, in % 0% 20% 40% 60% 80% 100%

Average for all areas 14.3 27.3 30.1 16.5 5.7 6.1

Policy indicators 16.8 27.1 30.7 15.9 3.6 5.9

Economy and finance 16.2 27.7 29.6 15.1 5.8 5.6

Regional statistics 15.6 26.1 26.1 20.2 6.9 5.0

Population and social conditions 15.4 28.9 28.4 16.1 6.2 5.0

Digital economy and society 15.2 24.8 36.0 14.4 2.4 7.2

Daily Industry, trade and services 12.5 25.9 33.3 18.6 2.9 6.8

Weekly International trade statistics 11.8 26.1 30.9 18.4 6.3 6.6 Monthly Transport statistics 11.4 26.8 31.7 18.7 4.9 6.5 Quarterly Science, technology and 10.4 25.6 26.2 20.1 8.5 9.1 innovation Annually

Energy statistics 10.0 26.1 32.2 17.2 6.7 7.8 Other intervals Other 9.5 23.8 28.6 11.9 14.3 11.9

Environment statistics 9.5 24.7 33.2 18.4 6.8 7.4

Agriculture statistics 9.4 30.0 31.9 13.8 7.5 7.5

Fishery statistics 8.9 26.7 33.3 6.7 11.1 13.3

Source: Eurostat 2019 user satisfaction survey Chart 15 illustrates the trend of the frequency of use between 2011 and 2019. More specifically, it shows the percentage of respondents who use Eurostat's statistics on daily, weekly or monthly basis. Overall, the use of the statistics has increased (66.4%), reaching almost the peak of 2012 and 2013 when two thirds of respondents (66.6% - 66.9%) used statistics at least on a monthly basis.

Chart 15. Frequency of use 2011-2019, in % 100%

90%

80%

70% 65.0% 66.6% 66.9% 66.4% 63.9% 62.9% 62.9% 60%

50%

40%

30%

20%

10%

0% 2011 2012 2013 2014 2015 2016 2019

Source: Eurostat 2011, 2012, 2013, 2014, 2015, 2016 and 2019 user satisfaction surveys

3.2 Information on quality aspects

In accordance with Eurostat’s mission statement, quality considerations play a central role in both its corporate management and day-to-day statistical operations. It is thus important to find out how users assess the quality of the European statistics produced and disseminated by Eurostat and of the products released by Eurostat. In addition to the overall data quality, the survey looked at three different aspects of data quality that are considered as the most important for Eurostat - timeliness, completeness and comparability.

3.2.1 What are the perceived quality and user friendliness of Eurostat products? Respondents were asked to assess the quality and friendliness of Eurostat products. As for quality, the highest evaluations were received by Eurostat press releases (65.9%) and Digital publications (65.5%), followed this year by the Eurostat database (63.5%). For all the other tools the rate of "very good/good" replies were also at around 55% or above.

Chart 16. Assessment of quality of Eurostat products, in % 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Other Eurostat publications 67.0 18.6 14.5

Eurostat press releases 65.9 15.5 18.7

Digital publications 65.5 16.0 18.6

Eurostat database 63.5 16.3 20.2 Very good/Good "What's New" articles 63.1 19.0 17.9 Adequate

Statistics Explained 62.2 16.5 21.4 Very poor/Poor

Eurostat micro data 58.4 21.9 19.7

Eurostat visualisation tools 56.4 23.4 20.2

Eurostat social media channels 55.2 23.5 21.3

Other products 53.6 33.0 13.4

Source: Eurostat 2019 user satisfaction survey Looking at the evolution over time of the assessment of the quality of products in Chart 17, a substantial stability can be observed with small variations each year for the two products released during the overall period of observation. Chart 17. Assessment of quality of products, 2011-2019, in % 100.0% 90.0% 80.0% 70.0% 60.0% 50.0% Statistics Explained 40.0% Press releases 30.0% 20.0% 10.0% 0.0% 2011 2012 2013 2014 2015 2016 2019

Source: Eurostat 2011, 2012, 2013, 2014, 2015, 2016 and 2019 user satisfaction surveys The perceived quality of Press releases in 2019 for the respondents in the USS 2019 has increased by 6.1 percentage points since 2016, up to 65.9%, which is now in line with the opinion reported in the survey for media users (68.1%), whereas in the past this difference was much wider.

As for the user friendliness of products, the highest evaluations were received by Digital publications (63.2%), followed by Eurostat press releases (63.1%) and “What’s New” articles (61.4%). For all the other products the rate of "very good/good" replies were also above 50%. Chart 18. Assessment of friendliness of Eurostat products, in % 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Digital publications 63.2 18.8 18.0

Eurostat press releases 63.1 18.7 18.1

"What's New" articles 61.4 19.5 19.2

Other Eurostat publications 59.7 24.6 15.7 Very good/Good Statistics Explained 58.7 19.7 21.6 Adequate

Eurostat social media channels 57.9 22.0 20.1 Very poor/Poor

Eurostat visualisation tools 56.3 23.4 20.3

Other products 53.8 31.1 15.1

Eurostat database 51.1 25.4 23.4

Eurostat micro data 51.0 26.6 22.3

Source: Eurostat 2019 user satisfaction survey

Respondents could also give comments on the quality of the products. Only few did so. A couple pointed to the difficulty to find the the data used for the statistics explained. It was also suggested to publish more info on social media.

3.2.2 Overall data quality As in the past, this year evaluations were generally positive with close to six out of 10 respondents viewing the overall quality of statistics as “very good” or “good”. As can be seen from Chart 19, the level of satisfaction with the overall quality of European statistics remained high, with 58.2% of all respondents considering the quality to be “very good” or “good” and 19.9% considering it as “adequate".

Chart 19. Assessment of overall quality per statistical area, in %

0% 20% 40% 60% 80% 100%

Average for all areas 58.2 19.9 16.2 5.7

Other 61.9 11.9 7.1 19.0

Population and social conditions 59.9 19.8 15.0 5.3

Economy and finances 59.7 16.0 17.9 6.3

Policy indicators 59.5 19.8 14.6 6.1

Digital economy and society 58.4 24.0 12.0 5.6 Very good/Good Regional statistics 57.8 21.6 16.1 4.6 Adequate

Fishery statistics 57.8 17.8 13.3 11.1 Poor/Very poor No opinion International trade statistics 57.4 21.3 18.4 2.9

Agriculture statistics 56.3 20.6 20.0 3.1

Transport statistics 56.1 26.0 12.2 5.7

Energy statistics 55.6 22.8 16.1 5.6

Industry, trade and services 54.2 19.7 18.4 7.7

Environment statistics 53.2 24.2 18.9 3.7

Science and technology and 53.0 27.4 14.0 5.5 innovation

Source: Eurostat 2019 user satisfaction survey At a more disaggregated level, “Population and social conditions” received the highest positive evaluation (59.9% of “very good/good” answers). “Economy and finance” and “Policy indicators” were also close to 60%, with shares of 59.7% and 59.5%, respectively. “Population and social conditions” was for once the highest rated area by overtaking “Economy and finance”, which continues to get the best scores for the other quality dimensions. Given the interest in economic, financial and social developments in Europe during the recent years and the fact that these domains are used most frequently, high evaluations represent positive views of European statistics users. A more detailed analysis of the two domains revealed that “Government finance statistics”, “Living conditions and social protection” and “Population” came to the top of the list receiving 65.9%, 62.8% and 62.0%, respectively, of “very good/good” assessments. On the other side of the spectrum, "Science, technology and innovation", “Environment statistics” and “Industry, trade and services" were among the ones with lowest share of positive views on overall quality, with 53.0%, 53.2% and 54.2%, respectively. Nevertheless,

the differences between all statistical domains (excluding “other statistics”) were the smallest registered since the survey started. When analysed by user types, intermediate and advanced users were more satisfied (60.4% and 58.0% of “very good” and “good” answers) compared to light users (53.7%). Compared to 2017, the share of those considering the overall quality at least good registered a small decrease of 1.4 percentage points. Chart 20 shows that there has not been a lot of difference with the overall data assessment in the period from 2011 to 2019, with similar values for all years. Chart 20. Overall data quality 2011-2019, in % 100 90 80 70 57.3 58 57.6 59.6 59.2 59.6 58.2 60 56.6 50 40 30 20 10 0 2011 2012 2013 2014 2015 2016 2017 2019

Source: Eurostat 2011, 2012, 2013, 2014, 2015, 2016, 2017 and 2019 user satisfaction surveys Given that there are several producers of European statistics, respondents were also asked to compare the quality of Eurostat’s data with that of national statistical institutes (NSIs) and other international organisations. The results are presented in Chart 21. Chart 21. Comparison with other statistical data producers by user types, in %

40 All users

35 Light users 30 Intermediate 25 users Advanced 20 users 15

0 Better Same Worse No opinion Source: Eurostat 2019 user satisfaction survey

As can be seen, the majority of participants consider the quality to be better or the same, resulting in a combined share of 66.8%. Among other positive sides of Eurostat, users highlighted better quality and reliability of the data provided, more complete, more timely and harmonised data, better coverage and comparability, better metadata, friendly and easier to use interface, and the independence from national politics. Only few respondents (6.1%) considered Eurostat’s data of a worse quality when compared to other sources. Respondents mentioned shorter time series, data changes not signalled, limited coverage of non-EU sources, data timeliness, missing data and a worse website as major drawbacks due to which they may prefer other data sources. It is interesting to note that on topics like data timeliness and the quality of the website, users may have contradictory opinions. It is also worth noting that more than a quarter (27.1%) of the respondents did not have an opinion on the issue, suggesting that a relatively large share of Eurostat statistics' users either do not use other data sources or find it hard to formulate such comparisons.

3.2.3 Timeliness The aspect of information timeliness reflects the length of time between its availability and the event or phenomenon it describes. According to the results, which are presented in Chart 22, on average 52.5% of users saw timeliness of European statistics as “very good” or “good”, 24.1% as “adequate” and 18.4% as “poor” or “very poor. Timeliness this time is not the quality dimension, of the three investigated, with the best performance as in the past, because respondents judged comparability slightly better. From a statistical domain perspective, “Economy and finance” was again rated as having the best timeliness across all areas, followed this time by “Policy indicators” and “Fishery statistics”, accounting for 57.4%, 56.1% and 55.6% of “very good/good” responses, respectively.

Chart 22. Assessment of timeliness per statistical area, in % 0 20 40 60 80 100

Average for all areas 52.5 24.1 18.4 5.1

Economy and finances 57.4 20.5 17.1 5.0

Policy indicators 56.1 24.3 13.9 5.7

Fishery statistics 55.6 26.7 8.9 8.9

International trade statistics 54.0 23.2 18.4 4.4

Digital economy and society 53.6 25.6 16.0 4.8

Agriculture statistics 53.1 26.3 18.8 1.9 Very good/Good Adequate Population and social… 51.7 23.0 20.5 4.8 Poor/Very poor 50.6 25.0 Science and technology and… 20.1 4.3 No opinion

Regional statistics 50.5 26.6 19.7 3.2

Environment statistics 48.9 26.3 21.1 3.7

Energy statistics 47.2 27.8 18.9 6.1

Transport statistics 45.5 33.3 16.3 4.9

Other 45.2 19.0 9.5 26.2

Industry, trade and services 44.9 28.1 20.9 6.1

Source: Eurostat 2019 user satisfaction survey Looking at the user types, as for all quality dimensions intermediate and advanced users are quite more satisfied than light users. The share of “very good” and “good” responses from intermediate users was 54.8%, from advanced users a similar 54.1% and from light users only 42.6%. The assessment of the overall timeliness this year is almost the same as in the previous survey in 2017. In fact, as Chart 23 demonstrates, there have been very limited variations during the entire period of observations.

Chart 23. Assessment of overall timeliness in 2011-2019, in % 100

60 53.2 53.7 53.2 51.3 50.9 51.4 52.4 52.5 50

0 2011 2012 2013 2014 2015 2016 2017 2019

Source: Eurostat 2011, 2012, 2013, 2014, 2015, 2016, 2017 and 2019 user satisfaction surveys

3.2.4 Completeness Completeness is the extent to which all statistics that are needed are available. It is usually described as a measure of the amount of available data from a statistical system compared to the amount that was expected to be obtained. Chart 24 presents the results of user views on data completeness in 2019. On average for all areas, 50.1% of users saw data completeness as “very good” or “good”, 24.1% thought it was “adequate” and 18.9% perceived it as “poor” or “very poor”. “Economy and finance” once again stood out as the best rated domain, followed by “Policy indicators” and "Population and social conditions" (54.5%, 54.1% and 51.3% of “very good/good” replies, respectively). The least performing area is this time “Transport statistics” with 39.8% of “very good” or “good”.

Chart 24. Assessment of completeness of European statistics per statistical area, in %

0 20 40 60 80 100

Average for all areas 50.1 24.1 18.9 6.9

Economy and finances 54.5 20.2 18.3 7.0

Policy indicators 54.1 20.9 18.0 7.0

Population and social… 51.3 25.3 17.0 6.4

International trade statistics 50.7 24.6 18.0 6.6

Fishery statistics 48.9 22.2 15.6 13.3

Agriculture statistics 47.5 25.6 21.3 5.6 Very good/Good Adequate Science and technology and… 47.0 24.4 22.6 6.1 Poor/Very poor Environment statistics 45.8 24.7 23.7 5.8 No opinion

Industry, trade and services 45.6 25.9 20.9 7.7

Digital economy and society 44.8 31.2 17.6 6.4

Regional statistics 44.5 28.9 21.6 5.0

Energy statistics 43.9 23.3 21.7 11.1

Transport statistics 39.8 30.9 23.6 5.7

Other 38.1 28.6 16.7 16.7

Source: Eurostat 2019 user satisfaction survey From the user type perspective, there is again a notable difference when comparing advanced and intermediate users with light users (51.5%, 51.0% and 44.1% of “very good/good” ratings, respectively). As Chart 25 shows, compared to 2017 there was a very small decrease (0.9 percentage points) in the “very good” and “good” assessments of data completeness this year, which makes completeness the quality dimension with the lowest overall score. Again, as can be seen in Chart 25, the differences in the user satisfaction with this indicator in the last nine years were quite small.

Chart 25. Assessment of overall completeness in 2011-2019, in % 100

60 52.2 51.4 49.6 49.9 50.9 49.3 51 50.1 50

0 2011 2012 2013 2014 2015 2016 2017 2019

Source: Eurostat 2011, 2012, 2013, 2014, 2015, 2016, 2017 and 2019 user satisfaction surveys

3.2.5 Comparability Comparability is the extent to which differences between statistics from different geographical areas, non-geographic domains or over time can be attributed to differences between the true values of statistics. As seen from Chart 26, comparability was the single quality dimension with the best score this time. The average of “very good/good” responses across all areas was 53.1%, 18.9% saw comparability as “adequate” and 15.7% did not feel positive about it. In this case “Economy and finance”, “Fishery statistics” and "Policy indicators" were the three domains with more than 55% of the respondents being satisfied, getting shares of 55.8%, 55.6% and 55.2% of “very good” and “good”, respectively. For this quality dimension, the differences among the domains were smaller than for other dimensions, "Regional statistics" having still 47.2% of satisfied respondents. For comparability, once more intermediate and advanced users were more positive than light users. 54.1% and 54.0% of respondents from the first two types judged comparability as “very good” or “good”, versus 48.4% of those identifying themselves as light users.

Chart 26. Assessment of comparability of European statistics per statistical area, in % 0 20 40 60 80 100

Average for all areas 53.1 18.9 15.7 12.3

Economy and finances 55.8 15.2 17.1 11.9

Fishery statistics 55.6 20.0 6.7 17.8

Policy indicators 55.2 17.5 14.6 12.7

Population and social… 54.1 19.6 15.1 11.3

International trade statistics 52.2 18.4 17.3 12.1

Environment statistics 52.1 18.9 16.8 12.1 Very good/Good Adequate Energy statistics 51.7 18.3 13.3 16.7 Poor/Very poor 51.2 19.5 Industry, trade and services 16.6 12.7 No opinion

Digital economy and society 51.2 22.4 12.8 13.6

Agriculture statistics 50.6 24.4 15.0 10.0

Transport statistics 48.0 24.4 13.0 14.6

Science and technology and… 47.6 22.0 17.1 13.4

Regional statistics 47.2 22.9 18.3 11.5

Other 42.9 21.4 9.5 26.2

Source: Eurostat 2019 user satisfaction survey Comparability was the quality dimension that saw the most notable variation compared to 2017, with an increase of 4.2 percentage points in the shares of “very good” and “good” responses. That makes the satisfaction share for 2019 the highest ever registered.

Chart 27. Assessment of overall comparability in 2011-2019, in % 100 90 80 70

60 53.1 50.3 50.5 48.1 47.5 48.9 49.5 48.9 50 40 30 20 10 0 2011 2012 2013 2014 2015 2016 2017 2019

Source: Eurostat 2011, 2012, 2013, 2014, 2015, 2016, 2017 and 2019 user satisfaction surveys

Some respondents gave also some general comments on the quality of European statistics, even if a specific request for such comments was not in the questionnaire, mostly adding them in the general comments. The majority of the comments referred to the necessity to improve the data timeliness, especially for those data which a have a delay of one year or more. However, some users recognised that timeliness had improved for some data compared to 2017. Eurostat should also try to minimise the number of missing data, due to lack of figures for some countries and to confidentiality. Data inconsistencies were also mentioned, over time, among regions and in mirror statistics. When the methodology changes users would like to have this reflected also in past data for comparability reasons.

3.3 Trust in European statistics

In a period when European citizens sometimes persist to be sceptic about the role and functioning of the EU institutions, it was interesting to check if users continue to trust the statistics produced by Eurostat. Results are presented in Chart 28. As in previous years, responses were overwhelmingly positive, with 96.0% of users stating they trusted European statistics greatly or tended to trust them. Only 3.1% said they did not trust statistics and 0.9% had no opinion. When looking at the distribution of responses by user types, the share of respondents trusting European statistics is very similar for all types, none going below 93.6% and with a peak at 98.0% for advanced users.

Chart 28. Trust in European statistics by user types, in %

0% 20% 40% 60% 80% 100%

All users 96.0 3.10.9 Trust them greatly / Tend to trust them Light users 93.6 4.91.5 Tend not to trust them / Distrust them greatly Intermediate users 95.0 3.91.1 No opinion

Advanced users 98.0 1.60.4

Source: Eurostat 2019 user satisfaction survey Despite the potential bias that comes from the fact that Eurostat's data users should generally trust the data they use, the constantly high rate of positive answers over time demonstrates a very good and encouraging sign on the confidence of users in the statistics disseminated by Eurostat. Looking at the responses, some of the reasons while people trust the statistics are that they are based on harmonised methodology and subject to quality standards and thorough validations. The fact that Eurostat is professional and is not politically influenced also helped to gain user trust. As in past years, the most recurrent comment of those few who tend not to trust European statistics is because they depend on national statistics. Some then pointed out discrepancies with national data and reported implausible data and errors. Few also felt the lack of a clear source for the original data. Users were also explicitly asked to suggest ways to improve trust. Common suggestions included more checks on the data provided by the countries and more transparency in the methodology used, including a better harmonisation of the methodology used by the countries. Few also suggested giving information on changes and updates in the data and explaining abnormal data and outliers. In one case, the peer reviews were mentioned as an important instrument to improve the quality and trust in European statistics. Between 2012 and 2016 there had been a continuous but very small decrease in trust in European statistics, whereas the survey 2019 shows an increase in this indicator, up to an overall 96.0%, the highest result ever (Chart 29).

Chart 29. Trust in European statistics in 2012-2019, in % 95.3% 100% 94.9% 94.4% 94.2% 93.8% 96.0% 90%

80%

70%

60%

50%

40%

30%

20%

10%

0% 2012 2013 2014 2015 2016 2019

Source: Eurostat 2012, 2013, 2014, 2015, 2016 and 2019 user satisfaction surveys To deepen our analysis on the trust in statistics, we have checked whether there is some relation between importance, trust and perceived quality of statistics. As can be seen in Chart 30 the degree of trust in European statistics depends on the importance that the statistics have for the users. Those respondents, for which the statistics are of greater value, trust more the statistics than those for whom statistics are not so important, who tend more often not to express an opinion. Chart 30. Trust in European statistics by importance, in %

0% 20% 40% 60% 80% 100%

They are essential 97.7 2.10.3

Trust them greatly They are important 95.1 3.91.0 / tend to trust them

They are of value as 96.2 2.71.1 background information Tend not to trust them / distrust them greatly They are of minor importance 90.5 7.1 2.4

No opinion They are of no use 66.7 0.0 33.3

Average 96.0 3.10.9

Source: Eurostat 2019 user satisfaction survey The respondents who trust more European statistics are also more convinced of their overall good quality, as it appears in Chart 31. In particular, those respondents who trust European statistics greatly are 11.4% points more satisfied with the data quality than the average of all users, while the few respondents who tend not to trust or distrust greatly the statistics, are also much more critical towards their quality.

Chart 31. Assessment of overall quality of European statistics by trust, in %

0% 20% 40% 60% 80% 100%

Average all users 58.2 19.9 16.2 5.7

Very Good/Good Trust them greatly 69.6 10.8 14.9 4.7 Adequate Poor/Very poor No opinion Tend to trust them 47.8 29.6 16.1 6.5

Tend not to trust them 19.8 29.7 46.5 4.0

Distrust them greatly 14.3 14.3 71.4 0.0

No opinion 0.0 33.3 16.7 50.0

Source: Eurostat 2019 user satisfaction survey

3.4 Information on dissemination aspects

This section covers a number of aspects concerning dissemination of European statistics (content and characteristics of the Eurostat website; access to the European statistics; release calendar; metadata; visualisation tools; experimental statistics; access to microdata and user support provided by Eurostat). Various aspect of the Eurostat website are investigated, starting with a general question on the satisfaction with the "Eurostat website" which is targeted to assess the more global level of satisfaction of the overall Eurostat dissemination offer. Indeed, for consumers of European statistics the term "Eurostat website" groups the various dissemination products and tools Eurostat publishes via the website. The degree of satisfaction expressed by those who gave an opinion is 58.1% satisfied and 34.9% partly satisfied, as presented in Chart 32. The rate of the respondents who declared to be satisfied went slightly down by 2.2 percentage points compared to 2016. This might indicate that the users would wish to get a more modern version of the website, which is, even with some improvements, still the one introduced in 2014, and indeed Eurostat is preparing a new website to be released soon. Furthermore, it is interesting to note that in the question on the "changes in perception of the overall quality of data and services provided by Eurostat", the website was the single item with the highest share of respondents (20.7%) perceiving that it had improved compared to the time of the previous survey. This could confirm that although users admit that the website has improved they would like to get a new one. In this case, the respondents identifying themselves as advanced users are the least satisfied (57.0%) even if the shares are almost equal for the three groups, with 58.5% for the light

users and 59.2% for the intermediate users. It might be that advanced users are also more exigent with the website. Chart 32. User satisfaction with the Eurostat website, in %

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

All users 58.1 34.9 7.0

Light users 58.5 35.0 6.6 Yes Partly No Intermediate users 59.2 33.5 7.3

Advanced users 57.0 36.0 7.0

Source: Eurostat 2019 user satisfaction survey

3.4.1 Access to and understanding of European statistics on the Eurostat Website More than half of the respondents (53.0%) found it easy to access and to understand the statistics on the Eurostat website and more than another third (35.4%) partly easy. 8.0% were not satisfied while the remaining 3.6% did not express an opinion. The results are practically identical to the ones of 2016. Here again intermediate and advanced users are more satisfied that light users. This is normal, as they should know better how to navigate the website and extract the statistics they need. Chart 33. Assessment of the access to and understanding of European statistics, in % (Is it easy to access and to understand European statistics?) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

All users 53.0 35.4 8.0 3.6

Light users 49.5 33.8 9.3 7.4 Yes

Partly

No Intermediate users 53.5 36.2 7.8 2.5

Advanced users 54.3 35.4 7.6 2.7

Source: Eurostat 2019 user satisfaction survey

Users were also asked to evaluate the content of the Eurostat database. As in previous years, responses were very positive (Chart 34). On average 67.7% of all respondents were satisfied with the content, which is 2.9 percentage points more than in 2016 and almost the highest value ever registered (Chart 35). Advanced users were quite more satisfied with the content of the website (71.5% of “Very Good/Good”) than those from the other two types (65.2% for the intermediate users and 63.7% for the light users). Chart 34. Assessment of the Eurostat website content by user types, in % 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

All users 67.7 24.1 5.6 2.6

Light users 63.7 25.5 3.9 6.9

Intermediate users 65.2 27.0 6.7 1.1

Advanced users 71.5 21.1 5.6 1.8

Very good / Good Adequate Poor / Very poor No opinion

Source: Eurostat 2019 user satisfaction survey Chart 35. Eurostat’s website content 2011-2019, in % 100%

90%

80% 68.0% 67.7% 63.8% 65.3% 70% 63.8% 63.0% 64.8% 60%

50%

40%

30%

20%

10%

0% 2011 2012 2013 2014 2015 2016 2019

Source: Eurostat 2011, 2012, 2013, 2014, 2015, 2016 and 2019 user satisfaction surveys In another question, users were requested to judge its technical characteristics (Chart 36). Results are slightly worse than in 2016 and uneven. Some technical characteristics, like clarity of information and performance and speed get more than half of "very good/good"

judgements (53.3% and 53.0% respectively) while for others like the help texts and facilities or the search facilities, the share of satisfied users does not reach 40% and even without taking into account those not giving an opinion, it would not reach 50%. It can be deduced that these attributes still require further attention and improvements. In the case of the alert and notification mechanisms almost half (49.8%) of the respondents did not give an opinion as many do not use or do not need this service. Chart 36. Assessment of technical characteristics of the Eurostat website, in % 0% 20% 40% 60% 80% 100%

Clarity of information 53.3 24.3 18.0 4.4

Performance / Speed 53.0 24.1 17.9 5.0

Database extraction tools 48.6 22.8 20.6 8.0 Very good / Good Adequate Navigation to required info 46.6 26.5 22.5 4.5 Poor / Very poor No opinion Help texts / Help facilities 38.8 27.8 18.2 15.3

Search facilities 36.7 27.2 24.4 11.8

Alert and notification 26.1 14.7 9.5 49.8

Source: Eurostat 2019 user satisfaction survey Chart 37 shows that the results have not changed much over time but going down this time, another sign of the necessity to renovate the website.

Chart 37. Assessment of technical characteristics of Eurostat’s website 2011-2019, in % 100.0%

90.0%

80.0%

70.0% Performance/speed 60.0% Database extraction tools

50.0% Navigation Search facilities 40.0% Help

30.0% Alert and notification

20.0%

10.0%

0.0% 2011 2012 2013 2014 2015 2016 2019

Source: Eurostat 2011, 2012, 2013, 2014, 2015, 2016 and 2019 user satisfaction surveys The following questions were to rate Eurostat's digital publications and visualisation tools. The satisfaction is presented in the Chart 38, and is generally very positive, with well more than 60% of respondents, who expressed an opinion, judging the tools as “very good/good”, up to the 71.1% of the digital publication on “Air traffic in the European Union”.

Chart 38. Assessment of Eurostat’s digital publications and visualisation tools, in %

0% 20% 40% 60% 80% 100%

Air traffic in the European 71.1 18.1 10.8 Union

Women and men in 69.4 18.6 12.0 Europe

Key figures on Europe 68.8 14.3 16.9

The EU economy since the Very good/ Good 68.1 20.0 11.9 start of the millennium Adequate Shedding light on energy Poor / Very poor 67.6 17.8 14.6 in Europe

Digital economy & society 65.2 18.8 16.0 in the EU

SDGs & me 63.8 24.9 11.3

Themes in the spotlight 63.1 24.3 12.5

Source: Eurostat 2019 user satisfaction survey It is worth noting that there is a considerable difference between all the survey respondents, the respondents who used Eurostat's digital publications and visualisation tools, and the respondents who used them and chose to express an opinion. As demonstrated in Chart 39, 30.1% to 58.9% of the survey respondents used the different publications and tools, “Key figures on Europe” being the most widely used, followed at distance by “Digital economy & society in the EU” (41.1%). However, as it can be seen in the same chart, the percentage of users who actually gave their opinion in the question concerned was about 9 - 12% points smaller than the number of users for each of the publications and tools. This means that in the case of “Air traffic in the European Union” the assessment was given by only 20% of users who filled in the survey, but which represents still a significant absolute number of 204 respondents. On an overall level, respondents from advanced users giving an opinion are again more satisfied with the publications and tools (71.6% of “very good/good”), compared to intermediate users (65.2%) and light users (63.0%). A comparison with the past surveys is not possible for this question as the tools and publications are almost all different from 3 years ago, a sign that Eurostat varies them according to the importance of the topics and to the interest of the users over time.

Chart 39. Users of Eurostat’s digital publications and visualisation tools, in %

100%

90%

80%

70% 58.9% 60% 49.9% Users expressing an 50% opinion 41.1% 39.1% 38.7% 38.0% 40% 35.7% 34.7% Users expressing an 31.0% 30.1% 29.8% 29.0% opinion + Users with 30% 26.1% 25.8% 25.1% no opinion 20.2% 20%

10%

Europe

SDGs & me & SDGs

spotlight

inEurope

Key figures on Key

Themes in the in Themes

Air traffic in the in Airtraffic

themillennium

European Union European

since de start of de since start

The EU economy The

energy inEurope energy

Women and Women men

Shedding light on light Shedding society in the society EU Digital economy & economy Digital Source: Eurostat 2019 user satisfaction survey The Eurostat website was, as usual, the section of the questionnaire that received the largest amount of comments, confirming the need to revise the website. Many respondents still found it rather difficult to find data, especially for new users or those who do not use the webpage frequently. Some felt that a clear overview was missing and that titles, definitions and units were not always clear. The size of the database and the high level of detail of data were also seen as a drawback by some users who found it hard to find the specific data they needed. The navigation tree was also criticised by some users who would like to have it organised differently. An index of all statistics was felt useful. This time several users also referred to the website and its interface as old-fashioned, with a too heavy front page, and not intuitive. Regarding data search, there were users dissatisfied with the search engine, some of whom would have preferred to a search targeted exclusively to databases rather than the whole website. Others complained about the downloading facilities, especially for big amount of data, which sometimes fail in the middle of the process. Several respondents suggested adding the possibility to customise the data extraction, using parameters, and allowing saving them. Other recurrent comments referred to the difficulty to find old data. Several users reported then problems with the changes in the database, which are not always clearly explained and on which users are not informed. Some asked if it was possible to add an alert mechanism on a set of data. Finally, some similar comments were made also on the specific database for

international trade statistics, COMEXT. One user also questioned why it was different from the rest of the database. It must considered that although the website attracted many negative comments, there were also users who described it in very positive terms, some considering it very user friendly and efficient and better than those of other similar organisations. A new question was added this year to the section of the survey on the website, on the usefulness of Eurostat’s experimental statistics, a domain that Eurostat has recently started to develop and that did not exist in 2016. As it was expected that only a minority of respondents had used at least some of the experimental statistics, in this case respondents could skip the question, declare that they had never used some experimental statistics or also that they had no opinion on their usefulness. In the end the shares of those who gave an opinion was around 10% - 20% of the respondents, from 11.6% (117 respondents) for “World heritage sites statistics” to 21.8% (220 respondents) for “Statistics on the joint distribution of income, consumption and wealth” and “Income and consumption: comparing social surveys and national accounts” (Chart 40). Chart 40. Users of Eurostat’s experimental statistics, in % 100%

90%

80%

70%

60%

50%

40% 28.8% 28.1% 29.0% 27.5% Users expressing an 30% 24.4% 21.8% 21.8% 23.1% opinion 21.2% 20.0% 21.0% 20.6% 19.7% 19.2% 18.9% 20% 16.6% 16.1% 13.7% 13.5% 12.1% 11.9% 11.6% 10% Users expressing an opinion + Users with

0% no opinion

statistics

(QALI)

Statistics on the joint Statisticson

distribution of income,… distributionof

Characteristics (STEC) Characteristics

andstructure their

Labour market transitions market Labour

Accounts for Research In… Research Accountsfor

Skills mismatches statistics mismatches Skills

Service Trade by Trade Enterprise Service

Income and Income consumption:

transmission along the food… the along transmission

Food price Price monitoring: Food

Full International and Global Full

comparing social surveys and… comparingsocial

World heritage sites statistics heritage sites World

Flash estimates of income estimates of Flash

Quality Adjusted Quality Input Labour Multinational enterprise groups Multinationalenterprise inequality and poverty indicators and poverty inequality Source: Eurostat 2019 user satisfaction survey The majority of those who gave an opinion on experimental statistics found them useful. The shares of “very good/good” answers went from 55.8% for the Quality Adjusted Labour Input (QALI) to 66.4% for the Labour market transitions statistics (Chart 41).

Chart 41. Usefulness of Eurostat’s experimental statistics, in %

0% 20% 40% 60% 80% 100%

Labour market transitions statistics 66.4 19.2 14.5

Flash estimates of income inequality 65.3 20.3 14.4 and poverty indicators

Skills mismatches statistics 63.5 24.0 12.6

World heritage sites statistics 63.2 20.5 16.2

Multinational enterprise groups and 63.2 25.7 11.0 Very good/ their structure Good Statistics on the joint distribution of 60.5 22.3 17.3 Adequate income, consumption and wealth Income and consumption: comparing 59.5 24.5 15.9 Poor / Very social surveys and national accounts poor Full International and Global 59.2 23.3 17.5 Accounts for Research In Input-… Service Trade by Enterprise 58.2 28.7 13.1 Characteristics (STEC) Food price monitoring: Price 56.2 30.2 13.6 transmission along the food supply…

Quality Adjusted Labour Input (QALI) 55.8 29.7 14.5

Source: Eurostat 2019 user satisfaction survey On an overall level, also for this question respondents from advanced users giving an opinion are more satisfied (65.1% of “very good/good”), compared to intermediate users (56.9%) and light users (63.2%). In their comments, users confirmed that they found experimental statistics quite interesting and useful and would like to get more of them in future. Some pointed out that they were not very visible and a few respondents said that they had discovered them thanks to the survey. To complete the section of the survey on the website, users were asked for the second time this year to rate the information on microdata access services on the Eurostat website. Almost half of the respondents (47.0%) gave an opinion, so showing that they use the microdata. The share was higher, as it could be expected, for the respondents from advanced users (52.7%) than for intermediate users (44.8%) and light users (38.2%). However, advanced users seem to be also more exigent for this kind of services, as their satisfaction (54.9% of “very good/good”) was in this case lower than for intermediate users (60.9%) and light users (59.0%) (Chart 42).

Chart 42. Assessment of the information on microdata access services on the Eurostat website, in % 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

All users 57.6 32.3 10.1

Very good Light users 59.0 35.9 5.1 / Good Adequate

Intermediate users 60.9 31.7 7.5 Poor / Very poor

Advanced users 54.9 31.5 13.6

Source: Eurostat 2019 user satisfaction survey Compared to 2016, the share of those considering the information on microdata access services as least good registered an increase of 2.8 percentage points (Chart 43). Chart 43. Assessment of the information on microdata access services on the Eurostat website in 2016 and 2019, in % 0 10 20 30 40 50 60 70 80 90 100

2019 57.6 32.3 10.1 Very good / Good Adequate Poor / Very poor 2016 54.8 35.3 9.9

Source: Eurostat 2016 and 2019 user satisfaction surveys In their comments respondents considered not only the information on microdata access but also the access procedure and the set of microdata available. They declared to appreciate the service and the availability of microdata. However, several pointed out that it is difficult to receive an answer when asking for microdata and the procedure for getting access is too complicate and long. On this, it can be noted that when getting a request Eurostat has to verify the criteria laid down in the applicable Regulation, which usually takes about one week. A few respondents said that they would like to get microdata also for other topics than those available and that they would appreciate more options to access the microdata.

3.4.2 Release calendar When asked about their awareness of Eurostat’s release calendar (Chart 44), which provides information on the dates and times of Euro indicators’ releases and other news releases and publications, a bit more than a third of the respondents seemed to be aware of it (34.6%), with a share increasing by 3.3 percentage points compared to 2016. Among user types, advanced and intermediate users, with respective shares of 38.8% and 36.2% were much more aware than light users (22.5%), which could be expected.

Chart 44. Awareness of the release calendar among user types, in % 0% 20% 40% 60% 80% 100%

All users 34.6 65.4

Light users 22.5 77.5 Aware Not aware Intermediate users 36.2 63.8

Advanced users 38.8 61.2

Source: Eurostat 2019 user satisfaction survey Within the entire surveying period, 2019 is the year with the highest degree of awareness. Chart 45. Awareness of release calendar 2011-2019, in % 100% 90% 80% 70% 60% 50% 40% 34.6% 31.3% 26.6% 30.9% 29.7% 29.2% 28.3% 30% 20% 10% 0% 2011 2012 2013 2014 2015 2016 2019

Source: Eurostat 2011, 2012, 2013, 2014, 2015, 2016 and 2019 user satisfaction surveys Those who were aware of the calendar were also asked to assess whether the release calendar had sufficient and relevant information to fulfil their needs (Chart 46). About two thirds of the respondents (67.3%), a share close to 2016, gave positive opinions, indicating that Eurostat release calendar continues to be of great value for those who are aware of it and use it for their needs. 18.6% of respondents said the calendar satisfied their needs partly.

Chart 46. Assessment of sufficiency and relevance of information in the release calendar by user types, in % 0% 20% 40% 60% 80% 100%

All users 67.3 18.6 2.6 11.5

Light users 69.6 21.7 0.0 8.7 Yes

Intermediate users 67.7 16.2 0.8 15.4 Partly

No Advanced users 66.5 19.7 4.6 9.2

Source: Eurostat 2019 user satisfaction survey For once, there were limited differences among the three user types, with the light users this time more satisfied (69.6%) than intermediate (67.7%) and advanced users (66.5%). In their comments, users expressed the wish to have more topics covered by the release calendar (social and environment statistics were mentioned), to include in the calendar the list of all data for which updates or releases are expected, and to respect the publications dates. After growing steadily until 2015, user satisfaction with the sufficiency and relevance of information in the release calendar seem to have stabilised (Chart 47). Chart 47. Sufficiency and relevance of information in the release calendar 2011-2019 100%

90%

80%

70% 67.6% 66.4% 67.3% 62.8% 64.2% 56.5% 58.4% 60%

50%

40%

30%

20%

10%

0% 2011 2012 2013 2014 2015 2016 2019

Source: Eurostat 2011, 2012, 2013, 2014, 2015, 2016 and 2019 user satisfaction surveys

3.4.3 Metadata and methodological information Eurostat publishes metadata in order to provide better background information about the data (definitions, methodology, classifications, nomenclature, etc.) and to explain their limitations.

Users were asked to indicate whether they used metadata provided by Eurostat. As seen from Chart 48, metadata was used by more than half of the respondents (57.6%), well more than in 2016 (48.5%). When asking about metadata usage there is a big difference among the three types of users. Only 30.4% of light users declared to use metadata, versus 56.0% of intermediate users and 71.3% of advanced users. Chart 48. Use of metadata by user types, in % 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

All users 57.6 42.4

Light users 30.4 69.6 Yes

No Intermediate users 56.0 44.0

Advanced users 71.3 28.7

Source: Eurostat 2019 user satisfaction survey With the biggest increase since 2011, in 2019 the share of the respondents using metadata is the highest ever registered. Chart 49. Usage of metadata 2011-2019 100%

90%

80%

70%

60% 57.6% 50.9% 48.9% 48.0% 48.5% 50% 43.5% 47.4%

40%

30%

20%

10%

0% 2011 2012 2013 2014 2015 2016 2019

Source: Eurostat 2011, 2012, 2013, 2015, 2016 and 2019 user satisfaction surveys Metadata users were also asked about their accessibility. Results of Chart 50 reveal that this year the share of respondents who find it easily accessible is slightly less than a half (48.4%). A share of 40.1% thought it was partly easy to find and 11.5% experienced difficulties.

Chart 50. Metadata accessibility, in %

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

All users 48.4 40.1 11.5

Yes Light users 38.7 48.4 12.9 Partly

Intermediate users 51.2 41.3 7.5 No

Advanced users 48.4 37.7 13.8

Source: Eurostat 2019 user satisfaction survey Intermediate users were surprisingly a bit more satisfied with the metadata accessibility (51.2%) than advanced users (48.4%), while light users were the least happy (38.7%). As can be seen from Chart 51, after the peak registered in 2015, which could have been due to the new website, user satisfaction with this aspect of the metadata has been slightly decreasing. Chart 51. Easiness of access to metadata 2011-2019, in % 100%

90%

80%

70%

60% 50.9% 52.8% 51.7% 45.3% 49.6% 48.4% 50% 47.4%

40%

30%

20%

10%

0% 2011 2012 2013 2014 2015 2016 2019

Source: Eurostat 2011, 2012, 2013, 2014, 2015, 2016 and 2019 user satisfaction surveys A bit less than half of the metadata users (45.2%) were also satisfied with its sufficiency (Chart 52). On average 45.2% found metadata sufficient for their purposes. This is 8.9 percentage points less than in 2016 and the largest registered decrease of all questions. Another 40.8% of users found metadata partly sufficient and 7.2% stated that metadata was not sufficient. In this case, the shares of intermediate and advanced users who were satisfied were almost identical (45.7% and 45.6%), while the one of light users was lower (41.7%).

Chart 52. Assessment of sufficiency of metadata for the different types of users, in % 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

All users 45.2 47.6 7.2

Light users 41.7 51.7 6.7 Yes Partly No Intermediate users 45.7 50.3 4.1

Advanced users 45.6 45.3 9.2

Source: Eurostat 2019 user satisfaction survey While some users reaffirmed in their comments that metadata are clear, complete and better than those of other data providers, have improved, others still found them not easy to access, not clear enough, too long or too technical. The main suggested further improvements included to provide some more basic metadata, easy to understand and in plain language for non-specialists, to provide metadata at more detailed level and for all indicators, and to give clearer and more complete definitions of all codes. Other respondents wished to get more information on the production of statistics and the used methodology, also to understand more easily the differences among countries. Finally, metadata should be consistent over time and among different statistics and always updated in case of changes in the methodology. As Chart 53 shows, 2019 proved to be the year when users were the least satisfied with this criterion after a period of stability. Chart 53. Sufficiency of metadata 2011-2019, in % 100%

90%

80%

70% 55.2% 57.2% 60% 54.9% 55.4% 55.3% 54.1%

50% 45.2%

40%

30%

20%

10%

0% 2011 2012 2013 2014 2015 2016 2019

Source: Eurostat 2011, 2012, 2013, 2014, 2015, 2016 and 2019 user satisfaction surveys

3.4.5 User support In the survey, users also had the opportunity to express their opinion on the support services offered by Eurostat. Results are presented in Chart 54. Leaving out those with no opinion or not aware of the user support function, the degree of satisfaction remains the highest of all services with 75.0% of the respondents saying that they were either “very satisfied” or “satisfied” with the support service provided by Eurostat. The share of unsatisfied users was 5.5% this year. All types of users were very satisfied. – Advanced users were the most satisfied (76.5%), a bit more than light users (74.0%) and intermediate users (73.5%). Chart 54. Satisfaction with user support, in % 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

All users 75.0 19.5 5.5

Very satisfied / Satisfied Light users 74.0 21.1 4.9 Neither satisfied not Intermediate 73.5 21.7 4.8 unsatisfied users Unsatisfied / Very unsatisfied Advanced users 76.5 17.2 6.3

Source: Eurostat 2019 user satisfaction survey Between 2011 and 2019, overall satisfaction with user support has always been very high, reaching its highest value this year, as shown in Chart 55. Chart 55. Satisfaction with user support 2011-2019 100%

90%

80% 75.0% 72.4% 72.7% 74.4% 70.4% 71.9% 72.5% 70%

60%

50%

40%

30%

20%

10%

0% 2011 2012 2013 2014 2015 2016 2019

Source: Eurostat 2011, 2012, 2013, 2014, 2015, 2016 and 2019 user satisfaction surveys

Users’ comments confirmed that they judge the service very good and the people really helpful. However, a few pointed out that the procedure to get the support can be problematic with the necessity to create an account.

3.5 Overall quality of data and services

Users were also asked to express their views on the overall quality of the data and services provided by Eurostat. As can be seen from Chart 56, the level of overall satisfaction remained quite high with 70.1% of all respondents evaluating data and services as “very good” or “good”, 19.5% as “adequate” and only 3.3% as “poor” or “very poor”. This share of “very good/good” was once again the highest for the advanced users (72.9%) compared to intermediate users (68.5%) and light users (66.7%). Chart 56. Overall satisfaction with the quality of the data and services, in % 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

All users 70.1 19.5 3.3 7.1

Very good / Light users 66.7 19.1 2.5 11.8 Good Adequate

Poor / Very Intermediate users 68.5 22.0 3.9 5.6 poor No opinion

Advanced users 72.9 17.7 3.1 6.3

Source: Eurostat 2019 user satisfaction survey As Chart 57 demonstrates, the assessment of the overall quality of data and services was the second highest since 2012 and the difference with 2017 was only due to the largest share of respondents not giving an opinion (7.1% in 2019 versus 2.5% in 2017). Chart 57. Overall quality of data and services 2012-2019, in % 100

80 73 66.9 70.1 70 66.1 66.5 67 65.3

0 2012 2013 2014 2015 2016 2017 2019

Source: Eurostat 2011, 2012, 2013, 2014, 2015, 2016 and 2019 user satisfaction surveys

3.6 Comparison with previous survey

It is interesting to note that more positive feedback came again, as in the past, from the comparison of overall evaluation of Eurostat’s data, products or services with those at the time of the previous survey (Chart 58). Contrary to what was expressed in response to some other questions, a number of respondents stated that they saw data quality components and services as “better” than in 2017 when looking at the bigger picture. The most striking evaluation is for the website, which was considered better than in 2017 by 20.7% of respondents, even if the judgement on its quality went slightly down when asked directly about it. This phenomenon might also be explained by a potential continuous increase in quality standards that users expect from Eurostat. Users may see improved data or service quality from previous years, but are not necessarily more satisfied with it. A high percentage of “no opinion” responses remained, even more than in the past, which can be partly explained by the fact that some users did not take part in the previous survey, did not recall their responses or simply did not have experience with the services. Chart 58. Changes in perception of Eurostat's data and services quality, in %

0% 20% 40% 60% 80% 100%

Overall 20.9 30.8 0.7 47.6

Website 20.7 28.9 1.7 48.7

Timeliness of data 20.4 30.0 1.1 48.5 Better Same

Completeness of data 19.4 31.6 1.3 47.7 Worse No opinion

Comparability of data 16.9 33.6 0.6 48.9

Metadata 13.8 28.7 0.6 56.9

Support services 12.0 27.4 0.8 59.9

Source: Eurostat 2019 user satisfaction survey At the end users could add more comments of a general nature. Many of them just repeated what already said in responding to the previous questions. What several users added is their wish to get more statistics, in particular more regional statistics, at different levels, and more disaggregated data. In particular, of all themes, transport statistics, waste statistics and statistics on migration were mentioned more than once for an increased coverage. Another recurrent wish is to have Eurostat’s website and publications in more languages.

4. Messages from the users

A list of suggestions for improvement was drawn taking into account both the quantitative analysis of the answers to different questions and the recurrent comments that respondents could give as a free text. Most of them have already been mentioned in the previous reports.

 To further improve the quality of statistical data especially by improving timeliness and reducing data gaps due to confidentiality and late sending of data for some countries.  To provide data at a more disaggregated level and at a more detailed regional level.  To correct data inconsistencies and provide explanations for abnormal data and outliers.  To provide more microdata and to make microdata more easy to access for the users.  To revise the Eurostat website, making it more modern, performant and user friendly, easier to navigate also for non-expert users.  To improve the search engine, data extraction and download facilities.  To provide an alert system when data are modified.  To expand the geographical coverage of provided data on non-EU countries.  To improve metadata by: (i) giving clear, easy to understand and less technical explanations, trying to avoid specialist language, (ii) providing metadata at a more detailed level, (iii) providing always definitions for all codes and explanations of methodology, (iv) regularly updating metadata (e.g. when the methodology changes).  To have more topics covered by the release calendar and to include all expected updates.  To lighten the procedure to get user support.  To have Eurostat’s website and publications in more languages.

Annex 1 - Statistical areas

1. Economy and finance, composed of: 1.1. National accounts (including GDP, main aggregates, input-output tables and European sector accounts) 1.2. Price statistics 1.3. Government finance statistics 1.4. Balance of payments 1.5. Financial accounts and monetary indicators

2. Population and social conditions, composed of: 2.1. Labour market (including labour force survey) 2.2. Population 2.3. Health 2.4. Education and training 2.5. Living conditions and social protection

3. Industry, trade and services, composed of 3.1. Structural business statistics 3.2. Short-term business statistics 3.3. Tourism

4. International trade statistics

5. Environment statistics

6. Agriculture statistics

7. Fishery statistics

8. Energy statistics

9. Transport statistics

10. Digital economy and society

11. Regional statistics

12. Policy indicators, composed of 12.1. Europe 2020 indicators 12.2. Sustainable Development indicators 12.3. Euro indicators / PEEIs (Principal European Economic Indicators) 12.4. Globalisation indicators 12.5. MIP (Macroeconomic Imbalances Procedure) indicators

13. Other

Annex 2 - Breakdown of respondents by country of work place

0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0 16.0 18.0

Belgium 17.7 Italy 9.7 Germany 8.1 Spain 7.1 France 4.7 Portugal 3.3 Netherlands 3.2 United Kingdom 3.1 Poland 3.0 Greece 2.9 Luxembourg 2.9 Austria 2.8 Romania 2.5 Denmark 2.4 Czechia 1.8 Bulgaria 1.5 Ireland 1.5 Croatia 1.5 Hungary 1.3 Sweden 1.3 Lithuania 1.0 Slovenia 0.8 Finland 0.8 Latvia 0.7 Slovakia 0.7 Cyprus 0.6 Estonia 0.3 Malta 0.1 Other (non-EU) 13.0

Annex 3 - Example of calculations for the question on overall quality

Step 1. Detailed results for all statistical areas

Q_10: How do you rate the overall quality of European statistics?

Overall Quality Very Good Adeq. Poor Very No Total good poor opin. National accounts (including GDP, main aggregates, input-output 103 150 63 55 27 27 425 tables and European sector accounts) Price statistics 48 75 41 30 13 19 226 Government finance statistics 46 64 26 13 10 8 167 Balance of payments 30 50 21 19 6 6 132 Financial accounts and monetary 22 38 17 11 4 6 98 indicators Structural business statistics 36 79 43 32 9 17 216 Short-term business statistics 26 52 23 18 5 8 132 Tourism 20 26 21 12 5 9 93 Labour market (including labour 78 137 68 35 16 21 355 force survey) Population 86 118 55 33 20 17 329 Health 38 57 42 14 9 9 169 Education and training 44 79 47 24 15 13 222 Living conditions and social 62 100 52 22 12 10 258 protection International trade statistics 55 101 58 32 18 8 272 Environment statistics 33 68 46 29 7 7 190 Agriculture statistics 24 66 33 24 8 5 160 Fishery statistics 9 17 8 4 2 5 45 Energy statistics 37 63 41 21 8 10 180 Transport statistics 22 47 32 11 4 7 123 Science and technology and 38 49 45 17 6 9 164 innovation Digital economy and society 29 44 30 8 7 7 125 Regional statistics 46 80 47 26 9 10 218 Europe 2020 indicators 43 64 29 19 9 12 176 Sustainable development indicators 42 55 40 14 7 7 165 Euro indicators / PEEIs (Principal 22 27 20 5 4 8 86 European Economic Indicators) Globalisation indicators 18 29 11 9 7 4 78 MIP (Macroeconomic Imbalances 17 16 11 5 3 3 55 Procedure) indicators Your other European statistics as 10 16 5 2 1 8 42 specified under Question 1 Step 2. Results are aggregated under bigger areas

Very Very No Overall Quality Good Adequate Poor Total good poor opinion Economy and finances 249 377 168 128 60 66 1048 Industry, trade and services 82 157 87 62 19 34 441 Population and social 308 491 264 128 72 70 1333 conditions International trade statistics 55 101 58 32 18 8 272 Environment statistics 33 68 46 29 7 7 190 Agriculture statistics 24 66 33 24 8 5 160 Fishery statistics 9 17 8 4 2 5 45 Energy statistics 37 63 41 21 8 10 180 Transport statistics 22 47 32 11 4 7 123 Science and technology and 38 49 45 17 6 9 164 innovation Digital economy and society 29 44 30 8 7 7 125 Regional statistics 46 80 47 26 9 10 218 Policy indicators 142 191 111 52 30 34 560 Other 10 16 5 2 1 8 42 Total 1084 1767 975 544 251 280 4901

Step 3. "Very good" and "Good" and "Very poor" and "Poor" are merged

Very Poor / Overall Quality good / Adequate No opinion Total Very poor Good Economy and finances 626 168 188 66 1048 Industry, trade and services 239 87 81 34 441 Population and social 799 264 200 70 1333 conditions International trade statistics 156 58 50 8 272 Environment statistics 101 46 36 7 190 Agriculture statistics 90 33 32 5 160 Fishery statistics 26 8 6 5 45 Energy statistics 100 41 29 10 180 Transport statistics 69 32 15 7 123 Science and technology and 87 45 23 9 164 innovation Digital economy and society 73 30 15 7 125 Regional statistics 126 47 35 10 218 Policy indicators 333 111 82 34 560 Other 26 5 3 8 42 Average for all areas 2851 975 795 280 4901

Step 4. Final table with calculated percentages

Very Poor/Very Overall Quality Adequate No opinion good/Good poor Economy and finances 59,7% 16,0% 17,9% 6,3% Industry, trade and services 54,2% 19,7% 18,4% 7,7% Population and social conditions 59,9% 19,8% 15,0% 5,3% International trade statistics 57,4% 21,3% 18,4% 2,9% Environment statistics 53,2% 24,2% 18,9% 3,7% Agriculture statistics 56,3% 20,6% 20,0% 3,1% Fishery statistics 57,8% 17,8% 13,3% 11,1% Energy statistics 55,6% 22,8% 16,1% 5,6% Transport statistics 56,1% 26,0% 12,2% 5,7% Science and technology and innovation 53,0% 27,4% 14,0% 5,5% Digital economy and society 58,4% 24,0% 12,0% 5,6% Regional statistics 57,8% 21,6% 16,1% 4,6% Policy indicators 59,5% 19,8% 14,6% 6,1% Other 61,9% 11,9% 7,1% 19,0% Average for all areas 58,2% 19,9% 16,2% 5,7%