Ways of Reducing Coverage and Sampling Error As Part of the Total Survey Error Framework for Establishment Surveys in Europe: Recent Developments

Ways of Reducing coverage and sampling error as part of the Total Survey Error Framework for Establishment Surveys in Europe: Recent Developments

Nikola Jovanovski - Sample Solutions BV Carsten Broich - Sample Solutions BV

CONTENT OVERVIEW

Enrichment Methodology Introduction How can we use a Big Data approach to Company Background decrease sampling error in European European Establishment Surveys 4 1 Establishment Surveys Big Data as a crucial part in sampling

Pros and Cons Research Sample challenges and Research Problems Beneﬁts and limitations from using the

2 ESRA 2019 Future considerations 5 enrichment approach

Establishment Sample sources Conclusion What are the diﬀerent sources of Research Problem address

6 3 establishment sample data for the EU27 Q&A

About Sample Solutions

Background Founded in the Netherlands, back in 2009 with the focus on Business & Consumer

Telephone Sample

Sample Survey Platform B2B module Specialized B2B Database designed Survey Research -instant counts and sample

ordering

Multi country B2B sample Projects Eurobarometer, London Economics, PwC, ABB, World Bank, American Express

www.companyname.com © 2016 Startup theme. All Rights Reserved. 6 "Big data" is a ﬁeld that treats ways to analyze, systematically extract information from, or otherwise deal with data sets that are too large or “ complex to be dealt with by traditional data-processing application software. “

Flash Eurobarometer (business survey part)

European Companies Survey (ECS)

European Survey of Enterprises on New and Emerging Risks

(ESENER)

B2B data vendors (ﬁnance, marketing,sales purposes)

National Business Register

Chamber of Commerce Directory

Sampling Challenge ~60% telephone Issues among the Sampling sources coverage No Phone number included

➔ Decentralized Sample data vending (1 vendor per country) ◆ Under representativeness among certain: ● Industries ● Company Sizes ● Area typologies ● Subsidiaries(Branches) ◆ Variable sample costs among countries ◆ Limited coverage of Contact details of decision makers Entire frame and oﬃcers ◆ No or limited coverage of email addresses Wrong/ Outdated Numbers

Establishment sample

➔ Oﬄine - phone

◆ Key breakdown values (Industry code, Size) Phone Email ◆ No direct contact point with decision maker

◆ Overall better response rates • Response rates • Direct approach

➔ Online - email • Coverage among • Desired role

Sole Traders • Convenient time ◆ Key breakdown values (Industry code, Size) • Wide availability of interview ◆ Direct email of Decision Maker • Rapid interview • Low Cost

◆ Targeting speciﬁc job roles based on research turnaround time • Non opt-in sampling

Research questions

What are the ways of reducing coverage and sampling error for Establishment

Surveys in EU27: 1. How can Big Data be used for increasing the telephone coverage in the Sampling Frame 2. How can the Sampling frame go online, for a mixed mode sampling approach? 3. How can we ensure sample data accuracy and validity in a international and longitudinal study?

From ESRA 2019 we have posted these future action steps:

➔ Use Big Data to Identify Company Size(no. of Employees) on businesses with unknown size ➔ Include a validation step in the enrichment process to lower data inaccuracy ➔ Include a Area Typology stratiﬁcation criteria in order to make a more representable sample to frame ➔ Design sample of all countries from a centralized source ➔ Source Contact person data and email

● Country, City, Postcode, Region, HQ Location, Branches 01 LOCATION Location

● SIC code, Keywords (from LinkedIn), NAICS code, NACE 02 INDUSTRY code

● Employee Range 03 COMPANY SIZE ● Revenue Range

● Founded date 04 COMPANY STATUS ● Operating status, Entity Type

● Email available, Phone available, Website URL, Decision 05 CONTACT OPTIONS maker (C-level)

06 TECHNOLOGY ● Technologies divided into categories

Three sources for Establishment sample sources:

TRADITIONAL ENRICHMENT GENERATED Designed Designed Designed by by by slidefusion slidefusion slidefusion

Email patterns Available lists Online lookups

Basic Business Verification of email Big Data information activity

National Registers Phone book directories

Open and free access for State or Privately run general public Categorized with NACE and SME sizes Commercial Data Chamber of Commerce vendors Rich in business information Paid access Can be Member only For Marketing, Credit report www.companyname.com and Sales purposes © 2016 Startup theme. All Rights Reserved. Rich in detail 18

CHARACTERISTICS OF TRADITIONAL SOURCES:

Slidefusion copyright

BRAND MANDATORY FREE ACCESS EMAIL ADDRESSES CONTACTS

National Business * Register * *

Phone book *

Commercial Data Vendor *

Chamber of Commerce * * * *

BIG DATA - ENRICHMENT SOURCES

Search Engines Directory lookup 01 02

Social Media Review sites 03 04

Enrichment sources and tools:

Directory lookups

Engine Google Snippet Reverse phone lookups Company name and address search Company Lookups( Company Name + Email address search Address)

Geographic lookup Contact person lookup

Social Media Review Sites

Facebook Company lookup

Phone lookup Geographic lookup Linkedin Contact and Emp. size lookup

SAMPLE DATA ENRICHMENT

Technology Contact personnel Sizes Telephone number Misc details Enrichment

Decision makers From employee count For missing records From a variety of VAT number,

Oﬃcers And Reports Incorrect/Outdated 300 Web BusinessDescription,

website 22

Which source can produce which data

Slidefusion copyright

FEATURES Employee size Phone number Contact person Email Address

Google

Facebook

Directory lookup

Yelp & Tripadvisor

Overall Enrichment rates among elements

75% 60% 40% 20%

Phone Employee size Contacts Email address number

Native encoding Incorrect Location

Wrong or no address Recognition and acceptance of information reduces letters such as Greek, Cyrillic, enrichment productivity Nordic etc.

Registration Name Inactive businesses

Input Data Sourced Company URL to create email domain Contact person Name

Generating Patterns Apply input data to the pattern Combinations eg. ﬁ[email protected] [email protected] ﬁ[email protected]

Validation of email activity Screen all combinations using a SMTP service to see whether the email server recognizes

the email addresses

Email Categorization

[email protected] Personal

[email protected] General

Categorization: Lookup table for CompanyEnrichmen general t

Email Name database for personal Freemail Provider list Freemail

[email protected] Personal

[email protected] General

VERIFIED NAME

EMAILS NOT VERIFIED NAME

NO STATUS DATA VERIFICATION NAME WORKING

NOT WORKING URLS

NAME REDIRECTING

NAME NO STATUS

European enterprises and establishments sampling featured in both online and oﬄine sampling modes Full Sampling method

➔ Mixed Mode Sample (Online and Oﬄine): deﬁned by:

◆ Direct and/or Company Email

◆ Company phone

➔ Merging and blending multiple business sample sources for a

country into one frame ensuring maximum coverage among the

EU27

➔ Collected email address, contact person data and phone number

Oﬄine(phone) Online(email) information from publicly available sources(URLs, Social Media

etc) Figure 2. Estimation of Coverage between oﬄine and online modes EU27 ➔ Generate Email addresses where not available

Business Register Online lookup Validation Enriching otherwise unavailable Merge and blend sample frame Ensuring generated data for business frame among multiple sources output is valid

Evaluate variables Generation available Data availability diﬀers Inhouse generating from source to source patterns for missing

Compile list Web Mining Enriched list Bulk web query to ﬁnd listing Blank records and incorrect The enriched list is of the company phone numbers are submitted delivered to the FW

Input Phone information number

wrong telephone data

without phone

Sources Cross-enrichment

Source validation Using multiple sources

Cross-validation Sustainability

Referencing one data fragment Keeping the data up to date

across multiple sources

BENEFITS

DETAILED RICH SAMPLE DATA Data contains more data elements than any other individual source 2 3

Veriﬁcation Centralized source for EU27 data

Data accuracy is One stop shop for 4 validated by multiple comprehensive data in all sources 27 countries 1 5

Sample going Online Increased coverage

Combining phone and online More sample data can be in survey ﬁeldwork accessed via phone or

Lack of data availability among Sole Traders

Sources limiting available information

Underrepresentation among certain industries ! that are fully oﬄine

Unintentional sourcing of inactive businesses

What are the ethical aspects that should

CONCLUSION

Research questions

What are the ways of reducing coverage and sampling error for Establishment

From ESRA 2019 we have posted these future action steps:

➔ Use Big Data to Identify Company Size(no. of Employees) on businesses with unknown size (COMPLETED) ➔ Include a validation step in the enrichment process to lower data inaccuracy (COMPLETED) ➔ Include a Area Typology stratiﬁcation criteria in order to make a more representable sample to frame (PENDING) ➔ Design sample of all countries from a centralized source (PENDING) ➔ Source Contact person data and email (COMPLETED)

SAMPLE SOLUTIONS WWW.SAMPLE.SOLUTIONS

ADDRESS E-MAIL TELEPHONE WEBSITE

Stationsplein 45 [email protected] Direct: 123-4567-9898 www.sample.solutions

3013 AK Rotterdam Fax: 123-4567-9898 The Netherlands