Optimization Summit 2013

Drafting a Testing Strategy How a subscription-based business increased conversions by 121%

Ben Eva Benjamin Huppertz Global Head of Conversion Management Senior Research Manager Euromoney Institutional Investor MECLABS Session speaker

Ben Eva @bevamail Global Head of Conversion Management

Headshot Euromoney Institutional Investor Ben Eva is an experienced digital marketing professional with more than 17 years of experience in developing strategies to grow revenues for a wide variety of B2B information businesses.

@BEvaMail @BEvaMail About:

• One of Europe’s largest business and financial media companies that has interests in financial publishing, training and event organization.

• It is 70% owned by The Daily Mail and General Trust Group.

• Publishes 200+ online information services plus global events and training courses.

• 2013 turnover: £404.7/$679.8 million

• 51% of revenue comes from subscription products, primarily online. History Graph: Conversion rates 2008-2012 @BEvaMail

Conversion to 8.5% Sale 6.8%

3% 1.4% .82% 2006 2007 2008 2009 2010 History Graph: Conversion rates 2008-2012 @BEvaMail

Conversion to 8.5% Sale 6.8%Why? 3% 1.4% .82% 2006 2007 2008 2009 2010 Why have conversions declined? @BEvaMail

External Factors Internal Factors • Higher product pricing • Poor development • Significant increase in cycle traffic volumes • Few UX resources • Poorer quality trials • Inability to run A/B testing Initial sales funnel

Anonymous Registered Initial sales funnel 2.) 60% of traffic Anonymous Registered dropping out of the trial funnel

1.) Less than 1% of traffic converts to 3.) Less a lead than 1% of leads convert to a sale Initial Sales Funnel: 6-month period Of these, 40K Anonymous Registered complete trial registration

For every 10M visitors, 100K go through the trial funnel Less than 1% of trials convert to a sale – 300 sales worth $1M Initial Sales Funnel: 6-month period Of these, 40K Anonymous Registered complete trial registration

For every 10M visitors, Time100K go to launch a new strategy! through the trial funnel Less than 1% of trials convert to a sale – 300 sales worth $1M Background: Channels and analytics @BEvaMail

Other 1%

Bounce Rate Conversion Rate Referral 16% Organic 35% None 24%

CPC 57% 25% 1% Traffic Source Background: Pre-analysis @BEvaMail

53% proceed to user details

75% proceed to free trial part 2

40% conversion @BEvaMail Background: Looking into the user experience @BEvaMail Strategy in three phases

Convert more traffic into the trial process

Increase the completion rate for trial sign-up process

Identify the best trials to convert to sales @BEvaMail Step 1: Strategy

Convert more traffic into the trial process

Increase the completion rate for trial sign-up process

Identify the best trials to convert to sales Experiment 1: Background @BEvaMail

Experiment ID: Metal Bulletin, paywall Location: Locked article Test Design: A/B

Research Notes: Background: Only 1% of anonymous traffic entering the trial process. Objective: To increase entry into the trial funnel. Primary Research Question: Can we reduce the bounce rate and take more people into the trial process? Test Elements: CTA, UX, design MetalBulletin: Control @BEvaMail

Instructions

Subscription or free trial CTAs

Banner with CTA

Free trial or subscribe CTAs

Free trial CTA

Variation 1 Variation 2 Variation 3 Results: MetalBulletin cumulative performance@BEvaMail

153% improvement in cumulative performance All variations outperformed the control

Conversion MetalBulletin.com Improvement Rate Control 1.7% -- Variation 1 2% 19.1% Variation 2 2% 17.6% Variation 3 4.2% 153%

What You Need to Understand: Variation 3 met and exceeded the target of ! 20% with a confidence level of 95%. Results: MetalBulletin cumulative performance@BEvaMail

153% improvement in cumulative performance All variations outperformed the control

Conversion Metal Bulletin Improvement Shared Learning:Rate Control 1.7% -- NowVariation rolling 1 out business2% 19.1%-wide Variation 2 2% 17.6% Variation 3 4.2% 153%

What You Need to Understand: Variation 3 met and exceeded the target of ! 20% with a confidence level of 95%. @BEvaMail Step 2: Strategy

Convert more traffic into the trial process

Increase the completion rate for trial sign-up process

Identify the best trials to convert to sales Experiment 2: Background @BEvaMail

Experiment ID: Institutional Investor, stage 1, trial step 1 Location: Trial Process

Background: 60% of traffic in the funnel is not completing the process. Objective: To increase step 1 and 2 conversion. Primary Research Question: Can we increase conversion in steps 1 and 2? Test Elements: CTA, UX, design Institutional Investor: Free trial @BEvaMail

Step 1: Variation 2 External Traffic

Step 1: Step 2 Step 3 Control

Internal Traffic Step 1: Variation 1 Institutional Investor: Control @BEvaMail Institutional Investor: Treatment 1 @BEvaMail

Rotating banners: Institutional Investor: Treatment 2 @BEvaMail Institutional Investor: Treatment 3 @BEvaMail Side by side @BEvaMail

Control Treatment 1 Treatment 2 Treatment 3 Results: KPI as conversion @BEvaMail

36.4% improvement over the control The single-panel page had the highest conversion rate.

Conversion InstitutionalInvestor.com Improvement Rate Control 32.3% -- Treatment 1 – Slideshow 38.2% 18.4% Treatment 2 – Form fields 41.1% 27.4% Treatment 3 – Single panel 44% 36.4%

What You Need to Understand: Variation 3 has a 95.4% chance of beating ! the baseline. Results: KPI as conversion @BEvaMail

36.4% improvement over the control The single panel page had the highest conversion rate.

Conversion Variation Improvement Shared Learning:Rate Control 32.3% -- NowTreatment 1rolling– Slideshow out over38.2% 11 18.4sites Treatment 2 – Form fields 41.1% 27.4 Treatment 3 – Single panel 44% 36.4%

What You Need to Understand: Variation 3 has a 95.4% chance of beating ! the baseline. Experiment 3: Background @BEvaMail

Experiment ID: EuroWeek, shared learning roll out, stage 1 trial process Location: Trial process

Background: 60% of traffic in the funnel not completing the process. Objective: To increase step 1 and 2 conversion. Primary Research Question: Can we increase conversion in steps 1 and 2? Test Elements: CTA, UX, design EuroWeek: Variation 1 @BEvaMail

Breadcrumbs

Form fill and features EuroWeek: Variation 2 @BEvaMail

Features

Form fill and CTA EuroWeek: Variation 3 @BEvaMail

Breadcrumbs

Picture upper left

Form fill and CTA Icons and features Variations: Side by side @BEvaMail

Variation 1 Variation 2 Variation 3 Results: Primary KPI as form submitted @BEvaMail

20% improvement in conversion rate None of the variations underperformed the control.

EuroWeek.com Conversion Improvement Control 45% -- Variation 1 54.7% 20% Variation 2 52.9% 16.8% Variation 3 45% 1% What You Need to Understand: The top performance tier target of 20% was ! reached and exceeded by Variation 1. Results: Primary KPI as form submitted @BEvaMail

20% improvement in conversion rate None of the variations underperformed the control SharedEuroweek Learning:Conversion Improvement Control 45% -- Variation 1 54.7% 20% NowVariation 2 rolling out over52.9% 1116.8% sites Variation 3 45% 1% What You Need to Understand: The top performance tier target of 20% was ! reached and exceeded by Variation 1. Experiment 4: Background @BEvaMail

Experiment ID: Hedge Fund Intelligence, combine steps 1 and 2, reducing steps in the funnel Location: Trial process

Background: Poor UX in stages 1, 2 and 3. Objective: To increase steps 1-3 conversion. Primary Research Question: Can we increase conversion in steps 1-3? Test Elements: CTA, UX, design Results analyzed @BEvaMail

68% Proceeded to email address

75% Proceeded to password Free trial sign-up process @BEvaMail

Step 1: Step 2: Control Control

Internal Step 3 Step 4 Traffic Step 1 and 2 of 4: Test Variation @BEvaMail Step 1: Step 2: Control Control Control

Internal Step 3 Step 4 Traffic Step 1 and 2 of 4: Test Variation

Email page Options page @BEvaMail Step 1: Step 2: Treatment Control Control

Internal Step 3 Step 4 Traffic Step 1 and 2 of 4: Test Variation

Features Header

Breadcrumbs CTA Contrast KPI: Conversion rate @BEvaMail

26.1% improvement in cumulative performance Combining steps 2 and 3 performed better than the control.

Conversion HedgeFundIntelligence.com Improvement Rate Control 40% -- Variation 1 50.5% 26.1%

What You Need to Understand: Variation 1 beat the control with a 93% level ! of confidence. KPI: Conversion rate @BEvaMail

26.1% improvement in cumulative performance CombiningShared steps 2 and 3 performed Learning: better than the control Conversion HedgeFund Intelligence Improvement Rate NowControl built into Web40% product-- Variation 1 50.5% 26.1%

Whatdevelopment You Need to Understand: Variation 1 beat process the control with a 93% level ! of confidence.

@bevamail @BEvaMail Step 3: Strategy

Convert more traffic into the trial process

Increase the completion rate for trial sign-up process

Identify the best trials to convert to sales Experiment 4: Background @BEvaMail

Experiment ID: Hedge Fund Intelligence, combine steps 1 and 2, reducing steps in the funnel Location: Trial process

Background: Poor UX in stages 1, 2 and 3. Objective: To increase step 1-3 conversion. Primary Research Question: Can we increase conversion in steps 1-3? Test Elements: CTA, UX, design Experiment 5: Background @BEvaMail

Experiment ID: Lead scoring /profiling – optimizing leads sent to Sales Location: All sites

Background: We don’t want to increase our sales team capacity or increase costs. Objective: Increase the quality of leads without increasing the volume. Primary Research Question: As we increase lead volumes, can we maintain sales team levels and identify the best leads? Test Design: A/B split test A/B testing lead scoring and engagement @BEvaMail

Total leads converting to sales 0.82% A/B testing lead scoring and engagement @BEvaMail

Segments Control Group A Group B

0.96% 1.73% 0.96% Conversion Conversion Conversion A/B testing lead scoring and engagement @BEvaMail

Segments Control Group A Group B Shared Learning: Now rolled out to all businesses

0.96% 1.73% 0.96% Conversion Conversion Conversion @BEvaMail New sales funnel* Of these, 60K Anonymous Registered complete trial registration

For every 10M visitors, 123K go

through the 38% of trials trial funnel are sales- ready, converting at 1.73%, worth *When all stages are rolled out over all products $1.22M @BEvaMail Top takeaways

Internal development was our roadblock, we had to find alternatives.

Find the “quick wins” that you can roll out as shared learning.

Break the problem down into manageable chunks. @BEvaMail

Questions? @BEvaMail Thanks to my team:

Ileana Lund and Darren Herbert Conversion Rate Optimization Team Latitude Digital Marketing [email protected] Thank you @BEvaMail Ben Eva Global Head of Conversion Management Headshot Euromoney Institutional Investor Plc. [email protected] @BEvaMail www.linkedin.com/in/beneva

Benjamin Huppertz Headshot Senior Research Manager MECLABS