Optimization Summit 2013
Total Page：16
File Type：pdf, Size：1020Kb
Drafting a Testing Strategy How a subscription-based business increased conversions by 121% Ben Eva Benjamin Huppertz Global Head of Conversion Management Senior Research Manager Euromoney Institutional Investor MECLABS Session speaker Ben Eva @bevamail Global Head of Conversion Management Headshot Euromoney Institutional Investor Ben Eva is an experienced digital marketing professional with more than 17 years of experience in developing strategies to grow revenues for a wide variety of B2B information businesses. @BEvaMail @BEvaMail About: • One of Europe’s largest business and financial media companies that has interests in financial publishing, training and event organization. • It is 70% owned by The Daily Mail and General Trust Group. • Publishes 200+ online information services plus global events and training courses. • 2013 turnover: £404.7/$679.8 million • 51% of revenue comes from subscription products, primarily online. History Graph: Conversion rates 2008-2012 @BEvaMail Conversion to 8.5% Sale 6.8% 3% 1.4% .82% 2006 2007 2008 2009 2010 History Graph: Conversion rates 2008-2012 @BEvaMail Conversion to 8.5% Sale 6.8%Why? 3% 1.4% .82% 2006 2007 2008 2009 2010 Why have conversions declined? @BEvaMail External Factors Internal Factors • Higher product pricing • Poor development • Significant increase in cycle traffic volumes • Few UX resources • Poorer quality trials • Inability to run A/B testing Initial sales funnel Anonymous Registered Initial sales funnel 2.) 60% of traffic Anonymous Registered dropping out of the trial funnel 1.) Less than 1% of traffic converts to 3.) Less a lead than 1% of leads convert to a sale Initial Sales Funnel: 6-month period Of these, 40K Anonymous Registered complete trial registration For every 10M visitors, 100K go through the trial funnel Less than 1% of trials convert to a sale – 300 sales worth $1M Initial Sales Funnel: 6-month period Of these, 40K Anonymous Registered complete trial registration For every 10M visitors, Time100K go to launch a new strategy! through the trial funnel Less than 1% of trials convert to a sale – 300 sales worth $1M Background: Channels and analytics @BEvaMail Other 1% Bounce Rate Conversion Rate Referral 16% Organic 35% None 24% CPC 57% 25% 1% Traffic Source Background: Pre-analysis @BEvaMail 53% proceed to user details 75% proceed to free trial part 2 40% conversion @BEvaMail Background: Looking into the user experience @BEvaMail Strategy in three phases Convert more traffic into the trial process Increase the completion rate for trial sign-up process Identify the best trials to convert to sales @BEvaMail Step 1: Strategy Convert more traffic into the trial process Increase the completion rate for trial sign-up process Identify the best trials to convert to sales Experiment 1: Background @BEvaMail Experiment ID: Metal Bulletin, paywall Location: Locked article Test Design: A/B Research Notes: Background: Only 1% of anonymous traffic entering the trial process. Objective: To increase entry into the trial funnel. Primary Research Question: Can we reduce the bounce rate and take more people into the trial process? Test Elements: CTA, UX, design MetalBulletin: Control @BEvaMail Instructions Subscription or free trial CTAs Login MetalBulletin: Variation 1 @BEvaMail Banner with CTA Login MetalBulletin: Variation 2 @BEvaMail Free trial or subscribe CTAs Login MetalBulletin: Variation 3 @BEvaMail Free trial CTA Login Side by side @BEvaMail Variation 1 Variation 2 Variation 3 Results: MetalBulletin cumulative performance@BEvaMail 153% improvement in cumulative performance All variations outperformed the control Conversion MetalBulletin.com Improvement Rate Control 1.7% -- Variation 1 2% 19.1% Variation 2 2% 17.6% Variation 3 4.2% 153% What You Need to Understand: Variation 3 met and exceeded the target of ! 20% with a confidence level of 95%. Results: MetalBulletin cumulative performance@BEvaMail 153% improvement in cumulative performance All variations outperformed the control Conversion Metal Bulletin Improvement Shared Learning:Rate Control 1.7% -- NowVariation rolling 1 out business2% 19.1%-wide Variation 2 2% 17.6% Variation 3 4.2% 153% What You Need to Understand: Variation 3 met and exceeded the target of ! 20% with a confidence level of 95%. @BEvaMail Step 2: Strategy Convert more traffic into the trial process Increase the completion rate for trial sign-up process Identify the best trials to convert to sales Experiment 2: Background @BEvaMail Experiment ID: Institutional Investor, stage 1, trial step 1 Location: Trial Process Background: 60% of traffic in the funnel is not completing the process. Objective: To increase step 1 and 2 conversion. Primary Research Question: Can we increase conversion in steps 1 and 2? Test Elements: CTA, UX, design Institutional Investor: Free trial @BEvaMail Step 1: Variation 2 External Traffic Step 1: Step 2 Step 3 Control Internal Traffic Step 1: Variation 1 Institutional Investor: Control @BEvaMail Institutional Investor: Treatment 1 @BEvaMail Rotating banners: Institutional Investor: Treatment 2 @BEvaMail Institutional Investor: Treatment 3 @BEvaMail Side by side @BEvaMail Control Treatment 1 Treatment 2 Treatment 3 Results: KPI as conversion @BEvaMail 36.4% improvement over the control The single-panel page had the highest conversion rate. Conversion InstitutionalInvestor.com Improvement Rate Control 32.3% -- Treatment 1 – Slideshow 38.2% 18.4% Treatment 2 – Form fields 41.1% 27.4% Treatment 3 – Single panel 44% 36.4% What You Need to Understand: Variation 3 has a 95.4% chance of beating ! the baseline. Results: KPI as conversion @BEvaMail 36.4% improvement over the control The single panel page had the highest conversion rate. Conversion Variation Improvement Shared Learning:Rate Control 32.3% -- NowTreatment 1rolling– Slideshow out over38.2% 11 18.4sites Treatment 2 – Form fields 41.1% 27.4 Treatment 3 – Single panel 44% 36.4% What You Need to Understand: Variation 3 has a 95.4% chance of beating ! the baseline. Experiment 3: Background @BEvaMail Experiment ID: EuroWeek, shared learning roll out, stage 1 trial process Location: Trial process Background: 60% of traffic in the funnel not completing the process. Objective: To increase step 1 and 2 conversion. Primary Research Question: Can we increase conversion in steps 1 and 2? Test Elements: CTA, UX, design EuroWeek: Variation 1 @BEvaMail Breadcrumbs Form fill and features EuroWeek: Variation 2 @BEvaMail Features Form fill and CTA EuroWeek: Variation 3 @BEvaMail Breadcrumbs Picture upper left Form fill and CTA Icons and features Variations: Side by side @BEvaMail Variation 1 Variation 2 Variation 3 Results: Primary KPI as form submitted @BEvaMail 20% improvement in conversion rate None of the variations underperformed the control. EuroWeek.com Conversion Improvement Control 45% -- Variation 1 54.7% 20% Variation 2 52.9% 16.8% Variation 3 45% 1% What You Need to Understand: The top performance tier target of 20% was ! reached and exceeded by Variation 1. Results: Primary KPI as form submitted @BEvaMail 20% improvement in conversion rate None of the variations underperformed the control SharedEuroweek Learning:Conversion Improvement Control 45% -- Variation 1 54.7% 20% NowVariation 2 rolling out over52.9% 1116.8% sites Variation 3 45% 1% What You Need to Understand: The top performance tier target of 20% was ! reached and exceeded by Variation 1. Experiment 4: Background @BEvaMail Experiment ID: Hedge Fund Intelligence, combine steps 1 and 2, reducing steps in the funnel Location: Trial process Background: Poor UX in stages 1, 2 and 3. Objective: To increase steps 1-3 conversion. Primary Research Question: Can we increase conversion in steps 1-3? Test Elements: CTA, UX, design Results analyzed @BEvaMail 68% Proceeded to email address 75% Proceeded to password Free trial sign-up process @BEvaMail Step 1: Step 2: Control Control Internal Step 3 Step 4 Traffic Step 1 and 2 of 4: Test Variation @BEvaMail Step 1: Step 2: Control Control Control Internal Step 3 Step 4 Traffic Step 1 and 2 of 4: Test Variation Email page Options page @BEvaMail Step 1: Step 2: Treatment Control Control Internal Step 3 Step 4 Traffic Step 1 and 2 of 4: Test Variation Features Header Breadcrumbs CTA Contrast KPI: Conversion rate @BEvaMail 26.1% improvement in cumulative performance Combining steps 2 and 3 performed better than the control. Conversion HedgeFundIntelligence.com Improvement Rate Control 40% -- Variation 1 50.5% 26.1% What You Need to Understand: Variation 1 beat the control with a 93% level ! of confidence. KPI: Conversion rate @BEvaMail 26.1% improvement in cumulative performance CombiningShared steps 2 and 3 performed Learning: better than the control Conversion HedgeFund Intelligence Improvement Rate NowControl built into Web40% product-- Variation 1 50.5% 26.1% Whatdevelopment You Need to Understand: Variation 1 beat process the control with a 93% level ! of confidence. @bevamail @BEvaMail Step 3: Strategy Convert more traffic into the trial process Increase the completion rate for trial sign-up process Identify the best trials to convert to sales Experiment 4: Background @BEvaMail Experiment ID: Hedge Fund Intelligence, combine steps 1 and 2, reducing steps in the funnel Location: Trial process Background: Poor UX in stages 1, 2 and 3. Objective: To increase step 1-3 conversion. Primary Research Question: Can we increase conversion in steps 1-3? Test Elements: CTA, UX, design Experiment 5: Background @BEvaMail Experiment ID: Lead scoring /profiling – optimizing leads