DOCSLIB.ORG

Engineering for a Science-Centric Experimentation Platform

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Engineering for a Science-Centric Experimentation Platform Engineering for a Science-Centric Experimentation Platform Nikos Diamantopoulos Jeffrey Wong David Issa Mattos [email protected] [email protected] [email protected] Netflix, Inc. Netflix, Inc. Chalmers University of Technology Los Gatos, California, USA Los Gatos, California, USA Gothenburg, Sweden Ilias Gerostathopoulos Matthew Wardrop Tobias Mao [email protected] [email protected] [email protected] Technical University of Munich Netflix, Inc. Netflix, Inc. Munich, Germany Los Gatos, California, USA Los Gatos, California, USA Colin McFarland [email protected] Netflix, Inc. Los Gatos, California, USA ABSTRACT symbiosis" of engineers and data scientists, each complementing Netflix is an internet entertainment service that routinely employs the skill sets of the other, in order to create a platform that is robust experimentation to guide strategy around product innovations. As and scalable, while also being readily extensible by data scientists. Netflix grew, it had the opportunity to explore increasingly spe- Netflix routinely uses online A/B experiments to inform strategy cialized improvements to its service, which generated demand for and operation discussions (e.g. [4, 18, 26, 28]), as well as whether deeper analyses supported by richer metrics and powered by more certain product changes should be launched. Over time these dis- diverse statistical methodologies. To facilitate this, and more fully cussions grew to be increasingly specialized, generating demand for harness the skill sets of both engineering and data science, Netflix more and richer metrics powered by extensible statistical method- engineers created a science-centric experimentation platform that ologies that are capable of answering diverse causal effects ques- leverages the expertise of data scientists from a wide range of back- tions. For example, it was becoming more common for teams to grounds by allowing them to make direct code contributions in the require bespoke metrics to assist in the analysis of specific ex- languages used by scientists (Python and R). Moreover, the same periments, such as when changes to Netflix’s UI architecture and code that runs in production is able to be run locally, making it video player design caused extra hard-to-isolate latency in play- straightforward to explore and graduate both metrics and causal back startup [18]; or to require bespoke statistical methodologies, inference methodologies directly into production services. such as when interleaving was used to garner additional statisti- In this paper, we utilize a case-study research method to provide cal power when trying to compare two already highly-optimised two main contributions. Firstly, we report on the architecture of this personalization algorithms [4]. platform, with a special emphasis on its novel aspects: how it sup- To support these ever-growing use-cases, Netflix made a strate- ports science-centric end-to-end workflows without compromising gic bet to make their experimentation science-centric; that is, to engineering requirements. Secondly, we describe its approach to place a heavy emphasis on supporting arbitrary scientific analyses. causal inference, which leverages the potential outcomes concep- To implement this science-centric vision, Netflix’s experimentation tual framework to provide a unified abstraction layer for arbitrary platform, Netflix XP, was reimagined around three key tenets: statistical models and methodologies. trustworthiness, scalability, and inclusivity. Trustworthiness is es- sential since results that are untrustworthy are not actionable. Scal- arXiv:1910.03878v1 [cs.SE] 9 Oct 2019 KEYWORDS ability is required to accommodate for Netflix’s growth. Inclusivity is a key tenet because it allows scientists from diverse backgrounds experimentation, A/B testing, software architecture, causal infer- such as biology, psychology, economics, mathematics, physics, com- ence, science-centric puter science and other disciplines to contribute to and enrich the 1 INTRODUCTION experimentation platform. The implications of these tenets on Netflix XP are wide-ranging, Understanding the causal effects of product and business decisions but perhaps chief among them are the resulting choices of language via experimentation is a key enabler for innovation and improve- and computing paradigm. Python was chosen as the primary lan- ment [13, 25, 29], and the gold-standard of experimentation is the guage of the platform; with some components in C++ and R as randomized controlled trial design (also known as A/B testing) needed to support performance and/or statistical models. This was [5, 39, 41]. a natural choice because it is familiar to many data scientists, and In this paper, we will be presenting a case-study of the A/B has a comprehensive collection of standard libraries supporting testing experimentation platform built by Netflix, a leading internet both engineering and data science use-cases. The platform also entertainment service. The innovations of this experimentation platform are interesting because they have resulted in a "technical adopted a non-distributed architecture in order to reduce the bar- Mann-Whitney test, or CUPED [7] are used to identify statistically rier of entry into the platform for new statistical methodologies. significant changes in the metrics and generate scorecards [12]. Since non-distributed architectures are not as trivially scaled, the These scorecards help product managers, engineers, and data scien- techniques employed by the platform in order to ensure scalability, tists to make informed decisions and identify a causal relationship i.e. compression and numerical performance optimizations, are a between the product change and the observed effect. RCT in web significant contribution of this work. systems is extensively discussed by Kohavi et al. [25]. The paper The reimagined Netflix XP has also had implications for its presents an in-depth guide on how to run controlled experiments stakeholders. Firstly, data science productivity has increased. It is on web systems, discussing types of experimental designs, statis- now straightforward for data scientists to reproduce and extend tical analysis, ramp-up, the effect of robots and some architecture the standard analyses performed by the experimentation platform considerations, such as assignment methods and randomization because they can run the production code in a local environment. algorithms. The code also permits ad hoc extensions, allowing scientists to Although most research in online experiments has focused on leverage their background and domain knowledge to easily deliver RCT, companies have been using other types of experimental de- customized scorecards [12]; for example, by including explorations signs to infer causal relations. For instance, Xu and Chen [40] of heterogenity or temporal effects. Secondly, data science work- describe the usage of quasi A/B tests to evaluate the mobile app flows have been enriched with a more extensive toolkit. Since the of LinkedIn. The paper details the characteristics of the mobile in- platform was reimagined, new statistical methodologies (such as frastructure that contribute to the need for designing and running quantile bootstrapping and regression) have been contributed to different experiment designs than RCT. the platform, which can then be used in combination with arbitrary metrics of the data scientists’ choice. Thirdly, engineers have been freed up to focus on the platform itself. Since data scientists are now 2.2 Experimentation Processes and Platforms responsible for contributing and maintaining their own metrics and To support and democratize experimentation across multiple depart- methodologies, engineers are now able to focus on aspects of the ments, products and use cases, Kaufman et al. [23] have identified platform in which they specialize, leading to greater scalability and the need for an experimentation platform to be generic and ex- trustworthiness. The effect of these implications has compounded tensible enough to allow the design, implementation, and analysis in rapid innovation cycles around ongoing strategy discussions, of experiments with minimal ad hoc work. They describe, in the which has changed the face of experimentation at Netflix. context of Booking.com, the usage of an extensible metric frame- In this paper, we utilize a case-study research method to provide work to provide flexibility for experiment owners to create new two main contributions. Firstly, we report on the architecture of this metrics. However, they do not describe the extensibility aspect in platform, with a special emphasis on its novel aspects: how it sup- the context of different experimental designs and analyses aswe ports science-centric end-to-end workflows without compromising do. the engineering requirements laid out in subsequent sections. Sec- Twitter discusses its experimentation platform and how it is ondly, we describe its approach to causal inference, which leverages capable of measuring and analyzing a large number of flexible the potential outcomes framework to provide a unified abstraction metrics [9]. The platform supports three types of metrics: built-in layer for arbitrary statistical models and methodologies. metrics that are tracked for all experiments, event-based metrics, The rest of this paper is organized as follows: Section 2 presents and metrics that are owned and generated by engineers. One of background information in online experiments and related works. the challenges
Load more

Recommended publications
  • Presto Customers to Enjoy No Data Download Limits on Foxtel
    Media Release: Wednesday, February 4, 2015 Presto customers to enjoy no data download limits on Foxtel Broadband Unbelievable entertainment now streaming on Presto Presto today confirmed that Foxtel Broadband subscribers will enjoy no data download limits when accessing Presto’s unbelievable TV and movie entertainment. The news comes on the heels of this week’s announcement by Foxtel that it has now launched Broadband and Home Phone bundles. Presto customers who take one of the new Foxtel Broadband bundles will now see even greater value from their combined subscriptions and can binge on their favourite movie and TV shows unrestricted by data download limits. Shaun James, Director of Presto and On Demand, said, “Presto subscribers are passionate about entertainment and enjoy bingeing on their favourite shows. The massive volume of hours viewed each week is testament to Presto’s popularity and today we’re thrilled to confirm that Foxtel Broadband subscribers can now indulge all they want on the programs they love free from data limits.” Presto recently launched Presto TV bringing an unbelievable selection of 1000s of hours ad-free television shows to Australian TV lovers for $9.99 per month*. Presto TV’s launch follows last year’s successful launch of Presto Movies, which features blockbusters and all-time favourite films from a constantly updated collection of great movies. For $9.99 per month Australians can subscribe to either Presto TV or Presto Movies or bundle both services for $14.99 per month with no ongoing commitment, and now it offers no data download limits for Foxtel Broadband subscribers.
    [Show full text]
  • Control Over Ammonium Nitrate Aerosol
    1 On the effectiveness of nitrogen oxide reductions as a 2 control over ammonium nitrate aerosol 3 4 S. E. Pusede1,*, K. C. Duffey1, A. A. Shusterman1, A. Saleh1, J. L. Laughner1, P. J. Wooldridge1, 5 Q. Zhang2, C. L. Parworth2, H. Kim3, S. L. Capps4, L. C. Valin5, C. D. Cappa6, A. Fried7, J. 6 Walega7, J. B. Nowak8, A. J. Weinheimer9, R. M. Hoff10, T. A. Berkoff11, A. J. Beyersdorf11, J. 7 Olson11, J. H. Crawford11, and R. C. Cohen1,12 8 9 [1]{Department of Chemistry, University of California Berkeley, Berkeley, CA, 94720, USA.} 10 [2]{Department of Environmental Toxicology, University of California at Davis, Davis, CA, 11 95616} 12 [3]{Center for Environment, Health and Welfare Research, Korea Institute of Science and 13 Technology, Seoul, Korea} 14 [4]{Department of Mechanical Engineering, University of Colorado Boulder, Boulder, CO, USA 15 80309} 16 [5]{Lamont-Doherty Earth Observatory, Columbia University, Palisades, NY, 10964} 17 [6]{Department of Civil and Environmental Engineering, University of California at Davis, 18 Davis, CA, 95616} 19 [7]{Institute of Arctic and Alpine Research, University of Colorado, Boulder, CO, 80309} 20 [8]{Aerodyne Research, Inc., Billerica, MA, 01821} 21 [9] {Atmospheric Chemistry Division, National Center for Atmospheric Research, Boulder, CO, 22 80307} 23 [10]{Department of Physics, University of Maryland Baltimore County, Baltimore, MD, 21250} 1 1 [11]{NASA Langley Research Center, Hampton, VA, 23681} 2 [12]{Department of Earth and Planetary Science, University of California Berkeley, Berkeley, 3 CA, 94720, USA.} 4 [*]{now at: Department of Environmental Sciences, University of Virginia, Charlottesville, VA, 5 22904} 6 7 Correspondence to: Ronald C.
    [Show full text]
  • Provider Directory
    Provider Directory How to use this directory Welcome Each health care provider in the directory has agreed to participate with Blue Cross Blue Shield of Wyoming for the Kid Care CHIP program. Services must be provided by the health care professionals, hospitals, and surgical centers listed in this directory. Please remember, should you see a health care provider who is not participating in Kid Care CHIP, services will not be covered by Blue Cross Blue Shield of Wyoming and you will be responsible for all charges. The status of the network is subject to change and providers listed are not a guarantee of payment for services. Although Blue Cross Blue Shield of Wyoming tries to keep this directory up to date, changes occur due to circumstances beyond our control. You may need to contact a provider directly to be sure they are currently accepting new patients. For specific benefit information, please refer to your Kid Care CHIP Handbook, or you may call member services at 1-800-209-9720. Using This We encourage you to select a primary care doctor from the Kid Directory Care CHIP network that you feel comfortable with, who knows your child's health history and is the provider you plan to see on an ongoing basis. This primary care provider can coordinate care with other health care providers. This is the best approach in using your health care benefits, and will provide your child the health care they need. Your child's health is important to all of us. Finding a The directory is separated into several sections.
    [Show full text]
  • Innovation for Age-Friendly Buildings, Cities and Environments
    A COMPILATION OF GOOD PRACTICES Action Group on Innovation for Age-friendly buildings, cities and environments This publication was prepared by the European Commission, DG SANCO and DG CONNECT and Funka Nu based on the material sent by members of the Action Group D4 in September-October 2013. The main contributors were from Funka NU, José Angel Martinez Usero, Frida Westholm, Louise Tengstrand and Eibhilin Manning, Jorge Pinto Antunes, Espen Kristoffersen and Horst Kraemer under the supervision of Maria Iglesia Gomez, Head of Unit Innovation for Health and Consumer and Ilias Iakovidis, Head of Unit for Digital Social Platforms. The publication was presented at the 2nd Conference of Partners of the European Innovation Partnership for Active and Healthy Ageing, which took place in Brussels on 25 November 2013 2 TABLE OF CONTENTS INTRODUCTION ................................................................................................. 4 LIST OF GOOD PRACTICES ................................................................................. 8 CLUSTER 1. LIVING ENVIRONMENTS ................................................................ 19 CLUSTER 2. ACTIVE AGEING IN THE COMMUNITY ........................................... 115 CLUSTER 3. ACTIVE & HEALTHY LIFESTYLES .................................................. 171 CLUSTER 4. DEMENTIA SUPPORTIVE ENVIRONMENTS .................................... 204 CONCLUSIONS ............................................................................................... 221 INDEX OF GOOD PRACTICES,
    [Show full text]
  • Presto: the Definitive Guide
    Presto The Definitive Guide SQL at Any Scale, on Any Storage, in Any Environment Compliments of Matt Fuller, Manfred Moser & Martin Traverso Virtual Book Tour Starburst presents Presto: The Definitive Guide Register Now! Starburst is hosting a virtual book tour series where attendees will: Meet the authors: • Meet the authors from the comfort of your own home Matt Fuller • Meet the Presto creators and participate in an Ask Me Anything (AMA) session with the book Manfred Moser authors + Presto creators • Meet special guest speakers from Martin your favorite podcasts who will Traverso moderate the AMA Register here to save your spot. Praise for Presto: The Definitive Guide This book provides a great introduction to Presto and teaches you everything you need to know to start your successful usage of Presto. —Dain Sundstrom and David Phillips, Creators of the Presto Projects and Founders of the Presto Software Foundation Presto plays a key role in enabling analysis at Pinterest. This book covers the Presto essentials, from use cases through how to run Presto at massive scale. —Ashish Kumar Singh, Tech Lead, Bigdata Query Processing Platform, Pinterest Presto has set the bar in both community-building and technical excellence for lightning- fast analytical processing on stored data in modern cloud architectures. This book is a must-read for companies looking to modernize their analytics stack. —Jay Kreps, Cocreator of Apache Kafka, Cofounder and CEO of Confluent Presto has saved us all—both in academia and industry—countless hours of work, allowing us all to avoid having to write code to manage distributed query processing.
    [Show full text]
  • Improving Access for All
    Improving access for all Submission to the Senate Community Affairs Committee, Inquiry into the delivery of outcomes under the National Disability Strategy 2010 – 2020 to build inclusive and accessible communities April 2017 Level 5, 175 Liverpool Street, Sydney NSW 2000 Phone: 61 2 8898 6500 • Fax: 61 2 8898 6555 • www.piac.asn.au Table of Contents 1. Introduction ..................................................................................................................................... 2 1.1 The Public Interest Advocacy Centre .......................................................................................... 2 2. Summary of recommendations ............................................................................................... 3 3. Access to transport ...................................................................................................................... 5 3.1 Access to public transport ............................................................................................................... 6 3.2 Transport Standards and reporting requirements ................................................................. 8 4. Access to communications and information systems ................................................. 10 4.1 Accessing online services, facilities and places ................................................................. 10 4.2 Broadcast television and audio description .......................................................................... 15 4.3 Video on demand services ...........................................................................................................
    [Show full text]
  • Accessing Subscription Video on Demand
    Accessing subscription video on demand: A study of disability and streaming television in Australia Accessing subscription video on demand: A study of disability and streaming television in Australia Katie Ellis, Mike Kent, Kathryn Locke and Melissa Merchant August 2016 Accessing subscription video on demand: A study of disability and streaming television in Australia Authored by Katie Ellis, Mike Kent, Kathryn Locke and Melissa Merchant Published in 2016 The operation of the Australian Communications Consumer Action Network is made possible by funding provided by the Commonwealth of Australia under section 593 of the Telecommunications Act 1997. This funding is recovered from charges on telecommunications carriers. Curtin University Website: www.curtin.edu.au Email: [email protected] Australian Communications Consumer Action Network Website: www.accan.org.au Email: [email protected] Telephone: 02 9288 4000 If you are deaf, or have a hearing or speech impairment, contact us through the National Relay Service: www.relayservice.gov.au ISBN: 978-1-921974-42-7 Cover image: Design by Richard Van Der Male with images from Shutterstock This work is copyright, licensed under the Creative Commons Attribution 4.0 International Licence. You are free to cite, copy, communicate and adapt this work, so long as you attribute the authors and “Curtin University, supported by a grant from the Australian Communications Consumer Action Network”. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ This work can be cited as: Ellis, K., Kent, M., Locke, K. & Merchant, M. 2016, Accessing subscription video on demand: A study of disability and streaming television in Australia, Australian Communications Consumer Action Network, Sydney.
    [Show full text]
  • TIVO EXPERIENCE ©2019 Tivo Inc
    QUICK GUIDE THE TIVO EXPERIENCE ©2019 TiVo Inc. All rights reserved. Your use of this product is subject to Android is a registered trademark of Google Inc. the TiVo User Agreement (available at ww.tivo.com/legal/terms) and the HBO is a registered trademark of Home Box Office, Inc. TiVo Privacy Policy (available at www.tivo.com/legal/privacy). Hulu is a registered trademarks of Hulu, LLC. Patented. U.S. patent numbers at www.tivo.com/patents. iOS is a registered trademarks of Apple Inc., registered in the U.S. and TiVo, the TiVo logo, the TiVo silhouette logo, OnePass, QuickMode, other countries. App Store is a service mark of Apple Inc. TrickPlay, WishList, the Jump logo, the Instant Replay logo, the Thumbs Up icons , the Thumbs Down icons , Overtime Netflix is a registered trademark of Netflix, Inc. Scheduler, Overlap Protection, the TiVo Circle logo, and the sounds YouTube is a registered trademark of Google Inc. used by the TiVo service are trademarks or registered trademarks of TiVo All other trademarks are the properties of their respective owners. Corp. or its subsidiaries worldwide, 2160 Gold Street, San Jose, CA 95002-2160. Manufactured under license from Dolby Laboratories. “Dolby” TiVo Experience Quick Guide 4.6 (July 2019) and the Double-D symbol are trademarks of Dolby Laboratories. ® HDMI, the HDMI logo, and High-Definition Multimedia Interface are trademarks or registered trademarks of HDMI Licensing LLC in the United States and other countries. WELCOME TABLE OF CONTENTS GETTING STARTED . .IV MANAGE . 26 Basic terms . iv To Do List . .26 THE HOME SCREEN .
    [Show full text]
  • Subscription Video on Demand: Viewing Preferences Among New Zealand Audiences
    Subscription Video on Demand: Viewing preferences among New Zealand audiences Rachel Hayley Daniels A thesis submitted to Auckland University of Technology in partial fulfilment of the requirements for the degree of Master of Communication Studies (MCS) 2017 School of Communication Studies Abstract With the development of digital television platforms in New Zealand and the launch of Subscription Video on Demand (SVOD) services such as Netflix NZ, Lightbox, Quickflix and Neon, a more discerning television audience is emerging. SVOD services are influencing a change within the television broadcasting landscape in New Zealand, fragmenting audiences away from traditional linear television, appealing to viewers in new and innovative ways and changing viewing behaviour. This thesis provides qualitative and quantitative analysis of SVOD viewers’ experiences, expectations, and behaviours with respect to viewing content on the digital platforms and services available to New Zealand subscribers. Quantitative data was collected from an online survey; participants were drawn from readers of the New Zealand Herald online. The quantitative data was collected in order to help position and supplement the qualitative data, which was obtained through focus group interviews. A thematic analysis was used to identify key themes and draw insight from the data sets. The thesis identifies that viewers place a high degree of preference and value on the freedom and opportunities that SVOD provides in personalizing their own viewing practices. Key preferences among these was the ability to control content selection and engage in the practice of anytime viewing, i.e. to choose from an increasingly broad selection of content wherever they are, whenever they like and on whatever device they prefer to view it on.
    [Show full text]
  • Presto Sets the Perfect Mood for February
    Media Release: Wednesday, 11 February 2015 Presto sets the perfect mood for February Celebrate 50 Shades arrival and Valentines Day with Presto special collections Just in time for Valentine’s Day and the Australian premier of 50 Shades of Grey, Presto, Australia’s premium entertainment streaming service, invites fans and romantics to get in the mood with two special collections to arouse their starry-eyed and steamier sides. With the curtain going up on one of the hottest big screen debuts in years, Presto is featuring ’50 Shades of Movie Seduction’ inspired by the book series and upcoming film. The collection includes Academy Award winners American Beauty, The Piano, and Dangerous Liaisons, along with passion filled favourites Body Heat, Notes on a Scandal, Secretary and more. For the more romantically inclined movie lover, Presto has assembled a series of great films to celebrate Valentine’s Day, offering its ‘Love is in the Air’ Valentine’s Day collection featuring Her, Love Story, About Time, Pretty in Pink, America’s Sweethearts, Four Weddings and a Funeral, Are We Officially Dating and many other love linked favourites. Presto TV also offers some of the most binge-worthy programming for a fun or romantic night in, including, HBO’s Girls, Sex and the City, True Blood and Entourage; as well as Love My Way, Satisfaction, Spirited, Winners and Losers and others. For $9.99 per month Australians can subscribe to either Presto TV or Presto Movies or bundle both services for $14.99 per month with no ongoing commitment, and now it offers no data download limits for Foxtel Broadband subscribers.
    [Show full text]
  • Nomination Press Release
    Outstanding Comedy Series 30 Rock • NBC • Broadway Video, Little Stranger, Inc. in association with Universal Television The Big Bang Theory • CBS • Chuck Lorre Productions, Inc. in association with Warner Tina Fey as Liz Lemon Bros. Television Veep • HBO • Dundee Productions in Curb Your Enthusiasm • HBO • HBO association with HBO Entertainment Entertainment Julia Louis-Dreyfus as Selina Meyer Girls • HBO • Apatow Productions and I am Jenni Konner Productions in association with HBO Entertainment Outstanding Lead Actor In A Modern Family • ABC • Levitan-Lloyd Comedy Series Productions in association with Twentieth The Big Bang Theory • CBS • Chuck Lorre Century Fox Television Productions, Inc. in association with Warner 30 Rock • NBC • Broadway Video, Little Bros. Television Stranger, Inc. in association with Universal Jim Parsons as Sheldon Cooper Television Curb Your Enthusiasm • HBO • HBO Veep • HBO • Dundee Productions in Entertainment association with HBO Entertainment Larry David as Himself House Of Lies • Showtime • Showtime Presents, Crescendo Productions, Totally Outstanding Lead Actress In A Commercial Films, Refugee Productions, Comedy Series Matthew Carnahan Circus Products Don Cheadle as Marty Kaan Girls • HBO • Apatow Productions and I am Jenni Konner Productions in association with Louie • FX Networks • Pig Newton, Inc. in HBO Entertainment association with FX Productions Lena Dunham as Hannah Horvath Louis C.K. as Louie Mike & Molly • CBS • Bonanza Productions, 30 Rock • NBC • Broadway Video, Little Inc. in association with Chuck Lorre Stranger, Inc. in association with Universal Productions, Inc. and Warner Bros. Television Television Alec Baldwin as Jack Donaghy Melissa McCarthy as Molly Flynn Two And A Half Men • CBS • Chuck Lorre New Girl • FOX • Chernin Entertainment in Productions Inc., The Tannenbaum Company association with Twentieth Century Fox in association with Warner Bros.
    [Show full text]
  • Apc Contractor Members
    APC CONTRACTOR MEMBERS Abbey Road Control (traffic control, flagging) 853 St. Johns Road Drums, PA 18222 570-788-3096 FAX: 570-788-2937 E-mail: [email protected] Charles Altmiller, President Abel Construction Co., Inc. (000671) (general contractor) P. O. Box 476 Mountville, PA 17554 717-285-3103 FAX: 717-285-2321 Web: www.abelconst.com Troy Abel, President & CEO E-mail: [email protected] Tony Kreider, Vice President- Business Development E-mail: [email protected] Chad Barshinger, Vice President- Project Management E-mail: [email protected] I. B. Abel, Inc. (004122) (electrical contractor) 620 Edgar Street York, PA 17403 717-845-1639 FAX: 717-846-0871 Web: www.ib-abel.com Patrick A. Kinsley, P.E., President E-mail: [email protected] Jeffrey L. Zarfoss, CFO Willard R. Wolf, Vice President Dennis L. Geiger, Vice President James R. Trebilcock, Vice President Mary C. Linebaugh, Secretary Advantage Steel and Construction (001409) (steel erection & steel repairs) 2300 South Noah Drive Saxonburg, PA 16056 724-352-4842 FAX: 724-352-4849 Web: www.advsteel.com Brian A. Hawk, General Manager E-mail: [email protected] Amelie Construction & Supply, LLC (001959) (DBE structural construction specializing in drilling and steel/concrete erection) 725 Saxonburg Blvd., Suite #2 Saxonburg, PA 16056 724-352-4700 FAX: 724-352-4710 Web: www.amelieconstruction.com Danielle A. Proctor, President E-mail: [email protected] James J. Anderson Construction Co., Inc. (001235) (general contractor) 6958 Torresdale Avenue, Suite 200 Philadelphia, PA 19135-1999 215-331-7150 FAX: 215-332-8350 Web: www.jjaconstruction.com James J. Anderson, President Ricke C.
    [Show full text]