Salience and Market-Aware Skill Extraction for Job Targeting
Total Page:16
File Type:pdf, Size:1020Kb
Salience and Market-aware Skill Extraction for Job Targeting Baoxu Shi Jaewon Yang LinkedIn Corporation, USA LinkedIn Corporation, USA [email protected] [email protected] Feng Guo Qi He LinkedIn Corporation, USA LinkedIn Corporation, USA [email protected] [email protected] ABSTRACT At LinkedIn, we want to create economic opportunity for every- one in the global workforce. To make this happen, LinkedIn offers a reactive Job Search system, and a proactive Jobs You May Be Interested In (JYMBII) system to match the best candidates with their dream jobs. One of the most challenging tasks for developing these systems is to properly extract important skill entities from job postings and then target members with matched attributes. In this work, we show that the commonly used text-based salience and Figure 1: Snippet of a machine learning engineer job posted market-agnostic skill extraction approach is sub-optimal because it on LinkedIn. Squared text are detected skill mentions. only considers skill mention and ignores the salient level of a skill and its market dynamics, i.e., the market supply and demand influ- ence on the importance of skills. To address the above drawbacks, job postings to quality applicants who are both qualified and will- we present Job2Skills, our deployed salience and market-aware skill ing to apply for the job. To serve this goal, LinkedIn offers two job extraction system. The proposed Job2Skills shows promising re- targeting systems, namely Jobs You May Be Interested In (JYM- sults in improving the online performance of job recommendation BII) [13], which proactively target jobs to the quality applicants, (JYMBII) (+1:92% job apply) and skill suggestions for job posters and Job Search [19], which reactively recommend jobs that the job (−37% suggestion rejection rate). Lastly, we present case studies seeker qualifies. to show interesting insights that contrast traditional skill recogni- Matching jobs to the quality applicants is a challenging task. Due tion method and the proposed Job2Skills from occupation, industry, to high cardinality (for LinkedIn, 645M members and more than country, and individual skill levels. Based on the above promising 20M open jobs), it is computationally intractable to define job’s results, we deployed the Job2Skills online to extract job targeting target member set by specifying individual members. For this rea- skills for all 20M job postings served at LinkedIn. son, many models match candidates by profile attributes [1, 25, 37]. At LinkedIn, we mostly use titles and skills to target candidates KEYWORDS (members). In other words, when a member comes to the job rec- job targeting, skill inference, skill recommendation ommendation page, we recommend job postings whose targeting skills (or titles) match the member’s skills (or titles). ACM Reference Format: In this paper, we study how to target job postings by identifying Baoxu Shi, Jaewon Yang, Feng Guo, and Qi He. 2020. Salience and Market- relevant skills. Given a job posting, we extract skills from the job aware Skill Extraction for Job Targeting. In Proceedings of ACM SIGKDD posting so that it can be shown to the members who have such skills. Conference on Knowledge Discovery and Data Mining (KDD’20). ACM, New York, NY, USA, 9 pages. https://doi.org/10.475/123_4 This task of mapping the job to a set of skills is very important because it determines the quality of applications for the job posting arXiv:2005.13094v1 [cs.IR] 27 May 2020 1 INTRODUCTION and affects hiring efficiency. To improve the quality of applicants, what should be the ob- LinkedIn is the world’s largest professional network whose vision jectives of extracting skills for job targeting? We argue that the is to “create economic opportunity for every member of the global extracted skills need to meet two criteria. First, the extracted skills workforce”. To achieve this vision, it is crucial for LinkedIn to match should not only be mentioned in the job posting, but also be rele- vant to the core job function. In other words, the skills should be Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed salient to the job posting. Second, the extracted skills should be for profit or commercial advantage and that copies bear this notice and the full citation able to reach out to enough number of members. In other words, on the first page. Copyrights for components of this work owned by others than ACM there should be enough supply for the skills in the job market. In must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a summary, we aim to build a machine learning model to extract fee. Request permissions from [email protected]. skills from job postings in a salience and market-aware way. KDD’20, August 2020, San Diego, California USA However, developing such salience and market-aware skill ex- © 2020 Association for Computing Machinery. ACM ISBN 123-4567-24-567/08/06...$15.00 traction model is a very challenging task. Not only because model- https://doi.org/10.475/123_4 ing the salience and market dynamic is hard, but also because the KDD’20, August 2020, San Diego, California USA B. Shi et al. lack of ground-truth data. Given a job posting, it is tricky to get way. We note that retraining existing NER models leads to sub- gold-standard labels for the skills that are both salient and have optimal results as they rely purely on the text information, ignoring strong supply. One may propose using crowdsourcing to annotate important signals such as how much supply the skill has in the job job postings, but there are two problems. First, crowdsourcing may market, how salient the skill is overall, and so on. Therefore, we not be the most cost-efficient way to collect a large amount oftrain- add signals representing the job market supplies and the salience ing examples. Second and more importantly, labeling salient and of skills in the model and significantly improve the performance in high-supply skills requires very solid domain knowledge about the the offline evaluation and online A/B test. job market and the job posting itself. As we will show later, the Present work: Product improvements. We deploy the Job2Skills in labels collected by crowdsourcing do not give us the most salient production and improve the product metrics across multiple ap- skills with the best job market supply. plications. First, we ask feedback from hiring experts in the job Figure 1 gives an illustration of the above challenge. Given a job creation systems and outperform the existing NER-based model by posting, it is relatively straightforward for human to label skills more than 30%. Second, we employ this new skill extraction model mentioned in the posting (rectangles), and an annotator in the in the job recommendation systems, one of the most important crowdsourcing platform would be able to do it. However, if the recommender systems at LinkedIn. We observe 2% member-job annotator aims to label which skills are more salient and have better interaction and 6% coverage improvement in proactive job recom- job market supply, the annotator needs to understand both the job mender (JYMBII) system. market and the job description well. For example, for the sake of Present work: Insights for Job Market. We note that Job2Skills’s market supply, the annotator should choose “deep learning” instead outputs capture the hiring trend in the job market for millions of of specific tools such as “Pytorch” or “Keras”, but this is impossible companies that post jobs at LinkedIn. Since Job2Skills is trained on if the annotator does not understand deep learning related skills the feedback from hiring experts and job markets, it learns what and the deep learning job market. On the other hand, skills like kinds of talent that each employer is trying to recruit. We argue “Communication Skill” have large supply but they are less salient that Job2Skills reveals employers’ intention better than traditional in the context of deep learning engineer jobs. information extraction methods that do not consider salience and Previous works usually treat this skill extraction task as a named market factors. We present extensive studies to showcase the power entity recognition (NER) [24] task and use the standard named of insights that can be gained with Job2Skills. For example, we show entity recognizer to identify skill mentions [19, 36]. In the example that Job2Skills’s results can forecast that Macy’s would expand a of Fig. 1, existing methods would identify all rectangles and use tech office in a new location in two months before the official them for job targeting with equal importance. This would lead to news article comes out. We also show that Job2Skills can vividly showing this deep learning job to members who have “Communi- capture the rising and fall of popularity of a certain skill. Lastly we cation Skills”. Since these methods do not consider the job market demonstrate that Job2Skills can be used to compare the required supply and salience of entities, they will lead to sub-optimal job skill sets across different regional markets. targeting performance, as we will show later. The contributions of this paper are summarized as follow: Present work: Data collection. In this paper, we tackle the prob- • lem of building salience and market-aware skill extraction model We propose a skill extraction framework to target job post- for job targeting. To collect the ground-truth labels, we note that ings by skill salience and market-awareness, which is differ- the ground-truth skills need to meet dual criteria: salience and ent from traditional entity recognition based method.