Collaborative Fashion Recommendation: a Functional Tensor Factorization Approach

Collaborative Fashion Recommendation: A Functional Tensor Factorization Approach Yang Hu†, Xi Yi‡, Larry S. Davis§ Institute for Advanced Computer Studies, University of Maryland, College Park, MD 20742 [email protected]†, [email protected]‡, [email protected]§ ABSTRACT With the rapid expansion of online shopping for fashion products, effective fashion recommendation has become an increasingly important problem. In this work, we study the problem of personalized outfit recommendation, i.e. automatically suggesting outfits to users that fit their personal (a) Users 1 fashion preferences. Unlike existing recommendation systems that usually recommend individual items, we suggest sets of items, which interact with each other, to users. We propose a functional tensor factorization method to model the interactions between user and fashion items. To effectively utilize the multi-modal features of the fashion items, we use a gradient boosting based method to learn nonlinear (b) Users 2 functions to map the feature vectors from the feature space into some low dimensional latent space. The effectiveness of the proposed algorithm is validated through extensive experiments on real world user data from a popular fashion- focused social network. Categories and Subject Descriptors (c) Users 3 H.3.3 [Information Search and Retrieval]: Retrieval models; I.2.6 [Learning]: Knowledge acquisition Figure 1: Examples of fashion sets created by three Keywords Polyvore users. Different users have different style preferences. Our task is to automatically recom- Recommendation systems; Collaborative filtering; Tensor mend outfits to users that fit their personal taste. factorization; Learning to rank; Gradient boosting 1. INTRODUCTION book3, where people showcase their personal styles and con- With the proliferation of social networks, people share al- nect to others that share similar fashion taste. With this most everything in their daily life online nowadays. They rising trend, a larger share of the fashion industry has moved share the dinners they had, the movies they watched, the online, which triggers a strong demand for intelligent fashion music they listened to, the places they visited, and also, analysis techniques. the outfits they wore. There are numerous fashion-focused Many recent works have begun to study fashion related online communities, such as Polyvore1, Chictopia2, Look- problems, e.g. clothing parsing [29, 28, 30, 7], clothing recognition [4, 13], clothing retrieval [26, 20], and clothing recom- 1http://www.polyvore.com/ 2 mendation [11, 18, 12]. In this work, we are interested in the http://www.chictopia.com/ problem of fashion recommendation, which is a key problem for promoting people’s interest and participation in online Permission to make digital or hard copies of all or part of this work for personal or shopping. There are two kinds of fashion recommendation classroom use is granted without fee provided that copies are not made or distributed problems. One is recommending whole outfits that people for profit or commercial advantage and that copies bear this notice and the full cita- may be interested in. The other is recommending fashion tion on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or re- items that make good matches with some given items. We publish, to post on servers or to redistribute to lists, requires prior specific permission mainly focus on the first problem. However the model we and/or a fee. Request permissions from [email protected]. learned can also be applied to solve the second task. MM’15, October 26–30, 2015, Brisbane, Australia. c 2015 ACM. ISBN 978-1-4503-3459-4/15/10 ...$15.00. 3 DOI: http://dx.doi.org/10.1145/2733373.2806239. http://lookbook.nu/ 129 One crucial point that has not been addressed in previ- ing, recognition, retrieval, as well as fashion recommenda- ous work is recommendation should be personal. Figure 1 tion. shows examples of fashion sets created by three users on Clothing parsing predicts pixel-wise labeling for garment Polyvore. It is obvious that different people have different items, which provides a foundation for other tasks. Several preferences of styles, which is a reflection of their ages, occu- solutions have been proposed in the literature [29, 28, 7, 30]. pations, cultural background, place of living, etc. Therefore, For clothing recognition, Chen et al. [4] described clothing for different users, an effective recommender system should appearance by semantic attributes using a CRF based ap- recommend different outfits. proach. Kiapour et al. [13] designed an online game to crowd To generate good recommendations, we need to learn from source human judgements of fashion styles and used the data people’s past behavior. The outfits a person has created or collected to train models for between-class and within-class worn reveal his or her fashion taste. However, the number classifications of styles. Clothing retrieval attempts to find of outfits we observe for a single person may be small. On similar clothing to a given query. Wang et al. [26] designed the other hand, there are a large number of outfits shared by a retrieval system by using color and attribute information. other people who have similar fashion taste. These outfits Liu et al. [20] considered the cross-scenario retrieval prob- could give a person a broader glimpse of the clothing in lem of finding similar clothing in online stores given a daily stock. Therefore, for effective recommendation, we should human photo captured in the wild. also learn from the behavior of others. This is the basic There are only a handful of attempts to solve the prob- idea of collaborative filtering, which is a common strategy lem of fashion recommendation. For item recommendation, for recommendation problems. Iwata et al. [11] proposed a probabilistic topic model for Collaborative filtering analyzes relationships between users learning information about fashion coordinates. In [12], and interdependencies among items to identify new user- Jagadeesh et al. proposed two classes of recommenders, item associations. However, previous work mostly restricted namely deterministic and stochastic fashion recommenders. attention to recommending individual items like movies and They mainly focused on color modeling for recommendation. music. We, on the other hand, want to recommend sets of Liu et al. [18] studied both outfit and item recommenda- items to users. In addition to matching user preferences, the tion problems. They designed a latent SVM based model fashion items in an outfit should also match with each other. for occasion-oriented clothing recommendation, i.e. given a Compared with movies and music, the number of fashion user-input occasion, suggest the most suitable clothing, or items is extremely huge. Each item may have only been recommend items to pair with the reference clothing. For chosen by very few or even no users. Therefore collabora- outfit recommendation, they only considered outfits with tive filtering methods that only use user-item associations two clothing items. None of these works considered the per- to characterize items are inapplicable. It is important to sonal issue, as we do. use auxiliary information about the fashion items to cap- Besides recommendation, there is increasing interest in ture the relationships amongst items. There is rich informa- building personalized models for other learning problems. tion about fashion items on the web. For example, visual Personalized tags are annotated for photos [24] and songs [23]. features related to style can be extracted from the images In [27] Weston et al. studied the collaborative retrieval prob- of the items. Sellers often provide properties of the fash- lem. Latent models were learned for recommending items ion products through text descriptions. Other information, to a user with respect to a given query. Yue et al. [31] pre- such as the price and the popularity of the items, may also sented a clustering analogue to collaborative filtering, called be available. It is important to design a model that can personalized collaborative clustering. effectively handle these heterogeneous features. Factorization techniques have shown great success in rec- With these considerations in mind, we propose a func- ommender systems. Various models have been proposed to tional tensor factorization model for fashion recommenda- factorize the user-item rating matrix [15]. To further en- tion. We decompose the high order interactions between hance the performance of the models, the use of auxiliary users and the fashion items into a set of pairwise interactions information in matrix factorization has been studied. In [1, in some latent space. We use the idea of gradient boosting 22, 27], linear functions have been used to map the feature to learn nonlinear functions to map the multi-modal feature vectors in the feature space into a latent space. Recently, vectors of the fashion items from the feature space into the Chen et al. [5] proposed using gradient boosting to automati- latent space. A learning to rank formulation is used to learn cally construct feature functions in matrix factorization. We the model. We collect real world user data from a popular extend Chen et al.’s work to tensor factorization. Also, in- fashion-focused social network. Comprehensive experiments stead of using regression, a learning to rank formulation is have been conducted to verify the effectiveness of the pro- used in our work. posed algorithm. Moving from individual items to item sets opens a new area for the studying of recommendation systems. Besides 3. COLLABORATIVE FASHION RECOM- fashion, it has potential applications to other problems. For MENDATION example, on Polyvore, users are also interested in creating collages of furniture. Using this data, the proposed method 3.1 Problem Formulation may also be used to help people design their rooms. In personalized outfit recommendation, we recommend a list of outfits, each of which consists of a set of fash- 2.

Load more