DOCSLIB.ORG

Collaborative Filtering: a Machine Learning Perspective by Benjamin Marlin a Thesis Submitted in Conformity with the Requirement

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Collaborative Filtering: a Machine Learning Perspective by Benjamin Marlin a Thesis Submitted in Conformity with the Requirement Collaborative Filtering: A Machine Learning Perspective by Benjamin Marlin A thesis submitted in conformity with the requirements for the degree of Master of Science Graduate Department of Computer Science University of Toronto Copyright c 2004 by Benjamin Marlin Abstract Collaborative Filtering: A Machine Learning Perspective Benjamin Marlin Master of Science Graduate Department of Computer Science University of Toronto 2004 Collaborative filtering was initially proposed as a framework for filtering information based on the preferences of users, and has since been refined in many different ways. This thesis is a comprehensive study of rating-based, pure, non-sequential collaborative filtering. We analyze existing methods for the task of rating prediction from a machine learning perspective. We show that many existing methods proposed for this task are simple applications or modifications of one or more standard machine learning methods for classification, regression, clustering, dimensionality reduction, and density estima- tion. We introduce new prediction methods in all of these classes. We introduce a new experimental procedure for testing stronger forms of generalization than has been used previously. We implement a total of nine prediction methods, and conduct large scale prediction accuracy experiments. We show interesting new results on the relative performance of these methods. ii Acknowledgements I would like to begin by thanking my supervisor Richard Zemel for introducing me to the field of collaborative filtering, for numerous helpful discussions about a multitude of models and methods, and for many constructive comments about this thesis itself. I would like to thank my second reader Sam Roweis for his thorough review of this thesis, as well as for many interesting discussions of this and other research. I would like to thank Matthew Beal for knowledgeably and enthusiastically answering more than a few queries about graphical models and variational methods. I would like to thank David Blei for our discussion of LDA and URP during his visit to the University this fall. I would also like to thank all the members of the machine learning group at the University of Toronto for comments on several presentations relating to collaborative filtering. Empirical research into collaborative filtering methods is not possible without suitable data sets, and large amounts of computer time. The empirical results presented in this thesis are based on the EachMovie and MovieLens data sets that have been generously made available for research purposes. I would like to thank the Compaq Computer Corporation for making the EachMovie data set available, and the GroupLens Research Project at the University of Minnesota for use of the MovieLens data set. I would like to thank Geoff Hinton for keeping us all well supplied with computing power. On a personal note, I would also like to thank my good friends Horst, Jenn, Josh, Kevin, Liam, and Rama for many entertaining lunches and dinners, and for making the AI lab an enjoyable place to work. I would like to thank my parents who have taught me much, but above all the value of hard work. Finally, I would like to thank my fiancee Krisztina who has given me boundless support, encouragement, and motivation. She has graciously agreed to share me with the University while I persue doctoral studies, and I thank her for that as well. iii Contents 1 Introduction 1 2 Formulations 4 2.1 A Space of Formulations . 4 2.1.1 Preference Indicators . 5 2.1.2 Additional Features . 6 2.1.3 Preference Dynamics . 7 2.2 Pure, Non-Sequential, Rating-Based Formulation . 7 2.2.1 Formal Definition . 8 2.2.2 Associated Tasks . 8 3 Fundamentals 10 3.1 Probability and Statistics . 11 3.2 Complexity Analysis . 13 3.3 Experimentation . 14 3.3.1 Experimental Protocols . 14 3.3.2 Error Measures . 15 3.3.3 Data Sets . 16 3.3.4 The Missing at Random Assumption . 18 4 Classification and Regression 20 iv 4.1 K-Nearest Neighbor Classifier . 21 4.1.1 Neighborhood-Based Rating Prediction . 22 4.1.2 Complexity . 24 4.1.3 Results . 25 4.2 Naive Bayes Classifier . 26 4.2.1 Naive Bayes Rating Prediction . 28 4.2.2 Complexity . 30 4.2.3 Results . 30 4.3 Other Classification and Regression Techniques . 31 5 Clustering 32 5.1 Standard Clustering . 33 5.1.1 Rating Prediction . 34 5.1.2 Complexity . 35 5.1.3 Results . 36 5.2 Hierarchical Clustering . 36 5.2.1 Rating Prediction . 38 6 Dimensionality Reduction 39 6.1 Singular Value Decomposition . 39 6.1.1 Weighted Low Rank Approximations . 40 6.1.2 Learning with Weighted SVD . 42 6.1.3 Rating Prediction with Weighted SVD . 43 6.1.4 Complexity . 44 6.1.5 Results . 45 6.2 Principal Components Analysis . 45 6.2.1 Rating Prediction with PCA . 46 6.3 Factor Analysis and Probabilistic PCA . 47 v 6.3.1 Rating Prediction with Probabilistic PCA . 49 7 Probabilistic Rating Models 51 7.1 The Multinomial Model . 52 7.1.1 Learning . 53 7.1.2 Rating Prediction . 53 7.1.3 Complexity . 54 7.1.4 Results . 55 7.2 Mixture of Multinomials Model . 55 7.2.1 Learning . 57 7.2.2 Rating Prediction . 60 7.2.3 Complexity . 61 7.2.4 Results . 61 7.3 The Aspect Model . 62 7.3.1 Learning . 65 7.3.2 Rating Prediction . 68 7.3.3 Complexity . 69 7.3.4 Results . 69 7.4 The User Rating Profile Model . 70 7.4.1 Variational Approximation and Free Energy . 72 7.4.2 Learning Variational Parameters . 75 7.4.3 Learning Model Parameters . 76 7.4.4 An Equivalence Between The Aspect Model and URP . 78 7.4.5 URP Rating Prediction . 80 7.4.6 Complexity . 81 7.4.7 Results . 82 7.5 The Attitude Model . 82 7.5.1 Variational Approximation and Free Energy . 85 vi 7.5.2 Learning . 86 7.5.3 Rating Prediction . 87 7.5.4 Binary Attitude Model . 88 7.5.5 Integer Attitude Model . 92 7.5.6 Complexity . 94 7.5.7 Results . 96 8 Comparison of Methods 98 8.1 Complexity . 98 8.2 Prediction Accuracy . 102 9 Conclusions 109 9.1 Summary . 109 9.2 Future Work . 111 9.2.1 Existing Models . 111 9.2.2 The Missing at Random Assumption . 113 9.2.3 Extensions to Additional Formulations . 114 9.2.4 Generalizations to Additional Applications . 116 9.3 The Last Word . 116 Bibliography 119 vii List of Tables 3.1 Each Movie and MovieLens Data Set Statistics . 17 4.1 PKNN-Predict: EachMovie Results . 26 4.2 PKNN-Predict: MovieLens Results . 26 4.3 NBClass-Predict: EachMovie Results . 31 4.4 NBClass-Predict: MovieLens Results . 31 5.1 K-Medians Clustering: EachMovie Results . 36 5.2 K-Medians Clustering: MovieLens Results . 36 6.1 wSVD-Predict: EachMovie Results . 45 6.2 wSVD-Predict: MovieLens Results . 45 7.1 MixMulti-Predict: EachMovie Results . 61 7.2 MixMulti-Predict: MovieLens Results . 61 7.3 Aspect-Predict: EachMovie Results . 70 7.4 Aspect-Predict: MovieLens Results . 70 7.5 URP: EachMovie Results . 82 7.6 URP: MovieLens Results . 82 7.7 AttBin-Predict: EachMovie Results . 97 7.8 AttBin-Predict: MovieLens Results . 97 8.1 Computational Complexity Of Learning and Prediction Methods . 99 viii 8.2 Space Complexity of Learned Representation . 101 8.3 EachMovie: Prediction Results . 103 8.4 MovieLens: Prediction Results . 103 ix List of Figures 3.1 EachMovie Rating Distributions . 18 3.2 MovieLens Rating Distributions . 18 4.1 Naive Bayes Classifier . 26 4.2 Naive.
Load more

Recommended publications
  • CRUC: Cold-Start Recommendations Using Collaborative Filtering in Internet of Things
    CRUC: Cold-start Recommendations Using Collaborative Filtering in Internet of Things Daqiang Zhanga,*, Qin Zoub, Haoyi Xiongc a School of Computer Science, Nanjing Normal University, Nanjing, Jiangsu Province, 210046,China b School of Remote Sensing and Information Engineering, Wuhan University, Wuhan, Hubei Province, 430072, China c Department of Telecommunication, Institute Telecome – Telecom SudParis, Evry, 91011, France Abstract The Internet of Things (IoT) aims at interconnecting everyday objects (including both things and users) and then using this connection information to provide customized user services. However, IoT does not work in its initial stages without adequate acquisition of user preferences. This is caused by cold-start problem that is a situation where only few users are interconnected. To this end, we propose CRUC scheme --- Cold-start Recommendations Using Collaborative Filtering in IoT, involving formulation, filtering and prediction steps. Extensive experiments over real cases and simulation have been performed to evaluate the performance of CRUC scheme. Experimental results show that CRUC efficiently solves the cold-start problem in IoT. Keywords: Cold-start Problem, Internet of Things, Collaborative Filtering 1. Introduction The Internet of Things (IoT) refers to a self-configuring network in which everyday objects are interconnected to the Internet [1] [2]. IoT deploys sensors in infrastructures (e.g., rooms and buildings) to get a heightened awareness of real-time events. It also employs sensors capturing contextual information about objects (e.g., user preferences) to achieve an enhanced situational awareness [3] [21-26]. Readings from a large number of sensors for various objects are enormous, but only a few of them are useful for a specific user.
    [Show full text]
  • Nonlinear Dimensionality Reduction by Locally Linear Embedding
    R EPORTS 2 ˆ 35. R. N. Shepard, Psychon. Bull. Rev. 1, 2 (1994). 1ÐR (DM, DY). DY is the matrix of Euclidean distanc- ing the coordinates of corresponding points {xi, yi}in 36. J. B. Tenenbaum, Adv. Neural Info. Proc. Syst. 10, 682 es in the low-dimensional embedding recovered by both spaces provided by Isomap together with stan- ˆ (1998). each algorithm. DM is each algorithm’s best estimate dard supervised learning techniques (39). 37. T. Martinetz, K. Schulten, Neural Netw. 7, 507 (1994). of the intrinsic manifold distances: for Isomap, this is 44. Supported by the Mitsubishi Electric Research Labo- 38. V. Kumar, A. Grama, A. Gupta, G. Karypis, Introduc- the graph distance matrix DG; for PCA and MDS, it is ratories, the Schlumberger Foundation, the NSF tion to Parallel Computing: Design and Analysis of the Euclidean input-space distance matrix DX (except (DBS-9021648), and the DARPA Human ID program. Algorithms (Benjamin/Cummings, Redwood City, CA, with the handwritten “2”s, where MDS uses the We thank Y. LeCun for making available the MNIST 1994), pp. 257Ð297. tangent distance). R is the standard linear correlation database and S. Roweis and L. Saul for sharing related ˆ 39. D. Beymer, T. Poggio, Science 272, 1905 (1996). coefficient, taken over all entries of DM and DY. unpublished work. For many helpful discussions, we 40. Available at www.research.att.com/ϳyann/ocr/mnist. 43. In each sequence shown, the three intermediate im- thank G. Carlsson, H. Farid, W. Freeman, T. Griffiths, 41. P. Y. Simard, Y. LeCun, J.
    [Show full text]
  • Comparison of Dimensionality Reduction Techniques on Audio Signals
    Comparison of Dimensionality Reduction Techniques on Audio Signals Tamás Pál, Dániel T. Várkonyi Eötvös Loránd University, Faculty of Informatics, Department of Data Science and Engineering, Telekom Innovation Laboratories, Budapest, Hungary {evwolcheim, varkonyid}@inf.elte.hu WWW home page: http://t-labs.elte.hu Abstract: Analysis of audio signals is widely used and this work: car horn, dog bark, engine idling, gun shot, and very effective technique in several domains like health- street music [5]. care, transportation, and agriculture. In a general process Related work is presented in Section 2, basic mathe- the output of the feature extraction method results in huge matical notation used is described in Section 3, while the number of relevant features which may be difficult to pro- different methods of the pipeline are briefly presented in cess. The number of features heavily correlates with the Section 4. Section 5 contains data about the evaluation complexity of the following machine learning method. Di- methodology, Section 6 presents the results and conclu- mensionality reduction methods have been used success- sions are formulated in Section 7. fully in recent times in machine learning to reduce com- The goal of this paper is to find a combination of feature plexity and memory usage and improve speed of following extraction and dimensionality reduction methods which ML algorithms. This paper attempts to compare the state can be most efficiently applied to audio data visualization of the art dimensionality reduction techniques as a build- in 2D and preserve inter-class relations the most. ing block of the general process and analyze the usability of these methods in visualizing large audio datasets.
    [Show full text]
  • Efficient Pseudo-Relevance Feedback Methods for Collaborative Filtering Recommendation
    Efficient Pseudo-Relevance Feedback Methods for Collaborative Filtering Recommendation Daniel Valcarce, Javier Parapar, and Álvaro Barreiro Information Retrieval Lab Computer Science Department University of A Coruña, Spain {daniel.valcarce,javierparapar,barreiro}@udc.es Abstract. Recently, Relevance-Based Language Models have been dem- onstrated as an effective Collaborative Filtering approach. Nevertheless, this family of Pseudo-Relevance Feedback techniques is computationally expensive for applying them to web-scale data. Also, they require the use of smoothing methods which need to be tuned. These facts lead us to study other similar techniques with better trade-offs between effective- ness and efficiency. Specifically, in this paper, we analyse the applicability to the recommendation task of four well-known query expansion tech- niques with multiple probability estimates. Moreover, we analyse the ef- fect of neighbourhood length and devise a new probability estimate that takes into account this property yielding better recommendation rank- ings. Finally, we find that the proposed algorithms are dramatically faster than those based on Relevance-Based Language Models, they do not have any parameter to tune (apart from the ones of the neighbourhood) and they provide a better trade-off between accuracy and diversity/novelty. Keywords: Recommender Systems, Collaborative Filtering, Query Expansion, Pseudo-Relevance Feedback. 1 Introduction Recommender systems are recognised as a key instrument to deliver relevant information to the users. Although the problem that attracts most attention in the field of Recommender Systems is accuracy, the emphasis on efficiency is increasing. We present new Collaborative Filtering (CF) algorithms. CF methods exploit the past interactions betweens items and users. Common approaches to CF are based on nearest neighbours or matrix factorisation [17].
    [Show full text]
  • Litrec Vs. Movielens a Comparative Study
    LitRec vs. Movielens A Comparative Study Paula Cristina Vaz1;4, Ricardo Ribeiro2;4 and David Martins de Matos3;4 1IST/UAL, Rua Alves Redol, 9, Lisboa, Portugal 2ISCTE-IUL, Rua Alves Redol, 9, Lisboa, Portugal 3IST, Rua Alves Redol, 9, Lisboa, Portugal 4INESC-ID, Rua Alves Redol, 9, Lisboa, Portugal fpaula.vaz, ricardo.ribeiro, [email protected] Keywords: Recommendation Systems, Data Set, Book Recommendation. Abstract: Recommendation is an important research area that relies on the availability and quality of the data sets in order to make progress. This paper presents a comparative study between Movielens, a movie recommendation data set that has been extensively used by the recommendation system research community, and LitRec, a newly created data set for content literary book recommendation, in a collaborative filtering set-up. Experiments have shown that when the number of ratings of Movielens is reduced to the level of LitRec, collaborative filtering results degrade and the use of content in hybrid approaches becomes important. 1 INTRODUCTION and (b) assess its suitability in recommendation stud- ies. The number of books, movies, and music items pub- The paper is structured as follows: Section 2 de- lished every year is increasing far more quickly than scribes related work on recommendation systems and is our ability to process it. The Internet has shortened existing data sets. In Section 3 we describe the collab- the distance between items and the common user, orative filtering (CF) algorithm and prediction equa- making items available to everyone with an Internet tion, as well as different item representation. Section connection.
    [Show full text]
  • Combining User-Based and Item-Based Collaborative Filtering Using Machine Learning
    Combining User-Based and Item-Based Collaborative Filtering Using Machine Learning Priyank Thakkar, Krunal Varma, Vijay Ukani, Sapan Mankad and Sudeep Tanwar Abstract Collaborative filtering (CF) is typically used for recommending those items to a user which other like-minded users preferred in the past. User-based collaborative filtering (UbCF) and item-based collaborative filtering (IbCF) are two types of CF with a common objective of estimating target user’s rating for the target item. This paper explores different ways of combining predictions from UbCF and IbCF with an aim of minimizing overall prediction error. In this paper, we propose an approach for combining predictions from UbCF and IbCF through multiple linear regression (MLR) and support vector regression (SVR). Results of the proposed approach are compared with the results of other fusion approaches. The comparison demonstrates the superiority of the proposed approach. All the tests are performed on a large publically available dataset. Keywords User-based collaborative filtering · Item-based collaborative filtering Machine learning · Multiple linear regression · Support vector regression P. Thakkar (B) · K. Varma · V. Ukani · S. Mankad · S. Tanwar Institute of Technology, Nirma University, Ahmedabad 382481, India e-mail: [email protected] K. Varma e-mail: [email protected] V. Ukani e-mail: [email protected] S. Mankad e-mail: [email protected] S. Tanwar e-mail: [email protected] © Springer Nature Singapore Pte Ltd. 2019 173 S. C. Satapathy and A. Joshi (eds.), Information and Communication Technology for Intelligent Systems, Smart Innovation, Systems and Technologies 107, https://doi.org/10.1007/978-981-13-1747-7_17 174 P.
    [Show full text]
  • Towards an Effective Crowdsourcing Recommendation System a Survey of the State-Of-The-Art
    2015 IEEE Symposium on Service-Oriented System Engineering Towards an Effective Crowdsourcing Recommendation System A Survey of the State-of-the-Art Eman Aldhahri, Vivek Shandilya, Sajjan Shiva Computer Science Department, The University of Memphis Memphis, USA {aldhahri, vmshndly, sshiva}@memphis.edu Abstract—Crowdsourcing is an approach where requesters can behaviors. User behavior can be interpreted as preferable call for workers with different capabilities to process a task for features.. Explicit profiles are based on asking users to complete monetary reward. With the vast amount of tasks posted every a preferable features form. Two techniques common in this area day, satisfying workers, requesters, and service providers--who are: a collaborative filtering approach (with cold start problems, are the stakeholders of any crowdsourcing system--is critical to scarcity and scalability in large datasets [6]), and a content- its success. To achieve this, the system should address three based approach (with problems due to overspecialization, etc.). objectives: (1) match the worker with a suitable task that fits the A hybrid approach that combines the two techniques has also worker’s interests and skills, and raise the worker’s rewards; (2) been used. give requesters more qualified solutions with lower cost and II. MOTIVATION time; and (3) raise the accepted tasks rate which will raise the The main objective in this study is to investigate various online aggregated commissions accordingly. For these objectives, we recommendation systems by analyzing their input parameters, present a critical study of the state-of-the-art in effectiveness, and limitations in order to assess their usage in recommendation systems that are ubiquitous among crowdsourcing systems.
    [Show full text]
  • Unsupervised Speech Representation Learning Using Wavenet Autoencoders Jan Chorowski, Ron J
    1 Unsupervised speech representation learning using WaveNet autoencoders Jan Chorowski, Ron J. Weiss, Samy Bengio, Aaron¨ van den Oord Abstract—We consider the task of unsupervised extraction speaker gender and identity, from phonetic content, properties of meaningful latent representations of speech by applying which are consistent with internal representations learned autoencoding neural networks to speech waveforms. The goal by speech recognizers [13], [14]. Such representations are is to learn a representation able to capture high level semantic content from the signal, e.g. phoneme identities, while being desired in several tasks, such as low resource automatic speech invariant to confounding low level details in the signal such as recognition (ASR), where only a small amount of labeled the underlying pitch contour or background noise. Since the training data is available. In such scenario, limited amounts learned representation is tuned to contain only phonetic content, of data may be sufficient to learn an acoustic model on the we resort to using a high capacity WaveNet decoder to infer representation discovered without supervision, but insufficient information discarded by the encoder from previous samples. Moreover, the behavior of autoencoder models depends on the to learn the acoustic model and a data representation in a fully kind of constraint that is applied to the latent representation. supervised manner [15], [16]. We compare three variants: a simple dimensionality reduction We focus on representations learned with autoencoders bottleneck, a Gaussian Variational Autoencoder (VAE), and a applied to raw waveforms and spectrogram features and discrete Vector Quantized VAE (VQ-VAE). We analyze the quality investigate the quality of learned representations on LibriSpeech of learned representations in terms of speaker independence, the ability to predict phonetic content, and the ability to accurately re- [17].
    [Show full text]
  • Unsupervised Speech Representation Learning Using Wavenet Autoencoders
    Unsupervised speech representation learning using WaveNet autoencoders https://arxiv.org/abs/1901.08810 Jan Chorowski University of Wrocław 06.06.2019 Deep Model = Hierarchy of Concepts Cat Dog … Moon Banana M. Zieler, “Visualizing and Understanding Convolutional Networks” Deep Learning history: 2006 2006: Stacked RBMs Hinton, Salakhutdinov, “Reducing the Dimensionality of Data with Neural Networks” Deep Learning history: 2012 2012: Alexnet SOTA on Imagenet Fully supervised training Deep Learning Recipe 1. Get a massive, labeled dataset 퐷 = {(푥, 푦)}: – Comp. vision: Imagenet, 1M images – Machine translation: EuroParlamanet data, CommonCrawl, several million sent. pairs – Speech recognition: 1000h (LibriSpeech), 12000h (Google Voice Search) – Question answering: SQuAD, 150k questions with human answers – … 2. Train model to maximize log 푝(푦|푥) Value of Labeled Data • Labeled data is crucial for deep learning • But labels carry little information: – Example: An ImageNet model has 30M weights, but ImageNet is about 1M images from 1000 classes Labels: 1M * 10bit = 10Mbits Raw data: (128 x 128 images): ca 500 Gbits! Value of Unlabeled Data “The brain has about 1014 synapses and we only live for about 109 seconds. So we have a lot more parameters than data. This motivates the idea that we must do a lot of unsupervised learning since the perceptual input (including proprioception) is the only place we can get 105 dimensions of constraint per second.” Geoff Hinton https://www.reddit.com/r/MachineLearning/comments/2lmo0l/ama_geoffrey_hinton/ Unsupervised learning recipe 1. Get a massive labeled dataset 퐷 = 푥 Easy, unlabeled data is nearly free 2. Train model to…??? What is the task? What is the loss function? Unsupervised learning by modeling data distribution Train the model to minimize − log 푝(푥) E.g.
    [Show full text]
  • Evaluation of Recommendation System for Sustainable E-Commerce: Accuracy, Diversity and Customer Satisfaction
    Preprints (www.preprints.org) | NOT PEER-REVIEWED | Posted: 2 January 2020 doi:10.20944/preprints202001.0015.v1 Article Evaluation of Recommendation System for Sustainable E-Commerce: Accuracy, Diversity and Customer Satisfaction Qinglong Li 1, Ilyoung Choi 2 and Jaekyeong Kim 3,* 1 Department of Social Network Science, KyungHee University, 26, Kyungheedae-ro, Dongdaemun-gu, Seoul, 02453, Republic of Korea; [email protected] 2 Graduate School of Business Administration, KyungHee University, 26, Kyungheedae-ro, Dongdaemun- gu, Seoul, 02453, Republic of Korea; [email protected] 3 School of Management, KyungHee University, 26, Kyungheedae-ro, Dongdaemun-gu, Seoul, 02453, Republic of Korea; [email protected] * Correspondence: [email protected]; Tel.: +82-2-961-9355 (F.L.) Abstract: With the development of information technology and the popularization of mobile devices, collecting various types of customer data such as purchase history or behavior patterns became possible. As the customer data being accumulated, there is a growing demand for personalized recommendation services that provide customized services to customers. Currently, global e-commerce companies offer personalized recommendation services to gain a sustainable competitive advantage. However, previous research on recommendation systems has consistently raised the issue that the accuracy of recommendation algorithms does not necessarily lead to the satisfaction of recommended service users. It also claims that customers are highly satisfied when the recommendation system recommends diverse items to them. In this study, we want to identify the factors that determine customer satisfaction when using the recommendation system which provides personalized services. To this end, we developed a recommendation system based on Deep Neural Networks (DNN) and measured the accuracy of recommendation service, the diversity of recommended items and customer satisfaction with the recommendation service.
    [Show full text]
  • Dimensionality Reduction and Principal Component Analysis
    Dimensionality Reduction Before running any ML algorithm on our data, we may want to reduce the number of features Dimensionality Reduction and Principal ● To save computer memory/disk space if the data are large. Component Analysis ● To reduce execution time. ● To visualize our data, e.g., if the dimensions can be reduced to 2 or 3. Dimensionality Reduction results in an Projecting Data - 2D to 1D approximation of the original data x2 x1 Original 50% 5% 1% Projecting Data - 2D to 1D Projecting Data - 2D to 1D x2 x2 x2 x1 x1 x1 Projecting Data - 2D to 1D Projecting Data - 2D to 1D new feature new feature Projecting Data - 3D to 2D Projecting Data - 3D to 2D Projecting Data - 3D to 2D Why such projections are effective ● In many datasets, some of the features are correlated. ● Correlated features can often be well approximated by a smaller number of new features. ● For example, consider the problem of predicting housing prices. Some of the features may be the square footage of the house, number of bedrooms, number of bathrooms, and lot size. These features are likely correlated. Principal Component Analysis (PCA) Principal Component Analysis (PCA) Suppose we want to reduce data from d dimensions to k dimensions, where d > k. Determine new basis. PCA finds k vectors onto which to project the data so that the Project data onto k axes of new projection errors are minimized. basis with largest variance. In other words, PCA finds the principal components, which offer the best approximation. PCA ≠ Linear Regression PCA Algorithm Given n data
    [Show full text]
  • Wiki-Rec: a Semantic-Based Recommendation System Using Wikipedia As an Ontology
    Wiki-Rec: A Semantic-Based Recommendation System Using Wikipedia as an Ontology Ahmed Elgohary, Hussein Nomir, Ibrahim Sabek, Mohamed Samir, Moustafa Badawy, Noha A. Yousri Computer and Systems Engineering Department Faculty of Engineering, Alexandria University Alexandria, Egypt {algoharyalex, hussein.nomier, ibrahim.sabek, m.samir.galal, moustafa.badawym}@gmail.com, [email protected] Abstract— Nowadays, satisfying user needs has become the Hybrid models try to overcome the limitations of each in main challenge in a variety of web applications. Recommender order to generate better quality recommendations. systems play a major role in that direction. However, as most Since most of the information on the web is present in a of the information is present in a textual form, recommender textual form, recommendation systems have to deal with systems face the challenge of efficiently analyzing huge huge amounts of unstructured text. Efficient text mining amounts of text. The usage of semantic-based analysis has techniques are, therefore, needed to understand documents in gained much interest in recent years. The emergence of order to extract important information. Traditional term- ontologies has yet facilitated semantic interpretation of text. based or lexical-based analysis cannot capture the underlying However, relying on an ontology for performing the semantic semantics when used on their own. That is why semantic- analysis requires too much effort to construct and maintain the based analysis approaches have been introduced [1] [2] to used ontologies. Besides, the currently known ontologies cover a small number of the world's concepts especially when a non- overcome such a limitation. The use of ontologies have also domain-specific concepts are needed.
    [Show full text]