DOCSLIB.ORG

Glove: Global Vectors for Word Representation

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Glove: Global Vectors for Word Representation GloVe: Global Vectors for Word Representation Jeffrey Pennington, Richard Socher, Christopher D. Manning Computer Science Department, Stanford University, Stanford, CA 94305 [email protected], [email protected], [email protected] Abstract the finer structure of the word vector space by ex- amining not the scalar distance between word vec- Recent methods for learning vector space tors, but rather their various dimensions of dif- representations of words have succeeded ference. For example, the analogy “king is to in capturing fine-grained semantic and queen as man is to woman” should be encoded syntactic regularities using vector arith- in the vector space by the vector equation king − metic, but the origin of these regularities queen = man − woman. This evaluation scheme has remained opaque. We analyze and favors models that produce dimensions of mean- make explicit the model properties needed ing, thereby capturing the multi-clustering idea of for such regularities to emerge in word distributed representations (Bengio, 2009). vectors. The result is a new global log- bilinear regression model that combines The two main model families for learning word the advantages of the two major model vectors are: 1) global matrix factorization meth- families in the literature: global matrix ods, such as latent semantic analysis (LSA) (Deer- factorization and local context window wester et al., 1990) and 2) local context window methods. Our model efficiently leverages methods, such as the skip-gram model of Mikolov statistical information by training only on et al. (2013c). Currently, both families suffer sig- the nonzero elements in a word-word co- nificant drawbacks. While methods like LSA ef- occurrence matrix, rather than on the en- ficiently leverage statistical information, they do tire sparse matrix or on individual context relatively poorly on the word analogy task, indi- windows in a large corpus. The model pro- cating a sub-optimal vector space structure. Meth- duces a vector space with meaningful sub- ods like skip-gram may do better on the analogy structure, as evidenced by its performance task, but they poorly utilize the statistics of the cor- of 75% on a recent word analogy task. It pus since they train on separate local context win- also outperforms related models on simi- dows instead of on global co-occurrence counts. larity tasks and named entity recognition. In this work, we analyze the model properties necessary to produce linear directions of meaning 1 Introduction and argue that global log-bilinear regression mod- Semantic vector space models of language repre- els are appropriate for doing so. We propose a spe- sent each word with a real-valued vector. These cific weighted least squares model that trains on vectors can be used as features in a variety of ap- global word-word co-occurrence counts and thus plications, such as information retrieval (Manning makes efficient use of statistics. The model pro- et al., 2008), document classification (Sebastiani, duces a word vector space with meaningful sub- 2002), question answering (Tellex et al., 2003), structure, as evidenced by its state-of-the-art per- named entity recognition (Turian et al., 2010), and formance of 75% accuracy on the word analogy parsing (Socher et al., 2013). dataset. We also demonstrate that our methods Most word vector methods rely on the distance outperform other current methods on several word or angle between pairs of word vectors as the pri- similarity tasks, and also on a common named en- mary method for evaluating the intrinsic quality tity recognition (NER) benchmark. of such a set of word representations. Recently, We provide the source code for the model as Mikolov et al. (2013c) introduced a new evalua- well as trained word vectors at http://nlp. tion scheme based on word analogies that probes stanford.edu/projects/glove/. 2 Related Work the way for Collobert et al. (2011) to use the full context of a word for learning the word represen- Matrix Factorization Methods. Matrix factor- tations, rather than just the preceding context as is ization methods for generating low-dimensional the case with language models. word representations have roots stretching as far Recently, the importance of the full neural net- back as LSA. These methods utilize low-rank ap- work structure for learning useful word repre- proximations to decompose large matrices that sentations has been called into question. The capture statistical information about a corpus. The skip-gram and continuous bag-of-words (CBOW) particular type of information captured by such models of Mikolov et al. (2013a) propose a sim- matrices varies by application. In LSA, the ma- ple single-layer architecture based on the inner trices are of “term-document” type, i.e., the rows product between two word vectors. Mnih and correspond to words or terms, and the columns Kavukcuoglu (2013) also proposed closely-related correspond to different documents in the corpus. vector log-bilinear models, vLBL and ivLBL, and In contrast, the Hyperspace Analogue to Language Levy et al. (2014) proposed explicit word embed- (HAL) (Lund and Burgess, 1996), for example, dings based on a PPMI metric. utilizes matrices of “term-term” type, i.e., the rows In the skip-gram and ivLBL models, the objec- and columns correspond to words and the entries tive is to predict a word’s context given the word correspond to the number of times a given word itself, whereas the objective in the CBOW and occurs in the context of another given word. vLBL models is to predict a word given its con- A main problem with HAL and related meth- text. Through evaluation on a word analogy task, ods is that the most frequent words contribute a these models demonstrated the capacity to learn disproportionate amount to the similarity measure: linguistic patterns as linear relationships between the number of times two words co-occur with the the word vectors. or and, for example, will have a large effect on Unlike the matrix factorization methods, the their similarity despite conveying relatively little shallow window-based methods suffer from the about their semantic relatedness. A number of disadvantage that they do not operate directly on techniques exist that addresses this shortcoming of the co-occurrence statistics of the corpus. Instead, HAL, such as the COALS method (Rohde et al., these models scan context windows across the en- 2006), in which the co-occurrence matrix is first tire corpus, which fails to take advantage of the transformed by an entropy- or correlation-based vast amount of repetition in the data. normalization. An advantage of this type of trans- formation is that the raw co-occurrence counts, 3 The GloVe Model which for a reasonably sized corpus might span 8 or 9 orders of magnitude, are compressed so as The statistics of word occurrences in a corpus is to be distributed more evenly in a smaller inter- the primary source of information available to all val. A variety of newer models also pursue this unsupervised methods for learning word represen- approach, including a study (Bullinaria and Levy, tations, and although many such methods now ex- 2007) that indicates that positive pointwise mu- ist, the question still remains as to how meaning tual information (PPMI) is a good transformation. is generated from these statistics, and how the re- More recently, a square root type transformation sulting word vectors might represent that meaning. in the form of Hellinger PCA (HPCA) (Lebret and In this section, we shed some light on this ques- Collobert, 2014) has been suggested as an effec- tion. We use our insights to construct a new model tive way of learning word representations. for word representation which we call GloVe, for Shallow Window-Based Methods. Another Global Vectors, because the global corpus statis- approach is to learn word representations that aid tics are captured directly by the model. in making predictions within local context win- First we establish some notation. Let the matrix dows. For example, Bengio et al. (2003) intro- of word-word co-occurrence counts be denoted by duced a model that learns word vector representa- X, whose entries Xi j tabulate the number of times tions as part of a simple neural network architec- word j occurs in the context of word i. Let Xi = P ture for language modeling. Collobert and Weston k Xik be the number of times any word appears (2008) decoupled the word vector training from in the context of word i. Finally, let Pi j = P( jji) = the downstream training objectives, which paved Xi j =Xi be the probability that word j appear in the Table 1: Co-occurrence probabilities for target words ice and steam with selected context words from a 6 billion token corpus. Only in the ratio does noise from non-discriminative words like water and fashion cancel out, so that large values (much greater than 1) correlate well with properties specific to ice, and small values (much less than 1) correlate well with properties specific of steam. Probability and Ratio k = solid k = gas k = water k = fashion P(kjice) 1:9 × 10−4 6:6 × 10−5 3:0 × 10−3 1:7 × 10−5 P(kjsteam) 2:2 × 10−5 7:8 × 10−4 2:2 × 10−3 1:8 × 10−5 P(kjice)=P(kjsteam) 8:9 8:5 × 10−2 1:36 0:96 context of word i. the information present the ratio Pik =Pjk in the We begin with a simple example that showcases word vector space. Since vector spaces are inher- how certain aspects of meaning can be extracted ently linear structures, the most natural way to do directly from co-occurrence probabilities. Con- this is with vector differences. With this aim, we sider two words i and j that exhibit a particular as- can restrict our consideration to those functions F pect of interest; for concreteness, suppose we are that depend only on the difference of the two target interested in the concept of thermodynamic phase, words, modifying Eqn.
Load more

Recommended publications
  • Classifying Relevant Social Media Posts During Disasters Using Ensemble of Domain-Agnostic and Domain-Specific Word Embeddings
    AAAI 2019 Fall Symposium Series: AI for Social Good 1 Classifying Relevant Social Media Posts During Disasters Using Ensemble of Domain-agnostic and Domain-specific Word Embeddings Ganesh Nalluru, Rahul Pandey, Hemant Purohit Volgenau School of Engineering, George Mason University Fairfax, VA, 22030 fgn, rpandey4, [email protected] Abstract to provide relevant intelligence inputs to the decision-makers (Hughes and Palen 2012). However, due to the burstiness of The use of social media as a means of communication has the incoming stream of social media data during the time of significantly increased over recent years. There is a plethora of information flow over the different topics of discussion, emergencies, it is really hard to filter relevant information which is widespread across different domains. The ease of given the limited number of emergency service personnel information sharing has increased noisy data being induced (Castillo 2016). Therefore, there is a need to automatically along with the relevant data stream. Finding such relevant filter out relevant posts from the pile of noisy data coming data is important, especially when we are dealing with a time- from the unconventional information channel of social media critical domain like disasters. It is also more important to filter in a real-time setting. the relevant data in a real-time setting to timely process and Our contribution: We provide a generalizable classification leverage the information for decision support. framework to classify relevant social media posts for emer- However, the short text and sometimes ungrammatical nature gency services. We introduce the framework in the Method of social media data challenge the extraction of contextual section, including the description of domain-agnostic and information cues, which could help differentiate relevant vs.
    [Show full text]
  • Injection of Automatically Selected Dbpedia Subjects in Electronic
    Injection of Automatically Selected DBpedia Subjects in Electronic Medical Records to boost Hospitalization Prediction Raphaël Gazzotti, Catherine Faron Zucker, Fabien Gandon, Virginie Lacroix-Hugues, David Darmon To cite this version: Raphaël Gazzotti, Catherine Faron Zucker, Fabien Gandon, Virginie Lacroix-Hugues, David Darmon. Injection of Automatically Selected DBpedia Subjects in Electronic Medical Records to boost Hos- pitalization Prediction. SAC 2020 - 35th ACM/SIGAPP Symposium On Applied Computing, Mar 2020, Brno, Czech Republic. 10.1145/3341105.3373932. hal-02389918 HAL Id: hal-02389918 https://hal.archives-ouvertes.fr/hal-02389918 Submitted on 16 Dec 2019 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Injection of Automatically Selected DBpedia Subjects in Electronic Medical Records to boost Hospitalization Prediction Raphaël Gazzotti Catherine Faron-Zucker Fabien Gandon Université Côte d’Azur, Inria, CNRS, Université Côte d’Azur, Inria, CNRS, Inria, Université Côte d’Azur, CNRS, I3S, Sophia-Antipolis, France I3S, Sophia-Antipolis, France
    [Show full text]
  • Information Extraction Based on Named Entity for Tourism Corpus
    Information Extraction based on Named Entity for Tourism Corpus Chantana Chantrapornchai Aphisit Tunsakul Dept. of Computer Engineering Dept. of Computer Engineering Faculty of Engineering Faculty of Engineering Kasetsart University Kasetsart University Bangkok, Thailand Bangkok, Thailand [email protected] [email protected] Abstract— Tourism information is scattered around nowa- The ontology is extracted based on HTML web structure, days. To search for the information, it is usually time consuming and the corpus is based on WordNet. For these approaches, to browse through the results from search engine, select and the time consuming process is the annotation which is to view the details of each accommodation. In this paper, we present a methodology to extract particular information from annotate the type of name entity. In this paper, we target at full text returned from the search engine to facilitate the users. the tourism domain, and aim to extract particular information Then, the users can specifically look to the desired relevant helping for ontology data acquisition. information. The approach can be used for the same task in We present the framework for the given named entity ex- other domains. The main steps are 1) building training data traction. Starting from the web information scraping process, and 2) building recognition model. First, the tourism data is gathered and the vocabularies are built. The raw corpus is used the data are selected based on the HTML tag for corpus to train for creating vocabulary embedding. Also, it is used building. The data is used for model creation for automatic for creating annotated data.
    [Show full text]
  • A Second Look at Word Embeddings W4705: Natural Language Processing
    A second look at word embeddings W4705: Natural Language Processing Fei-Tzin Lee October 23, 2019 Fei-Tzin Lee Word embeddings October 23, 2019 1 / 39 Overview Last time... • Distributional representations (SVD) • word2vec • Analogy performance Fei-Tzin Lee Word embeddings October 23, 2019 2 / 39 Overview This time • Homework-related topics • Non-homework topics Fei-Tzin Lee Word embeddings October 23, 2019 3 / 39 Overview Outline 1 GloVe 2 How good are our embeddings? 3 The math behind the models? 4 Word embeddings, new and improved Fei-Tzin Lee Word embeddings October 23, 2019 4 / 39 GloVe Outline 1 GloVe 2 How good are our embeddings? 3 The math behind the models? 4 Word embeddings, new and improved Fei-Tzin Lee Word embeddings October 23, 2019 5 / 39 GloVe A recap of GloVe Our motivation: • SVD places too much emphasis on unimportant matrix entries • word2vec never gets to look at global co-occurrence statistics Can we create a new model that balances the strengths of both of these to better express linear structure? Fei-Tzin Lee Word embeddings October 23, 2019 6 / 39 GloVe Setting We'll use more or less the same setting as in previous models: • A corpus D • A word vocabulary V from which both target and context words are drawn • A co-occurrence matrix Mij , from which we can calculate conditional probabilities Pij = Mij =Mi Fei-Tzin Lee Word embeddings October 23, 2019 7 / 39 GloVe The GloVe objective, in overview Idea: we want to capture not individual probabilities of co-occurrence, but ratios of co-occurrence probability between pairs (wi ; wk ) and (wj ; wk ).
    [Show full text]
  • Cw2vec: Learning Chinese Word Embeddings with Stroke N-Gram Information
    The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18) cw2vec: Learning Chinese Word Embeddings with Stroke n-gram Information Shaosheng Cao,1,2 Wei Lu,2 Jun Zhou,1 Xiaolong Li1 1 AI Department, Ant Financial Services Group 2 Singapore University of Technology and Design {shaosheng.css, jun.zhoujun, xl.li}@antfin.com [email protected] Abstract We propose cw2vec, a novel method for learning Chinese word embeddings. It is based on our observation that ex- ploiting stroke-level information is crucial for improving the learning of Chinese word embeddings. Specifically, we de- sign a minimalist approach to exploit such features, by us- ing stroke n-grams, which capture semantic and morpholog- ical level information of Chinese words. Through qualita- tive analysis, we demonstrate that our model is able to ex- Figure 1: Radical v.s. components v.s. stroke n-gram tract semantic information that cannot be captured by exist- ing methods. Empirical results on the word similarity, word analogy, text classification and named entity recognition tasks show that the proposed approach consistently outperforms Bojanowski et al. 2016; Cao and Lu 2017). While these ap- state-of-the-art approaches such as word-based word2vec and proaches were shown effective, they largely focused on Eu- GloVe, character-based CWE, component-based JWE and ropean languages such as English, Spanish and German that pixel-based GWE. employ the Latin script in their writing system. Therefore the methods developed are not directly applicable to lan- guages such as Chinese that employ a completely different 1. Introduction writing system. Word representation learning has recently received a sig- In Chinese, each word typically consists of less charac- nificant amount of attention in the field of natural lan- ters than English1, where each character conveys fruitful se- guage processing (NLP).
    [Show full text]
  • Effects of Pre-Trained Word Embeddings on Text-Based Deception Detection
    2020 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress Effects of Pre-trained Word Embeddings on Text-based Deception Detection David Nam, Jerin Yasmin, Farhana Zulkernine School of Computing Queen’s University Kingston, Canada Email: {david.nam, jerin.yasmin, farhana.zulkernine}@queensu.ca Abstract—With e-commerce transforming the way in which be wary of any deception that the reviews might present. individuals and businesses conduct trades, online reviews have Deception can be defined as a dishonest or illegal method become a great source of information among consumers. With in which misleading information is intentionally conveyed 93% of shoppers relying on online reviews to make their purchasing decisions, the credibility of reviews should be to others for a specific gain [2]. strongly considered. While detecting deceptive text has proven Companies or certain individuals may attempt to deceive to be a challenge for humans to detect, it has been shown consumers for various reasons to influence their decisions that machines can be better at distinguishing between truthful about purchasing a product or a service. While the motives and deceptive online information by applying pattern analysis for deception may be hard to determine, it is clear why on a large amount of data. In this work, we look at the use of several popular pre-trained word embeddings (Word2Vec, businesses may have a vested interest in producing deceptive GloVe, fastText) with deep neural network models (CNN, reviews about their products.
    [Show full text]
  • Smart Ubiquitous Chatbot for COVID-19 Assistance with Deep Learning Sentiment Analysis Model During and After Quarantine
    Smart Ubiquitous Chatbot for COVID-19 Assistance with Deep learning Sentiment Analysis Model during and after quarantine Nourchène Ouerhani ( [email protected] ) University of Manouba, National School of Computer Sciences, RIADI Laboratory, 2010, Manouba, Tunisia Ahmed Maalel University of Manouba, National School of Computer Sciences, RIADI Laboratory, 2010, Manouba, Tunisia Henda Ben Ghézala University of Manouba, National School of Computer Sciences, RIADI Laboratory, 2010, Manouba, Tunisia Soulaymen Chouri Vialytics Lautenschlagerstr Research Article Keywords: Chatbot, COVID-19, Natural Language Processing, Deep learning, Mental Health, Ubiquity Posted Date: June 25th, 2020 DOI: https://doi.org/10.21203/rs.3.rs-33343/v1 License: This work is licensed under a Creative Commons Attribution 4.0 International License. Read Full License Noname manuscript No. (will be inserted by the editor) Smart Ubiquitous Chatbot for COVID-19 Assistance with Deep learning Sentiment Analysis Model during and after quarantine Nourch`ene Ouerhani · Ahmed Maalel · Henda Ben Gh´ezela · Soulaymen Chouri Received: date / Accepted: date Abstract The huge number of deaths caused by the posed method is a ubiquitous healthcare service that is novel pandemic COVID-19, which can affect anyone presented by its four interdependent modules: Informa- of any sex, age and socio-demographic status in the tion Understanding Module (IUM) in which the NLP is world, presents a serious threat for humanity and so- done, Data Collector Module (DCM) that collect user’s ciety. At this point, there are two types of citizens, non-confidential information to be used later by the Ac- those oblivious of this contagious disaster’s danger that tion Generator Module (AGM) that generates the chat- could be one of the causes of its spread, and those who bots answers which are managed through its three sub- show erratic or even turbulent behavior since fear and modules.
    [Show full text]
  • Easyboot Glove Soft, Easyshoe (Unlimited Miles Per Week)
    EasyCare Inc. The Ultimate in Hoof Protection 2018 Product Lineup www.easycareinc.com PERFORMANCE Easyboot Easyboot Easyboot Glove Glove Soft Epic Easyboot Easyboot Easyboot LC Glue-On Flip Flop PLEASURE Easyboot Easyboot Easyboot Back Country Trail Original Trail Original Easyboot New Old Mac’s G2 Easyboot Mini Mac THERAPY Easyboot Easyboot Easyboot Rx Cloud Stratus Therapy Click Easyboot EasySoaker System Zip EASYSHOE NEW EasyShoe EasyShoe EasyShoe Flex Performance Performance N/G EasyShoe EasyShoe Sport Compete 2 The Performance line is made for aggressive riding, long distance and speed. This line offers low profile, secure fitting hoof protection, designed to reduce interference and allow for increased training time. The Performance line offers both boots and glue-on hoof protection. Contributing to more top finishes at the world’s toughest endurance races than any other brand, EasyCare’s performance line has proven to hold up under the most challenging conditions. Our Performance line includes the 2016 Glove, Glove Soft, Epic, Glue-On, LC and Flip Flop. The Pleasure line-up is built with easy application and removal in mind. Any boot in this category is ideal for trail riding, as a replacement for a lost shoe or protection for a sore-footed horse. Our Pleasure line includes the Trail, Trail Original, New Mac, Old Mac’s G2, Back Country, Mini and Original Easyboot. The Therapy line offer a range of innovative therapeutic boot for comfort, support, rehab and medicating. If your horse suffers from chronic lameness, founder, laminitis, thin soles, abscesses, shipping stress or any other hoof and lower limb problems, we have a boot that will provide instant and ongoing relief.
    [Show full text]
  • A Comparison of Word Embeddings and N-Gram Models for Dbpedia Type and Invalid Entity Detection †
    information Article A Comparison of Word Embeddings and N-gram Models for DBpedia Type and Invalid Entity Detection † Hanqing Zhou *, Amal Zouaq and Diana Inkpen School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa ON K1N 6N5, Canada; [email protected] (A.Z.); [email protected] (D.I.) * Correspondence: [email protected]; Tel.: +1-613-562-5800 † This paper is an extended version of our conference paper: Hanqing Zhou, Amal Zouaq, and Diana Inkpen. DBpedia Entity Type Detection using Entity Embeddings and N-Gram Models. In Proceedings of the International Conference on Knowledge Engineering and Semantic Web (KESW 2017), Szczecin, Poland, 8–10 November 2017, pp. 309–322. Received: 6 November 2018; Accepted: 20 December 2018; Published: 25 December 2018 Abstract: This article presents and evaluates a method for the detection of DBpedia types and entities that can be used for knowledge base completion and maintenance. This method compares entity embeddings with traditional N-gram models coupled with clustering and classification. We tackle two challenges: (a) the detection of entity types, which can be used to detect invalid DBpedia types and assign DBpedia types for type-less entities; and (b) the detection of invalid entities in the resource description of a DBpedia entity. Our results show that entity embeddings outperform n-gram models for type and entity detection and can contribute to the improvement of DBpedia’s quality, maintenance, and evolution. Keywords: semantic web; DBpedia; entity embedding; n-grams; type identification; entity identification; data mining; machine learning 1. Introduction The Semantic Web is defined by Berners-Lee et al.
    [Show full text]
  • Linked Data Triples Enhance Document Relevance Classification
    applied sciences Article Linked Data Triples Enhance Document Relevance Classification Dinesh Nagumothu * , Peter W. Eklund , Bahadorreza Ofoghi and Mohamed Reda Bouadjenek School of Information Technology, Deakin University, Geelong, VIC 3220, Australia; [email protected] (P.W.E.); [email protected] (B.O.); [email protected] (M.R.B.) * Correspondence: [email protected] Abstract: Standardized approaches to relevance classification in information retrieval use generative statistical models to identify the presence or absence of certain topics that might make a document relevant to the searcher. These approaches have been used to better predict relevance on the basis of what the document is “about”, rather than a simple-minded analysis of the bag of words contained within the document. In more recent times, this idea has been extended by using pre-trained deep learning models and text representations, such as GloVe or BERT. These use an external corpus as a knowledge-base that conditions the model to help predict what a document is about. This paper adopts a hybrid approach that leverages the structure of knowledge embedded in a corpus. In particular, the paper reports on experiments where linked data triples (subject-predicate-object), constructed from natural language elements are derived from deep learning. These are evaluated as additional latent semantic features for a relevant document classifier in a customized news- feed website. The research is a synthesis of current thinking in deep learning models in NLP and information retrieval and the predicate structure used in semantic web research. Our experiments Citation: Nagumothu, D.; Eklund, indicate that linked data triples increased the F-score of the baseline GloVe representations by 6% P.W.; Ofoghi, B.; Bouadjenek, M.R.
    [Show full text]
  • United States Patent (19) 11) Patent Number: 5,025,502 Raymond Et Al
    United States Patent (19) 11) Patent Number: 5,025,502 Raymond et al. (45) Date of Patent: Jun. 25, 1991 (54) PUFF-OFF GLOVE 4,099,270 7/1978 Jabour . 4,195,365 4/1980 Eyman et al. ...................... 2/161 A (76) Inventors: Douglas W. Raymond, 23 Martha 4,304,008 12/1981 Joung. Rd., Orinda, Calif. 94563; Thomas W. 4,345,335 8/1982 Shih-Li............................... 2/161 A Smith, 60 Orinda Cir., Pittsburg, 4,499,154 2/1985 James . Calif. 94565 4,503,565 3/1985 Lipatt . 21) Appl. No.: 408,805 FOREIGN PATENT DOCUMENTS 22 Filed: Sep. 18, 1989 04.584 7/1938 Australia ................................... 2A18 3619180 12/1987 Fed. Rep. of Germany ... 2/DIG. 1 51) Int. C. ............................................. A41D 19/00 872596 7/1961 United Kingdom ................ 2/2.1 R (52) U.S.C. ........................................ 2/159; 2/161 A 58 Field of Search ........ 2/2.1 R, 16, 18, 159-161 A, Primary Examiner-Allan N. Shoap 2/161 R, 164, DIG. 1, DIG. 3, 159, 161 A; Assistant Examiner-Scott Cummings 223/211 Attorney, Agent, or Firm-Schapp and Hatch (56) References Cited 57 ABSTRACT U.S. PATENT DOCUMENTS A glove is equipped with a mouthpiece containing an 600,779 3/1898 Frazier ...................................... 2/8 air passage. The mouthpiece allows the wearer to puff 1,938,685 12/1933 Breuls et al. ........................ 22.3/11 air into the space between the glove and the skin of the 1,996,377 4/1935 Hinchen .............................. 223/111 hand. The puffed up glove is much easier to doff and 2,842,771 7/1958 Foti ................................
    [Show full text]
  • Jacket Socks Hat Sweater Coat Shirt
    Adult Learner ESOL CURRICULUM ESOL Program jacket coat socks hat shirt sweater Adult Learner ESOL CURRICULUM ESOL Program skirt shoes pants scarf sneakers dress Adult Learner ESOL CURRICULUM ESOL Program gloves belt T-shirt underwear bathing suit suit Clothes Quiz 1. Name 2 pieces of clothing starting 11. What is worn by women and is with the letter P. very short? (see picture above) 2. Which item of clothing has 12. Name something made of rubber ‘fingers’? worn on your feet. 3. Name a popular hat that is also 13. What do you wear on your neck used in a ball sport. when it’s cold? 4. Give another name for a woman’s 14. Name three pieces of clothing shirt. (begins with b) starting with the letter T. 5. Name 5 things you could wear to 15. What do you call the plastic or go to the beach. metal thing you can hang clothes on? (see picture above) 6. What do you call a glove that has 16. Name 3 pieces of clothing that no fingers? are often made of leather. 7. Name five things a businessman 17. What do you wear for bed? would wear to work. 8. What do you call the ‘strings’ used 18. Name 3 things that can be made to tie your shoes? of wool? 9. You find these on a shirt. They’re 19. Shirts can have short or long used to ‘close’ the shirt. ones. What are they? 10. I keep you warm. I usually have a 20. Name 5 items of clothing women zipper I’m short wear but men don’t.
    [Show full text]