sign in sign up
<<
Home , Artificial leather, Bandage dress, Bermuda shorts, Blouson, Court shoe, Crew neck

Towards an Interoperable Data Protocol Aimed at Linking the Industry with AI Companies

Mohammed Al-Rawi Joeran Beel [email protected] [email protected] ADAPT Centre, Trinity College Dublin ADAPT Centre, Trinity College Dublin Dublin, Ireland Dublin, Ireland

Wholesale / Retail ddoif Customer ddoif Manufacturer - 1 Store - 2 ddoif

User APP

ddoif ddoif Online Retailer - 1 Online Retailer - 2

Clothing Manufacturer - 2 Store - 1 ddoif Customer ddoif ddoif ddoif ddoif ddoif

Fashion Design Company-2 Search Engine ddoif ddoif = Data of a Fashion ddoif Company - 1 ddoif item in ddoif exchange ddoif protocol AI Hub format Data Centre

Figure 1: Business workflow that makes use of the Digital Data Exchange and Organisation in Fashion (DDOIF) protocol. There are many scenarios that DDOIF provides help with, for example; (1) the DDOIF file is generated at the clothing design stage and may be shipped with the item to the retailer and then to the customers app; (2) AI companies can either directly obtain clothing items in DDOIF format from manufacturers / retailers or use the DDOIF protocol to annotate their own data; (3) customers can use handset apps that host their electronic wardrobe items in DDOIF format, e.g., helping them picking the right clothing and style automatically; and (4) search engines can use the rich information that DDOIF files possess to provide accurate search results. See use-cases for more details. ABSTRACT among other things, functions for importing and exporting a file The fashion industry is looking forward to use artificial intelligence built according to the DDOIF protocol that stores all information technologies to enhance their processes, services, and applications. about a single item of clothing. To this end, we identified over 1000 class and subclass names used to name fashion items and use them

arXiv:2009.03005v1 [cs.CY] 7 Sep 2020 Although the amount of fashion data currently in use is increasing, there is a large gap in data exchange between the fashion industry to build the DDOIF dictionary. We make DDOIF publicly available and the related AI companies, not to mention the different structure to all interested users and developers and look forward to engaging used for each fashion dataset. As a result, AI companies are relying more collaborators to improve and enrich it. on manually annotated fashion data to build different applications. Furthermore, as of this writing, the terminology, vocabulary and KEYWORDS methods of data representation used to denote fashion items are Clothing, Artificial Intelligence, Data Exchange, Recommender Sys- still ambiguous and confusing. Hence, it is clear that the fashion tems, Personalized Fashion, ICT Standards industry and AI companies will benefit from a protocol that allows them to exchange and organise fashion information in a unified way. To achieve this goal we aim (1) to define a protocol called DDOIF 1 INTRODUCTION that will allow interoperability of fashion data; (2) for DDOIF to Nowadays, the fashion industry and related retail industries are contain diverse entities including extensive information on clothing racing to develop applications to serve them and their customers. and accessories attributes in the form of text and various media This is usually done with the help of AI companies to provide differ- formats; and (3)To design and implement an API that includes, ent services; for example, recommender systems, AI based fashion 1 Mohammed Al-Rawi and Joeran Beel

ddoif V0.1 material media info clothing color post-processing # of media brand solid histogram media types barcode vest heel horiz-stripes hsv-cluster wrinkle quality city vertic-stripes color-names resolution country -tee flowery color-degradation shred fake-leather device date ... grid color-standard rip fur location manufacturer mens-pant strech zebra ... description womens-pant accessory ...... hawaiian ...... anatomy ... fabric pesonal-char bag sustainability care annotation age recycled washing skirt caption skin-color grade bleaching t-shirt bounding-box ethnic-group mens- slow-fashion ironinig ... cord mask size-and-fitting jewllery fast-fashion dry-cleaning ... fleece 3D- primary-lang ... organic water-temp preferred-color ......

Figure 2: DDOIF classes. Due to lack of space, entries of classes are truncated. Best viewed in color, as colors mimic connections between classes shown in subsequent figures. design, applications enabling sustainability and slow fashion, trend interoperability and the use of artificial intelligence (AI) in fashion forecasting, sizing and fitting, and brand protection3 [ , 13, 14]. In systems. AI companies involved in fashion have identified this data addition, organizations are heavily involved in research related to limitation problem without a clear view of how to solve it [21]. fashion aiming to develop different applications, e.g., to retrieve a In this work we aim to (1) design a Digital Data Exchange and particular fashion item based on a text query of clothing attributes Organisation in Fashion (DDOIF) protocol; (2) collect and organise and/or user preferences [3, 10, 12, 18, 27]. The impact of deep learn- the fashion vocabulary and terminology into DDOIF dictionary; (3) ing and the development of other AI methods provide companies create relevant API (DDOIF API) that enables handling, importing with unprecedented means to achieve their goals. Amazon and and exporting the fashion data according to DDOIF; and (4) enable StitchFix are providing their customers with the so called "Personal the API to encapsulate fashion data of a single item into a single file. Styling" shopping, which is semi-assisted by fashion stylists [5, 6]. A generic business workflow making use of DDOIF is illustrated in Facebook is building a universal product understanding system Figure 1, where DDOIF files can be shipped to different stakeholders. where fashion is at its core [3, 4]. Zalando’s researchers proposed in It is clear that the information one piece of clothing might contain [17] a model for finding pieces of clothing worn by a person in full- is immense. The main classes that DDOIF may contain are depicted body or half-body images with neutral backgrounds. Some other in Figure 2. It may therefore be impractical to use crowdsourcing companies are dedicated to fashion sizing and fitting [9, 24, 26]. to non-uniformly, and possibly inaccurately, annotate items that However, it is unfortunate that these companies rely solely on AI have already been fabricated according to very specific details and to predict the features of fashion items that were already present attributes. This is because all features and annotations are available during the design stage. These AI companies have to allocate non- at the design stage. Shipping these attributes in DDOIF format with trivial resources to forecast features that have been eliminated after the item will save a lot of time and effort, and will provide more design. accurate item description. Moreover, a protocol that can be used to The fashion industry, which now enjoy enormous benefits from build AI based applications should contain vital information about the information boom, still lack a proper data exchange format and each item; e.g., vocabulary and terminology, media, visual attributes, relevant software that enable them exchange data interoperably. In design patterns and colours, material, type and texture of the fabric, fact, the fashion industry is currently using a diversity of in-house / colour and style matching criteria, recommendations according to proprietary data structures to identify and describe fashion articles fashion stylists, customer preferences, description, outfit type (e.g., and the vocabulary and terminology used to label their data are outdoor, casual, ), etc. The existence of such information incongruous. While this is understandable due to the descriptive will help the industry build much better services that are accurate complexity of the fashion items, this diversity hinders cooperation and robust. within and between companies, complicates and constrains data

2 Towards an Interoperable Data Protocol Aimed at Linking the Fashion Industry with AI Companies

Part 1. Introduction and Overview

Part 15. Security Part 20. Imaging Part 14. Grayscale Part 16. Content Part 17. Part 18. Part 19. Part 13. Print and System Reports using HL7 Standard Display Mapping Explanatory Web Application (Retired) Management Clinical Document Function Resource Information Services Hosting Profiles Architecture

Part 4. Service Class Specifications

Part 3. - Information Object Definitions Part 6. Data Dictionary

Part 6. Data Structure and Encoding Part 2.Conformance Part 7. Message Exchange

Part 8. Network Part 10. Media Part 9. Point to Point Part 11. Media Part 21. Transformations Part 22. Communication Storage and File Coummunication Storage Application between DICOM and Real-Time Support for Format for Media (Retired) Profiles other Representations Communication Message Exchange Interchange

Figure 3: DICOM Structure.

2 LITERATURE REVIEW is DICONDE (Standard Practice for Digital Imaging and Communi- 1. The way AI companies build applications. AI companies cation in Nondestructive Evaluation) of the American Society for build applications via a data-driven approach. Their main first Testing and Materials [8]. To provide an example of the complexity step is to build a fashion dataset by annotating the fashion im- of ICT standards, the DICOM standard contains more than 4,000 ages [10, 12, 18, 27]. The annotation is usually done manually to tags and corresponding values. Figure 3 illustrates the structure of segment the images and / or define their main attributes using some DICOM and the relationship between its parts. crowdsourcing platform, such as the Amazon Mechanical Turk [7]. To our knowledge, there is no ICT standard or protocol that Although these annotated datasets provide good benchmarks for comprehensively deals with fashion terminology, vocabulary, and implementing AI fashion research and other related applications, media formats. There are, however, a few ISO standards for the the annotation is restricted to a very limited clothing categories, garment industry, to name a few: (1) ISO/TC 133 Clothing Sizing costly, and prone to human error. Moreover, the naming and cate- Systems - Size Designation, Size Measurement Methods and Digital gories of fashion items across different datasets are not and Fittings [19]; and (2) ISO 18163:2016 Clothing âĂŤ Digital fittings may vary. Nonetheless, the exact and accurate naming and clothing âĂŤ Vocabulary and Terminology used for the Virtual Garment [15]. attributes were already available at the time of design and/or man- ISO/TC 133 is a standardization of a system of size designations ufacturing. Unfortunately. the information that the manufacturer resulting from the establishment of one or more sizing systems sends to retailers is not only limited, but also not uniform across for clothes. It is based on size designation, body size measurement different manufacturers because it does not follow a unified proto- methods for clothing and for digital garment fitting. ISO 18163:2016 col. It would be highly beneficial to standardise this information specifies the data attributes and formats required for the creation and ship it along the clothing item to the retailers, and customers’ of virtual garments, facilitating clear and synchronized communi- apps as well. cation of terminology. The major advantage of this standard is that 2. The current situation of fashion standards. ICT standards it supports online consumers, fashion designers, manufacturers are clearly very useful in many areas, and the fashion community and retailers who have an interest in the style and fit of apparel. can benefit in a similar way. Among the many useful ICT standards While ISO/TC 133 is only dedicated to sizing and fitting, the short- there is the JPEG standard ([22]; Rec. ITU T.81 | ISO/IEC 10918), ages of ISO 18163:2016 and ISO/TC 133, which may be closest to which is one of the most successful multimedia standards to date. the protocol that we aim to propose lacks, among other things, The JPEG standard (the Joint Photographic Experts Group) speci- detailed subclass names and anatomy of garments. Both standards fies the basic encoding technology and includes several options for lack associating the vocabulary and attributes to media formats; encoding images, compliance testing, defining a set of extensions and importantly, how these items can be connected to the emerging to the coding technologies, specifying the compression types, file AI applications. Moreover, these ISO are just guiding standards exchange format, registration authorities, etc. Another relevant and not like the DICOM or DICONDE that provide comprehensive ICT standard is the DICOM standard [16, 20] (Digital Imaging and solutions to the related industries. Having some ICT protocols or Communications in Medicine). DICOM aims to store, retrieve, print, standards for fashion analogous to DICOM or DICONDE will result process, and display medical imaging information; thus, making in a very big impact on the fashion industry. medical imaging information interoperable. Another ICT standard

3 Mohammed Al-Rawi and Joeran Beel

and so on. For illustration, first-level classes are shown in Figure 2 material (the colored field names), second-level classesi.e ( . subclasses) are Acrylic Llama also listed under each first-level class. Then in Figure 7, we demon- Alpaca Merino strate the clothing class down to the third-level. As an example, the Angora clothing class (first-level) has subclasses Dress and Skirt (second- Asbestos Nylon Camel Poly Cotton level). Then, Dress class has third-level classes according to the Cashgora Polymers type name of , e.g. A-line Dress and Dress, etc. If Cashmere Polypropylene / Elastane the A-line dress class has some distinctive features, it will generate Cotton other subclasses. Hence, each level m entry in the DDOIF dictionary Diolen Qiviut Artificial Fur Ryon / Artificial Silk has the following building block: Fiberglass Silk class − levelm Fur class1 − levelm−1 Glass fibre Stretch Cotton class − level − 2 m 1 Lambswool ... Leather Vicuna classi − levelm−1 / Flax Visocse ... classn − levelm−1 descriptionm Figure 4: Material class. where class −levelm, which is the parent class, contains all the pos- sible child / sub classes in addition to ’description’ that may contain fabric some information about class − levelm. Each (child / sub) class is then entitled to act as a parent class, and so on. We collected several Denim post-processing Jersey fashion class and subclass names from many Internet sources, e.g., Ribbed Bleach fashion style and online retail sites [2, 11, 23, 25]. We also used ISO Softshell 18163:2016 [15] to ensure that the main classes comply with ISO, Hardshell Pleat Lace Rinse if available and/or needed. It must be noted that ISO 18163:2016 Fleece Rip only and partially supports first-level classes. We designate all first- Chiffon Shred Strech Blends level classes in lowercase letters, otherwise, the first letters of the Tin Cord Wash subclasses are capitalized. We then built the DDOIF dictionary ac- Wrinkle cording to the structure described above using YAML, JSON, and Needlecord Cotton and Blends XML. Future releases will contain additional entries by dynamically adding more classes as they appear, depending on fashion evolution. Figure 6: Post- We present in Figures 2, 4, 5, 6, 7, 8, and 9 some DDOIF classes. Processing class. All classes can be accessed in [1]. These classes give an idea about Figure 5: Fabric class. 85 classes list the terminology and vocabulary that DDOIF uses. In all diagrams, truncated due to lack of space. we use colors to mimic connections between main and sub classes. Interestingly, the number of classes of women’s clothing items are 3 THE DDOIF PROTOCOL much higher than those of men, see Figure 7 pink versus light DDOIF facilitates the interchange of information between applica- classes. This accords with women’s interest and passion for fashion. tions in the fashion industry environments by specifying: This is also clear in Figure 10 where the frequency distribution of classes is well presented. • Naming: The vocabulary and terminology of fashion articles, and relevant AI annotations. • Format: Fashion items characteristics and associated media 3.2 DDOIF API formats. The API is a software used to enable importing and exporting We therefore conceptualize DDOIF by defining its terminology fashion items according to the DDOIF protocol. Consequently, we and vocabulary structure in one part (DDOIF Dictionary) and a designed the DDOIF file as follows: file structure (DDOIF File) that encapsulates a fashion item based on the dictionary part. While the first part can be used by auser 3.2.1 DDOIF File Header. We designed the DDOIF file to start with as a reference to ensure fashion items are correctly named and a 10-byte signature (i.e. Magic Number or Magic Bytes) containing structured, the second part can be used to exchange information the following: between different parties. (1) 0x89 has the high bit set to detect systems that do not support one byte transmission data. 3.1 DDOIF Dictionary (2) 0x44 ’0x44’ ’0x4f’ ’0x49’ ’0x46’ in ASCII, the letters We defined the fashion item classes into levels; the first-level is DDOIF enable identifying the format easily if it is viewed in the main class, the second-level is a subclass from the first-level, a text editor. 4 Towards an Interoperable Data Protocol Aimed at Linking the Fashion Industry with AI Companies

clothing womens-pant

vest jacket vest Baggy Pants womens-shorts dress Bell Bottoms Travel Vest Anorak Jacket jacket Capris Pants Shorts A-line (Aline) Dress Vest Barbour Ashby Jacket Board Shorts Apron Dress Vest Barbour Beaufort Jacket Dress pants Booty Shorts Asymmetrical Dress Quilted Vest Barbour Bedale Jacket Dungarees Pants Baby doll Dress Shooting Vest Biker Jacket sweatshirt Eastern Pants Boy Shorts Ballgown Dress Fleece Vest Jacket Fatigue Capri Boyfriend Shorts Balloon Dress Polo Vest Jacket sweater Cargo Shorts Seven Casual Vest Bomber Jacket Chino Shorts Bardot Dress Tuxedo Vest Crombie Coat womens-pant Jodhpur Pants off Shorts Bell-Sleeve Dress Research Vest Cropped Jacket Jamaican Shorts Blazer Dress Fishing Vest Denim Jacket womens-shorts / Knee hugger Shorts Blouson Dress Puffed Vest Drawstring Jacket Palazzos Pants Knee length Shorts Bodycon Dress Hunting Vest Hooded Jacket dress Peg-Leg Pants Pedal pusher Shorts Bouffant Dress Long Vest Long Bomber Jacket Punk Pants Slipshort Dress Hooded Vest skirt Sailor Pants Swimming trunk Shorts Dress High Neck Vest Stirrup Pants Toreador Shorts Coat Dress Fringe Vest Jacket top-tee Stovepipe / drainpipe Shorts pants Dress Puffer /Down Jacket Debutante Dress sweater gown Sweat Pants skirt Quilted Jacket Denim Dress Toreador Pants Dress Boyfriend Sweater Jacket A-Line Skirt Draped Dress Sweater Trousers Pants Accordion Skirt Sport Jacket romper Empire Dress Sweater Jacket Asymmetrical Skirt Empire line Dress Grandpa Sweater Asymmetrical Skirt Track Jacket mens-shirt gown Fit and flare Dress Half-Zip Sweater Skirt Gathered Dress Mock Turtleneck Sweate Skirt Trucker Jacket mens-pant A-Line Gown Granny Dress Raglan Sleeve Sweater Tuxedo Suit Bell Shaped Skirt Dress Roll Neck Sweater Asymmetrical Gown Bondage Skirt t-shirt Ball Gown Handkerchief Hem Dress Set-in Sleeve Sweater Broomstick Skirt Handkerchief hem Dress Collar Sweater hoodie Bias cut Gown Bubble Skirt mens-shorts Bouffant skirt Gown Harem Dress Bustled Skirt High-Low Dress Turtleneck Sweater Baja Hoodie Gown Skirt Black Hoodie Circular skirt Gown Kaftan Dress Circular / Circle Skirt Dress Designer Hoodie top-tee Full Skirt Gown Cowl Skirt Fur Hoodie Mermaid Gown Culottes Skirt Long Sleeve Dress Over-the-Head Hoodie Asymmetrical Top Panel Gown Polo Hoodie Sheath Gown Low or drop waist Dress Bardot / Off-Shoulder T Divided Skirt Maxi Dress Skate Hoodie Batwing (Magyar/ Dolm Tiered Gown Draped / -Draped S Skull Hoodie Mermaid Silhouette Dress Bralette Top Drindl Skirt Midi Dress Slim-Fit Hoodie Boxy Top Fitted / Pencil / Tube Skirt Zip-Up Hoodie romper Mini Dress Blouson Top Fix Box Skirt Off Shoulder Dress Top Back Cut Out Romper Flared Skirt Off the Shoulder mens-shorts Bustier Top Backless Romper Skirt One shoulder Dress Camisole Top Bandage Romper Skirt Party Dress Cape Top Romper Gored Skirt Peasant Dress Board Shorts Cardigans & Bardot Romper High Waisted Skirt Pencil Dress Cargo Shorts Top Beach Romper Peplum Dress Chino Shorts Corset Top Beaded Romper Inverted Pleat Skirt Pillowcase Dress Collar Top Knife Pleated Skirt Denim Shorts mens-pant Butterfly Sleeve Rompe Dress Flat Front Shorts Cold-shouldered Top Caged Romper Layered Skirt Pinafore / Jumber Dress Draped Blouse Empire Maxi Skirt Leather Shorts Khakis Pants Cami Romper Pouf Dress Flashdance Top Mermaid Skirt Plaid Shorts Pleated Pants Cargo Romper Princess Silhouette Dress Halter Top Micro Mini Skirt Pleated Shorts Cartographer Romper Princess Dress Henley Neck Top Mini Skirt Chinos Pants Choker Romper Qipao Dress High Low Top Paneled Skirt Tennis Shorts Flat Fronts Pants Cold-shoulder Romper Kaftan Top Colorblock Romper Peasant Skirt Sheath Dress Keyhole Top Contemporary Romper Pegged Skirt Shift Dress mens-shirt t-shirt Lace Up Top Criss Cross Romper Pelmet Skirt Shirt Dress Layered Top Crochet Romper Shirt waist Dress Aloha Shirt Basic Half Sleeve T-Shirt Maxi Top Cross Back Romper Peplum Skirt Skater Dress Baseball Shirt Long Sleeve Crew Neck T-Shir One Shoulder Top Cut Out Elbows Rompe Pleated Skirt Dress Camp Shirt Polo Collar T-Shirt Peasant Top Denim Romper Ruffled Skirt Smock Dress Casual Shirt V-neck T-Shirt Top Dip Hem Skirt Romper Sarong Skirt Sports Shirt Douche-Bag / Scoop Neck T-S Peplum Top Dress Style Romper Sheath Skirt Sun Dress Henley Collar T-Shirt Top Fluted Sleeve Romper Skater Skirt Sun Dress Epaulette Shirt Baseball T-Shirt Raglan Sleeve Top Frill Romper Skirt Sweater Dress / Lumberjack S Raglan / Long Sleeve T-Shirt Spaghetti Top (fabric) Romp Straight Skirt Swing Dress Granddad Shirt Turtle Neck T-Shirt Sweatshirt Top Halter Neck Romper Tiered /Gypsy Skirt T-shirt Dress Golf Shirt Ringer T-Shirt Swing Top High Neck Romper Trouser Skirt Tea Length Dress Sleeve T-Shirt Shirt Top Kimono Sleeve Rompe Trumpet Skirt Ivy league Shirt Half (Singlet) T-Shirt Smock Top Lace Romper Tulip Skirt Trumpet / Mermaid Dress Shirt Muscle T-Shirt Shell Top Lace Up Romper Skirt Tube Dress Jersey Shirt Half Muscle T-Shirt Tie Front Top Lace Up Sleeve Romp Skirt Dress Night Shirt Longline T-Shirt Tank Top Leather Romper Under Skirt Tutu Dress Western Shirt T-Shirt Maxi Romper Up Skirt Wrap around Dress Baby Doll T-Shirt Tunic Top Off-shoulder Romper Wrap / Wrap-Around Skirt Xray Dress SweatShirt Wide Neck off Shoulder T-Shirt T-Shirt / Surplice Top One Shoulder Romper Wrap Skirt Dress Tunic Shirt Yoke Neck T-Shirt X-ray Top Overlay Romper Yoke Skirt Tuxedo / Formal Shirt Yoke Top Peplum Romper

Figure 7: Clothing Classes. Romper items are truncated due to lack of space. Best viewed in color, as colors mimic connections between classes. The full list can be accessed in [1].

5 Mohammed Al-Rawi and Joeran Beel

footwear annatomy

shoe collar shoe neckline boot neckline collar Athletic Shoe Ballet Flats pocket sandal heel Asymmetric Neckline Ascot Collar Bateau / Boat-Neck Neckline Barbett Collar vent heel Ankle Strap Heel Collared Neckline Bateau Collar Bootie Shoe Heel Collared Neckline Bertha Round Collar Boyfriend Shoe Basic Pumps Block Heel Cowl Neckline Bertha Square Collar Chunky Heel Crew Neckline Bib With Frill Collar sandal sleeve Comma Heel Crossover Halter Style Necklin Bishop Collar Cone Heel Florentine Neckline Boat Collar Flat Sandal slit Corset Heel Grecian Neckline Butterfly Collar Leather Sandal D'Orsay Shoe Cuban Heel Halter Neckline Button Down Collar Metallic Sandal keypoints Shoe Cut Out Heel Halter Strap Neckline Button Down Collar Sandal Flip-flops Decorative Heel High Neck Illusion Neckline Cape Collar Rubber Sandal opening Gladiator Shoe Espadrille Heel Jewel Neckline Carcaille Collar Tatami Sandal High-Top Shoe Fantasy Heel Keyhole Neckline Cascade Collar Sandal overlap Loafer Flare Heel Notch Neckline Chinese V-Neck Collar Wood Sandal Off Shoulder / Bardot Neckline Clover Mary Janes Shoe Flatform Heel French Heel Pentagon Neckline Convertible Closed Collar Portrait Neckline Convertible Open Collar boot High Heel Queen Anne Neckline Cossack Collar Motorcycle Shoe High Heel Sabrina Neckline Cowl Collar Mules Scoop Neckline Danton Collar Lace-Up Heel Semi Sweetheart Neckline keypoints Detachable Collars Peep-Toe Shoe Louis / Pompadour Neckline Dog Ear Collar Crotch-High Boot Medium Heel Square Neckline Armpit Left Double Collar Go-Go Boot Peep Toe Heel Straight Neckline Armpit Right Dutch Collar Pumps Shoe Gumboots Platform Heel Strapless Neckline Bottom Left In Embellished Collar Platform Ankle Boo Sculptural Heel Sweetheart Neckline Bottom Left Out Embroidered Collar Shoe Paddock Boot Slingback Heel V Neck Wedding Band Necklin Bottom Right In Eton Collar Slip-on Platform Boot Bottom Right Out Falling Band Collar Rain Boot Stacked Heel Center Front Fichu Collar Soccer Shoe sleeve Crotch Fish Mouth Revers Sport Shoe Slouchy Boot T-Strap Heel Cuff Left In Flape Collar Toe Shoe Wedge Heel Angel / Dalmation Sleeve Cuff Left Out Flat Collar Bag Sleeve Cuff Right In Flip Collar Balloon Sleeve Cuff Right Out Flower Petal Collar Batwing Sleeve Left Frilled Stand Collar Figure 8: Footwear Classes. Best viewed in color, as colors Bell Sleeve Hemline Right Giraffe Collar Bishop Sleeve Neckline Left Gorget Collar mimic connections between classes. Bracelet / Three-Fourth Sleev Neckline Right Graphic Collar Butterfly Sleeve Shoulder Left Half Collar Cap Sleeve Shoulder Right Italian Collar Cap-Extended Sleeve Top Hem Left Johnny Collar Cape Sleeve Top Hem Right L-Revers Collar (3) 0x0D 0x0A a DOS- ending (CRLF) to detect DOS- Circle Sleeve / Flutter Sleeve Left Lace Collar Cold-shouldered Sleeve Waistband Right Mandarin Collar Unix line ending conversion of the data. / Magyar Sleeve Left Modesty Collar (4) 0x1A A byte that stops display of the file under DOS when the Draped Sleeve Waistline Right Napoleon High Coat Collar Drawstring Puff Sleeve Nehru / Mao / Cadet Collar Notch Collar command type has been usedâĂŤthe end-of-file character. Elbow Patched Sleeve vent Epaulet Sleeve Notch Shawl Collar (5) 0x0A A Unix-style line ending (LF) to detect Unix-DOS line Ornamented Collar Frill / Flute Sleeve Centre Vent Collar ending conversion. Gauntlet Sleeve (Pointed) Side / Double Vent Gibson Girl Sleeve No Vent Hanging Sleeve Pierrot Collar 3.2.2 Reserved bytes. 16 bytes are reserved for future additions, in Juliet Sleeve Polo/ Roll/ Turtle Collar case needed. Currently filled with 0 values. Kimono Sleeve ruffle Portrait Collar Lantern Sleeve Puritan Collar / Pilgrim Collar Leg Of Mutton Sleeve Single Edge Ruffles Collar 3.2.3 Data Chunks. We designed DDOIF file in a manner that Long Bell Sleeve Double Edge Ruffles Revers Collar Mahoitres Sleeve Circular Ruffles Ring Collar encapsulates textual data denoting fashion attributes, media types Man's Shirt Sleeve With Cuff Double Width Ruffles Rolled Collar of a single piece of clothing and the other information according to Marmaluke / Virago Sleeve Water Fall Ruffles Round Bertha Collar Melon Sleeve Rounded Wing Tipped Collar Normal Sleeve Collar the DDOIF dictionary. The textual data chunc consists of: cuff Over Sleeve Sailor Collar Padded Shouldered Sleeve Shaped Shawl Collar (1) 4 bytes store the length in bytes (let the length be denoted by Convertible Square Cuff Pagoda Sleeve Shawl Collar French Cuff N) of the DDOIF textual description according to the DDOIF Peasant Sleeve Shirt Collar Long One Button Barrel Cuf Petal /Tulip / Lapped Sleeve Side Collar dictionary. One Button Barrel Cuff Poet Sleeve Stand Collar With A Knot One Button Mitered Cuff Puff Sleeve Straight Band Collar (2) N bytes store the classes and attributes of the item according Rounded French Cuff Raglan Sleeve Tab Collar Soft Long One Button Cuff to the DDOIF dictionary. Slashed Sleeve Tie / Bow Collar Soft One Button Cuff Square Armhole Sleeve Turtleneck Collar Soft Square Two Button Cuf It is well known that the design of fashion items may articulate Strapped Or Banded Sleeve Tuxedo Collar Two Button Mitered Cuff Tailored Sleeve Van Dyke Collar different types of media data; for example, 3D objects, 2D images at Two Button Rounded Cuff different views (front, back, side, etc.) and even videos. Each media format will constitute a data chunk in the DDOIF file. Consequently, Figure 9: Anatomy Classes. Best viewed in color, as colors we store the media data chunks sequentially, and each will contain mimic connections between classes. the following stream of bytes: (1) 8 bytes store the format name (i.e type) of the media, e.g., JPG / JPEG , PNG, TIFF, STL, OBJ, 3DS, etc. This string will 6 Towards an Interoperable Data Protocol Aimed at Linking the Fashion Industry with AI Companies

4 USE CASES In addition to interoperability, DDOIF can be used in several fashion use-cases, including but not limited to:

Figure 10: DDOIF Clothing, Accessory, and Anatomy Name • Cloud Illustration. We only show the highest frequencies. Generating datasets with rich annotations. The amount of Colors have no meaning here other than providing a good details, classes and attributes included in a DDOIF file will visualisation. enable AI companies take advantage and build novel fine- grained classification systems that can be applied to solve different problems. This is because DDOIF not only usesa unified fashion dictionary and highly structured file, butcan be needed to know the type of stored media and to decode it also have a vintage approach as a bottom-up hierarchy that once it is read. We choose 8 bytes instead of 4 to enable future starts from the early stages of fashion design. As an exam- extension to formats that may have more than 4 letters. ple, AI companies usually use fashion datasets containing (2) 4 bytes to store the length of the Media Buffer, let it be images. There are two cases where fashion data are anno- denoted as M. tated; the first is for images scrapped from (or provided by) (3) M bytes store the media buffer. We refer to the encoded online retailers, such as Amazon [12]; the second is for im- media data as "Media Buffer". The Media Buffer is a stream ages scrapped from social media or uploaded by some users of bytes that may be encoded into a compressed format. as in [27]. Although our approach aims at encapsulating (4) 4 bytes store the CRC (cyclic redundancy code/checksum. fashion data into a single file, it cant still be used to provide The CRC is a network-byte-order CRC-32 computed over accurate annotation. For the social media example the user the Media Buffer. can (1) grab the DDOIF information from a dedicated mobile Whenever there are several media data, even of different formats, we / desktop app, or (2) scan the barcode or QR-code attached store them in a sequential manner each having the aforementioned to the piece of clothing and the DDOIF information can then structure. be imported from a DDOIF centralized data centre. • 3.2.4 DDOIF filename extension. We add the ".ddof" extension to Improved design, styling and development of fashion items. any DDOIF filename. Fashion design knowledge and items will be available at a For illustration, we used a set of images of a sport jacket at different large scale. This will help stylists and designers make the views and conditions to populate a single DDOIF file. Via DDOIF best of use of the technology, and dedicated fashion design API, we stored all these images into as single file in DDOIF format, software will also make use of the technology. • along with classes defining attributes of the jacket taken from Clothing online retrieval based on text and / or images DDOIF dictionary. The file jacket.ddof is made publicly available queries. Compared to the simple annotation currently in at https://github.com/morawi/ddoif/tree/master/Examples and any use, the rich context, based on vast clothing attributes that user can use the DDOIF API to load it. We designed the file format we are proposing in DDOIF, will enable AI companies build in a way that accepts an unlimited number of media data. better predicting systems. • Fashion data archiving and retrieval. Interoperability is the most important aspect here, and archiving and retrieval will be spread across relevant sectors and other interested parties. • Fashion evolution analysis. Storing fashion items in DDOIF format will enable using them in a longitudinal manner if backward compatibility is maintained. The analysis involves repeated observations of the same variables over short or long periods of time. This way, one can investigate how those fashion items have changed over time, which can be helpful for different applications. • Recommender Systems. This is one of the ultimate applica- tions that can make use of DDOIF format, as the latter will Figure 11: Sport jacket at different views and conditions. All provide comprehensive, interoperable fashion data that are these images have been stored into (a single file) DDOIF structured according to a standardised protocol. format along with classes defining attributes of the jacket • Fashion forensics and brand protection. We propose fashion taken from DDOIF dictionary. See jacket.ddof in https:// forensics as a mean to be used for investigating theft of github.com/morawi/ddoif/tree/master/Examples. fashion intellectual properties, which to our knowledge first time to be proposed. The aim of these tools is to enhance the role of intellectual property rights in the fashion business. 7 Mohammed Al-Rawi and Joeran Beel

5 DISCUSSION AND CONCLUSIONS styles, these tags do not work well in the application scenarios such 5.1 Data interoperability as product recommendation – consumers do not care about these tags (they are more driven by visual contents). DDOIF provides The protocol we propose will have a significant impact on Data a practical solutions to this problem as the tags are generated at Management and Software Interoperability used in fashion. It will the design phase, and shipped in DDOIF format to the customers enhance the capability of different fashion companies to exchange along with the clothing item. Tags can be retrieved via bar code that data via a common set of exchange formats, to read and write the accesses some central database, or a tiny chip shipped with the item. same file format, and to use the same protocols. That said, itwill Hence, each customer will possess the clothing item along with help the fashion industry and the IT companies build platforms that all the tags, and she/he can use these tags in application scenarios, have the ability to collect, categorise, store fashion data and provide as depicted in Fig. 1. Importantly, the protocol is not intended for different tools useful to marketers. Hence, not only that DDOIF customers per se, but also for fashion and AI companies that provide aims to provide interoperability within and between the fashion enhanced services for customers. industry and third-party companies, but it can also be used to solve the problem of fashion annotation as well as providing accurate and 5.5 Usefulness of information collected from rich annotations that can never be achieved through crowdsourc- customers ing. DDOIF dictionary ensures portability, completeness, efficiency, comparability, compatibility, interchangeability, freedom of legal For recommender systems, DDOIF tags, alongside the visual con- restrictions, easiness, and integrity. We make DDOIF publicly avail- tent and media of the items, will provide rich information. However, able in [1] (MIT License). many fashion related businesses have recently been interested in studying how consumers perceive fashion items. They are especially 5.2 Impact on the fashion market interested in the tags related to subjective feeling (e.g., whether a shoe looks modern, fantastic or outdated). Companies usually col- The protocol will also encourage building a central fashion data lect this information from customers. Such information can also be management platforms, giving the stakeholders, e.g., marketers, fed to recommender system learning and update, as these informa- executives, designers, stylists, etc., the most accurate business and tion will be updated to DDOIF via the feedback tags. Retailers have customer information available. Moreover, it will not only pro- already developed applications to collect such personal preferences, vide an opportunity improvement of fashion data collection and see [5, 6] for details. exchange, but also interoperability across different platforms and fashion industries that are making use of the AI boom. This, to 5.6 Future aspects our knowledge, is the first effort aimed at drafting such a protocol for fashion data. We therefore rely on the relevant scientific and Research user community, which includes industry and academia, professional community to provide feedback and contribute to im- is heavily involved in fashion AI. Their interest in DDOIF is likely proving, modifying, and updating this living document and tool. to be prominent and grow further in the next few years. This is We are thus keen and open to receive cooperation from the fashion clearly expected as the the fashion industry, which dominates the online market, is a huge business currently worth around US$2.4 industry and other interested parties to review, enrich and expand 1 DDOIF. trillion . Although DDOIF can help with physical shopping via handset apps, it will make a huge difference to online shoppers as more than 2.14 billion people worldwide are expected to purchase 5.3 Fashion data standardisation goods and services online. As we aim to boost fashion data organisation and exchange, DDOIF will also enable non-trivial research activity aligned with the infor- ACKNOWLEDGMENTS mation and knowledge management. The rich textual and media This work has received funding from the Elite-S project which information it provides will make knowledge representation and is co-funded by ADAPT Centre, Trinity College Dublin and the reasoning from fashion data possible. As a resource, DDOIF will European UnionâĂŹs Horizon 2020 programme under the Marie simultaneously provide powerful research for fashion AI and foster SkÅĆodowska-Curie Grant Agreement No. 801522. completely new areas as well. As a living tool, dictionary and API, we plan to curate and update DDOIF; hopefully, it will one day REFERENCES become an ICT standard. This is a long way to go as it heavily [1] Mohammed Al-rawi. 2020. ddoif - Digital Data Organization and Exchange depend on collaboration of many partners from both the fashion in Fashion. ADAPT, Trinity College Dublin. Retrieved May 26, 2020 from industry, AI companies and academia. It is not hard to conjecture https://github.com/morawi/ddoif that DDOIF will be beneficial for a very long time, as it relates to [2] Amazon. 2020. Amazon Fashion. Amazon.com, Inc. Retrieved May 26, 2020 from https://www.amazon.com/fashion prominent human needs, clothes and accessories. [3] Sean Bel, Yiqun Liu, Sami Alsheikh, Yina Tang, Ed Pizzi, M. Henning, Karun Singh, Omkar Parkhi, and Fedor Borisyuk. 2020. GrokNet: Unified Computer Vision Model Trunk and Embeddings For Commerce. Retrieved May 25, 2020 5.4 How customers will benefit from DDOIF from https://ai.facebook.com/research/publications/groknet-unified-computer- vision-model-trunk-and-embeddings-for-commerce/ What we suggest might be questioned whether users would find [4] Tamara Berg, Sean Bell, Manohar Paluri, Andrei Chtcherbatchenko, Harry Chen, this protocol useful in real applications. It is really a challenging Francis Ge, and Bo Yin. 2020. Powered by AI: Advancing product understanding task to tag fashion products such as clothes at the retailer end. Even if clothes are tagged with their materials, colour or general 1These numbers were collected before COVID-19. 8 Towards an Interoperable Data Protocol Aimed at Linking the Fashion Industry with AI Companies

and building new shopping experiences. Retrieved July 22, 2020 from https: [16] Charles E. Kahn, John A. Carrino, Michael Joseph Flynn, Donald J. Peck, and //tinyurl.com/yyomybc9 Steven C. Horii. 2007. DICOM and radiology: past, present, and future. Journal [5] Bethany Biron. 2019. Amazon launched a new personal-styling service that works of the American College of Radiology : JACR 4 9 (2007), 652–7. a lot like Fix. Retrieved July 22, 2020 from https://tinyurl.com/yy86ajhj [17] Julia Lasserre, Christian Bracher, and Roland Vollgraf. 2018. Street2Fashion2Shop: [6] Stich Fix co. 2020. Personal Styling for Everybody. Retrieved August 22, 2020 Enabling Visual Search in Fashion e-Commerce Using Studio Images. In Pattern from https://www.stitchfix.com/ Recognition Applications and Methods - 7th International Conference, ICPRAM 2018, [7] Kevin Crowston. 2012. Amazon Mechanical Turk: A Research Tool for Organiza- January 16-18, 2018, Revised Selected Papers (Lecture Notes in Computer Science), tions and Information Systems Scholars. In Shaping the Future of ICT Research. Maria De Marsico, Gabriella Sanniti di Baja, and Ana L. N. Fred (Eds.), Vol. 11351. Methods and Approaches, Anol Bhattacherjee and Brian Fitzgerald (Eds.). Springer Springer, Madeira, Portugal, 3–26. https://doi.org/10.1007/978-3-030-05499-1_1 Berlin Heidelberg, Berlin, Heidelberg, 210–221. [18] Xiaodan Liang, Liang Lin, Wei Yang, Ping Luo, Junshi Huang, and Shuicheng [8] ASTM E2339. 2016. Standard Practice for Digital Imaging and Communication Yan. 2016. Clothes Co-Parsing Via Joint Image Segmentation and Labeling With in Nondestructive Evaluation (DICONDE). American Society for Testing and Application to Clothing Retrieval. IEEE Transactions on Multimedia 18 (2016), Materials. Retrieved May 10, 2020 from https://www.astm.org/Standards/E2339. 1175–1186. htm [19] et. al. Mr Jinchun Yang. 2016. Clothing sizing systems - size designation, size mea- [9] EasySize. 2020. Retrieved July 21, 2020 from https://www.easysize.me/ surement methods and digital fittings. International Organization for Standardiza- [10] Yuying Ge, Ruimao Zhang, Xiaogang Wang, Xiaoou Tang, and Ping Luo. 2019. tion. Retrieved May 4, 2020 from https://www.iso.org/committee/52374.html DeepFashion2: A Versatile Benchmark for Detection, Pose Estimation, Segmenta- [20] NEMA. 2020. NEMA PS3 / ISO 12052, Digital Imaging and Communications in tion and Re-Identification of Clothing Images. In The IEEE Conference on Computer Medicine (DICOM) Standard. National Electrical Manufacturers Association, Vision and Pattern Recognition (CVPR). Computer Vision Foundation / IEEE, Long Rosslyn, VA, USA. Retrieved May 19, 2020 from http://www.dicomstandard.org Beach, California, United States, 5337–5345. [21] Charles Ollion. 2018. Why computer vision APIs wonâĂŹt do the trick for vertical- [11] Google. 2020. Googel Art and Culture. Google LLC. Retrieved May 22, 2020 from ized applications. HeuritechâĂŹs take in Fashion. Zalando SEs. Retrieved May 22, https://artsandculture.google.com/project/fashion-making-of 2020 from https://bit.ly/3dcCsT6 [12] Xiaoxiao Guo, Hui Qing Wu, Yupeng Gao, Steven J. Rennie, and Rogério Schmidt [22] William B. Pennebaker and Joan L. Mitchell. 1992. JPEG Still Image Data Com- Feris. 2019. The Fashion IQ Dataset: Retrieving Images by Combining Side pression Standard. Van Nostrand Reinhold, New York. Information and Relative Natural Language Feedback. ArXiv abs/1905.12794 [23] PopOptiq. 2020. PopOptiq Fashion. PopOptiq. Retrieved May 26, 2020 from (2019), 00–00. https://www.popoptiq.com/fashion/ [13] Min Hou, Le Wu, Enhong Chen, Zhi Li, Vincent Wenchen Zheng, and Qi Liu. 2019. [24] SizerMe. 2020. Retrieved July 21, 2020 from https://sizer.me/ Explainable Fashion Recommendation: A Semantic Attribute Region Guided [25] Zalando. 2020. Zalando SEs. Zalando SEs. Retrieved May 22, 2020 from https: Approach. In IJCAI. International Joint Conference on Artificial Intelligence, //www.zalando.com/ Macao, , 211–228. [26] ZeeKit. 2020. Retrieved July 23, 2020 from https://zeekit.me/ [14] Amazon Technologies Inc. 2018. On demand apparel manufacturing. Amazon. [27] Shuai Zheng, Fan Yang, M. Hadi Kiapour, and Robinson Piramuthu. 2018. Retrieved May 16, 2020 from https://patents.google.com/patent/US9623578B1/en ModaNet: A Large-Scale Street Fashion Dataset with Polygon Annotations. In [15] ISO. 2016. ISO 18831:2016(en) Clothing âĂŤ Digital fittings âĂŤ Attributes of virtual Proceedings of the 26th ACM International Conference on Multimedia (Seoul, Re- garments. International Organization for Standardization. Retrieved May 2, 2020 public of Korea) (MM âĂŹ18). Association for Computing Machinery, New York, from https://www.iso.org/obp/ui/#iso:std:iso:18831:ed-1:v1:en NY, USA, 1670âĂŞ1678. https://doi.org/10.1145/3240508.3240652

9