DOCSLIB.ORG

Amazon Mechanical Turk Developer Guide API Version 2017-01-17 Amazon Mechanical Turk Developer Guide

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Amazon Mechanical Turk Developer Guide API Version 2017-01-17 Amazon Mechanical Turk Developer Guide Amazon Mechanical Turk Developer Guide API Version 2017-01-17 Amazon Mechanical Turk Developer Guide Amazon Mechanical Turk: Developer Guide Copyright © Amazon Web Services, Inc. and/or its affiliates. All rights reserved. Amazon's trademarks and trade dress may not be used in connection with any product or service that is not Amazon's, in any manner that is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who may or may not be affiliated with, connected to, or sponsored by Amazon. Amazon Mechanical Turk Developer Guide Table of Contents What is Amazon Mechanical Turk? ........................................................................................................ 1 Mechanical Turk marketplace ....................................................................................................... 1 Marketplace rules ............................................................................................................... 2 The sandbox marketplace .................................................................................................... 2 Tasks that work well on Mechanical Turk ...................................................................................... 3 Tasks can be completed within a web browser ....................................................................... 3 Work can be broken into distinct, bite-sized tasks .................................................................. 3 Task supports clear instructions and outcomes ...................................................................... 3 Examples of common uses of Mechanical Turk ....................................................................... 3 Core concepts ............................................................................................................................ 4 Requesters and workers ...................................................................................................... 4 Marketplace ....................................................................................................................... 4 Task or HIT ....................................................................................................................... 4 Assignment ....................................................................................................................... 5 Reward and bonus ............................................................................................................. 5 Qualifications .................................................................................................................... 5 Best practices ............................................................................................................................ 5 Allow workers to be as efficient as possible ........................................................................... 5 Build tasks with family and friends in mind ........................................................................... 5 Include an optional feedback field ....................................................................................... 5 Test your HITs ................................................................................................................... 6 Start small ........................................................................................................................ 6 Keep HIT type attributes consistent ...................................................................................... 6 Specify that links open new browser, windows or tabs ............................................................ 7 Limit your use of worker blocks ........................................................................................... 7 Include clear reasons for rejections and blocks ....................................................................... 7 Frequently asked questions ......................................................................................................... 7 Why aren't my tasks being completed? ................................................................................. 7 How do I pull down HITs I created by mistake? ...................................................................... 8 I expired my HITs. Why am I still getting submissions from workers? ......................................... 8 Why are some of my task fields missing from my results? ....................................................... 8 Can I make some fields in my task interface required? ............................................................ 8 How can I test my task interface? ........................................................................................ 8 What is the difference between a HIT and an assignment? ....................................................... 8 Can I view the HITs I create with the API in the requester website? ........................................... 9 I published HITs in the sandbox environment. Why they aren't being completed? ........................ 9 I incorrectly rejected some assignments. Can I reverse the rejection? ......................................... 9 How do I filter the workers eligible to work on my task? ......................................................... 9 How do I create a custom qualification? ................................................................................ 9 Can I restrict how many HITs a worker can complete for my project? ......................................... 9 Can I post HITs in languages other than English? ................................................................... 9 Additional Mechanical Turk Resources ................................................................................. 10 Set up Mechanical Turk ..................................................................................................................... 11 Step 1: Create a Mechanical Turk account .................................................................................... 11 Step 2: Link your AWS account to your Mechanical Turk requester account ....................................... 11 Step 3: Select a payment option ................................................................................................ 12 Step 4: Get an AWS access key .................................................................................................. 12 Step 5: Configure Your Credentials ............................................................................................. 13 Step 6: Set up the developer sandbox ........................................................................................ 13 Access Mechanical Turk ..................................................................................................................... 14 Use the Mechanical Turk Requester UI ........................................................................................ 14 Use the Mechanical Turk API ...................................................................................................... 14 Use the AWS CLI ...................................................................................................................... 14 Download and configure the AWS CLI ................................................................................. 15 API Version 2017-01-17 iii Amazon Mechanical Turk Developer Guide Use the AWS CLI with Mechanical Turk ............................................................................... 15 Get Started with Mechanical Turk ....................................................................................................... 17 Prerequisites ............................................................................................................................ 17 Step 1: Create a task ................................................................................................................ 17 Question definition ........................................................................................................... 17 Task attributes ................................................................................................................. 18 Post the task ................................................................................................................... 18 Step 2: Check task status .......................................................................................................... 20 Step 3: Retrieve results ............................................................................................................. 23 Step 4: Approve Assignments .................................................................................................... 27 Creating and managing tasks (HITs) .................................................................................................... 29 Define questions ...................................................................................................................... 29 Use HTML to define questions ........................................................................................... 29 Question definitions ......................................................................................................... 33 Requester website layouts ................................................................................................. 35 HIT attributes .........................................................................................................................
Load more

Recommended publications
  • Timeline 1994 July Company Incorporated 1995 July Amazon
    Timeline 1994 July Company Incorporated 1995 July Amazon.com Sells First Book, “Fluid Concepts & Creative Analogies: Computer Models of the Fundamental Mechanisms of Thought” 1996 July Launches Amazon.com Associates Program 1997 May Announces IPO, Begins Trading on NASDAQ Under “AMZN” September Introduces 1-ClickTM Shopping November Opens Fulfillment Center in New Castle, Delaware 1998 February Launches Amazon.com Advantage Program April Acquires Internet Movie Database June Opens Music Store October Launches First International Sites, Amazon.co.uk (UK) and Amazon.de (Germany) November Opens DVD/Video Store 1999 January Opens Fulfillment Center in Fernley, Nevada March Launches Amazon.com Auctions April Opens Fulfillment Center in Coffeyville, Kansas May Opens Fulfillment Centers in Campbellsville and Lexington, Kentucky June Acquires Alexa Internet July Opens Consumer Electronics, and Toys & Games Stores September Launches zShops October Opens Customer Service Center in Tacoma, Washington Acquires Tool Crib of the North’s Online and Catalog Sales Division November Opens Home Improvement, Software, Video Games and Gift Ideas Stores December Jeff Bezos Named TIME Magazine “Person Of The Year” 2000 January Opens Customer Service Center in Huntington, West Virginia May Opens Kitchen Store August Announces Toys “R” Us Alliance Launches Amazon.fr (France) October Opens Camera & Photo Store November Launches Amazon.co.jp (Japan) Launches Marketplace Introduces First Free Super Saver Shipping Offer (Orders Over $100) 2001 April Announces Borders Group Alliance August Introduces In-Store Pick Up September Announces Target Stores Alliance October Introduces Look Inside The BookTM 2002 June Launches Amazon.ca (Canada) July Launches Amazon Web Services August Lowers Free Super Saver Shipping Threshold to $25 September Opens Office Products Store November Opens Apparel & Accessories Store 2003 April Announces National Basketball Association Alliance June Launches Amazon Services, Inc.
    [Show full text]
  • Amazon Dynamodb
    Dynamo Amazon DynamoDB Nicolas Travers Inspiré de Advait Deo Vertigo N. Travers ESILV : Dynamo Amazon DynamoDB – What is it ? • Fully managed nosql database service on AWS • Data model in the form of tables • Data stored in the form of items (name – value attributes) • Automatic scaling ▫ Provisioned throughput ▫ Storage scaling ▫ Distributed architecture • Easy Administration • Monitoring of tables using CloudWatch • Integration with EMR (Elastic MapReduce) ▫ Analyze data and store in S3 Vertigo N. Travers ESILV : Dynamo Amazon DynamoDB – What is it ? key=value key=value key=value key=value Table Item (64KB max) Attributes • Primary key (mandatory for every table) ▫ Hash or Hash + Range • Data model in the form of tables • Data stored in the form of items (name – value attributes) • Secondary Indexes for improved performance ▫ Local secondary index ▫ Global secondary index • Scalar data type (number, string etc) or multi-valued data type (sets) Vertigo N. Travers ESILV : Dynamo DynamoDB Architecture • True distributed architecture • Data is spread across hundreds of servers called storage nodes • Hundreds of servers form a cluster in the form of a “ring” • Client application can connect using one of the two approaches ▫ Routing using a load balancer ▫ Client-library that reflects Dynamo’s partitioning scheme and can determine the storage host to connect • Advantage of load balancer – no need for dynamo specific code in client application • Advantage of client-library – saves 1 network hop to load balancer • Synchronous replication is not achievable for high availability and scalability requirement at amazon • DynamoDB is designed to be “always writable” storage solution • Allows multiple versions of data on multiple storage nodes • Conflict resolution happens while reads and NOT during writes ▫ Syntactic conflict resolution ▫ Symantec conflict resolution Vertigo N.
    [Show full text]
  • Amazon Web Services: Overview of Security Processes
    Amazon Web Services: Overview of Security Processes March 2020 For the latest technical content, see Best Practices for Security, Identity & Compliance https://aws.amazon.com/architecture/ security-identity-compliance Archived Notices Customers are responsible for making their own independent assessment of the information in this document. This document: (a) is for informational purposes only, (b) represents current AWS product offerings and practices, which are subject to change without notice, and (c) does not create any commitments or assurances from AWS and its affiliates, suppliers or licensors. AWS products or services are provided “as is” without warranties, representations, or conditions of any kind, whether express or implied. The responsibilities and liabilities of AWS to its customers are controlled by AWS agreements, and this document is not part of, nor does it modify, any agreement between AWS and its customers. © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved. Archived Contents Introduction .......................................................................................................................... 1 Shared Security Responsibility Model ................................................................................ 1 AWS Security Responsibilities ......................................................................................... 2 Customer Security Responsibilities ................................................................................. 2 AWS Global Infrastructure Security
    [Show full text]
  • Performance at Scale with Amazon Elasticache
    Performance at Scale with Amazon ElastiCache July 2019 Notices Customers are responsible for making their own independent assessment of the information in this document. This document: (a) is for informational purposes only, (b) represents current AWS product offerings and practices, which are subject to change without notice, and (c) does not create any commitments or assurances from AWS and its affiliates, suppliers or licensors. AWS products or services are provided “as is” without warranties, representations, or conditions of any kind, whether express or implied. The responsibilities and liabilities of AWS to its customers are controlled by AWS agreements, and this document is not part of, nor does it modify, any agreement between AWS and its customers. © 2019 Amazon Web Services, Inc. or its affiliates. All rights reserved. Contents Introduction .......................................................................................................................... 1 ElastiCache Overview ......................................................................................................... 2 Alternatives to ElastiCache ................................................................................................. 2 Memcached vs. Redis ......................................................................................................... 3 ElastiCache for Memcached ............................................................................................... 5 Architecture with ElastiCache for Memcached ...............................................................
    [Show full text]
  • Amazon Documentdb Deep Dive
    DAT326 Amazon DocumentDB deep dive Joseph Idziorek Antra Grover Principal Product Manager Software Development Engineer Amazon Web Services Fulfillment By Amazon © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Agenda What is the purpose of a document database? What customer problems does Amazon DocumentDB (with MongoDB compatibility) solve and how? Customer use case and learnings: Fulfillment by Amazon What did we deliver for customers this year? What’s next? © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Purpose-built databases Relational Key value Document In-memory Graph Search Time series Ledger Why document databases? Denormalized data Normalized data model model { 'name': 'Bat City Gelato', 'price': '$', 'rating': 5.0, 'review_count': 46, 'categories': ['gelato', 'ice cream'], 'location': { 'address': '6301 W Parmer Ln', 'city': 'Austin', 'country': 'US', 'state': 'TX', 'zip_code': '78729'} } Why document databases? GET https://api.yelp.com/v3/businesses/{id} { 'name': 'Bat City Gelato', 'price': '$', 'rating': 5.0, 'review_count': 46, 'categories': ['gelato', 'ice cream'], 'location': { 'address': '6301 W Parmer Ln', 'city': 'Austin', 'country': 'US', 'state': 'TX', 'zip_code': '78729'} } Why document databases? response = yelp_api.search_query(term='ice cream', location='austin, tx', sort_by='rating', limit=5) Why document databases? for i in response['businesses']: col.insert_one(i) db.businesses.aggregate([ { $group: { _id: "$price", ratingAvg: { $avg: "$rating"}} } ]) db.businesses.find({
    [Show full text]
  • A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk
    A Data-Driven Analysis of Workers’ Earnings on Amazon Mechanical Turk Kotaro Hara1,2, Abi Adams3, Kristy Milland4,5, Saiph Savage6, Chris Callison-Burch7, Jeffrey P. Bigham1 1Carnegie Mellon University, 2Singapore Management University, 3University of Oxford 4McMaster University, 5TurkerNation.com, 6West Virginia University, 7University of Pennsylvania [email protected] ABSTRACT temporarily out-of-work engineers to work [1,4,39,46,65]. A growing number of people are working as part of on-line crowd work. Crowd work is often thought to be low wage Yet, despite the potential for crowdsourcing platforms to work. However, we know little about the wage distribution extend the scope of the labor market, many are concerned in practice and what causes low/high earnings in this that workers on crowdsourcing markets are treated unfairly setting. We recorded 2,676 workers performing 3.8 million [19,38,39,42,47,59]. Concerns about low earnings on crowd tasks on Amazon Mechanical Turk. Our task-level analysis work platforms have been voiced repeatedly. Past research revealed that workers earned a median hourly wage of only has found evidence that workers typically earn a fraction of ~$2/h, and only 4% earned more than $7.25/h. While the the U.S. minimum wage [34,35,37–39,49] and many average requester pays more than $11/h, lower-paying workers report not being paid for adequately completed requesters post much more work. Our wage calculations are tasks [38,51]. This is problematic as income generation is influenced by how unpaid work is accounted for, e.g., time the primary motivation of workers [4,13,46,49].
    [Show full text]
  • A Motion Is Requested to Authorize the Execution of a Contract for Amazon Business Procurement Services Through the U.S. Communities Government Purchasing Alliance
    MOT 2019-8118 Page 1 of 98 VILLAGE OF DOWNERS GROVE Report for the Village Council Meeting 3/19/2019 SUBJECT: SUBMITTED BY: Authorization of a contract for Amazon Business procurement Judy Buttny services Finance Director SYNOPSIS A motion is requested to authorize the execution of a contract for Amazon Business procurement services through the U.S. Communities Government Purchasing Alliance. STRATEGIC PLAN ALIGNMENT The goals for 2017-2019 includes Steward of Financial Sustainability, and Exceptional, Continual Innovation. FISCAL IMPACT There is no cost to utilize Amazon Business procurement services through the U.S. Communities Government Purchasing Alliance. RECOMMENDATION Approval on the March 19, 2019 Consent Agenda. BACKGROUND U.S. Communities Government Purchasing Alliance is the largest public sector cooperative purchasing organization in the nation. All contracts are awarded by a governmental entity utilizing industry best practices, processes and procedures. The Village of Downers Grove has been a member of the U.S. Communities Government Purchasing Alliance since 2008. Through cooperative purchasing, the Village is able to take advantage of economy of scale and reduce the cost of goods and services. U.S. Communities has partnered with Amazon Services to offer local government agencies the ability to utilize Amazon Business for procurement services at no cost to U.S. Communities members. Amazon Business offers business-only prices on millions of products in a competitive digital market place and a multi-level approval workflow. Staff can efficiently find quotes and purchase products for the best possible price, and the multi-level approval workflow ensures this service is compliant with the Village’s competitive process for purchases under $7,000.
    [Show full text]
  • Amazon Elasticache Deep Dive Powering Modern Applications with Low Latency and High Throughput
    Amazon ElastiCache Deep Dive Powering modern applications with low latency and high throughput Michael Labib Sr. Manager, Non-Relational Databases © 2020, Amazon Web Services, Inc. or its Affiliates. Agenda • Introduction to Amazon ElastiCache • Redis Topologies & Features • ElastiCache Use Cases • Monitoring, Sizing & Best Practices © 2020, Amazon Web Services, Inc. or its Affiliates. Introduction to Amazon ElastiCache © 2020, Amazon Web Services, Inc. or its Affiliates. Purpose-built databases © 2020, Amazon Web Services, Inc. or its Affiliates. Purpose-built databases © 2020, Amazon Web Services, Inc. or its Affiliates. Modern real-time applications require Performance, Scale & Availability Users 1M+ Data volume Terabytes—petabytes Locality Global Performance Microsecond latency Request rate Millions per second Access Mobile, IoT, devices Scale Up-out-in E-Commerce Media Social Online Shared economy Economics Pay-as-you-go streaming media gaming Developer access Open API © 2020, Amazon Web Services, Inc. or its Affiliates. Amazon ElastiCache – Fully Managed Service Redis & Extreme Secure Easily scales to Memcached compatible performance and reliable massive workloads Fully compatible with In-memory data store Network isolation, encryption Scale writes and open source Redis and cache for microsecond at rest/transit, HIPAA, PCI, reads with sharding and Memcached response times FedRAMP, multi AZ, and and replicas automatic failover © 2020, Amazon Web Services, Inc. or its Affiliates. What is Redis? Initially released in 2009, Redis provides: • Complex data structures: Strings, Lists, Sets, Sorted Sets, Hash Maps, HyperLogLog, Geospatial, and Streams • High-availability through replication • Scalability through online sharding • Persistence via snapshot / restore • Multi-key atomic operations A high-speed, in-memory, non-Relational data store. • LUA scripting Customers love that Redis is easy to use.
    [Show full text]
  • Transform Your Business with Amazon Web Services and Salesforce Collaborative Solutions to Innovate Customer Experience
    AWS and Salesforce Transform Your Business with Amazon Web Services and Salesforce Collaborative Solutions to Innovate Customer Experience AWS and Salesforce Together Enterprises around the world need to continually transform their business and the experience they provide their customers to thrive. Doing so however, requires the agility to experiment frequently and the resources to innovate quickly. Amazon Web Services and Salesforce have collaborated to help you fulfll these requirements. By bringing together the world’s most comprehensive and broadly adopted cloud platform with the leading Customer Relationship Management (CRM) ofering, you are able to leverage fully featured services for compute, storage, databases, networking, analytics, machine learning and artifcial intelligence (AI), the Internet of Things (IoT), mobile, security, hybrid, virtual and augmented reality (VR and AR), media, and application development, deployment, and management in tandem with the backbone for your customer engagements. A shared culture of customer-driven innovation means you’ll continually get the latest technology advances at your disposal. Key Benefts AWS and Salesforce share a strategic relationship, encompassing technical alignment and joint development. Many Salesforce products run on AWS. In addition, Salesforce customers can build on AWS to extend their Salesforce capabilities. Whether you are simply using Salesforce SaaS applications that run on AWS, or leveraging AWS to extend Salesforce capabilities, this collaboration provides a multitude
    [Show full text]
  • Dspace Cover Page
    Human-powered sorts and joins The MIT Faculty has made this article openly available. Please share how this access benefits you. Your story matters. Citation Adam Marcus, Eugene Wu, David Karger, Samuel Madden, and Robert Miller. 2011. Human-powered sorts and joins. Proc. VLDB Endow. 5, 1 (September 2011), 13-24. As Published http://www.vldb.org/pvldb/vol5.html Publisher VLDB Endowment Version Author's final manuscript Citable link http://hdl.handle.net/1721.1/73192 Terms of Use Creative Commons Attribution-Noncommercial-Share Alike 3.0 Detailed Terms http://creativecommons.org/licenses/by-nc-sa/3.0/ Human-powered Sorts and Joins Adam Marcus Eugene Wu David Karger Samuel Madden Robert Miller {marcua,sirrice,karger,madden,rcm}@csail.mit.edu ABSTRACT There are several reasons that systems like MTurk are of interest Crowdsourcing marketplaces like Amazon’s Mechanical Turk (MTurk) database researchers. First, MTurk workflow developers often imple- make it possible to task people with small jobs, such as labeling im- ment tasks that involve familiar database operations such as filtering, ages or looking up phone numbers, via a programmatic interface. sorting, and joining datasets. For example, it is common for MTurk MTurk tasks for processing datasets with humans are currently de- workflows to filter datasets to find images or audio on a specific sub- signed with significant reimplementation of common workflows and ject, or rank such data based on workers’ subjective opinion. Pro- ad-hoc selection of parameters such as price to pay per task. We de- grammers currently waste considerable effort re-implementing these scribe how we have integrated crowds into a declarative workflow operations because reusable implementations do not exist.
    [Show full text]
  • Integration with Amazon S3
    Integration with Amazon S3 YourDataConnect has developed a solution to ingest metadata from the AWS S3 file system into YourDataConnect. Amazon S3 can store multiple object types which enables storage for Internet applications, backup and recovery, disaster recovery, data archives, data lakes for analytics, and hybrid cloud storage. Figure 1 shows an AWS S3 bucket and folders. Figure 1: AWS S3 bucket and folders. Figure 2 shows a preview of the csv file in AWS S3. Figure 2: Preview of data in AWS Copyright © 2020, YourDataConnect. All rights reserved. Integration of Amazon S3 with YourDataConnect has developed a solution to import the file system, S3 bucket, directory, filegroup, file, and field metadata from Amazon S3 intoYourDataConnect while maintaining the hierarchy between S3 objects (see Figure 3). Figure 3: End-to-end traceability of Amazon S3 objects in YourDataConnect To learn more about this solution, please request a demo by contacting [email protected] or visit our website at yourdataconnect.com. Copyright © 2020, YourDataConnect. All rights reserved. This document is provided for information purposes only, and the contents hereof are subject to change without notice. This document is not warranted to be error-free, nor is it subject to any other warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or fitness for a particular purpose. We specifically disclaim any liability with respect to this document, and no contrac tual obligations are formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose, without our prior written permission..
    [Show full text]
  • Amazon Dossier
    Fluch und Segen für Markenhersteller Markus Fost Inhalt des Dossiers amazon – Fluch und Segen für Markenhersteller 1 Facts & Figures von Amazon 2 Das Amazon Geschäftsmodell – ein komplexes Ökosystem 3 Technische Infrastruktur von Amazon 4 Relevanz von Amazon im Herstellerumfeld 5 Das Amazon Ranking – Entscheidend für die Sichtbarkeit von Markenhersteller 6 Ausblick und Thesen Seite . 2 Dossier | amazon – Fluch und Segen für Markenhersteller | © FOSTEC Commerce Consultants Vorstellung Leistungsspektrum FOSTEC Commerce Consultants ANALYSE . Pragmatische Analyse von Strategie, USP, Geschäftsmodell und Prozessen mit dem Fokus auf die digitale Welt mittels einer Umfeld- und Customer Journey Analyse . Ergebnis: Status quo, Handlungsoptionen und Optimierungsansätze ZIELDEFINITION . Auf Basis Ihrer Ziele mit unserem Know-How gemeinsames Entwickeln eines Best-in-Class Digitalstrategie . Ergebnis: Blueprint Ihrer Digitalstrategie inklusive (E-Commerce) Distributionsstrategie, Quick-wins, Systemevaluation UMSETZUNG . Mit Projekterfahrung und Methodenkompetenz unterstützen wir Ihre Umsetzung zur digitalen Transformation . Ergebnis: Nachhaltig erfolgreiches Geschäftsmodell Seite . 3 Dossier | amazon – Fluch und Segen für Markenhersteller | © FOSTEC Commerce Consultants Vorstellung Warum FOSTEC der richtige Diskussionspartner ist . Erfahrene Experten beraten Sie individuell und auf Augenhöhe . Persönliche Betreuung durch ein inhabergeführtes Beratungsunternehmen . Wir wählen effiziente Methoden, um Sie schnellstmöglich an das Ziel zu bringen . Hohe
    [Show full text]