How to Plan Your MTurk Study

Successful HITs require planning, especially if the project is managed by a team of researchers.  I find it helpful to answer the following questions before I start creating projects and HITs:

Properties - Describing the HIT

• What kind of project would you like to create?

• What is the title of the project?  Only you will see this title.

• What is the title of the HIT?  Workers will see this title.

• How would you describe the task?  What are you asking Workers to do?

• What keywords will help Workers find your HIT?

Properties - Setting up the HIT

• How much will you pay Workers per assignment?

• How many Workers do you want to complete your HIT?

• How much time will you give Workers to complete your HIT?

• When do you want the HIT to expire?

• Do you want submissions to be automatically approved within a certain time period? If so, when?

Properties - Advanced Settings

• Do you plan to use Master Workers? 

• What qualifications do Workers need to meet in order to complete the HIT?
• MTurk provides a few system qualification types:  location, number of HITs approved, and approval rating (%) for all HITs.
• You may want to create your own qualification type(s) to assign to the HIT. 

• Do you want only qualified Workers to be able to preview the HIT?

Design Layout

• What steps does it take to complete the HIT? 
• How long does it take to complete?
• Are there special instructions you need to highlight? Anything Workers need to pay special attention to?
• What would lead you to reject a Worker's submission?
• Are you awarding bonuses? If so, what would lead you to award a bonus?
• Who should the Workers contact if they have questions or problems?

• Do you need to communicate that some Workers are ineligible to complete the HIT (even though they may be able to preview it)?  Click here for more information.

• Are you planning to use external websites to collect information (e.g., Qualtrics, Survey Monkey, etc.)?

• Are you planning to ask Workers for feedback?