CastingWords Platform

Castingwords has been working with Mechanical Turk for years, and in that time we've developed and extensive platform with a rich feature set our competitors can't match.  
 

CastingWords Workforce Trust Graph

We know from experience that if you don't handle bad or fraudulent work, you will eventually get poor results from MTurk. Fraudulent Workers have plenty of incentive, as they know they will receive real money they can take to the bank just by submitting false work - if it goes unnoticed by the business approving the work. 

Knowing which workers you can trust and which workers you need to double check is one of the most important parts of the platform. Our metrics are like a performance review of each and every Worker. Our trust graph rates every worker on their track record, quality of work performed, and overall trustworthiness. All of the Workers are interconnected in the Trust Graph because we have the Workers continually review and rate each others work. It's our secret sauce that prevents fraudulent workers from giving us bad results. 

By comparison, Amazon only provides some very basic metrics around approval rate which starts at %100 for new Workers. They tell you nothing about how long the worker has been around or how many other tasks the worker has completed.

CastingWords Workflow Building Blocks

Workers can and do make mistakes, and over the years the CastingWords platform had to be engineered to handle all types of submissions from Workers - from great, to lazy, to unintelligible, to downright purposely fraudulent. Along with the above Trust Graph, workflows are the best way to ensure quality results.

We have a set of pre-defined workflows that you can start with, but we also offer a flexible set of building blocks that you can connect together to create your own custom workflow. 

The Workflow Building Blocks include:

• Sub-Tasks: The approval of the original task is dependent on the approval of the sub tasks
• Forked Sub-Task: A sub-task is generated where the approval of the original is not dependent on this sub-task. Used to generate new types of task that needs to be processed in parallel with the original task.
• Quorum Task: Ask a set of Workers the same question and approve the ones with the majority answer
• Decoy Task: This is a task where you know the answer to the task. It is used as an additional quality control measure to ensure the Workers are submitting quality answers
• Verification Task via Web Hooks: Like a sub-task, but approval is dependent on your side via an API call. You get all the information so that you can process the data and either automatically or manually approve the results
• Dispute Rejection Task: A process where workers can dispute rejections if they think they were right, further adding to the quality of the results

CastingWords Task Templates

A task like a web page and is comprised of HTML and JavaScript that defines a form that a Worker complete and submit to MTurk. These forms must include all the instructions needed for new Workers to understand the task as well as the form that is filled out when doing the actual work. We have developed a variety of templates that you can start with and modify to suite your needs.