Stimulating Skill Evolution in Market-based CrowdsourcingBenjamin Satzger, Harald Psaier, Daniel Schall, and Schahram DustdarDistributed Systems Group, Vienna University of Technology Argentinierstrasse 8/184-1, A-1040 Vienna, Austria satzger,psaier,schall,dustdarinfosys.tuwien.ac.at, WWW home page: http:/www.infosys.tuwien.ac.atAbstract. Crowdsourcing has emerged as an important paradigm in hu- man problem-solving techniques on the Web. One application of crowd-sourcing is to outsource certain tasks to the crowd that are difficult toimplement in software. Another potential benefit of crowdsourcing isthe on-demand allocation of a flexible workforce. Businesses may out- source tasks to the crowd based on temporary workload variations. A major challenge in crowdsourcing is to guarantee high-quality processing of tasks. We present a novel crowdsourcing marketplace that matches tasks to suitable workers based on auctions. The key to ensuring high quality lies in skilled members whose capabilities can be estimated cor- rectly. We present a novel auction mechanism for skill evolution that helps to correctly estimate workers and to evolve skills that are needed. Evaluations show that this leads to improved crowdsourcing.Keywords: Human-centric BPM, Crowdsourcing, Online communities, Task markets, Auctions, Skill evolution1IntroductionToday, ever changing requirements force in-house business processes to rapidly adapt to changing situations in order to stay competitive. Changes involve not only the need for process adaptation, but also, require an additional inclusion of new capabilities and knowledge, previously unavailable to the company. Thus, outsourcing of parts of business processes became an attractive model. This work, in particular, focuses on a distinguished recent type of outsourcing called crowd- sourcing. The term crowdsourcing describes a new web-based business model that harnesses the creative solutions of a distributed network of individuals 4, 18. This network of humans is typically an open Internet-based platform thatfollows the open world assumption and tries to attract members with different knowledge and interests. Large IT companies such as Amazon, Google, or Yahoo! have recognized the opportunities behind such mass collaboration systems 8 for both improving their own services and as business case. In particular, Amazon fo- cuses on a task-based marketplace that requires explicit collaboration. The mostprominent platform they currently offer is Amazon Mechanical Turk (AMT) 3.2Requesters are invited to issue human-intelligence tasks (HITs) requiring a cer-tain qualification to the AMT. The registered customers post mostly tasks withminor effort that, however, require human capabilities (e.g., transcription, clas-sification, or categorization tasks 11).Reasons for companies to outsource tasks to a crowd platform are flexibility,high availability, and low costs. Workers from all over the world and different timezones are allowed to register with the platform. With 90% of tasks processedat a cost of $0.10 and less, the same task is usually also offered to multiple AMT workers. However, crowdsourcing platforms are loosely-coupled networkswith members of different interests, working style, and cultural background.The flexible crowd structure allows workers to join and leave at any time. This hampers to predict or guarantee the quality of a tasks result. There are crowd providers such as CrowdFlower 6 that broker crowd resources to customers to overcome quality and reliability issues. However, managing and adapting the crowds skills and resources in an au- tomated manner remains challenging. Crowd customers prefer fully automated deployment of their tasks to a crowd, just as in common business process models. In this paper, we base our solution on the service-oriented architecture (SOA) paradigm. SOAs are an ideal grounding for distributed environments. With their notion of the participants as services and registries, resources can be easily and even automatically discovered for composing whole business processes. A plethora of standards supports seamless integration and registration of new ser- vices, and provides protocols for communication, interaction and control of the components. Altogether, we believe SOAs provide an intuitive and convenient technical grounding to automate large-scale crowdsourcing environments. Im- portant to note, today SOA not only includes software-based services, but also Human-Provided Services 16 and BPEL4People 2 for human interactions in business processes and allow to host mass collaboration environments. The main challenges addressed in this work relate to building and manag-ing an automated crowd platform. It is not only of importance to find suitable workers for a task and to provide the customer with satisfying quality, but also, to maintain a motivated base of crowd members and provide stimulus for learn-ing required skills. Only a recurrin