Collabmap: Crowdsourcing Maps for Emergency PlanningSarvapali D. Ramchurn,1Trung Dong Huynh,1Matteo Venanzi,1Bing Shi2 1Electronics and Computer Science, University of Southampton, United Kingdom sdr,tdh,mv1g10ecs.soton.ac.uk2School of Computer Science and Technology, Wuhan University of Technology Wuhan, China bingshiwhut.edu.cnABSTRACT In this paper, we present a software tool to help emergency planners at Hampshire County Council in the UK to createmaps for high-fidelity crowd simulations that require evac- uation routes from buildings to roads.The main feature of the system is a crowdsourcing mechanism that breaks down the problem of creating evacuation routes into micro- tasks that a contributor to the platform can execute in less than a minute.As part of the mechanism we developeda concensus-based trust mechanism that filters out incor- rect contributions and ensures that the individual tasks are complete and correct. To drive people to contribute to theplatform, we experimented with different incentive mecha-nisms and applied these over different time scales, the aimbeing to evaluate what incentives work with different types of crowds, including anonymous contributors from Amazon Mechanical Turk. The results of the in the wild deploymentof the system show that the system is effective at engaging contributors to perform tasks correctly and that users re-spond to incentives in different ways. More specifically, we show that purely social motives are not good enough to at- tract a large number of contributors and that contributors are averse to the uncertainty in winning rewards.When taken altogether, our results suggest that a combination of incentives may be the best approach to harnessing the maxi- mum number of resources to get socially valuable tasks (such for planning applications) performed on a large scale.Categories and Subject Descriptors C.5 World Wide Web: Crowdsourcing1.INTRODUCTIONThe creation of high fidelity scenarios for disaster simulation is a major challenge for a number of reasons. First, in the UK, the maps supplied by existing map providers (e.g., Ord- nance Survey, TeleAtlas) tend to provide only road or build- ing shapes and do not accurately model open spaces which people use to evacuate buildings, homes, or industrial facili-ties (e.g. the space around a stadium or a commercial centreboth constitute evacuation routes of different shapes and sizes). Secondly, even if some of the data about evacuation routes is available, the real-world connection points between these spaces and roads and buildings is usually not well de-fined unless data from buildings owners can be obtained (e.g. building entrances, borders, and fences).Finally, in order to augment current maps with accurate spatial data, it would require either a good set of training data (which is not available to our knowledge) for a computer vision algo-rithm to define evacuation routes using pictures (working onaerial maps) or a significant amount of manpower to directly survey a vast area.Against this background, we developed a novel model of geospatial data creation, called CollabMap1, that relies on human computation. CollabMap is a crowdsourcing tool to get users to perform micro-tasks that involve augmenting ex- isting maps (e.g. Google Maps or Ordnance Survey) by draw- ing evacuation routes, using satellite imagery from Google Maps and panoramic views from Google StreetView. In a similar vein to 12, 4, we use human computation to com- plete tasks that are hard for a computer vision algorithm to perform or to generate training data that could be used bya computer vision algorithm to automatically define evac- uation routes. Compared to other community-driven plat- forms such as OpenStreetMap and Googles MapMaker, Col- labmap allows inexperienced and anonymous users to per- form tasks without them needing the expertise to integrate the data into the system (as in OpenStreetMap) and does not rely on having experts verifying the tasks (as in Map- Maker) in order to generate meaningful results.To ensure that individual contributions are correct and com- plete, we build upon the Find-Fix-Verify (FFV) pattern 1to develop a novel adaptive workflow that includes concensus- based trust metrics and allows the creation of new tasks where no ground-truth is known. Our trust metrics allow users to rate and correct each others contributions while ourworkflow is adaptive in the sense that it allows the system designer to improve the performance of the crowd accord- ing to both the number and types of contributions into the system. As we show in our results, this approach was ef- fective in preventing workers from getting bored and taking full advantage of users motivation to contribute.Given our implementation of the platform, we deployed our1www.collabmap.orgsystem to help map the area around the Fawley Oil refin- ery next to the city of Southampton in the UK over three months. The area covered over 5,000 buildings