Evaluating Low-Cost in Internal Crowdsourcing for Software Engineering: The Case of Feature Location in an Industrial Environment

Abstract

Internal crowdsourcing in software engineering is a mechanism for recruiting engineers to carry out more ef ciently software engineering tasks. However, engineers are busy resources and time is a valuable asset in industry, which hinders internal crowdsourcing in software engineering from becoming a widespread practice. In this work, we propose a low-cost variant of internal crowdsourcing for locating features in models, which limits the time that engineers can spend for providing knowledge. Our approach uses the knowledge provided by the internal crowd to automatically reformulate an initial feature description. The result is taken as input to automatically locate the relevant model fragment using Latent Semantic Indexing. We evaluate our approach using four query reformulation techniques in a real-world case study from our industrial partner.We compare the results of our approach in terms of recall, precision and F-measure with a baseline by means of statistical methods to show that the impact of the results of our approach is signi cant. Despite the limitation of time, the results showthat low-cost in internal crowdsourcing improves signi cantly the results in an industrial context where engineers' availability is scarce.