We rarely see a software engineering paper that aims to prove a point from multiple views. Even better, this study not only refutes prior finds in the fault localization research field, but also proposes a combination of techniques that harness existing ones.

Fault localization is a technique to list parts of source code that are likely to contain errors. This method starts from a given fault program to help software developers find errors faster, driving them to look at the most probable erroneous parts first.

The study initially reports an independent replication of seven well-known fault localization approaches. So far, earlier research explored artificial faults, all geared toward producing more accurate results, even though only three out of ten prior results are shown to be reproducible. Then, the study turns to real-world program faults; worse, none of the prior claims are shown to be significant. Irreproducibility could happen for many legitimate reasons. However, the authors provide their implementations to the community along with the data used to enable other researchers to repeat, improve, or refute their results.

Finally, the study builds upon existing techniques and suggests several improvements on ranking erroneous code effectively.

Several software engineering venues are now dealing with the increasing concern of the lack of reproducibility of key research findings because it could hinder scientific progress. These efforts are laudable, and this paper shows how to explore fault localization technique replications and improve the state of the art.