Case study of comparing command line tools
Case study of comparing command line tools Background Recently I ran into an issue where one of the tools I was using no longer produced the output I was expecting. I had used the tool just a couple of days ago where it provided me the desired results. The source code of the tool had not changed. I had changed the Linux distribution on which the tool was built and running. After some debugging I concluded that the different distribution enabled the compiler flag “-faggressive-loop-optimizations” by default. Compilation did not fail and unfortunately this flag removed an important loop in the code altering the behavior of the tool. This got me thinking what other alterations could the optimizer have made? How would I tell? How would I catch this at scale? What if my distro had built the binary, would they have caught the issue? In “Forensics tools” where are your tests? I emphasized that “ The absence of documented tests and results renders reproducibility impossible. ” Unfortuna...