Pkg.test("Gadfly") evaluates all of the files in
Gadfly/test/testscripts. Any errors or warnings are printed to the REPL. In addition, the figures that are produced are put into either the
cachedoutput/ sub-directories. Nominally, the former represents the changes in a pull request while the latter are used for comparison. Specifically,
runtests.jl examines the currently checked out git commit, and sets the output directory to
cachedoutput/ if it is the HEAD of the master branch or if it is detached. Otherwise, it assumes you are at the tip of a development branch and saves the figures to
gennedoutput/. After running the tests on both of these branches, executing
compare_examples.jl displays differences between the new and old figures. This script can dump a diff of the files to the REPL, open both figures for manual comparison, and/or, for SVG and PNG files, display a black and white figure highlighting the spatial location of the differences.
So the automated regression analysis workflow is then as follows:
In a branch other than master,
develop your new feature or fix your old bug,
commit all your changes,
Pkg.add("ArgParse")and, for B&W images, Cairo, Fontconfig, Rsvg, and Images as well,
check for differences with `julia test/compare_examples.jl [–diff] [–two]
[–bw] [-h] [filter]
. For example,julia test/compare_examples.jl -bw .js.svg` will show black and white images hightlighting the differences between the svg test images.