Creating your own "interestingness" tests

The first argument will the file whose interestingness is in question. If Lithium gets extra arguments on the command line, it will pass them to the test.

The test should return 0 ("success") to indicate that the input is interesting, and nonzero to indicate that the input is uninteresting.

Tests can be written in bash, python, or any other language that lets you handle command-line arguments and set exit codes.

It's up to you whether your test will take extra arguments, perhaps making it more reusable, or whether you hard-code things, perhaps making it take less typing to use.

Ideas

• "The testcase is displayed differently by version X and version Y of Firefox." This could be an easy way to make reduced testcases for regressions that affect how web pages are displayed.
• Faster test for hangs: automatically reduce the timeout based on how long successful runs take, or assume it's hanging if it outputs nothing for two seconds.
• "Makes Firefox crash with function X at the top of the crash stack trace"
• An "outputs" test that doesn't require restarting Firefox (for reducing non-fatal assertion bugs).
• Interactive, if a bug can only be observed by a human. (Using Lithium might be faster than manual reduction even in this case!)