I've just implemented another Python idea: doctest. This is a function that will test snippets of shell sessions. It gets its name from its use testing these snippets that appear in documentation comments, but it turns out to be a very convenient way to write tests more generally.
For example, say you've written a new function hello(). I usually go to the shell and play with it to make sure it works correctly:
js> function hello(greetee) {
> return "hello, " + greetee;
> }
js> hello("world");
hello, world
js> hello(3)
hello, 3
js> hello()
hello, undefined
For Rhino we've used a set of JavaScript language tests developed by mozilla.org over many years. They were run by a Perl driver that was recently ported to Java. Until recently I'd would have tested the above code with a test that would be run by that framework. These tests involved a bunch of setup code and then taking each call to hello() above, saving the result value, and calling a comparison function with the actual and expected values. Doctest does this all for me.
In my Rhino implementation of Doctest in addition to a shell function I've written a JUnit runner that finds files with a .doctest extension and runs them. So now all I need to do is copy the shell session above, paste it into hello.doctest, and put it in the right directory and I have a JUnit test! It's much more convenient to write tests, which greatly increases the chances that tests actually get written.
This is available now in the latest version of Rhino (ftp) if you'd like to try it out.
6 comments:
I'm using this to implement a @doc.test tag for javadoc, and it's very nice -- thank you!
One minor issue: it's hard to write doc tests that are *supposed* to throw exceptions. I can't set the error reporter in the context, since Global.runDoctest overrides it. I *suspect* that ToolErrorReporter.reportException(RhinoException) should invoke we.getWrappedException().printStackTrace(err) instead of we.printStackTrace(err)? That gives me the actual Java exception involved -- and then Global.doctestOutputMatches would need to implement either doctest.ELLIPSIS or doctest.IGNORE_EXCEPTION_DETAIL to allow the body of the stack trace to be elided.
Here's a link to JDoctest, based on your Rhino work. Thanks again!
This is cool. Will future versions of rhino distinguish between 'true' and true in console output (or ['foo'] and 'foo' etc.)? Seems like that would make these tests more useful.
tschaub: You're right that true and "true" print the same. I'll often write my doctests to print out the value and the type:
js> function f() { return true; }
js> var x = f();
js> x
true
js> typeof x
boolean
Beyond that, it would be generally useful to have a more descriptive output format for the shell; perhaps a special global function that, if defined, will be called to print the results of each expression on the command line (similar to the prompts variable now).
Can doctest be run in another scope? I can't figure out how to get at runDoctest from the shell, but I think that provides what I would like to do (basically, run doctest without polluting global - and call it multiple times with the same scope). Thanks for any tips.
tschaub: Have you looked at org.mozilla.javascript.tests.DoctestsTest? This is a JUnit test that calls runDoctest. Does that do what you want?
Post a Comment