Re: Testcases for append! in greg style.

[Jim Blandy <jimb at red-bean dot com>]

What I was going to say (and forgot to in my last post) is that I
think Greg and (test-suite lib) could probably learn a lot from each
other.  Frankly, I doubt they are large enough or different enough
that we couldn't profitably combine them.

> > - the ability to mark independent groups of tests, so that if one
> > test hits an error, we don't just abandon the whole file
> 
> An error in a Greg test doesn't cause a file to be abandoned and
> therefore doesn't need an extra facility to say that tests shouldn't
> be abandoned - though you do have the ability to omit a group of
> tests based on the result of an earlier test.

That's something that (test-suite lib) doesn't have any provision for;
there's no way to find the result of a previous test.

The rationale for this feature comes from my experiences with the GDB
test suite.  In that situation, you're actually talking to a running
GDB, setting breakpoints, stepping into functions, and so on.  Very
often, if a test fails, that means some action didn't occur, but
subsequent tests will assume that it did.  For example, if you fail to
step into a function, you can't check the values of its arguments.  So
you'll often see a string of failures that turns out to be a single
problem.  However, if a later test actually restarts the program
afresh (which some do), there's no reason you couldn't continue to run
those.

The idea is that you can wrap a bunch of tests that depend on each
other in a `catch-test-errors' form; if there's an error, that form
will return, but you'll continue on with the rest of the test suite.

However, I'm not at all sure this is the most useful feature to
support the problem.  For example, it might be better to have
something which jumps out if any test *fails*, not if any test
signals an error.


> > - the ability to group a bunch of tests and specify a common prefix
> > to use in their names.  Every test needs to have a distinct name, or
> > else you can't figure out what code failed; this is a natural and
> > convenient way to help do that.
> 
> Greg tells you which test in which file and in which 'tool' (a concept
> borrowed from DejaGNU) failed but I can see that additional named groups
> within a file might be a useful facility to add if you don't want to use
> unique test names within the file.

They save typing, and make it easier to keep names consistent.  You
can have a group name that says you're testing "string ports", and
then use short test names like "display text" or "write/read sexpr".


> > - the ability to test whether a piece of code signals an error, not
> > simply whether some expression is true or false
> 
> Greg handles this for you automatically.

That's probably better.