[MERGE] build-test: fixed err detection and added multi-test support

From: Alex Rousskov <rousskov_at_measurement-factory.com>
Date: Mon, 09 Mar 2009 10:41:22 -0600

Hello,

    When working on SourceLayout I got a few false errors from
build-test.sh when it could not find some string in the make output. The
attached patch fixes that by relying on make exit code rather than some
magic string some make targets may print in the middle of the make
process. The build is still considered failed if error strings are found
in the log.

I was also annoyed on how the test would quit on the first error,
wasting a lot of test time if the first error gets unnoticed. The
attached patch adds a --keep-going option, similar to "make -k". The
option is off by default so I am not changing the default behavior here.

Finally, I made test spec names processing a little bit more flexible.
It is now possible to explicitly specify multiple tests, similar to
"make all check distcheck".

More detailed comments are below. Please review.

Thank you,

Alex.
bb:approve

------- change log -----------
Support multiple test spec arguments. If at least one test fails, the script
exits with a non-zero code (but possibly not immediately, see --keep-going).

Each test spec is a test config file name or a well-known config name
(no path
or extension!). If no specs are given, all known test specs are used (as
before). The same happens if the only test spec given is 'all'. The
following are now equivalent:

    ./test-builds.sh
    ./test-builds.sh all
    ./test-builds.sh btlayer-00-default btlayer-01-minimal
btlayer-02-maximus
    ./test-builds.sh test-suite/buildtests/layer-*

You can mix file names and spec names, but not the 'all' macro: There is
currently no support for using 'all' together with other test cases.

Tolerate individual test errors if --keep-going is specified. This helps
when
one wants to find more errors than just the first one, especially when tests
are long and are running without a human watching.

When detecting test failures, rely on test-suite/buildtest.sh exit
status code
rather than on the presence of error-like strings in the log file.

Added and polished comments.
-----------------------------------

Received on Mon Mar 09 2009 - 16:41:26 MDT

This archive was generated by hypermail 2.2.0 : Fri Mar 13 2009 - 12:00:04 MDT