synopsis.
The testdriver class is a subclass of the rundriver_o class and it is closely related. Whereas the parent, rundriver_o, is intended to provide a general-purpose framework for testing software programs. the testdriver class was designed specifically to include testing capability ("Software Quality Assurance - SQA"). SQA, or just "QA", is abig part of computer programming. It is by nature subordinate to devleopment - you need to write the thing before you can test it; and hence, the area is often somewhat neglected.

Most Z Directory objects have corresponding source code for testing the object. A framework for testing objects has evolved at Vettrasoft. The need for testing Z Directory components led to the development of the testdriver class, which itself is a Z Directory component. Protocols are now in place to QA a component. There are typically 3 types of operations handled by testdriver_o, 2 of which are done in the base class (rundriver_o):

• validation - the object may always do the same operation, providing fixed, immutable results. We write code to do the specific task, get the output, and compare it with the expected results. For full validation, every concievable combination should be tried. That is to say, not every value of data. If we are testing an adding machine, it may be enough to test, say, 7 + 13 and compare the result to 20. Validation tests should test a wide range of the object's functionality, typically testing each one once (or perhaps a few times). Continuing the example, instead of going on to also test 7 + 14, or 8 + 13, the next test would move on to a different operation, such as subtraction or doing square roots. If we do addition again, we should test another aspect, such as extreme values (ie, 4,294,967,295 + 8,589,934,591). If all tests produce the expected results, the object has passed and is validated. . See the validate() function (below) for a list of return codes and exit values relating to validation.

Note that in the discussion above, with the exception of the calculator example, absolutely no functionality of any specific object was used. Thus, the SQA process envisioned here is generic and universal to any object, domain, and subject. This begets the concept of code for creating a generel-purpose testing harness and that is what testdriver_o does. Since it is a layer 1 object, the implementation is rather simple, but sufficient for many different class objects.

At first glance it may be hard to see what differences lie between rundriver_o and testdriver_o. both have member functions cmdline(), help(), usage(), description(), reset(), and run(). All of these function signitures are the same (indeed, they are virtual functions in rundriver_o).

The rundriver provides a place to store the object name, but since rundriver is not specific to objects, it does nothing with this internal string variable. The job of using it is deferred to sub-classes such as the testdriver. Differences in similar-sounding member functions will be pointed out in the function list on this page.

member functions (primary)

testdriver_o()
SIGNATURE: testdriver_o ()
SYNOPSIS: creates an empty testdriver object. the bulk of the work for this constructor is done in the base class.
 

do_validate()
SIGNATURE: boolean do_validate () const
SYNOPSIS: informational: tells if the object is in validation mode. run() will call the validate() function.
TRAITS: this function is inline
 

do_autoexit()
SIGNATURE: boolean do_autoexit () const
SYNOPSIS:
tells if the program will exit (by calling exit()) after validation is done. this applies only to validation mode.
TRAITS: this function is inline
 

in_testmode()
SIGNATURE: boolean in_testmode () const
SYNOPSIS:
if TRUE, th testdriver_o instance is to run a validation ccheck on itself. This function is intended to be used only by Vettrasoft for internal testing purposes.
TRAITS: this function is inline
 

validate_toend()
SIGNATURE: boolean validate_toend() const
SYNOPSIS:
applies only to validation mode. if this returns TRUE, then when an error is encountered, the validation code should call this function to determine if the validation check should go on. if it returns FALSE, validation should stop, with a return code of -1.
TRAITS: this function is inline
 

valid_stop_onerr()
SIGNATURE: boolean valid_stop_onerr() const
SYNOPSIS: this function isd a simple alias for the converse of validate_toend().
TRAITS: this function is inline
 

arg_isvalidate()
SIGNATURE: boolean arg_isvalidate (const char *, int * = NULL)
TRAITS: this is a static function. it must be called as testdriver_o::arg_isvalidate().
 

isvalid_arg()
SIGNATURE: boolean isvalid_arg (const char *, int * = NULL)
SYNOPSIS:
this is a simplified version of the member function is_testparam(), with a more generic name. it calls the latter function. it is basically a convenience function.
TRAITS: this is a static function. it must be called as testdriver_o::isvalid_arg().
 

is_testparam()
SIGNATURE: boolean is_testparam (const char *nam, boolean &warg, boolean &wdb, int *pi = NULL)
SYNOPSIS:
this tells if the input parameter "nam" is a command-line argument that is recognized by the testdriver object, or by the [parent] rundriver class object. This function internall calls rundriver_o::is_runparam(), so it is a "superset" of that function.
Since this function is static, it should be called like so:
boolean td_var = testdriver_o::is_testparam ("-volume", more);
The leading dash ('-') is mandatory.
PARAMETERS

• nam: the name of the command-line parameter. The first character of this string can have a leading dash ('-'), or it can be just the name, without the '-'.
• warg: [required output parameter] a boolean variable that tells if the parameter specified by "nam" takes an argument.
• wdb: [required output parameter] a boolean variable that tells if the parameter specified by "nam" is a "database type" argument.
• pi: [optional output] error indicator flag. The value of this variable is 0, if the parameter is specific to the testdriver object, or exactly that of rundriver_o::is_runparam(), if the parameter belongs to the rundriver_o class.

TRAITS: this is a static function. it must be called as testdriver_o::is_testparam().
 

set_validation()
SIGNATURE: void set_validation (boolean x = TRUE)
SYNOPSIS: sets the object's mode to "validation".
TRAITS: this function is inline
 

enable_cleanup()
SIGNATURE: void enable_cleanup ()
SYNOPSIS:
if this function is called, then the member function clean_up() will be called at the end of the run. clean-up is not considered a fundamental testdriver operation, as it may be called after validate() is called.
TRAITS: this function is inline
 

disable_cleanup()
SIGNATURE: void disable_cleanup ()
SYNOPSIS: turns of doing a clean-up. the converse of 'enable_cleanup()'.`
TRAITS: this function is inline
 

set_autoexit()
SIGNATURE: void set_autoexit (boolean x = TRUE)
SYNOPSIS:
"automatic exit" is done by default after validation is done. this applies only to validation. if it is turned off (by setting 'x' to FALSE), the program will not exit, and run() will return normally.
TRAITS: this function is inline
 

shutoff_autoexit()
SIGNATURE: void shutoff_autoexit ()
SYNOPSIS: turns off automatic program exit. the converse of set_autoexit(). this applies only to validation.
TRAITS: this function is inline
 

set_testmode()
SIGNATURE: void set_testmode (boolean x = TRUE)
SYNOPSIS: sets the testdriver_o instance to "test mode". this function is intended only for Vettrasoft and should not be used.
TRAITS: this function is inline
 

set_validate_nonstop()
SIGNATURE: void set_validate_nonstop (boolean x = TRUE)
SYNOPSIS:
this applies only to validation. if this function is called prior to calling validate()`, then the validation should not abort upon encountering an error. when an error is hit, validate_toend() or valid_stop_onerr() should be called to check if the validation test should go on or not..
TRAITS: this function is inline
 

reset()
SIGNATURE: int reset()
SYNOPSIS: resets the object state. the parent class (rundriver_o) reset() function is also invoked.
 

set_up()
SIGNATURE: int set_up (const int argc, const char **argv, int *pi = NULL)
SYNOPSIS:
set_up() is similar to run() (and is called by it). It calls/does the following:

• z_start()


• [explicitly] calls the testdriver's cmdline() function. this is done to make sure the local command-line processor is invoked, instead of the base class.


• calls cmdline(). If the function has not been redefined, rundriver's version will be invoked.


• version()


• validate()

For the last two functions, version() and validate(), if they are done, then nothing else is done by set_up(). This function does slightly less than run(). Most applications will use run() instead of this function.
 

run()
SIGNATURE: int run (const int argc, const char **argv, int *pi = NULL)
SYNOPSIS:
Runs whatever the user has configured the object to do according to command-line parameters given to it. The following standard functions are automatically invoked within this function:

function name(s)	description
usage(), help()	show help information. text is dumped to the console (ie stdout)
version()	show program version info (incl. build date)
validate()	run a validation test.
interactive()	go into "interactive mode"
batch()	run a batch operation

This function is intended to be invoked once. To call run() multiple times, reset() should be invoked before calling run() again.
PARAMETERS

• argc: the number of command-line arguments given to the object
• argv: an array of character strings holding the command-line arguments. The first element ('argv[0]') is the name of the program. The first command-line argument is put into 'argv[1]'. The structure of this parameter is the same as that passed into 'main()'.
• pi: [output] an optional error indicator variable. values:
  0: no errors detected

DESCRIPTION:
First, all the actions in set_up() will be done. Note that multiple modes (eg "batch" and "interactive") cannot be performed in 1 all to run().
Typically, one [only] of the following actions will be performed by calling this function:

help() - shows a help menu, printed out to the console window;
version() - shows version information;
validate() - runs validation operations, which should QA the object or components the test driver addresses;
interactive() - run a menu-driven, console-based "interactive" session with the user (this function "belongs" to base class rundriver_o);
batch() - run a set of instructions in "batch mode". Like interactive(), this function "belongs" to base class rundriver_o. The purpose of this function can be for the developer to store example code for the underlying object/components, or perhaps to run another set of validation tests (less formally than running 'validation'), or even for implementing operations that the object is to do as a traditional computer batch run, which can then be periodically called, say via a nightly cron job.

RETURNS:
0: a testdriver task was done (eg, validation; batch; interactive)
1: no standard "testdriver" operations were invoked; no errors
-1: an error occurred (see value of output var 'pi' for more info)
TRAITS: this function is virtual
 

validate()
SIGNATURE: int validate (const char *name, count_t &ne, int *pi = NULL)
SYNOPSIS: this member function is the place where the application has code can 'validate' the correctness of the underlying object.
DESCRIPTION:
validate() is a virtual function, so one should sub-class from testdriver_o and implement this function. As is, in the testdriver_o class, it does nothing and returns 1. A return code of 1 means that validation has not been implemented for the object (see the discussion in the "quick &6mp; dirty" group - the parent page of this one).
You should always do the following in this function:

• set 'ne' to 0 at the start of the function;
• increment 'ne' whenever an error is encountered;
• upon hitting an error, call validate_toend() to check if the test should contiue. if so (if it returns TRUE), just increment 'ne'. You can print a warning message to stdout. if validate_toend() returns FALSE, the function should return -1 upon hitting the error.

After the tests are done, show_validation_results() will be called. to "announce" the results of the validation test. if the component has completed all its tests successfully, show_validation_results() (which see) should be called with a result value of 0.
RETURNS:
This should return an integer code indicating the result of testing the component. The value should be one of the following:

return code	exit code	meaning
0	0	test passed
-1	1	test failed
1	2	validation has not been implemented
2	3	validation cannot be implemented
3	4	a configuration problem exists
4	5	dry mode

See the main discussion for the "quickdirt" group for more information about the meaning of validation exit codes.
TRAITS: this function is virtual
 

show_validation_results()
SIGNATURE: void show_validation_results (const char *progname, count_t ne, int x)
SYNOPSIS:
prints the results of validation() to the console. this function is automatically called immediately after validation(); that is, after tests are completed. It announces the test results by printing to the console (stdout).
PARAMETERS

• progname: the name of the program, as it is to be displayed.
• ne: the number of errors found in validate(). note that if the validation test runs in "abort on error" mode, this number will be either 0 or 1.
• x: ??

TRAITS: this function is virtual
 

cmdline()
SIGNATURE: int cmdline (const int argc, const char **argv, int *pi = NULL)
SYNOPSIS:
Processes command-line arguments. Its first 2 parameters should be provided from the same arguments as used by main(), (or whatever is the entry point of your program). They can be "synthetic" - you can process the real command-line list and modify it before calling this function, or make up your own.
DESCRIPTION:
This member function will call rundriver_o::cmdline() internally. Besides what the base class (rundriver_o) does, it also checks for and handles "batch" and "interactive" modes. You can override this virtual function with your own implementation - that is typical. If you do make your own: If all the command-line parameters are ok for your application, it should return 0. If an unknown option is encountered, your cmdline() should return 1. If it returns 0, this signals to the testdriver object that all is well, and run() will return with 0. Otherwise, amy options valid to your application but unknown to the testdriver class will result in an "unknown option mode" causing the testdriver to return -1 from run().
 

show_validation_results()
SIGNATURE: void show_validation_results (const char *nam, count_t n, int x)
SYNOPSIS: this member function is called after 'validate()'.
 

help()
SIGNATURE: void help (const char *progname = NULL) const
SYNOPSIS:
displays a help message. This function is intended to print out information to the console a general description of what the program or object does. This is invoked with the command-line parameter "-h" or "-help".
 

usage()
SIGNATURE: void usage (const char * = NULL) const
SYNOPSIS:
This is an alternative to help(), intended to be a place where you can provide usage information about your test driver program (such as a list of the command-line parameters with their descriptions).
 

description()
SIGNATURE: void description (const char * = NULL) const
SYNOPSIS: like help() or usage(), but more detailed. this is mated to the command-line parameter "-help".
 

usage.
The member function validate() is virtual. The code in the testdriver_o class actually contains code to validate this class. But, in real applications, in order to meaningfully use the testdriver_o class, you need to sub-class it. A validation test in the Z Directory context is implemented to result in a simple pass / fail outcome. there are no grades assigned - either a component works as expected, or not. Validation can terminate as soon as an error is encountered, or keep going. The user can control whether or not to abort on error, or run to the end, tallying an error count. This is controlled by the standard rundriver flags -nonstop and -abort_onerr.

The following table summarizes protocols for validation:

Note: sometimes it is not possible or practical to do validation. A case in point is the barecomputer object: this object simply returns information about a computer, but there is nothing to base validation on. If barecomputer reports that the computer name is XYZ, how can you prove it? One can write system calls that retrieve the name, but that simply duplicates the code inside the barecomputer object. One needs a "lateral technique" that verifies the results.

Another case in point is baredisk_o, which can retrieve the capacity of a hard disk. Suppose your object reports that your main hard drive can hold 120 GB. You can test that by writing to the disk until it fills up, then counting the number of bytes used to perform the test. However, this is a destructive, impractical test - akin to testing bombs (eg, how to test a bomb without exploding it?).

Note: an example of a configuration problem that prevents validation from running is a component that requires a database, but none is available (or the program at least cannot find or access one).

Note that the program exit code is not the same as the return code from the validation function. It is your responsibility to ensure that the function return code is correct.

Announcement of results:
If volume is set 0, validation will not report any results. You should not print anything in inside validate().
If volume is 10 or greater, validation will print results to the console.
If volume is in the range 10..20, your validation code (inside validate()) may print minimal (milestone) announcements and failures. There should not be more than 4 output lines.
If volume is in the range 20..30, your validation code (inside validate()) may print all messages from below volumde level 20, and a few detail lines (not to exceed 15 total, roughly).
If volume is in the range 30..50, your validation code (in validate()) may print summaries of each test item.
If volume is in the 60 or above, your validation code can print copious, detailed output.
The suggested format for A typical error message in validation would be "VALIDATION" followed by the compoenent name in parenthesis, followed by a message ID #, then the message. For example,

const char *progname = "my_object";
if (volume() > 20)
    std::cout << "VALIDATION (" << progname << "): [01] get() failed\n";

limitations.
It is entire up to the application to implement the member function interactive(). Note that this object has no connection with the [layer 0] menuloop_o class, which is conceptually equivalent to interactive(). The testdriver_o class may one day in the future recieve some upgrades to work with the menuloop_o class.