Test harness

KCML provides a mechanism whereby a number of tests can be automatically run under the control of a test harness in the form of a special TRY block. The tests consist of a collection of DEFSUBs in the same program which share a common name prefix. The details of the tests are encapsulated in these functions which must perform any setup, perform the test and check its results. Incorrect results should be reported with a STOP or PANIC. All the tests must be of the same form taking the same number and types of arguments and returning the same result type.

To mark a TRY as a test harness, the built in function 'KCML_SetTestCallback() should be executed in the body of the TRY. It will construct an array of all the DEFSUBs in the nominated library that have a common prefix and return it to the application which can then call the test functions in the body of the TRY. The library to be searched is specified by the last argument to the call and is either the user specified name from LIBRARY ADD or blank for the current program.

The third argument to the call nominates a local DEFSUB callback function that will be called if the test fails due to one of these reasons:

The callback function will be passed a BYREF reference to a _KCML_TEST_INFO record with details of the reason for the error. It executes in the context of the failure so in the case of an error $ERR will report the text of error message. In the case of a STOP or PANIC $ERR will return the optional text associated with that statement. A stack trace can be obtained with 'KCML_Debug_GetReturnStack(). The application can log these errors as it sees fit. When the callback function returns, execution will resume with the next test. To abandon the tests altogether the callback can use a THROW ERR statement to throw a user error such as _KCML_USER_ERROR. This will cause execution to switch to the CATCH part of the TRY block.

If the callback function itself suffers a runtime error or executes a STOP then it will be handled normally forcing either a fatal PANIC or perhaps forcing the program into the workbench on a development system. If it executes a PANIC then the PANIC be generated but the program will not terminate.

The callback function must make a call to 'KCML_ClearTestState() as the last statement to clear internal flags used to prevent recursion.

Example

PUBLIC DEFSUB 'DoTest()
	LOCAL DIM n, pfnTests(1) AS SYM('x), i
	TRY
		n = 'KCML_SetTestCallback("TEST_", BYREF pfnTests(), SYM('ErrorCallback), "")
		PRINT "found";n;"tests"
		FOR i = 1 TO n
			PRINT "running test";i;SYMNAME(pfnTests(i))
			'SYM(*pfnTests(i))()
		END FOR
	CATCH
		PRINT "Test abandoned"
	END TRY
	STOP
END SUB
PRIVATE DEFSUB 'ErrorCallback(BYREF e$_KCML_TEST_INFO)
	PRINT "ERROR callback type=";FLD(e$.KCML_TEST_Reason$)
	'KCML_ClearTestState()
END SUB
PRIVATE DEFSUB 'Test_test1()
	LOCAL DIM n
	n = 2 + 2
	IF (n != 4) THEN PANIC
END SUB
PRIVATE DEFSUB 'Test_test2()
	PRINT "C62 error coming up"
	PRINT 2/0
END SUB

Compatibility

This feature was introduced in KCML 7.02.