Skip to content

TSTool / Command / StartRegressionTestResultsReport

Overview

The StartRegressionTestResultsReport command starts a report file (and optionally results table) to be written to as regression tests are run. The CreateRegressionTestCommandFile command automatically inserts this command and is the preferred way to set up automated tests. The RunCommands commands will write to this file (and optionally results table) if available.

Command Editor

The following dialog is used to edit the command and illustrates the command syntax.

StartRegressionTestResultsReport

StartRegressionTestResultsReport Command Editor (see also the full-size image)

Command Syntax

The command syntax is as follows:

StartRegressionTestResultsReport(Parameter="Value",...)

Command Parameters

Parameter                           Description Default          
OutputFile
required		 The regression results report file to write, as an absolute path or relative to the command file. Can use ${Property}. None - must be specified.
TestResultsTableID The identifier of an output table to be created. The table will be created. No table will be output.

Examples

See the Quality Control chapter for how to set up a regression test.

Each of the command files that are run with RunCommands commands should produce expected time series results, without warnings. If any command file unexpectedly produces a warning, a warning will also be visible in TSTool. The issue can then be evaluated to determine whether a software or configuration change is necessary. An example of the output file is:

# File generated by...
# program:      TSTool 10.20.00 (2013-04-10)
# user:         sam
# date:         Sat Apr 20 13:36:05 MDT 2013
# host:         AMAZON
# directory:    C:\Develop\TSTool_SourceBuild\TSTool\test\regression\TestSuites\commands_general\run
# command line: TSTool
#  -home test/operational/CDSS
#
# Command file regression test report from StartRegressionTestResultsReport() and RunCommands()
#
# Explanation of columns:
#
# Num: count of the tests
# Enabled: TRUE if test enabled or FALSE if "#@enabled false" in command file
# Run Time: run time in milliseconds
# Test Pass/Fail:
#    The test status below may be PASS or FAIL (or blank if disabled).
#    A test will pass if the command file actual status matches the expected status.
#    Disabled tests are not run and do not count as PASS or FAIL.
#    Search for *FAIL* to find failed tests.
# Commands Expected Status:
#    Default is assumed to be SUCCESS.
#    "#@expectedStatus Warning|Failure" comment in command file overrides default.
# Commands Actual Status:
#    The most severe status (Success|Warning|Failure) for each command file.
#
#    |       |Test  |Commands  |Commands   |
#    |       |Pass/ |Expected  |Actual     |
# Num|Enabled|Fail  |Status    |Status     |Command File
#----+-------+------+----------+-----------+----------------------------------------------------------------------------------------
    1| TRUE  | PASS |SUCCESS   |SUCCESS    |C:\Develop\TSTool_SourceBuild\TSTool\test\regression\commands\general\ARMA\Test_ARMA_Day.TSTool
    2| TRUE  | PASS |SUCCESS   |SUCCESS    |C:\Develop\TSTool_SourceBuild\TSTool\test\regression\commands\general\ARMA\Test_ARMA_Legacy.TSTool
    3| TRUE  | PASS |SUCCESS   |SUCCESS    |C:\Develop\TSTool_SourceBuild\TSTool\test\regression\commands\general\ARMA\Test_ARMA_Legacy_Ast.TSTool
    4| TRUE  | PASS |SUCCESS   |SUCCESS    |C:\Develop\TSTool_SourceBuild\TSTool\test\regression\commands\general\ARMA\Test_ARMA_Legacy…
...
...
...
...
#----+-------+-------+------+----------+-----------+----------------------------------------------------------------------------------------
FAIL count     = 0, 0.000%
PASS count     = 17, 100.000%
Disabled count = 1
#--------------------------------
Total          = 18

Troubleshooting

See Also