Skip to content

TSTool / Command / CheckTimeSeries

Overview

The CheckTimeSeries command checks time series data values against criteria, for example to identify missing, erroneous, or extreme data values. A warning is generated for each match and time series values optionally can be flagged, which allows annotation on graphs and reports. Values that meet the check criteria also can be removed (if irregular interval), or set to missing. Check results can be saved to an output table for output and further processing. The WriteCheckFile command also can be used to write a summary of the warnings based on command messages. The CheckTimeSeriesStatistic command checks a statistic for the entire time series (e.g., missing value count). See also the Delta command, which creates new time series as the change between each value – this command may be necessary in cases where data periodically reset to a starting value, prior to performing a Change> check, for example.

Command Editor

The following dialog is used to edit the command and illustrates the syntax of the command for time series parameters.

CheckTimeSeries

CheckTimeSeries Command Editor for Time Series Parameters (see also the full-size image)

The following dialog is used to edit the command and illustrates the syntax of the command for criteria and action parameters.

CheckTimeSeries Criteria

CheckTimeSeries Command Editor for Check Criteria and Action Parameters (see also the full-size image)

The following dialog is used to edit the command and illustrates the syntax of the command for analysis period and window parameters.

CheckTimeSeries Time

CheckTimeSeries Command Editor for Analysis Period and Window Parameters (see also the full-size image)

The following dialog is used to edit the command and illustrates the syntax of the command for output table parameters.

CheckTimeSeries Table

CheckTimeSeries Command Editor for Output Table Parameters (see also the full-size image)

The following dialog is used to edit the command and illustrates the syntax of the command for output property parameters.

CheckTimeSeries Properties

CheckTimeSeries Command Editor for Output Properties Parameters (see also the full-size image)

Command Syntax

The command syntax is as follows:

CheckTimeSeries(Parameter="Value",...)

Command Parameters

Parameter                                            Description Default                           
TSList Indicates the list of time series to be processed, one of:
  • AllMatchingTSID – all time series that match the TSID (single TSID or TSID with wildcards) will be processed.
  • AllTS – all time series before the command.
  • EnsembleID – all time series in the ensemble will be processed (see the EnsembleID parameter).
  • FirstMatchingTSID – the first time series that matches the TSID (single TSID or TSID with wildcards) will be processed.
  • LastMatchingTSID – the last time series that matches the TSID (single TSID or TSID with wildcards) will be processed.
  • SelectedTS – the time series are those selected with the SelectTimeSeries command.
 AllTS
TSID The time series identifier or alias for the time series to be processed, using the * wildcard character to match multiple time series. Can be specified using ${Property}. Required if TSList=*TSID
EnsembleID The ensemble to be processed, if processing an ensemble. Can be specified using ${Property}. Required if TSList=*EnsembleID
CheckCriteria The criteria that is checked, one of the following. Missing values are skipped except for cases where the statistic is specific to missing values.
  • AbsChange> – check for absolute change from one value to the next value > Value1
  • AbsChangePercent> – check for absolute change in percent from one value to the next value > Value1.
  • Change> – check for change > Value1.
  • Change< – check for change < Value1.
  • InRange – check for value >= Value1 and <= Value2.
  • OutOfRange – check for value < Value1 or > Value2.
  • Missing – check for missing values.
  • Repeat – check for Value1 repeating values (i.e., if Value1=2, then the check will detect 2 adjacent values that are the same). If the flag or action are specified, values Value1+ in the sequence are modified (i.e., if Value1=2, the 2nd and subsequent repeating values will be modified by the action).
  • < – check for values < Value1.
  • <= – check for values <= Value1.
  • > – check for values > Value1.
  • >= – check for values >= Value1.
  • == – check for values equal to Value1.
    		 None – must be specified.
    Value1 A parameter that is used for specific CheckCriteria values.
    Value2 A parameter that is used for specific CheckCriteria values.
    ProblemType The problem type that will be shown in warning messages. CheckCriteria
    MaxWarnings The maximum number of warnings to list for each time series, useful if analysis results in many warnings. List all warnings.
    Flag A string to use for a flag on values that are detected during the check, which will be shown in the HTML summary report. No flag.
    FlagDesc Description for the flag. No description.
    Action Action to take for matched values, in addition to generating warnings:
    • Remove – remove the values. For irregular interval time series the values will be removed. For regular interval time series the values will be set to missing.
    • SetMissing – set the values to missing.
    		 No action is taken.
    AnalysisStart The date/time to start analyzing data. Can be specified using processor ${Property}. Analyze full period.
    AnalysisEnd The date/time to end analyzing data. Can be specified using processor ${Property}. Analyze full period.
    AnalysisWindowStart The calendar date/time for the analysis start within each year. Specify using the format MM, MM-DD, MM-DD hh, or MM-DD hh:mm, consistent with the time series interval precision. A year of 2000 will be used internally to parse the date/time. Use this parameter to limit data processing within the year, for example to analyze only a season. Analyze the full year.
    AnalysisWindowEnd Specify date/time for the analysis end within each year. See AnalysisWindowStart for details. Analyze the full year.
    TableID Identifier for output table to contain check results. Specify an existing table or new table to create. Can be specified using processor ${Property}. No table output.
    TableTSIDColumn Table column name for time series TSID. Required for table.
    TableTSIDFormat The specification to format the time series identifier to insert into the TSID column. Use the format choices and other characters to define a unique identifier. Required for table.
    TableDateTimeColumn Table column name for date/time. Column is not output.
    TableValueColumn Table column name for time series data values. Column is not output.
    TableValuePrecision Precision for values in TableValueColumn column. 4
    TableFlagColumn Table column name for time series data flag values. Column is not output.
    TableCheckTypeColumn Table column name for data check type. Column is not output.
    TableCheckMessageColumn Table column name for data check message. Column is not output.
    CheckCountProperty Name of processor property to set with count of values that meet the criteria. Can use processor ${Property} and time series % or ${ts:Property}. No property is set.
    CheckCountTimeSeriesProperty Name of time series property to set with count of values that meet the criteria. Can use processor ${Property} and time series % or ${ts:Property}. No property is set.

    Examples

    See the automated tests.

    Troubleshooting

    See Also