TSTool / Release Notes / Version 13

Changes in Version 13.04.00
Changes in Version 13.03.00
Changes in Version 13.02.00
Changes in Version 13.00.00 - 13.01.00
Release notes for all versions

Changes in Version 13.04.00

This is the final 32-bit Java release. It is recommended to use the 64-bit version that follows this release.

[13.04.00] Remove the ColoradoWaterHBGuest datastore from software and configuration files since data access is no longer available.
[13.04.00] Remove the ColoradoWaterSMS datastore from software and configuration files since data access is no longer available.
[13.04.00] Fix the user interface so that the message Wait...initializing data connections... is removed sooner, rather than being present behind other components.
[13.04.00] Fix the DeleteTableRows command where operators with equal sign were not being handled properly in command editing.
[13.04.00] Fix the FillMOVE2 command, which was skipping the fill operation.
[13.04.00] Fix the FillRegression command, which was not handling some checks on the statistics due to how the installer was created
[13.04.00] Fix the ReadTableFromExcel command to handle date/time formula cell.
[13.04.00] The testing repository folders were simplified to remove regression and general folders and therefore all command documentation Examples section links have been updated. Examples links in older documentation will be broken.
[13.04.00] Update the CreateRegressionTestCommandFile command to have ExcludeTestSuite parameter to exclude test suites
[13.04.00] Update the TableMath command:
- add =, Cumulate, and Delta operators
- add Condition and ProcessRows parameters to indicate which rows to process
- improve logic for handling data types and provide better feedback when input prevents calculations
[13.04.00] Update the ReadStateCUB command to allow ${Property} notation for input file, TSID, and period
[13.04.00] Update the UnzipFile command to have RemoveOutputFolder parameter to control whether the output folder is removed before unzipping
[13.04.00] Add an appendix to the documentation that contains table concepts and provides documentation that is shared by multiple commands. This documentation will be expanded over time.

Changes in Version 13.03.00

This release includes many changes accumulated during an extended development period.

[13.03.00] Change so that user's .tstool/NN/system/TSTool.cfg file properties are recognized whether or not [TSTool] is at the top of the file.
[13.03.00] Fix the CheckTimeSeriesStatistic command to fix typos in parameters that resulted in invalid command
[13.03.00] Fix the ReadNrcsAwdb command to read heightOfDepthValue from web services (previously had a typo in the property)
[13.03.00] Update the ReadStateCU and ReadStateMod commands to better handle ${Property} in the InputFile parameter.
[13.03.00] Update the RunProgram command to work on Linux. Also add the EnvVars, IfNonZeroExitCode, StderrFile, and StdoutFile parameters.
[13.03.00] Update the TableToTimeSeries command to list time series in discovery mode if an alias is specified, for use in other commands.
[13.03.00] Update the WriteStateCU and WriteStateMod commands to enable TSID to handle ${Property}, as was previously documented.
[13.00.00] Fix the WriteTimeSeriesToGeoJSON command to output the correct bounding box features for each point (previously was using entire layer).
[13.03.00] Update the tstool script used for Linux to parse command line options, for example to run in headless mode on Linux.
[13.03.00] All command editors have been updated to implement ..., Abs, and Rel buttons to select files and folders and switch between absolute and relative paths, with default being to use relative path. The command reference documentation has been updated accordingly.
[13.03.00] Lowercase command file extension tstool is now included in file selectors and is the default enforced when saving. Mixed case TSTool extension is still supported.
[13.03.00] Time series graphs can now be saved as JSON format time series product file - this preliminary format is being evaluated to integrate with web applications.
[13.03.00] Fix a number of issues where table arrays were not cleanly supported.
[13.03.00] Update the Add command AnalysisStart and AnalysisEnd parameters to support using ${Property}.
[13.03.00] Update the AppendTable command to accept property notation in the ColumnFilters parameter.
[13.03.00] Update the ChangePeriod command NewStart and NewEnd parameters to support using ${Property}.
[13.03.00] Update the CloseDataStore command to have IfNotFound parameter to control error handling
[13.03.00] Update the # Comment command to support:
- @fixme - to indicate an item to fix
- @order - to control order of command files for automated testing
- @require - to check the software and datastore version
- @todo - to indicate an item to do
[13.03.00] Change the CompareFiles command:
- update Visual Diff feature to work when files use ${Property} in the name
- add ability to compare file properties such as size and modification time
- check the full length of each file when computing the number of different lines
[13.03.00] Update the CompareTimeSeries command to better support comparing irregular time series, add CompareFlags to control whether data flags are also compared.
[13.03.00] Update the CreateEnsembleFromOneTimeSeries command to have TransferDataHow to control how input time series are transferred to output time series traces.
[13.03.00] Update the CreateDataStoreDataDictionary to handle optional metadata for database software that do not provide metadata
[13.03.00] Update the CreateRegressionTestCommandFile command
- support multiple patterns to match files
- add TestResultFile parameter to set the output file in the created command file
[13.03.00] Update the DeleteTableRows command to allow * to delete all rows, additional functionality has been added for strings, and string operations have been made case-dependent.
[13.03.00] Update the FillInterpolate command to fill the ends of the fill period even if non-missing values are outside of that period
[13.03.00] Update the For command:
- List parameter support using ${Property}.
- Add ability to iterate over a list of time series.
[13.03.00] Update the ManipulateTableString to handle conversion of floating point numbers (as strings) to integers and properly round. Also add ToLowerCase, ToMixedCase, and ToUpperCase conversions.
[13.03.00] Update the NewTable command to support arrays.
[13.03.00] Update the NewTimeSeries command SetStart and SetEnd parameters to support using ${Property}.
[13.03.00] Update the ProcessTSProduct command to output the time series product file to reflect the full product definition.
[13.03.00] Update the ReadStateCU command to have Alias parameter to assign an alias to time series.
[13.03.00] Update the ReadStateModB command to add IncludeDataTypes and ExcludeDataTypes parameters, which limits the time series that are read
[13.03.00] Update the ReadTableFromDBF command to covert column data types.
[13.03.00] Update the RunningStatisticTimeSeries command to add statistics: Change, ChangeAbs, ChangeFraction, ChangeFractionAbs, ChangePercent, ChangePercentAbs, NewMax, and NewMin.
[13.03.00] Update the ReadTableFromDelimitedFile command to have ColumnNames parameter to specify column names when they are not in the file
[13.03.00] Update the ReadTableFromDataStore command to allow modifiers after table names in OrderBy.
[13.03.00] Update the RunCommands command:
- status messages for the command file that is run are more clearly displayed in the command status, which increases efficiency debugging issues
[13.03.00] Update the SetDataValue command TSID and EnsembleID parameters to support using ${Property}.
[13.03.00] Update the SetProperty command:
- allow a property to be set as a simple math operation
- allow setting a property from an environment variable
- allow setting a property from a Java system property
[13.03.00] Update the SetConstant command SetStart and SetEnd parameters to support using ${Property}.
[13.03.00] Update the SetFromTS command SetStart and SetEnd parameters to support using ${Property}.
[13.03.00] Update the SetTableValues command to add Column and Value parameters to set a single column value, useful for complex values such as arrays
[13.03.00] Update the StartLog command to have MaxSize parameter to limit log file size.
[13.03.00] Update the TableMath command input and output parameters to support properties.
[13.03.00] Update the Wait command to allow parameters to be specified as properties.
[13.03.00] Update the WebGet command:
- add the EncodeURI parameter to implement URL percent-encoding
- add the ConnectTimeout and ReadTimeout parameters to control timeouts
- add the RetryMax and RetryWait parameters to control retry attempts
- add the IfHttpError parameter to control HTTP status
[13.03.00] Update the WriteDelimitedFile command to have HeaderComments parameter to provide comments
[13.03.00] Update the WriteTableToDelimitedFile command parameters to provide more flexibility in controlling the file format and support array columns. Also enhance performance of the command.
[13.03.00] Update the PostgreSQL JDBC driver to version 42.2.9.
[13.03.00] If an output file created by TSTool that is listed in the Results area has tstool file extension, allow the file to be opened in the command list.
[13.03.00] Add new command line options:
- Property==Value can be used to set processor property to use for each command file run
- --datastore-substitute=oldname,newname can be used to substitute a datastore name for another, which allows command files without changing datastore names, such as in automated testing
[13.03.00] Add the Break command to break out of For loop block
[13.03.00] Add the experimental ChangeIntervalIrregularToRegular command to change interval of irregular interval time series to regular interval, simpler than the ChangeInterval command
[13.03.00] Add the CheckFile command to check file content and properties
[13.03.00] Add the Continue command to jump to end of For loop block
[13.03.00] Add the CreateFolder command to create a folder
[13.03.00] Add the FormatFile command to format a file for output, including for web
[13.03.00] Add the OpenDataStore command to open a datastore from command file
[13.03.00] Add the RunR command to run R scripts.
[13.03.00] Add the NewSQLiteDatabase command to create a SQLite database
[13.03.00] Add the TextEdit command to edit text files

Changes in Version 13.02.00

[13.02.00] Most menu items now have tooltips that provide additional information to users. The tooltips will be modified as feedback on their interpretation is received.
[13.02.00] The table command menu Manipulate Table Values has been resorted to facilitate use, with commands grouped by column, row, and other manipulation functionality.
[13.02.00] The AppendTable command has been updated with new ColumnData parameter to set constant data on appended rows.
[13.02.00] The DeleteTableColumns command has been updated with new KeepColumns parameter to streamline indicating which columns should be deleted.
[13.02.00] The DeleteTableRows command has been updated with new Condition parameter to filter which rows are deleted.
[13.02.00] The ReadColoradoHydroBaseRest command now reads DivComment time series, to facilitate troubleshooting diversion records. The data value is set to zero if no water was taken in a year.
[13.02.00] The ReadHydroBase and ReadColoradoHydroBaseRest commands have been updated to provide FillDivRecordsCarryForward parameter to allow control (rather than default). The behavior of both commands is consistent in terms of default parameter values. The WaterClass parameter has been added to allow matching a specific water class. A number of usability issues have also been fixed.
[13.02.00] The ReadTableFromDataStore and RunSql commands have been updated to run procedures that have parameters and set the return status as a property. Procedures that have the same name with different parameters are not yet handled.
[13.02.00] The StartLog command has been updated to write log messages to clarify the hand-off between closed and newly opened log file.
[13.02.00] Add RenameTableColumns command.
[13.02.00] Dialogs related to saving command files have been updated to provide an option to compare the in-memory (unsaved) command file with latest command file saved to disk. The View / Command File Diff tool has been implemented to show differences. This is useful when deciding whether to save or discard command file edits.

Changes in Versions 13.00.00 - 13.01.00

[13.00.00] Remove RiversideDB datastore built-in features. The features can be added later as a plugin if necessary.
[13.01.01] Fix bug where Find Command(s) popup menu was not working.
[13.01.00] Fix bug where highlighting line on graph by clicking on legend was not working.
[13.00.00] Fix bug where StateMod binary files over 2GB could not be read.
[13.00.00] Fix bug to allow StateMod time series to have up to 1500 years in *.xop time series files, to support climate change modeling.
[13.01.00] Improve datastore information for HydroBase to facilitate troubleshooting.
[13.01.00] Finish converting Word documentation to online format for remaining commands and datastores.
[13.00.00] Enable open source licenses for software and documentation.
[13.00.00] Migrate to new configuration folders:
- Installation files now include datastores and plugins folders, consistent with user files.
- User files are located in under .tstool/N folder with datastores, logs, plugins, and system folder, where N is the major TSTool version.
- Also automatically copy the command-file-history.txt file to new major version folder if a new installation.
[13.00.00] Ignore ColoradoWaterHBGuest and ColoradoWaterSMS web service datastores because the services are no longer available from the State of Colorado.
[13.00.00] Improve the Tools / Options information to explain TSTool configuration.
[13.00.00] Improve ColoradoHydroBaseRest web services features:
- Display additional information in the time series list.
- The query period has been enabled for all supported time series data types.
- Time series properties are set for station, structure, and other object data.
- The dataUrl property is set to allow accessing data on the website.
[13.00.00] Enhance the ManipulateTableString command to allow $ for replace and remove actions using \$. The $ without escape alone represents end of line, as previously implemented.
[13.00.00] Enhance the WriteTableToGeoJSON command to specify the GeoJSON version (default is RFC7946) via new Version parameter. Also add the CoordinatePrecision parameter to control precision on coordinates and bounding box.
[13.00.00] Enhance the WriteTimeSeriesToGeoJSON command by adding the CoordinatePrecision parameter to control precision on coordinates and bounding box.
[13.01.00] Add scripts to streamline creating Linux installer from Windows install.