TSTool / Datastore Reference / Overview
Introduction
This reference section of the documentation provides information about datastores:
- databases:
- must be accessible via ODBC/JDBC connection, using a database driver software distributed with TSTool
- a datastore corresponding to a database is can be queried using Structured Query Language (SQL),
such as with the
ReadTableFromDataStorecommand - general support for additional database software can be added relatively easily if an ODBC/JDBC driver is available for the database
- requires additional software development and integration with TSTool for time series identifier and read/write commands
- web services:
- typically REST web services using a published API
- SOAP web services can also be supported
- requires software development and integration with TSTool
- file databases:
- ideally uses an open specification
- typically optimized for a use, such as HEC-DSS used with Army Corps software
- adding a new file database requires implementing software that is able to read the file format
- currently are handled as Input Type in the TSTool software but in the future will be migrated to a "file datastore" design
- files:
- ideally uses an open specification
- simple files, including general DateValue time series file and model files such as the State of Colorado's StateMod model
- currently are handled as Input Type in the TSTool software but in the future will be migrated to a "file datastore" design
A "datastore" is a data management component that saves time series and other data. TSTool originally used the term "input type" for some formats such as data files and HydroBase database. However, the newer "datastore" design has since been implemented to provide more flexibility in configuring data access. Newer databases and web services are treated as "datastores", whereas older files are treated as "input types". The "input type" notation is retained in the software, pending a full migration to datastores. Datastores and input types can be selected in the TSTool interface when browsing data. A software design goal is to move all input types to "file datastores" so that terminology is consistent.
Time series, tables, and other data objects can be read from datastores using Read commands.
Some datastores can also be written to, using Write commands.
Datastores containing time series have general characteristics that can be handled with general software features.
For example time series identifier commands (TSIDs) are used to uniquely identify time series in
a datastore and result in an automatic data read from the datastore.
However, some datastores have specific characteristics that are difficult to support in a general way,
in which case Read command parameters or datastore configuration properties are used to control software behavior.
Datastore List
The following datastores (and input types) are listed in alphabetical order by datastore name. Any datastores or other features that are obsolete may have already been removed from TSTool or may be removed in the future. If necessary, such features can be reimplemented as a plugin datastore and used on specific installations of TSTool.
| Datastore (link to documentation) | Technology | Spatial Extent | Contents |
|---|---|---|---|
| ColoradoHydroBaseRest | Web service (REST) | Colorado | Historical and real-time data |
| Colorado SMS (legacy) OBSOLETE - replaced with ColoradoHydroBaseRest web services |
Database (SQL Server) | Colorado | Real-time data |
| ColoradoWaterHBGuest OBSOLETE - replaced with ColoradoHydroBaseRest web services |
Web service (SOAP) | Colorado | Historical data |
| ColoradoWaterSMS OBSOLETE - replaced with ColoradoHydroBaseRest web services |
Web service (SOAP) | Colorado | Real-time data |
| DateValue | File (text) | No limit | General time series format |
| Delft FEWS PI XML | File (text) | No limit | Delft FEWS software file |
| GenericDatabase | Database (any ODBC) | No limit | Any SQL database |
| HEC-DSS File | File database (binary) | No limit | General time series database |
| HydroBase | Database (SQL Server) | Colorado | Colorado water resources (datastore version) |
| HydroBase (legacy, input type version) | Database (SQL Server) | Colorado | Colorado water resources (input type version). See the HydroBase datastore documentation for latest documentation. |
| HydroJSON | File (text) | No limit | HydroJSON time series format |
| MODSIM | File (text) | No limit | MODSIM modeling software data file |
| NRCS AWDB | Web service (SOAP) | USA | Snow, reservoir, climate, other |
| NDFD OBSOLETE - need new equivalent |
File (text) | No limit | National Digital Forecast Database |
| NWS Card OBSOLETE - NWS uses other formats |
File (text) | No limit | General time series format (hour interval) |
| NWSRFS ESP Trace Ensemble OBSOLETE |
File (binary) | No limit | Ensembles of time series (hour interval) |
| NWSRFS FS5Files OBSOLETE |
File database (binary) | No limit | National Weather Service River Forecast System |
| RCC ACIS | Web service (REST) | USA | Precipitation, temperature, other climate |
| Reclamation HDB | Database (Oracle) | USA | Water resources |
| Reclamation Pisces | Database (MySQL) | USA | Water resources |
| RiversideDB OBSOLETE |
Database (SQL Server) | No limit | Hydrologic modeling database. Has been removed from TSTool as of version 13. |
| RiverWare | File (text) | No limit | RiverWare modeling software data file |
| StateCU | File (text) | No limit | StateCU modeling software data file |
| StateCUB - StateCU Output | File (binary) | No limit | StateCU modeling software data file |
| StateMod | File (text) | No limit | StateMod modeling software data file |
| StateModB - StateMod Output | File (binary) | No limit | StateMod modeling software data file |
| USGS NWIS Daily | Web service (REST) | No limit | Water and other data (day interval) |
| USGS NWIS Groundwater | Web service (REST) | No limit | Groundwater and other data |
| USGS NWIS Instantaneous | Web service (REST) | No limit | Water and other data (instantaneous interval) |
| USGS NWIS RDB | File (text) | No limit | RDB format for USGS data (see USGS above) |
| WaterML | File (text) | No limit | WaterML format for USGS data (see USGS above) |
| WaterML2 | File (text) | No limit | WaterML format for USGS data (see USGS above) |
| WaterOneFlow prototype |
Web service (SOAP) | No limit | Web service for CUAHSI |
Datastore Configuration
Datastores are configured using datastore configuration files, which are described in the specific appendix.
The system/TSTool.cfg file located in the software installation folder provides properties to enable/disable
datastores globally, which turns off software features (such as Commands menu for commands
related to the datastore).
A similar file can be created in users's files (.tstool/NN/system/TSTool.cfg) to configure TSTool for each major version.
Built-in (installation) datastore configuration files are located in the software installation system/datastores folder.
User datastore configuration files are located in the .tstool/NN/datastores folder.
The user configuration files override the installation configuration files.
For example, the State of Colorado HydroBase datastore configuration file .tstool/13/datastores/HydroBase.cfg
defining a datastore named HydroBase will override a similarly-named datastore in
configuration file C:\Users\user\CDSS\TSTool-13.00.00\datastores\HydroBase.cfg.
Note that the Name property in the datastore configuration file defines the name, not the file name.
User datastore configuration files allow providing a more specific configuration for:
- database version for HydroBase datastore database installed on user's computer
- API key for ColoradoHydroBaseRest web services for the specific user
- user-specific datastore files used by software developers
Use the View / Datastores menu in TSTool to view datastores that are enabled, in particular to review configuration errors and to see which configuration file was used for a datastore.
Use the Tools / Options menu in TSTool to change TSTool configuration properties. If necessary, edit configuration files with a text editor.