Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • gwu479/Summa-Actors
  • numerical_simulations_lab/actors/Summa-Actors
2 results
Show changes
Commits on Source (459)
359 additional commits have been omitted to prevent performance issues.
Hide whitespace changes
Inline Side-by-side
Showing
with 1474 additions and 136 deletions
.gitignore 0 → 100644
View file @ 8e5f43c0
bin/summa_actors
build/cmake/build
build/summa
build/summa-sundials
build/.vscode
.vscode/settings.json
.DS_Store
laugh_tests
build/summa_sundials
build/summa_original
build/source/testing/containers/output_container/main
bin/submission_script.sh
build/summa_kinsol
bin/summa_kinsol
build/flibs-0.9/
build/cmake/build_be/
build/cmake/build_kinsol/
bin/data_be.csv
bin/data_kinsol.csv
bin/submission_copern.sh
bin/summa_be
bin/data_kinsol_state_vec.csv
bin/plot_resuduial.py
bin/state.png
bin/submission_script_array.sh
build/source/testing/containers/output_container/out.txt
build/summa_old
bin/config.json
bin/summa_sundials
build/cmake_build/
build/source/engine/access_write.f90 → .gitlab-ci.yml
View file @ 8e5f43c0
File moved
README.md
View file @ 8e5f43c0
# **NOTICE**
**This repository has been frozen. SUMMA-Actors project development has moved to: https://github.com/uofs-simlab/Summa-Actors**
# SUMMA-Actors: Structure for Unifying Multiple Modeling Alternatives with Actors
SUMMA-Actors is a modified version of the already existing SUMMA software that can be
found [here](https://github.com/CH-Earth/summa#readme). SUMMA-Actors is a modification
of SUMMA that uses the Actor Model to increase scalability and fault-tolerance. The actor which is known as the basic unit of concurrent computation is at the heart of this
software. SUMMA-Actors is built using the [C++ Actor Framework](https://github.com/actor-framework/actor-framework).
SUMMA-Actors is a modified version of the already existing SUMMA software that can be found [here](https://github.com/CH-Earth/summa#readme). SUMMA-Actors uses the Actor Model to increase scalability and fault-tolerance. It is built using the [C++ Actor Framework](https://github.com/actor-framework/actor-framework).
## Documentation
A more in-depth documentation can be found [here](https://summa-actors.readthedocs.io/en/latest/)
## Compiling Summa-Actors
SUMMA-Actors is written in C++ and FORTRAN and can be compiled with a C++ and FORTRAN
compiler from the GNU Compiler Collection. We have compiled SUMMA-Actors with the
following compilers:
A more in-depth documentation can be found [here](https://git.cs.usask.ca/numerical_simulations_lab/actors/Summa-Actors/-/wikis/home). SUMMA-Actors depends on many files from the [original SUMMA repo](https://github.com/CH-Earth/summa). Below is a quick start guide for compiling and running SUMMA-Actors, please consult our wiki for more in-depth documentation. Consider creating an issue for any missing documentation, asking questions, or providing suggestions.
## Preliminaries
SUMMA-Actors is meant to wrap around the existing SUMMA implementation. Therefore you will need to clone the SUMMA repo into the SUMMA-Actors build directory. SUMMA-Actors works with the master branch of SUMMA, which can be found [here](https://github.com/CH-Earth/summa). Below the directory structure is explained in more detail and the steps to compiling SUMMA-Actors are explained.
## Directory Structure
SUMMA-Actors is set up with the following sub-directories, we will consider the top level Summa Actors directory the `root_dir`:
- bin
- build
- includes
- source
- summa (https://github.com/KyleKlenk/summa/tree/summa-actors, summa-actors branch)
- CMakeLists.txt
- utils
- README.md
First clone Summa-Actors to your workstation. Then cd into `build/` and clone `summa` into Summa-Actor's build directory as folder summa.
## Compiling SUMMA-Actors
### Dependencies
SUMMA-Actors has only one additional dependency, the [C++ Actor Framework](https://github.com/actor-framework/actor-framework), specifically the [0.18.6
release](https://github.com/actor-framework/actor-framework/archive/refs/tags/0.18.6.tar.gz).
The following steps can be used to install the C++ Actor Framework on a Linux system:
```bash
wget https://github.com/actor-framework/actor-framework/archive/refs/tags/0.18.6.tar.gz
tar -xzf 0.18.6.tar.gz
cd actor-framework-0.18.6/
./configure --prefix=/path/to/install
cd build
make # [-j]
make install # [as root if necessary]
```
Additional dependencies required by both SUMMA and SUMMA actors are:
* g++
* gfortran
SUMMA-Actors depends on the following Libraries:
* [NetCDF-Fortran](https://github.com/Unidata/netcdf-fortran)
* [OpenBLAS](https://github.com/xianyi/OpenBLAS)
* [C++ Actor Framework](https://github.com/actor-framework/actor-framework)
Once the following libraries have been installed SUMMA-Actors can be compiled in
one of two ways. The first way is to modify the makefile directly and the second
is to invoke the makefile by shellscript:
1. Makefile:
Changes need to be made to the following variables in the Makefile:
- F_MASTER = directory/above/build
- FC = gfortran
- CC = g++
- INCLUDES = Path/to/netcdf/includes
- LIBRARIES = Path/to/netcdf/lib & Path/to/openblas
-lnetcdff -lopenblas
- ACTORS_INCLUDES = $INCLUDES & Path/to/CAF/includes
- ACTORS_LIBRARIES = $LIBRARIES & PATH/to/CAF/lib & PATH/to/summa.so
-lcaf_core -lcaf_io -lsumma -lopenblas -lnetcdff
Once these changes have been made SUMMA-Actors can be called with `make` from
the build/ directory.
Note: SUMMA is compiled as a shared library (libsumma.so) and the main program
will need to know where this library is located in order to properly link it.
By compiling both libsumma.so and summaMain in the same directory (build/) should be
enough. However if there are issues, specifing where libsumma.so will be compiled can be done
by adding the path to `ACTORS_LIBRARIES` in the makefile this should rectify the issues.
1. ShellScript:
In the build directory exists a example_compile.sh script that can be modified.
This is usually used for HPC computing environments and includes which modules
to load for Compute Canada or the University of Saskatchewan's Copernicus.
example_compile.sh contains instructions on what parts to modify and how to
invoke the makefile from the script.
Once the shellscript has been modified running it with `source your_script.sh` will compile
SUMMA-Actors.
After SUMMA-Actors is compiled you should be left with a libsumma.so and a summaMain file
in your build directory.
It is important to set the `LD_LIBRARY_PATH` environment variable before attempting to run
summa. This variable needs to point to the location of libsumma.so. If you compiled using the
shellscript using the `source` command and following the directions within the example_compile.sh
this should already be set for you.
## Running Summa-Actors
Once the binary is compiled it can be run like the following example command:
./summaMain -g 1 -n 10 -c /path/to/config/directory --config-file=/path/to/actors/config/file
-g = starting index of the first GRU to compute
-c = number of grus to run
-m = path to the file manager
OPTIONAL: --config-file = /path/to/config/file
This config file specifies to the C++ Actor Framework how many threads to use when executing the program. If left out the C++ Actor Framework will automatically set this value based on your system.
#### Config File ####
SUMMA-Actors settings can be modified from a JSON file provided in config/Summa_Actors_Settings.json.
There are three types of actors that can be configured:
* SummaActor
- OutputStructureSize = The number of timesteps in which an HRU can hold before needing to contact
the file_access_actor to write the data to a file.
- maxGRUPerJob = The number of GRUs that will be attemtpted to run at once. For example, if this value
is set to 500 and you invoke the program with ./summaMain -g 1 -n 1000 -c /path/to/config/directory.
SUMMA-Actors will only spawn 500 actors at a time and compute all 1000 in two batches.
Both of the above setting control the amount of RAM SUMMA-Actors uses. Larger numbers can cause your
job to run out of memory. We have found that setting both to 500 uses around 20GB of RAM for reference.
* JobActor
- FileManagerPath = Path the the fileManager.txt file needed by SUMMA. This is remained relativley
unchanged from the original version of SUMMA. With two additions. An example file is provided in the
config/ directory called fileManager_example.txt
- outputCSV = Boolean value for if you would like individual HRU run-time statsicts when they complete.
- csvPath = The path that the csv file will be written to.
* HRUActor
- printOutput = Boolean value for if you would like each HRU to print information on where it is in
its computation. ie. what timestep it is on and some other timing information.
- outputFrequency = The frequency in which you would like an HRU printing to stdout. The number specified is the interval in timesteps in which an HRU will print. Note: Lower numbers can see decreased performance as stdout will begin to lag the more that needs to be printed.
* [SUNDIALS V6.6](https://github.com/LLNL/sundials/releases/tag/v6.6.0) (optional)
### Steps To Compile
```
git clone https://git.cs.usask.ca/numerical_simulations_lab/actors/Summa-Actors.git
cd Summa-Actors/build
git clone -b summa-actors https://github.com/KyleKlenk/summa.git
cmake -B cmake_build -S . -DCMAKE_BUILD_TYPE=Build_Type
cmake --build cmake_build --target all -j
```
Available build types are:
* BE or BE_Debug: Builds for local machine
* BE_Cluster or BE_Cluster_Debug: See below section (Modules for Alliance Machines)
### Modules for Alliance Machines
To compile on Alliance machines the following modules can be loaded:
```bash
module load StdEnv/2020
module load gcc/9.3.0
module load openblas/0.3.17
module load netcdf-fortran/4.5.2
module load caf
```
## Running SUMMA-Actors
SUMMA-Actors can be run in two modes, distributed and non-distributed. The distributed mode used to create ad-hoc clusters. The command line arguments are kept as close to the original SUMMA as possible. However, there are options not yet available in SUMMA-Actors that are in the original SUMMA. Here is a full list of the available and unavailable options:
```
Usage: summa_actors -m master_file [-g startGRU countGRU] [-c config_file] [-b backup_server] [-s server_mode]
Available options:
-m, --master: Define path/name of master file (can be specified in config)
-g, --gru: Run a subset of countGRU GRUs starting from index startGRU
-c, --config: Path name of the Summa-Actors config file (optional but recommended)
--gen-config: Generate a config file
-b, --backup-server: Start backup server, requires a server and config_file
-s, --server-mode: Enable server mode
-h, --help: Print this help message
Unimplemented Options:
-n --newFile Define frequency [noNewFiles,newFileEveryOct1] of new output files
-s --suffix Add fileSuffix to the output files
-h --hru Run a single HRU with index of iHRU
-r --restart Define frequency [y,m,d,e,never] to write restart files
-p --progress Define frequency [m,d,h,never] to print progress
-v --version Display version information of the current build
```
Instructions for using each mode are provided below, if you just want to skip ahead and get started using the default options.
### Non-Distributed Mode (Similar to original SUMMA)
Using SUMMA-Actors in non-distributed mode is like running the normal SUMMA. The difference is that HRUs will run concurrently in SUMMA-Actors and maximize resource use. Usage is very close to SUMMA, and the same input files are required. In depth documentation for configuring a SUMMA run can be found [here](https://summa.readthedocs.io/en/latest/).
Usage is as follows: ./summa_actors -m master_file [-g startGRU countGRU] [-c config_file]
The master_file for SUMMA-Actors can either be defined explicitly on the command line or it can be included in the new to SUMMA-Actors config_file. The config_file is optional but it is highly recommended as it does allow users to fine tune how SUMMA-Actor will perform. We are working to automate these aspects but for now its best to define it for the domain and compute environment in use. Using `./summa_actors --gen-config` will generate a configuration file that can then be filled in. See the section on config_file in this readme for more information.
### Distributed Mode
Using SUMMA-Actors in distributed mode allows SUMMA-Actors to solve batches and dymically assign them to nodes and reassign them in the event of node failures.
To use this feature there are 3 actors that can be spawned to cluster nodes together and add additional redundancy to the system. All settings are to be defined in the config_file for this mode. In the config file the user must set `distributed_mode` to true, define a `port`, and configure the domain to run. The domain is defined by how many total GRUs the user wishes to compute and the number of GRUs that should be assembled into a batch. Finally, the `server_list` should be set to the hostname of the backup_servers that are started.
To start the system one start the server first with `./summa_actors -c -s`
Backup servers can be added with `./summa_actors -c -b`
Clients can simply be added with `./summa_actors -c`
NOTE: Each system will need a copy of the forcing data or input data, or the data should be in a singular location like on a cluster system.
### Config File
The config file is a JSON file that is used to configure SUMMA-Actors. It is highly recommended to use a config file as it allows the user to fine tune how SUMMA-Actors will perform. Using `./summa_actors --gen-config` will generate a configuration file that can then be filled in. Below is a list of the available options and their descriptions.
#### Distributed_Settings
The distributed settings are used to configure the distributed mode of SUMMA-Actors. The following options are available:
- distributed_mode: (true/false) Enables distributed mode
- servers_list: (list of strings) List of hostnames for backup servers
- port: (int) Port to use for communication
- total_hru_count: (int) Total number of HRUs in the entire domain
- num_hru_per_batch: (int) Number of HRUs to assemble into a batches
#### Summa_Actor
The Summa_Actor settings are used to restrict how many HRU actor can run at once. The following options are available:
- max_gru_per_job: (int) Maximum number of HRUs that can be run at once
#### Job_Actor
The Job_Actor settings can specify the file manager path so it does not need to be specified on the command line, and the maximum number of times to attempt an HRU before giving up. The following options are available:
- file_manager_path: (string) Path to the file manager file
- max_run_attempts: (int) Maximum number of times to attempt an HRU before giving up
#### File_Access_Actor
The File_Access_Actor settings are used to configure the file access actor. The following options are available:
- num_partitions_in_output_buffer: (int) Number of partitions in the output buffer
- num_timesteps_in_output_buffer: (int) Number of timesteps in the output buffer
#### HRU_Actor
The HRU_Actor settings are used to configure the HRU actor. The following options are available:
- print_output: (true/false) Print output to the screen
- output_frequency: (int) Frequency to print output to the screen
- dt_init_factor: (int) Factor to multiply dt_init by
- rel_tol: (float) Relative tolerance for the HRU actor (Requires Sundials)
- abs_tol: (float) Absolute tolerance for the HRU actor (Requires Sundials)
## Credits
The inital implementation of SUMMA is credited to the inital publications below. These
The initial implementation of SUMMA is credited to the initial publications below. These
publications can be found in [Water Resources Research](http://onlinelibrary.wiley.com/journal/10.1002/(ISSN)1944-7973).
* Clark, M. P., B. Nijssen, J. D. Lundquist, D. Kavetski, D. E. Rupp, R. A. Woods, J. E. Freer, E. D. Gutmann, A. W. Wood, L. D. Brekke, J. R. Arnold, D. J. Gochis, R. M. Rasmussen, 2015a: A unified approach for process-based hydrologic modeling: Part 1. Modeling concept. _Water Resources Research_, [doi:10.1002/2015WR017198](http://dx.doi.org/10.1002/2015WR017198).<a id="clark_2015a"></a>
......
actorsChanges.txt 0 → 100644
View file @ 8e5f43c0
driver
/summaActors_alarms.f90 -- remove
/summaActors_globalData.-- could combine and remove, renamed for now
/summaActors_type.f90 -- could combine and remove, renamed for now, adds a new function, but don't use both
/summaActors_util.f90 -- could combine and remove, renamed for now, adds a new function, but don't use both
/summaversion.inc -- remove
dshare
/csv_file_1d.f90 -- remove?? not seeing it used anywhere?
/csv_file_2d.f90 -- remove?? not seeing it used anywhere?
/csv_file.f90 -- remove?? IS this only in kinsol? do we need?
/data_types.f90 -- has a lot
/globalData.f90 -- remove with ACTORS_ACTIVE adds forcing stuff and tmZoneOffsetFracDay, fracJulDay, yearLength not as global
/varLookup.f90 -- remove with ACTORS_ACTIVE uses a iso_c binding call
engine
can remove sundials folder shouldn't need with SUNDIALS_ACTIVE
/sundials/coupled_em.f90 -- remove with ACTORS_ACTIVE needs fracJulDay, yearLength
/sundials/mDecisions.f90 -- remove with ACTORS_ACTIVE called with iso_c binding
/sundials/t2enthalpy.f90 -- remove
/alloc_fileAccess.f90 -- new
/allocspace[Actors].f90 -- this is a replacement, rename, and see how changes CHECK IF NECESSARY!
/check_icondActors.f90-- could combine and remove, renamed for now, adds a new function, but don't use both
/checkStruc.f90 -- remove
/coupled_em.f90 -- remove shouldn't need the non-sundials version
/derivforce.f90 -- remove with ACTORS_ACTIVE tmZoneOffsetFracDay needs to be variable not global
/ffile_info.f90 -- very different, call this ffile_infoActors?
/mDecisions.f90 -- remove shouldn't need the non-sundials version
/read_dimension.-- remove, part of read_attrb, - some deallocation changes, use this in all (not actually used in actors, ??)
- also Actors assumes don't have aspect variable .. why?
/read_pinit.f90 -- remove
/vegPhenlgy.f90 -- remove with ACTORS_ACTIVE needs fracJulDay, yearLength
SUMMA_FILEACCESS_INTERFACE = \
output_structure.f90 \
cppwrap_fileAccess.f90 \
read_attrb.f90 \
read_force.f90 \
read_param.f90 \
read_icondFromStructure.f90 \
ARE THESE ALL SORT OF REPEATS?? DO WE EVEN USE ALL OF THEM? specifically read_icondFromStructure ... what is that ... do we call its
Can we give them the same names as the regular summa files if they are repeats, so we know to change them??
changed calls in run_oneHRU so can use (modded) currently in actors /derivforce.f90 /coupled_em.f90 /vegPhenlgy.f90 as is
hookup
/ascii_util.f90 -- use Kyle's version everywhere!! (so remove)
/summaActors_FileManager.-- could combine and remove, renamed for now??
lapack -- remove
/Makefile -- remove
/README -- remove
netcdf
/def_output.f90 -- has a lot ... use Kyle's? Hard to tell which is Actors specific and which is just changed
/netcdf_util.f90 -- remove
/read_icondActors.-- has a lot ... could combine and remove, renamed for now,
/writeOutput.f90 -- part of modelwrite.f90 -- renamed to modelwrite so can replace, and then IS THIS VERSION BETTER can we change the other version and put a few compiler directives in?
/writeRestart.f90 -- remove, same in modelwrite, part of it
build/CMakeLists.txt 0 → 100644
View file @ 8e5f43c0
cmake_minimum_required(VERSION 3.10 FATAL_ERROR)
project(summa_actors LANGUAGES CXX Fortran)
enable_language(C)
SET (CMAKE_Fortran_COMPILER gfortran)
include(FortranCInterface)
FortranCInterface_VERIFY(CXX)
get_filename_component(PARENT_DIR "${CMAKE_CURRENT_SOURCE_DIR}" DIRECTORY)
set(EXEC_DIR ${PARENT_DIR}/bin) # set the output directory for executables
SET(F_MASTER ${PARENT_DIR}/build/summa)
# Add options for build type
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${EXEC_DIR})
set(CMAKE_CONFIGURATION_TYPES BE BE_Debug BE_Cluster BE_Cluster_Debug)
# Set Compiler Options
if(CMAKE_BUILD_TYPE MATCHES Debug)
message("\nSetting Debug Options\n")
add_compile_definitions(DEBUG)
set(FLAGS_NOAH -g -O0 -fbacktrace -fbounds-check -ffree-form -ffree-line-length-none -fmax-errors=0 -fPIC -Wfatal-errors ${FLAGS_OPT})
set(FLAGS_ALL -g -O0 -fbacktrace -fbounds-check -ffree-line-length-none -fmax-errors=0 -fPIC -Wfatal-errors -cpp ${FLAGS_OPT})
set(FLAGS_CXX -g -O0 -fbounds-check -Wfatal-errors -std=c++17 ${FLAGS_OPT})
else()
message("\nSetting Release Options")
set(FLAGS_NOAH -O3 -ffree-form -ffree-line-length-none -fmax-errors=0 -fPIC -Wfatal-errors ${FLAGS_OPT})
set(FLAGS_ALL -O3 -ffree-line-length-none -fmax-errors=0 -fPIC -Wfatal-errors -cpp ${FLAGS_OPT})
set(FLAGS_CXX -O3 -Wfatal-errors -std=c++17 ${FLAGS_OPT})
endif()
find_package(CAF COMPONENTS core io REQUIRED)
set(CAF_INCLUDES ${CAF_INCLUDE_DIRS})
set(CAF_LIBRARIES CAF::core CAF::io)
set(EXEC_NAME summa_be)
if (CMAKE_BUILD_TYPE MATCHES Cluster)
find_package(OpenBLAS REQUIRED)
# Set include directories
set(INCLUDES $ENV{EBROOTNETCDFMINFORTRAN}/include ${netCDF_INCLUDES} ${OpenBLAS_INCLUDES})
set(LIBRARIES SUMMA_NOAHMP ${OpenBLAS_LIBRARIES} -lnetcdff)
set(INC_ACTORS
${CAF_INCLUDES}
${PARENT_DIR}/build/includes/global
${PARENT_DIR}/build/includes/summa_actor
${PARENT_DIR}/build/includes/job_actor
${PARENT_DIR}/build/includes/file_access_actor
${PARENT_DIR}/build/includes/hru_actor)
set(LIB_ACTORS
${CAF_LIBRARIES}
-lnetcdf
-lcaf_core
-lcaf_io)
else()
# Set include and library for Fortran portion of the code
set(INCLUDES "/usr/include" "/usr/local/include")
set(LIBRARIES -lnetcdff -lopenblas SUMMA_NOAHMP)
# Set include and library for C++ portion of the code
set(INC_ACTORS CAF::core CAF::io ${INCLUDES}
"${PARENT_DIR}/build/includes/global"
"${PARENT_DIR}/build/includes/summa_actor"
"${PARENT_DIR}/build/includes/job_actor"
"${PARENT_DIR}/build/includes/file_access_actor"
"${PARENT_DIR}/build/includes/hru_actor")
link_directories("/usr/local/lib")
set(LIB_ACTORS CAF::core CAF::io -lopenblas -lnetcdff -lnetcdf)
endif()
# Define directories that contains source code
set(DRIVER_DIR ${F_MASTER}/build/source/driver)
set(DSHARE_DIR ${F_MASTER}/build/source/dshare)
set(ENGINE_DIR ${F_MASTER}/build/source/engine)
set(HOOKUP_DIR ${F_MASTER}/build/source/hookup)
set(NETCDF_DIR ${F_MASTER}/build/source/netcdf)
set(NOAHMP_DIR ${F_MASTER}/build/source/noah-mp)
# Define Actors specific directories
set(ACTORS_DIR ${PARENT_DIR}/build/source/actors)
set(FILE_ACCESS_DIR ${ACTORS_DIR}/file_access_actor)
set(JOB_ACTOR_DIR ${ACTORS_DIR}/job_actor)
set(HRU_ACTOR_DIR ${ACTORS_DIR}/hru_actor)
# NOAHMP modules
set(NOAHMP
${NOAHMP_DIR}/module_model_constants.F
${NOAHMP_DIR}/module_sf_noahutl.F
${NOAHMP_DIR}/module_sf_noahlsm.F
${NOAHMP_DIR}/module_sf_noahmplsm.F)
# Free versions of numerical recipes utilities for NOAH-MP modules
set(NRUTIL
${ENGINE_DIR}/f2008funcs.f90
${ENGINE_DIR}/nr_utility.f90
${ENGINE_DIR}/nrtype.f90)
# Free versions of numerical recipes procedures for SUMMA modules
set(NRPROC
${ENGINE_DIR}/expIntegral.f90
${ENGINE_DIR}/spline_int.f90)
# Hook-up modules
set(HOOKUP
${HOOKUP_DIR}/ascii_util.f90
${HOOKUP_DIR}/summaFileManager.f90)
# Data modules
set(DATAMS
${DSHARE_DIR}/data_types.f90
${ACTORS_DIR}/global/actor_data_types.f90
${DSHARE_DIR}/flxMapping.f90
${DSHARE_DIR}/get_ixname.f90
${DSHARE_DIR}/globalData.f90
${DSHARE_DIR}/multiconst.f90
${DSHARE_DIR}/outpt_stat.f90
${DSHARE_DIR}/popMetadat.f90
${DSHARE_DIR}/var_lookup.f90)
# Utility modules
set(UTILMS
${ENGINE_DIR}/matrixOper.f90
${ENGINE_DIR}/mDecisions.f90
${ENGINE_DIR}/snow_utils.f90
${ENGINE_DIR}/soil_utils.f90
${ENGINE_DIR}/time_utils.f90
${ENGINE_DIR}/updatState.f90)
# NetCDF routines
set(NETCDF
${NETCDF_DIR}/def_output.f90
${NETCDF_DIR}/netcdf_util.f90
${NETCDF_DIR}/read_icond.f90)
# Preliminary modules
set(PRELIM
${ENGINE_DIR}/allocspace.f90
${ENGINE_DIR}/check_icond.f90
${ENGINE_DIR}/checkStruc.f90
${ENGINE_DIR}/childStruc.f90
${ENGINE_DIR}/conv_funcs.f90
${ENGINE_DIR}/convE2Temp.f90
${FILE_ACCESS_DIR}/ffile_info.f90
${ENGINE_DIR}/read_pinit.f90
${ENGINE_DIR}/read_attrb.f90
${ENGINE_DIR}/paramCheck.f90
${ENGINE_DIR}/pOverwrite.f90
${ENGINE_DIR}/sunGeomtry.f90
${ENGINE_DIR}/read_param.f90)
# Model run support modules
set(MODRUN
${ENGINE_DIR}/canopySnow.f90
${ENGINE_DIR}/derivforce.f90
${ENGINE_DIR}/getVectorz.f90
${ENGINE_DIR}/indexState.f90
${ENGINE_DIR}/layerMerge.f90
${ENGINE_DIR}/layerDivide.f90
${ENGINE_DIR}/qTimeDelay.f90
${ENGINE_DIR}/snowAlbedo.f90
${ENGINE_DIR}/snwCompact.f90
${ENGINE_DIR}/tempAdjust.f90
${ENGINE_DIR}/updateVars.f90
${ENGINE_DIR}/var_derive.f90
${ENGINE_DIR}/volicePack.f90)
# Solver main modules
set(SOLVER
${ENGINE_DIR}/bigAquifer.f90
${ENGINE_DIR}/computFlux.f90
${ENGINE_DIR}/computJacob.f90
${ENGINE_DIR}/computResid.f90
${ENGINE_DIR}/coupled_em.f90
${ENGINE_DIR}/diagn_evar.f90
${ENGINE_DIR}/eval8summa.f90
${ENGINE_DIR}/groundwatr.f90
${ENGINE_DIR}/opSplittin.f90
${ENGINE_DIR}/snowLiqFlx.f90
${ENGINE_DIR}/soilLiqFlx.f90
${ENGINE_DIR}/ssdNrgFlux.f90
${ENGINE_DIR}/stomResist.f90
${ENGINE_DIR}/summaSolve.f90
${ENGINE_DIR}/systemSolv.f90
${ENGINE_DIR}/varSubstep.f90
${ENGINE_DIR}/vegLiqFlux.f90
${ENGINE_DIR}/vegNrgFlux.f90
${ENGINE_DIR}/vegPhenlgy.f90
${ENGINE_DIR}/vegSWavRad.f90)
set(DRIVER
${DRIVER_DIR}/summa_alarms.f90
${DRIVER_DIR}/summa_globalData.f90)
# Actors interface modules
set(INTERFACE
${ACTORS_DIR}/global/cppwrap_auxiliary.f90
${ACTORS_DIR}/global/cppwrap_datatypes.f90
${ACTORS_DIR}/global/cppwrap_metadata.f90)
set(FILE_ACCESS_INTERFACE
${FILE_ACCESS_DIR}/cppwrap_fileAccess.f90
${FILE_ACCESS_DIR}/output_structure.f90
${FILE_ACCESS_DIR}/read_force.f90
${FILE_ACCESS_DIR}/fileAccess_writeOutput.f90)
set(JOB_INTERFACE
${JOB_ACTOR_DIR}/job_actor.f90)
set(HRU_INTERFACE
${HRU_ACTOR_DIR}/hru_init.f90
${HRU_ACTOR_DIR}/hru_read.f90
${HRU_ACTOR_DIR}/hru_modelRun.f90
${HRU_ACTOR_DIR}/hru_writeOutput.f90)
# Actors actual actor modules
set(ACTORS_GLOBAL
${ACTORS_DIR}/global/auxiliary.cpp
${ACTORS_DIR}/global/global.cpp
${ACTORS_DIR}/global/message_atoms.cpp
${ACTORS_DIR}/global/settings_functions.cpp
${ACTORS_DIR}/global/timing_info.cpp)
set(SUMMA_ACTOR
${ACTORS_DIR}/summa_actor/batch.cpp
${ACTORS_DIR}/summa_actor/batch_container.cpp
${ACTORS_DIR}/summa_actor/client.cpp
${ACTORS_DIR}/summa_actor/client_container.cpp
${ACTORS_DIR}/summa_actor/summa_actor.cpp
${ACTORS_DIR}/summa_actor/summa_backup_server.cpp
${ACTORS_DIR}/summa_actor/summa_client.cpp
${ACTORS_DIR}/summa_actor/summa_server.cpp)
set(FILE_ACCESS_ACTOR
${ACTORS_DIR}/file_access_actor/file_access_actor.cpp
${ACTORS_DIR}/file_access_actor/forcing_file_info.cpp
${ACTORS_DIR}/file_access_actor/output_container.cpp)
set(JOB_ACTOR
${ACTORS_DIR}/job_actor/GRU.cpp
${ACTORS_DIR}/job_actor/job_actor.cpp)
set(HRU_ACTOR
${ACTORS_DIR}/hru_actor/hru_actor.cpp)
#=========================================================================================
# COMPILE PART 3: Collect the subroutines into build groups depending on build type
#=========================================================================================
set(COMM_ALL
${NRPROC}
${HOOKUP}
${DATAMS}
${UTILMS}
${INTERFACE})
set(SUMMA_ALL
${NETCDF}
${PRELIM}
${MODRUN}
${SOLVER}
${DRIVER})
set(SUMMA_ALL
${SUMMA_ALL}
${FILE_ACCESS_INTERFACE}
${JOB_INTERFACE}
${HRU_INTERFACE})
set(MAIN_ACTOR ${ACTORS_DIR}/main.cpp)
# Define version number, not working correctly
set(VERSIONFILE ${DRIVER_DIR}/summaversion.inc)
execute_process(COMMAND " ${GIT_EXECUTABLE} tag | tail -n 1" OUTPUT_VARIABLE VERSION)
execute_process(COMMAND "date" OUTPUT_VARIABLE BULTTIM)
execute_process(COMMAND " ${GIT_EXECUTABLE} describe --long --all --always | sed -e's/heads\///'" OUTPUT_VARIABLE GITBRCH)
execute_process(COMMAND " ${GIT_EXECUTABLE} rev-parse HEAD" OUTPUT_VARIABLE GITHASH)
#=========================================================================================
# COMPILE PART 4: Do the compilation
#=========================================================================================
# update version information, not working correctly
file(WRITE ${VERSIONFILE} "character(len=64), parameter :: summaVersion = '${VERSION}'\n")
file(APPEND ${VERSIONFILE} "character(len=64), parameter :: buildTime = ''\n")
file(APPEND ${VERSIONFILE} "character(len=64), parameter :: gitBranch = '${GITBRCH}'\n")
file(APPEND ${VERSIONFILE} "character(len=64), parameter :: gitHash = '${GITHASH}'")
# Build SUMMA_NOAHMP Object
add_library(SUMMA_NOAHMP OBJECT ${NOAHMP} ${NRUTIL})
target_compile_options(SUMMA_NOAHMP PRIVATE ${FLAGS_NOAH})
# Build SUMMA_COMM Object
add_library(SUMMA_COMM OBJECT ${COMM_ALL})
target_compile_options(SUMMA_COMM PRIVATE ${FLAGS_ALL})
target_include_directories(SUMMA_COMM PRIVATE ${INCLUDES})
target_link_libraries(SUMMA_COMM PUBLIC SUMMA_NOAHMP ${FLAGS_ALL}) # added flags to the link step
add_library(summaactors SHARED ${SUMMA_ALL})
target_compile_options(summaactors PRIVATE ${FLAGS_ALL})
target_include_directories(summaactors PUBLIC ${INCLUDES})
target_link_libraries(summaactors PUBLIC ${LIBRARIES} SUMMA_NOAHMP SUMMA_COMM)
add_executable(${EXEC_NAME}
${MAIN_ACTOR}
${ACTORS_GLOBAL}
${FILE_ACCESS_ACTOR}
${JOB_ACTOR}
${HRU_ACTOR}
${SUMMA_ACTOR}
${SUMMA_CLIENT}
${SUMMA_SERVER})
set_property(TARGET ${EXEC_NAME} PROPERTY LINKER_LANGUAGE Fortran)
target_compile_options(${EXEC_NAME} PUBLIC ${FLAGS_CXX})
target_include_directories(${EXEC_NAME} PUBLIC ${INC_ACTORS})
target_link_libraries( ${EXEC_NAME} ${LIB_ACTORS} summaactors)
\ No newline at end of file
build/example_compile.sh deleted 100644 → 0
View file @ b6e0281b
#!/bin/bash
#### load modules if using Compute Canada or Copernicus ####
# module load gcc/9.3.0
# module load netcdf-fortran
# module load openblas
# module load caf
#### Specifiy Master Directory, parent of build directory
# export F_MASTER=/path/to/summaActors
#### Specifiy Compilers ####
# export FC=gfortran
# export CC=g++
#### Includes and Libraries ####
# export INCLUDES = NETCDF and OPENBLAS
# export LIBRARIES = NETCDF and OPENBLAS
# export ACTORS_INCLUDES = C++ Actor Framework
# export ACTORS_LIBRARIES = C++ Actor Framework and
# The directory in which libsumma.so resides
#### Compile with the Makefile ####
# make -f ${F_MASTER}/build/makefile lib # libsumma.so part
# mv libsumma.so ${F_MASTER}/bin # optional move of libsumma (just ensure that summaMain knows where to find it)
# make -f ${F_MASTER}/build/makefile main # summaMain part
# mv summaMain ${F_MASTER}/bin # optional move, cleans things up
# export LD_LIBRARY_PATH=Path/to/libsumma.so
\ No newline at end of file
build/includes/file_access_actor/file_access_actor.hpp 0 → 100644
View file @ 8e5f43c0
#pragma once
#include "caf/actor.hpp"
#include "forcing_file_info.hpp"
#include "timing_info.hpp"
#include "output_container.hpp"
#include "settings_functions.hpp"
#include "fortran_data_types.hpp"
#include "auxilary.hpp"
#include "global.hpp"
/*********************************************
* File Access Actor Fortran Functions
*********************************************/
extern "C" {
void fileAccessActor_init_fortran(void* handle_forcing_file_info, int* num_forcing_files, int* num_timesteps,
int* num_timesteps_output_buffer, void* handle_output_ncid, int* startGRU,
int* numGRU, int* numHRU, int* err);
void writeOutput_fortran(void* handle_ncid, int* num_steps, int* start_gru, int* max_gru,
bool* writeParamFlag, int* err);
void read_forcingFile(void* forcFileInfo, int* currentFile, int* stepsInFile,
int* startGRU, int* numGRU, int* err);
void FileAccessActor_DeallocateStructures(void* handle_forcFileInfo, void* handle_ncid);
}
/*********************************************
* File Access Actor state variables
*********************************************/
namespace caf {
struct file_access_state {
// Variables set on Spawn
caf::actor parent;
int start_gru;
int num_gru;
void *handle_forcing_file_info = new_handle_file_info(); // Handle for the forcing file information
void *handle_ncid = new_handle_var_i(); // output file ids
int num_vectors_in_output_manager;
int num_steps;
int stepsInCurrentFile;
int numFiles;
int filesLoaded;
int err;
int num_output_steps;
Output_Container* output_container;
File_Access_Actor_Settings file_access_actor_settings;
std::vector<Forcing_File_Info> forcing_file_list; // list of steps in file
// Timing Variables
TimingInfo file_access_timing;
};
// called to spawn a file_access_actor
behavior file_access_actor(stateful_actor<file_access_state>* self, int startGRU, int numGRU,
File_Access_Actor_Settings file_access_actor_settings, actor parent);
/*********************************************
* Functions for the file access actor
*********************************************/
/* Setup and call the fortran routine that writes the output */
void writeOutput(stateful_actor<file_access_state>* self, Output_Partition* partition);
} // end namespace
\ No newline at end of file
build/includes/file_access_actor/forcing_file_info.hpp 0 → 100644
View file @ 8e5f43c0
#pragma once
#include <vector>
class Forcing_File_Info {
private:
int file_ID;
int num_steps;
bool is_loaded;
public:
Forcing_File_Info(int file_ID);
int getNumSteps();
bool isFileLoaded();
void updateIsLoaded();
void updateNumSteps(int num_steps);
};
struct Forcing_Info {
int num_vars;
int num_timesteps;
std::vector<int> index_forc_var;
std::vector<int> ncid_var;
};
\ No newline at end of file
build/includes/file_access_actor/output_container.hpp 0 → 100644
View file @ 8e5f43c0
#pragma once
#include "caf/actor.hpp"
#include <optional>
#include <cmath>
#include "fortran_data_types.hpp"
#include <vector>
#include <iostream>
/*
* This class manages a portion of the HRUs in the model.
* All HRUs are grouped into partitions/objects of this class.
*/
class Output_Partition {
private:
int start_local_gru_index; // The index of the first GRU in the partition
int end_local_gru_index; // The index of the last GRU in the partition
int num_local_grus; // The number of GRUs in the partition
int num_active_grus; // The number of GRUs that have not failed
int num_timesteps_simulation; // The number of timesteps in the simulation
int num_stored_timesteps; // The number of timesteps held within the partition
bool write_params = true; // Flag to write the parameters to the output file (only performed once)
std::vector<caf::actor> ready_to_write_list;
std::vector<int> failed_gru_index_list; // The list of GRUs that have failed
public:
Output_Partition(int start_local_gru_index, int num_local_grus, int num_timesteps_simulation, int num_timesteps_buffer);
~Output_Partition();
// Set the GRU to ready to write
void setGRUReadyToWrite(caf::actor gru_actor);
// Check if all GRUs are ready to write
bool isReadyToWrite();
// Get the max index of the GRUs in the partition
int getMaxGRUIndex();
// Get the number of timesteps stored in the partition
int getNumStoredTimesteps();
// Get the start gru index
int getStartGRUIndex();
// Update the number of timesteps remaining in the simulation
void updateTimeSteps();
// Get the list of GRUs that have written so we can send them the next set of timesteps
std::vector<caf::actor> getReadyToWriteList();
// Reset the list of GRUs that are ready to write
void resetReadyToWriteList();
// Add a GRU index to the list of failed GRUs
void addFailedGRUIndex(int local_gru_index);
std::vector<int> getFailedGRUIndexList();
int getNumActiveGRUs();
int getNumLocalGRUs();
int getRemainingTimesteps();
bool isWriteParams();
};
/*
* This class is used to store informaiton about when
* HRUs are ready to write. This class does not store
* the data of the HRUs only the information about if
* HRUs are ready to write and which HRUs should be
* written to the output file.
*/
class Output_Container {
private:
int num_partitions; // The number of partitions in the model
int num_grus_per_partition; // The average number of GRUs per partition
int num_grus; // The number of GRUs in the model
int num_timesteps_simulation; // The number of timesteps in the simulation
int num_stored_timesteps; // The number of timesteps a partion can hold before needing to write
bool rerunning_failed_hrus = false;
std::vector<Output_Partition*> output_partitions; // This is the main output partition
std::vector<int> failed_gru_index_list; // The list of GRUs that have failed
// Private Method
public:
Output_Container(int num_partitions, int num_grus, int num_timesteps_simulation, int num_timesteps_buffer);
~Output_Container();
int findPartition(int local_gru_index);
int getNumPartitions();
// The output container needs to be restructured when rerunning the failed GRUs.
void reconstruct();
Output_Partition* getOutputPartition(int local_gru_index);
std::vector<int> getFailedGRUIndexList();
};
build/includes/global/auxilary.hpp 0 → 100644
View file @ 8e5f43c0
#pragma once
// Setters
void set_flagVec(std::vector<int>& arr_i, void* handle);
void set_var_i(std::vector<int>& arr_i, void* handle);
void set_var_d(std::vector<double> &arr_d, void* handle);
void set_var_i8(std::vector<long int>& arr_i, void* handle);
void set_i8length(std::vector<long int> &arr_i8length, void* handle);
void set_ilength(std::vector<int> &arr_ilength, void* handle);
void set_dlength(std::vector<double> &arr_dlength, void* handle);
void set_var_flagVec(std::vector<std::vector<int> > &mat, void* handle);
void set_var_ilength(std::vector<std::vector<int> > &mat, void* handle);
void set_var_i8length(std::vector<std::vector<long int> > &mat, void* handle);
void set_var_dlength(std::vector<std::vector<double> > &mat, void *handle);
// Getters
std::vector<int> get_flagVec(void* handle);
std::vector<int> get_var_i(void* handle);
std::vector<double> get_var_d(void* handle);
std::vector<long int> get_var_i8(void* handle);
std::vector<long int> get_i8length(void* handle);
std::vector<int> get_ilength(void* handle);
std::vector<double> get_dlength(void* handle);
std::vector<std::vector<int> > get_var_flagVec(void* handle);
std::vector<std::vector<int> > get_var_ilength(void* handle);
std::vector<std::vector<long int> > get_var_i8length(void* handle);
std::vector<std::vector<double> > get_var_dlength(void* handle);
std::vector<double> get_attr_struct(void* handle);
std::vector<int> get_type_struct(void* handle);
std::vector<std::vector<double>> get_mpar_struct_array(void* handle);
std::vector<double> get_bpar_struct(void* handle);
\ No newline at end of file
build/source/interface/fortran_dataTypes.h → build/includes/global/fortran_data_types.hpp
View file @ 8e5f43c0
#ifndef FORTRAN_DATATYPES_H_
#define FORTRAN_DATATYPES_H_
#pragma once
extern "C" {
// flagVec
......@@ -15,6 +14,8 @@ extern "C" {
void set_data_var_i(void* handle, const int* array, int size);
void get_size_data_var_i(void* handle, int* size);
void get_data_var_i(void* handle, int* array);
void get_size_data_typeStruct(void* handle, int* size);
void get_data_typeStruct(void* handle, int* array);
// var_i8
void* new_handle_var_i8();
......@@ -29,6 +30,10 @@ extern "C" {
void set_data_var_d(void* handle, const double* array, int size);
void get_size_data_var_d(void* handle, int* size);
void get_data_var_d(void* handle, double* array);
void get_data_attrStruct(void* handle, double* array);
void get_size_data_attrStruct(void* handle, int* size);
void get_data_bparStruct(void* handle, double* array);
void get_size_data_bparStruct(void* handle, int* size);
// ilength
void* new_handle_ilength();
......@@ -51,6 +56,7 @@ extern "C" {
void get_size_data_dlength(void* handle, int* size);
void get_data_dlength(void* handle, double* array);
// var_flagVec
void* new_handle_var_flagVec();
void delete_handle_var_flagVec(void* handle);
......@@ -82,6 +88,9 @@ extern "C" {
void get_size_var_dlength(void* handle, int* num_var);
void get_size_data_var_dlength(void* handle, int* num_var, int* num_dat);
void get_data_var_dlength(void* handle, double* array);
void get_size_var_mparStruct(void* handle, int* num_var);
void get_size_data_mparStruct(void* handle, int* num_var, int* num_dat);
void get_data_mparStruct(void* handle, double* array);
// var_dlength_array
void* new_handle_dlength_array();
......@@ -97,6 +106,12 @@ extern "C" {
void* new_handle_file_info();
void delete_handle_file_info(void* handle);
}
// zLookup
void* new_handle_z_lookup();
void* delete_handle_z_lookup(void* handle);
// hru_type
void* new_handle_hru_type();
void delete_handle_hru_type(void* handle);
#endif
\ No newline at end of file
}
\ No newline at end of file
build/includes/global/global.hpp 0 → 100644
View file @ 8e5f43c0
#pragma once
#include <chrono>
/**
* Return the time between to time points
*/
double calculateTime(std::chrono::time_point<std::chrono::system_clock> start,
std::chrono::time_point<std::chrono::system_clock> end);
struct serializable_netcdf_gru_actor_info {
double run_time;
double init_duration;
double forcing_duration;
double run_physics_duration;
double write_output_duration;
int successful; // 0 = false, 1 = true
int num_attempts;
double rel_tol;
double abs_tol;
};
template<class Inspector>
bool inspect(Inspector& f, serializable_netcdf_gru_actor_info& x) {
return f.object(x).fields(f.field("run_time", x.run_time),
f.field("init_duration", x.init_duration),
f.field("forcing_duration", x.forcing_duration),
f.field("run_physics_duration", x.run_physics_duration),
f.field("write_output_duration", x.write_output_duration),
f.field("successful", x.successful),
f.field("num_attempts", x.num_attempts),
f.field("rel_tol", x.rel_tol),
f.field("abs_tol", x.abs_tol));
}
build/includes/global/message_atoms.hpp 0 → 100644
View file @ 8e5f43c0
#pragma once
#include "batch.hpp"
#include "batch_container.hpp"
#include "client.hpp"
#include "client_container.hpp"
#include <vector>
#include "settings_functions.hpp"
#include "global.hpp"
#include "caf/all.hpp"
enum class hru_error : uint8_t {
run_physics_unhandleable = 1,
run_physics_infeasible_state = 2,
};
enum class file_access_error : uint8_t {
writing_error = 1,
unhandleable_error = 2,
mDecisions_error = 100,
};
// HRU Errors
std::string to_string(hru_error err);
bool from_string(caf::string_view in, hru_error& out);
bool from_integer(uint8_t in, hru_error& out);
template<class Inspector>
bool inspect(Inspector& f, hru_error& x) {
return caf::default_enum_inspect(f, x);
}
// File Access Actor
std::string to_string(file_access_error err);
bool from_string(caf::string_view in, file_access_error& out);
bool from_integer(uint8_t in, file_access_error& out);
template<class Inspector>
bool inspect(Inspector& f, file_access_error& x) {
return caf::default_enum_inspect(f, x);
}
CAF_BEGIN_TYPE_ID_BLOCK(summa, first_custom_type_id)
// Sender: job_actor
// Reciever: summa_actor
// Summary: job_actor finished job
CAF_ADD_ATOM(summa, done_job)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, err)
CAF_ADD_ATOM(summa, err_atom)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, init_gru)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, done_init_gru)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, init_hru)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, done_init_hru)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, done_write)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, done_hru)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, run_failure)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, access_forcing)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, access_forcing_internal)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, write_output)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, deallocate_structures)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, write_param)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, restart_failures)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, get_attributes_params)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, run_hru)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, start_hru)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, dt_init_factor)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, connect_to_server)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, connect_as_backup)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, compute_batch)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, done_batch)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, time_to_exit)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, update_with_current_state)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, new_assigned_batch)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, no_more_batches)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, update_backup_server_list)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, client_removed)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, new_client)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, is_lead_server)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, yes)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, no)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, new_forcing_file)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, num_steps_before_write)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, get_num_output_steps)
// Sender:
// Reciever:
// Summary:
CAF_ADD_ATOM(summa, finalize)
// Struct Types
CAF_ADD_TYPE_ID(summa, (Distributed_Settings))
CAF_ADD_TYPE_ID(summa, (Summa_Actor_Settings))
CAF_ADD_TYPE_ID(summa, (File_Access_Actor_Settings))
CAF_ADD_TYPE_ID(summa, (Job_Actor_Settings))
CAF_ADD_TYPE_ID(summa, (HRU_Actor_Settings))
CAF_ADD_TYPE_ID(summa, (serializable_netcdf_gru_actor_info))
// Class Types
CAF_ADD_TYPE_ID(summa, (Client))
CAF_ADD_TYPE_ID(summa, (Client_Container))
CAF_ADD_TYPE_ID(summa, (Batch))
CAF_ADD_TYPE_ID(summa, (Batch_Container))
CAF_ADD_TYPE_ID(summa, (std::vector<std::vector<double>>))
CAF_ADD_TYPE_ID(summa, (std::vector<std::vector<int>>))
CAF_ADD_TYPE_ID(summa, (std::vector<int>))
CAF_ADD_TYPE_ID(summa, (std::vector<double>))
CAF_ADD_TYPE_ID(summa, (std::vector<long int>))
CAF_ADD_TYPE_ID(summa, (std::vector<std::tuple<caf::actor, std::string>>))
CAF_ADD_TYPE_ID(summa, (std::vector<serializable_netcdf_gru_actor_info>))
// GRU Parameter/Attribute Vectors
CAF_ADD_TYPE_ID(summa, (std::tuple<std::vector<double>,
std::vector<int>,
std::vector<long int>,
std::vector<double>,
std::vector<double>,
std::vector<std::vector<double>>>))
// File_Access_Actor Read/Write times
CAF_ADD_TYPE_ID(summa, (std::tuple<double, double>))
CAF_ADD_TYPE_ID(summa, (std::optional<caf::strong_actor_ptr>))
// error types
CAF_ADD_TYPE_ID(summa, (hru_error))
CAF_ADD_TYPE_ID(summa, (file_access_error))
CAF_END_TYPE_ID_BLOCK(summa)
CAF_ERROR_CODE_ENUM(hru_error)
CAF_ERROR_CODE_ENUM(file_access_error)
\ No newline at end of file
build/includes/global/settings_functions.hpp 0 → 100644
View file @ 8e5f43c0
#pragma once
#include <string>
#include <vector>
#include <thread>
#include <optional>
#include "json.hpp"
#include <bits/stdc++.h>
#include <sys/stat.h>
using json = nlohmann::json;
struct File_Access_Actor_Settings;
struct Job_Actor_Settings;
struct HRU_Actor_Settings;
// ####################################################################
// Distributed Settings
// ####################################################################
struct Distributed_Settings;
struct Distributed_Settings {
bool distributed_mode; // flag for starting summa in distributed mode
std::vector<std::string> servers_list; // the hostname of the server actor
int port; // the port number of the server actor
int total_hru_count;
int num_hru_per_batch;
};
template<class Inspector>
bool inspect(Inspector& inspector, Distributed_Settings& distributed_settings) {
return inspector.object(distributed_settings).fields(
inspector.field("distributed_mode", distributed_settings.distributed_mode),
inspector.field("servers_list", distributed_settings.servers_list),
inspector.field("port", distributed_settings.port),
inspector.field("total_hru_count", distributed_settings.total_hru_count),
inspector.field("num_hru_per_batch",distributed_settings.num_hru_per_batch));
}
Distributed_Settings readDistributedSettings(std::string json_settings_file);
// ####################################################################
// SUMMA Actor Settings
// ####################################################################
struct Summa_Actor_Settings;
struct Summa_Actor_Settings {
int max_gru_per_job;
};
template<class Inspector>
bool inspect(Inspector& inspector, Summa_Actor_Settings& summa_actor_settings) {
return inspector.object(summa_actor_settings).fields(
inspector.field("max_gru_per_job", summa_actor_settings.max_gru_per_job));
}
Summa_Actor_Settings readSummaActorSettings(std::string json_settings_file);
// ####################################################################
// File Access Actor Settings
// ####################################################################
struct File_Access_Actor_Settings {
int num_partitions_in_output_buffer;
int num_timesteps_in_output_buffer;
};
template<class Inspector>
bool inspect(Inspector& inspector, File_Access_Actor_Settings& file_access_actor_settings) {
return inspector.object(file_access_actor_settings).fields(
inspector.field("num_partitions_in_output_buffer",
file_access_actor_settings.num_partitions_in_output_buffer),
inspector.field("num_timesteps_in_output_buffer",
file_access_actor_settings.num_timesteps_in_output_buffer));
}
File_Access_Actor_Settings readFileAccessActorSettings(std::string json_settings_file);
// ####################################################################
// Job Actor Settings
// ####################################################################
struct Job_Actor_Settings {
std::string file_manager_path;
int max_run_attempts; // maximum number of times to attempt to run each HRU in a job
};
template<class Inspector>
bool inspect(Inspector& inspector, Job_Actor_Settings& job_actor_settings) {
return inspector.object(job_actor_settings).fields(
inspector.field("file_manager_path", job_actor_settings.file_manager_path),
inspector.field("max_run_attempts", job_actor_settings.max_run_attempts));
}
Job_Actor_Settings readJobActorSettings(std::string json_settings_file);
// ####################################################################
// HRU Actor Settings
// ####################################################################
struct HRU_Actor_Settings {
bool print_output;
int output_frequency;
int dt_init_factor; // factor to multiply the initial timestep by
double rel_tol;
double abs_tol;
double relTolTempCas;
double absTolTempCas;
double relTolTempVeg;
double absTolTempVeg;
double relTolWatVeg;
double absTolWatVeg;
double relTolTempSoilSnow;
double absTolTempSoilSnow;
double relTolWatSnow;
double absTolWatSnow;
double relTolMatric;
double absTolMatric;
double relTolAquifr;
double absTolAquifr;
};
template<class Inspector>
bool inspect(Inspector& inspector, HRU_Actor_Settings& hru_actor_settings) {
return inspector.object(hru_actor_settings).fields(
inspector.field("print_output", hru_actor_settings.print_output),
inspector.field("output_frequency", hru_actor_settings.output_frequency),
inspector.field("dt_init_factor", hru_actor_settings.dt_init_factor),
inspector.field("rel_tol", hru_actor_settings.rel_tol),
inspector.field("abs_tol", hru_actor_settings.abs_tol));
}
HRU_Actor_Settings readHRUActorSettings(std::string json_settings_file);
// ####################################################################
// Non Actor Specific Settings
// ####################################################################
int checkFileExists(std::string file_path);
/**
* @brief Get the Settings from json
* Template function that can be used with retrieving any singular type from the settings file
*/
template <typename T>
std::optional<T> getSettings(std::string json_settings_file, std::string key_1, std::string key_2,
T return_value) {
json settings;
std::ifstream settings_file(json_settings_file);
if (!settings_file.good()) return {};
settings_file >> settings;
settings_file.close();
// find first key
try {
if (settings.find(key_1) != settings.end()) {
json key_1_settings = settings[key_1];
// find value behind second key
if (key_1_settings.find(key_2) != key_1_settings.end()) {
return key_1_settings[key_2];
} else
return {};
} else {
return {}; // return none in the optional (error value)
}
} catch (json::exception& e) {
std::cout << e.what() << "\n";
std::cout << key_1 << "\n";
std::cout << key_2 << "\n";
return {};
}
}
// Get settings from settings file in array format
std::optional<std::vector<std::string>> getSettingsArray(std::string json_settings_file, std::string key_1, std::string key_2);
// Check the settings - Print them out to stdout
void check_settings_from_json(Distributed_Settings &distributed_settings,
Summa_Actor_Settings &summa_actor_settings, File_Access_Actor_Settings &file_access_actor_settings,
Job_Actor_Settings &job_actor_settings, HRU_Actor_Settings &hru_actor_settings);
// Output a default configuration file
void generate_config_file();
\ No newline at end of file
build/includes/global/timing_info.hpp 0 → 100644
View file @ 8e5f43c0
#pragma once
#include <chrono>
#include <optional>
#include <vector>
#include <string>
using chrono_time = std::chrono::time_point<std::chrono::system_clock>;
/**
* Class to manage timing information. This allows the user to add an arbitrary amount of timing variables.
* The timing variables are accessed through their named string and will keep a running duration of the amount
* of time spent through multiple calls to updateStartPoint and updateEndPoint
*/
class TimingInfo {
private:
std::vector<std::optional<chrono_time>> start;
std::vector<std::optional<chrono_time>> end;
std::vector<double> duration;
std::vector<std::string> name_of_time_point; // the name you want for the time point (ie. reading, writing, duration)
int num_time_points;
std::optional<double> calculateDuration(int index);
std::optional<int> getIndex(std::string time_point_name);
public:
TimingInfo();
~TimingInfo();
void addTimePoint(std::string time_point_name);
void updateStartPoint(std::string time_point_name);
void updateEndPoint(std::string time_point_name);
std::optional<double> getDuration(std::string time_point_name); // returns duration in seconds
};
\ No newline at end of file
build/includes/gru_actor/gru_actor.hpp 0 → 100644
View file @ 8e5f43c0
#pragma once
#include "caf/all.hpp"
#include "fortran_data_types.hpp"
#include "timing_info.hpp"
#include "global.hpp"
#include <vector>
#include <array>
#include <chrono>
#include <string>
#include "var_lookup.hpp"
namespace caf{
struct gru_state {
// GRU Initalization
int ref_gru;
int indx_gru;
std::string config_path;
caf::actor file_access_actor;
int output_struc_size;
caf::actor parent;
// HRU information
std::vector<caf::actor> hru_list;
int num_hrus;
int hrus_complete = 0;
// Global Data
int nTimeDelay = 2000; // number of hours in the time delay histogram (default: ~1 season = 24*365/4)
struct iLookVarType var_type_lookup;
// Data Structure local to the GRU
int num_bpar_vars; // number of variables in the fortran structure for bpar_struct
std::vector<double> bpar_struct;
std::vector<int> bpar_struct_var_type_list; // The types to the variable at each element in bpar_struct
int num_bvar_vars; // number of variables in the fortran structure for bvar_struct
std::vector<std::vector<double>> bvar_struct;
std::vector<int> bvar_struct_var_type_list;
int num_var_types;
std::vector<int> i_look_var_type_list; // The types to the variable at each element in bvar_struct
};
behavior gru_actor(stateful_actor<gru_state>* self,
int ref_gru,
int indx_gru,
std::string config_path,
caf::actor file_access_actor,
caf::actor parent);
}
\ No newline at end of file
build/includes/gru_actor/gru_actor_subroutine_wrappers.hpp 0 → 100644
View file @ 8e5f43c0
#pragma once
extern "C" {
void getVarSizes(int* num_var_types, int* num_bpar_vars, int* num_bvar_vars);
void initVarType(void* var_type_lookup);
void fillVarTypeLists( int* num_bpar_vars, int* num_bvar_vars,
void* bpar_struct_var_type_list, void* bvar_struct_var_type_list, int* err);
int getNumHRU(int* indx_gru);
}
\ No newline at end of file
build/includes/gru_actor/gru_data_structure.hpp 0 → 100644
View file @ 8e5f43c0
#pragma once
build/includes/hru_actor/hru_actor.hpp 0 → 100644
View file @ 8e5f43c0
#pragma once
#include "caf/all.hpp"
#include "fortran_data_types.hpp"
#include "auxilary.hpp"
#include "timing_info.hpp"
#include "settings_functions.hpp"
#include <string>
#include "message_atoms.hpp"
#include "global.hpp"
/*********************************************
* HRU Actor Fortran Functions
*********************************************/
extern "C" {
// Initialize HRU data_structures
void initHRU(int* indxGRU, int* num_steps, void* hru_data, int* err);
void setupHRUParam( int* indxGRU, int* indxHRU, void* hru_data, double* upArea, int* err);
// Setup summa_readRestart File if this option has been chosen
void summa_readRestart(int* indxGRU, int* indxHRU, void* hru_data, double* dtInit, int* err);
// Run the model for one timestep
void RunPhysics(int* id, int* stepIndex, void* hru_data, double* dt, int* dt_int_factor, int* err);
void hru_writeOutput(int* index_hru, int* index_gru, int* timestep, int* output_step, void* hru_data, int* err);
void setTimeZoneOffset(int* iFile, void* hru_data, int* err);
void HRU_readForcing(int* index_gru, int* iStep, int* iRead, int* iFile, void* hru_data, int* err);
void get_sundials_tolerances(void* hru_data, double* relTol, double* absTol);
void set_sundials_tolerances(void* hru_data, double* relTol, double* absTol);
void setIDATolerances(void* hru_data, double* relTolTempCas, double* absTolTempCas, double* relTolTempVeg,
double* absTolTempVeg, double* relTolWatVeg, double* absTolWatVeg, double* relTolTempSoilSnow,
double* absTolTempSoilSnow, double* relTolWatSnow, double* absTolWatSnow, double* relTolMatric,
double* absTolMatric, double* relTolAquifr, double* absTolAquifr);
}
/*********************************************
* HRU Actor state variables
*********************************************/
namespace caf {
struct hru_state {
// Actor References
caf::actor file_access_actor;
caf::actor parent;
// Info about which HRU we are and our indexes
// into global structures in Fortran
int indxHRU; // index for hru part of derived types in FORTRAN
int indxGRU; // index for gru part of derived types in FORTRAN
int refGRU; // The actual ID of the GRU we are
// Variables for forcing structures
int stepsInCurrentFFile; // number of time steps in current forcing file
int num_steps_until_write; // number of time steps until we pause for FA_Actor to write
// HRU data structures (formerly summa_type)
void *hru_data = new_handle_hru_type();
// Misc Variables
int timestep = 1; // Current Timestep of HRU simulation
int forcingStep = 1; // index of current time step in current forcing file
int num_steps = 0; // number of time steps
int iFile = 1; // index of current forcing file from forcing file list
int dt_init_factor = 1; // factor of dt_init (coupled_em)
int output_structure_step_index = 1; // index of current time step in output structure
double dt_init; // used to initialize the length of the sub-step for each HRU
double upArea; // area upslope of each HRU
int err = 0;
// Sundials variables
double rtol = -9999; // -9999 uses default
double atol = -9999; // -9999 uses default
// Settings
HRU_Actor_Settings hru_actor_settings;
~hru_state() {
delete_handle_hru_type(hru_data);
}
};
behavior hru_actor(stateful_actor<hru_state>* self, int refGRU, int indxGRU,
HRU_Actor_Settings hru_actor_settings, caf::actor file_access_actor,
caf::actor parent);
/*********************************************
* Functions for the HRU Actor
*********************************************/
/** Function to initalize the HRU for running */
void Initialize_HRU(stateful_actor<hru_state>* self);
/** Function runs all of the hru time_steps */
int Run_HRU(stateful_actor<hru_state>* self);
}
\ No newline at end of file