SUMMA-Actors: Structure for Unifying Multiple Modeling Alternatives with Actors

SUMMA-Actors is a modified version of the already existing SUMMA software that can be found here. SUMMA-Actors is a modification of SUMMA that uses the Actor Model to increase scalability and fault-tolerance. The actor which is known as the basic unit of concurrent computation is at the heart of this software. SUMMA-Actors is built using the C++ Actor Framework.

Documentation

A more in-depth documentation can be found here

Compiling Summa-Actors

SUMMA-Actors is written in C++ and FORTRAN and can be compiled with a C++ and FORTRAN compiler from the GNU Compiler Collection. We have compiled SUMMA-Actors with the following compilers:

• g++
• gfortran

SUMMA-Actors depends on the following Libraries:

• NetCDF-Fortran
• OpenBLAS
• C++ Actor Framework

Once the following libraries have been installed SUMMA-Actors can be compiled in one of two ways. The first way is to modify the makefile directly and the second is to invoke the makefile by shellscript:

1. Makefile: Changes need to be made to the following variables in the Makefile:
   • F_MASTER = directory/above/build
   • FC = gfortran
   • CC = g++
   • INCLUDES = Path/to/netcdf/includes
   • LIBRARIES = Path/to/netcdf/lib & Path/to/openblas -lnetcdff -lopenblas
   • ACTORS_INCLUDES = $INCLUDES & Path/to/CAF/includes
   • ACTORS_LIBRARIES = $LIBRARIES & PATH/to/CAF/lib & PATH/to/summa.so -lcaf_core -lcaf_io -lsumma -lopenblas -lnetcdff

Once these changes have been made SUMMA-Actors can be called with make from the build/ directory.

Note: SUMMA is compiled as a shared library (libsumma.so) and the main program will need to know where this library is located in order to properly link it. By compiling both libsumma.so and summaMain in the same directory (build/) should be enough. However if there are issues, specifing where libsumma.so will be compiled can be done by adding the path to ACTORS_LIBRARIES in the makefile this should rectify the issues.

1. ShellScript: In the build directory exists a example_compile.sh script that can be modified. This is usually used for HPC computing environments and includes which modules to load for Compute Canada or the University of Saskatchewan's Copernicus. example_compile.sh contains instructions on what parts to modify and how to invoke the makefile from the script.

Once the shellscript has been modified running it with source your_script.sh will compile SUMMA-Actors.

After SUMMA-Actors is compiled you should be left with a libsumma.so and a summaMain file in your build directory.

It is important to set the LD_LIBRARY_PATH environment variable before attempting to run summa. This variable needs to point to the location of libsumma.so. If you compiled using the shellscript using the source command and following the directions within the example_compile.sh this should already be set for you.

Running Summa-Actors

Once the binary is compiled it can be run like the following example command:

./summaMain -g 1 -n 10 -c /path/to/config/directory --config-file=/path/to/actors/config/file

-g = starting index of the first GRU to compute -c = number of grus to run -m = path to the file manager OPTIONAL: --config-file = /path/to/config/file This config file specifies to the C++ Actor Framework how many threads to use when executing the program. If left out the C++ Actor Framework will automatically set this value based on your system.

Config File

SUMMA-Actors settings can be modified from a JSON file provided in config/Summa_Actors_Settings.json. There are three types of actors that can be configured:

• SummaActor

  • OutputStructureSize = The number of timesteps in which an HRU can hold before needing to contact the file_access_actor to write the data to a file.
  • maxGRUPerJob = The number of GRUs that will be attemtpted to run at once. For example, if this value is set to 500 and you invoke the program with ./summaMain -g 1 -n 1000 -c /path/to/config/directory. SUMMA-Actors will only spawn 500 actors at a time and compute all 1000 in two batches.

  Both of the above setting control the amount of RAM SUMMA-Actors uses. Larger numbers can cause your job to run out of memory. We have found that setting both to 500 uses around 20GB of RAM for reference.

• JobActor

  • FileManagerPath = Path the the fileManager.txt file needed by SUMMA. This is remained relativley unchanged from the original version of SUMMA. With two additions. An example file is provided in the config/ directory called fileManager_example.txt
  • outputCSV = Boolean value for if you would like individual HRU run-time statsicts when they complete.
  • csvPath = The path that the csv file will be written to.

• HRUActor

  • printOutput = Boolean value for if you would like each HRU to print information on where it is in its computation. ie. what timestep it is on and some other timing information.
  • outputFrequency = The frequency in which you would like an HRU printing to stdout. The number specified is the interval in timesteps in which an HRU will print. Note: Lower numbers can see decreased performance as stdout will begin to lag the more that needs to be printed.

Credits

The inital implementation of SUMMA is credited to the inital publications below. These publications can be found in Water Resources Research.

• Clark, M. P., B. Nijssen, J. D. Lundquist, D. Kavetski, D. E. Rupp, R. A. Woods, J. E. Freer, E. D. Gutmann, A. W. Wood, L. D. Brekke, J. R. Arnold, D. J. Gochis, R. M. Rasmussen, 2015a: A unified approach for process-based hydrologic modeling: Part 1. Modeling concept. Water Resources Research, doi:10.1002/2015WR017198.

• Clark, M. P., B. Nijssen, J. D. Lundquist, D. Kavetski, D. E. Rupp, R. A. Woods, J. E. Freer, E. D. Gutmann, A. W. Wood, D. J. Gochis, R. M. Rasmussen, D. G. Tarboton, V. Mahat, G. N. Flerchinger, D. G. Marks, 2015b: A unified approach for process-based hydrologic modeling: Part 2. Model implementation and case studies. Water Resources Research, doi:10.1002/2015WR017200.

We also credit the original creators of the C++ Actor Framework which allowed us to implement the actor model into SUMMA-Actors. Links to their research work can be found below.

• Charousset, D., Schmidt, T. C., Hiesgen, R., Wählisch, M., 2013: Native actors: a scalable software platform for distributed, heterogeneous environments. AGERE!, doi:10.1145/2541329.2541336.

• Charousset, D., Schmidt, T. C., Hiesgen, R., 2016: Revisiting actor programming in C++. Computer Languages, Systems & Structures, [doi:10.1016/j.cl.2016.01.002](http:// dx.doi.org/10.1016/j.cl.2016.01.002)