ATPESC 2021
- TBA
Globus will be presenting in this year's Argonne training program on extreme-scale computing.
Automating data management for computation at (exa)scale with Globus
We will briefly review the challenges of managing research data at scale, using HPC modeling and simulation as exemplars. We will describe how data can be staged to support computational tasks, and how the resulting data products can be shared with collaborators. We will describe the Globus Automate service and illustrate how it facilitates automation of compute tasks at scale. We will also introduce funcX, a service for running codes on diverse classes of compute resources. We will demonstrate the core capabilities of the Globus service using a Jupyter notebook to compose and execute an automated flow. Participants will experiment with building flows to describe and analyze data, and share the results with collaborators. We will provide templates as a starting point for hands-on exercises. Participants should be familiar with Python; knowledge of web programming (calling REST APIs in particular) is desirable but not required.