Future cyber-infrastructures are increasingly expected to revolve around the integration of big data, computation, and high performance networking. The High Performance Computing with Data and Networking Acceleration (HPCDNA) project focuses on these issues in the context of campus science and computational requirements. Many researchers have application specific data and a variety of desired compute environments, including purpose built small lab compute resources, medium scale campus High Performance Computing (HPC), cloud computing, and national scale distributed or centralized resources. The inability to flexibly and seamlessly get data to the most appropriate compute resource is often the defining factor determining where the computation is run and what computation is performed. The HPCDNA project is developing technologies to improve the ability to utilize common data sets across this diverse set of computational resources. The HPCDNA architecture and technologies include a Network Embedded Storage (NES) system based on a distributed high performance parallel Ceph based file system. This system is deployed at the edge of the Mid-Atlantic Crossroads regional network and the UMD campus infrastructure. This system is enhanced for high performance via tight integration with campus HPC, campus networks, and wide area networks. The HPCDNA system has been enhanced with the RAINS MRML and computational intelligence technologies to develop a “Software Defined ScienceDMZ (SD-SDMZ)” which enables on-demand application specific hybrid services to be constructed.
