IOFSL

"A project to design, build, and distribute a scalable, unified high-end computing I/O forwarding software layer that would be adopted and supported by DOE Office of Science and NNSA.

The application I/O calls get translated to ZOIDFS—an efficient parallel file I/O API. ZOIDFS calls are forwarded over an interconnecting network to the I/O nodes. I/O nodes run ZOIDFS server code that efficiently translates the calls to the native API of the underlying parallel filesystem.

IOFSL leverages existing projects including:

• ZOIDFS is a filesystem-independent protocol for I/O forwarding, somewhat similar to NFSv3. Highlights include stateless servers, file handles (rather than file descriptors) that can be freely exchanged between the clients, and maximally flexible read/write operations that can deal with multiple file regions and memory buffers in one call.
• SYSIO is a flexible, user-space implementation of the virtual file system abstraction, developed for Cray XT. It supplies all of the normal POSIX calls to the application and assembles the local name space on the compute node based on global configuration information. We are augmenting SYSIO for this project to use our I/O forwarding framework underneath for access to file systems.
• ROMIO is the foundation of most MPI-IO implementations in use. We are implementing support for the ZOIDFS client interface in ROMIO, so that applications using MPI-IO can take advantage of our I/O forwarding infrastructure without modifications.
• BMI is a portable communication layer developed for PVFS, which has support for MX, sockets, Portals, and others. It serves as a communication interface between the compute nodes and the I/O nodes.
• ZOID (the ZeptoOS I/O Daemon) is an extensible, high-performance function call forwarding infrastructure developed for IBM Blue Gene within the ZeptoOS project. ZOID is used as a low-level transfer layer for the Blue Gene driver of BMI.

http://www.mcs.anl.gov/research/projects/iofsl/

https://trac.mcs.anl.gov/projects/iofsl/

https://www.mcs.anl.gov/software/data-intensive-software

Dependencies

ROMIO/MPICH2 - https://www.mpich.org/

OpenPA - http://trac.mpich.org/projects/openpa/

PVFS2 - https://en.wikipedia.org/wiki/Parallel_Virtual_File_System

FUSE - https://github.com/libfuse/libfuse

libsysio - https://sourceforge.net/projects/libsysio/