pC++/streams: a Library for I/O on Complex Distributed Data Structures

Abstract

The design and implementation of portable, efficient, and expressive mechanisms for I/O on complex distributed data structures—such as found in adaptive parallel applications—is a challenging problem that we address in this paper.We describe the design, programmer interface, implementation, and performance of pC++/streams, a library that provides an expressive mechanism for I/O on distributed arrays of variable-sized objects in pC++, an object-parallel language. pC++/streams is intended for developers of parallel programs requiring efficient high-level I/O abstractions for checkpointing, scientific visualization, and debugging.pC++/streams is an implementation of d/streams, a language-independent abstraction for buffered I/O on distributed data structures. We describe the d/streams abstraction and present performance results on the Intel Paragon and SGI Challenge showing that d/streams can be implemented efficiently and portably.