Towards HDF5: Encapsulation of Large and/or Complex Astronomical Data

The size and complexity of astronomical data are growing at relentless rates. This increase is especially apparent in the radio community as evidenced by the data challenges faced by many of the SKA pathfinders and other major new radio telescopes such as LOFAR, EVLA, ALMA, ASKAP, MeerKAT, MWA, LWA, and eMERLIN. Enormous data rates are also becoming a challenge for large optical projects that are currently ramping up including Pan-Starrs and LSST. As progress towards meeting these challenges, ASTRON and the LOFAR project are currently exploring the use of the Hierarchical Data Format, version 5 (HDF5) for LOFAR radio data encapsulation. In this session, we brought together scientists and developers struggling with large and complex datasets as well as those groups currently exploring HDF5 implementations. This paper briefly summarizes the BoF presentations, material discussed and future goals.