Parallel Access to NetCDF Data
Parallel access to netCDF data files can speed read/write times from paralleled code - that is, code that is written with the MPI library to run on (many) multiple cores. How Much Does it Help? The amount of improvement you can get for parallel access depends heavily on your hardware. Running tests on a high-powered, multi-core linux box, I can see improvements of 4X or even greater. On HPC systems with a parallel file system, you should be able to do better. But parallel IO performance (IMHO), does not scale as well as processor performance. On systems with a large number of cores, you will max out the hardward channels to your storage before you are using all the cores for IO. Using a subset of the cores for IO is a good solution, but annoying to program. This functionality is provided by the PIO library (see below). Building NetCDF with Parallel Access In order to use parallel access with netCDF, you must build netCDF correctly. The following must be true: All l...