tags:

views:

1101

answers:

2

I am currently working on (or at least planning) a couple of projects that work with large amounts of repetitive data. The kind of data that works well in a spreadsheet or database, but is nasty in XML. :)

NetCDF seems like a great option for a file format. However, my work is in C# and there is no "official" NetCDF implementation available. There is an "official" Python version that I could potentially use with IronPython. Another option is the "NetCDF Library for .Net" on CodePlex but it seems pretty quiet (http://www.codeplex.com/netcdf).

Has anyone worked with either of these libraries? Can you provide any recommendations?

+2  A: 

First, are you sure that NetCDF is the right choice? If you want to interact with other programs that need to read in large amounts of data and they already support NetCDF, then it's probably a great choice. There aren't that many standard and well-supported file formats that support large multidimensional arrays. But if you're only going to be reading and writing files from C#, it may not be such a good choice.

I'm a big fan of the "classic" NetCDF file format. It's compact and extremely simple, but flexible enough to support lots of common kinds of multidimensional well-structured data. It only took me one day to write a complete parser for classic NetCDF, and it only took an hour to write a program to output a well-formed special case of a classic NetCDF file. You could implement a pure C# NetCDF library yourself and it wouldn't be much trouble. You could easily start by implementing only the features you need. Here's the specification.

Unfortunately, NetCDF-4 chose to use HDF-5 as its data format. It adds a lot of complexity and makes it much more difficult to write a complete NetCDF parser in another language. HDF-5 is very general-purpose and in my opinion, it was overengineered - it tries to be too many things to too many people. I would not recommend trying to work with it directly unless you plan to spend a month writing unit tests. If you must use netCDF-4 / HDF-5 from C#, your only realistic option would be to wrap the C library using SWIG or something like that.

Note that NetCDF for Python is just a wrapper around the C code, so it's not really all that helpful; if you're going to use a wrapped C library you may as well just write a C# wrapper rather than use Python as a middle layer.

dmazzoni
+1 Another option besides SWIG would be to use the PInvoke interop assistant. It was written by the .NET framework team. http://clrinterop.codeplex.com/releases/view/14120
MarkJ
A: 

In a project we are using the ucar netcdf implementation in c# using ikvm. IKVM can be used to 'convert' java projects into .Net libraries without a use for java VM. I have not done any performance check but it is a simple way to get netcdf in c# :).

http://www.ikvm.net/stories.html

http://www.unidata.ucar.edu/downloads/netcdf/netcdf-java-4/index.jsp