Support NetCDF format

Details

  • Type: New Feature New Feature
  • Status: Closed Closed
  • Priority: Minor Minor
  • Resolution: Won't Fix
  • Affects Version/s: 2.1.M0, 2.4-M2
  • Fix Version/s: 2.7-M3
  • Component/s: None
  • Labels:
    None

Description

NetCDF (network Common Data Form) is an interface for array-oriented data access and a library that provides an implementation of the interface. The netCDF library also defines a machine-independent format for representing scientific data. Together, the interface, library, and format support the creation, access, and sharing of scientific data. The netCDF software was developed at the Unidata Program Center in Boulder, Colorado.

More detail can be found at:
http://www.unidata.ucar.edu/packages/netcdf/index.html

A LGPL Java library already exists at:
http://www.unidata.ucar.edu/packages/netcdf-java/

NOTE Added 6/25/2005 by Bryce:
A review of the current form of the netCDF java library, as well as reviewing the direction in which netCDF is heading reveals:

  • Java is the prototype language for which the new features in NetCDF4 are tested.
  • NetCDF4 implements a "Common Data Model" (CDM) which is the result of generalizing the NetCDF3, HDF5, and OpenDAP data models.
  • The NetCDF-Java library automatically recognizes many of the common ways of storing geographic information in netCDF files and abstracts to the "GeoGrid" object.
  • The NetCDF-Java library is a 100% java library which can read/write NetCDF3 files, and can read a number of other formats (HDF5, GRIB, etc.)

This should alleviate some of the concerns expressed in the comments below and suggests the following:

  • GeoTools should provide an adapter from the GeoGrid abstraction to the GeoAPI framework.
  • NetCDF conventions should be added to the netCDF-java library.

Issue Links

depends upon

Task - A task that needs to be done. GEOT-2117 Review GeoToolkit fixes

  • Minor - Minor loss of function, or other problem where easy workaround is present.
  • Open - The issue is open and ready for the assignee to start work on it.
is duplicated by

Task - A task that needs to be done. GEOT-505 Create netCDF/WRF GridCoverageReader Plugin

  • Minor - Minor loss of function, or other problem where easy workaround is present.
  • Closed - The issue is considered finished, the resolution is correct. Issues which are not closed can be reopened.
is related to

Sub-task - The sub-task of the issue GEOT-757 Implement WRF nD coverage plugin

  • Major - Major loss of function.
  • Closed - The issue is considered finished, the resolution is correct. Issues which are not closed can be reopened.

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Martin Desruisseaux added a comment -
We already have some work done in this regards. I will have to look at that when I will have a chance.
Show
Martin Desruisseaux added a comment - We already have some work done in this regards. I will have to look at that when I will have a chance.
Hide
Permalink
Ed Yu added a comment -
I'll be doing some with creating a polygon shapefile from a NetCDF file. In what context are we going to "support NetCDF format"? In other words, is there a better description on what do we want to accomplish here?
Show
Ed Yu added a comment - I'll be doing some with creating a polygon shapefile from a NetCDF file. In what context are we going to "support NetCDF format"? In other words, is there a better description on what do we want to accomplish here?
Hide
Permalink
James Macgill added a comment -
"Support NetCDF" referes to creating an implementation of the DataStore api that is able to work with NetCDF files.

A NetCDF -> Shapefile program would not actualy be direct, instead it would read from a NetCDF DataStore implementation into the internal GeoTools feature object model and then be written back out via the Shapefile DataStore.

You can find full details on the DataStore API, how to use it and how to write a new implementation in this tutorial:
http://docs.codehaus.org/display/GEOTOOLS/DataStore+Walkthrough

If you need further details feel free to ask!

It would be excelent if you could undertake this task, read the tutorial and let me know if you think you can take it on.
Show
James Macgill added a comment - "Support NetCDF" referes to creating an implementation of the DataStore api that is able to work with NetCDF files. A NetCDF -> Shapefile program would not actualy be direct, instead it would read from a NetCDF DataStore implementation into the internal GeoTools feature object model and then be written back out via the Shapefile DataStore. You can find full details on the DataStore API, how to use it and how to write a new implementation in this tutorial: http://docs.codehaus.org/display/GEOTOOLS/DataStore+Walkthrough If you need further details feel free to ask! It would be excelent if you could undertake this task, read the tutorial and let me know if you think you can take it on.
Hide
Permalink
Ed Yu added a comment -
Hi James,

Thanks for answering. Although I'm not an expert in NetCDF, I think NetCDF is more suitable for storing primitive data types in fixed multi dimensions (only a single dimension/attribute can grow). As a result, storing gridded data set is pretty common in NetCDF format.

IMHO, using the NetCDF as data store to persist features is not a natural fit. Fortunately, there is a set of DODS API bundling with (uses) the NetCDF java API which allow growing structures (possibly nested) to be stored because the OpenDAP data model is a superset of NetCDF. There are quite a good bit of work ongoing to gel DODS (OpenDAP data sets) with GIS. Anyway, if it is ok, I'll look at the DODS API when I have a chance.
Show
Ed Yu added a comment - Hi James, Thanks for answering. Although I'm not an expert in NetCDF, I think NetCDF is more suitable for storing primitive data types in fixed multi dimensions (only a single dimension/attribute can grow). As a result, storing gridded data set is pretty common in NetCDF format. IMHO, using the NetCDF as data store to persist features is not a natural fit. Fortunately, there is a set of DODS API bundling with (uses) the NetCDF java API which allow growing structures (possibly nested) to be stored because the OpenDAP data model is a superset of NetCDF. There are quite a good bit of work ongoing to gel DODS (OpenDAP data sets) with GIS. Anyway, if it is ok, I'll look at the DODS API when I have a chance.
Hide
Permalink
Chris Holmes added a comment -
Well the Data Store does not need to persist features, it would be useful as a read only datastore. Indeed if a netCDF datastore was created then geotools could theoretically (and hopefully soon in practice to) be used to persist it to another datastore, such as shapefile or postgis.
Show
Chris Holmes added a comment - Well the Data Store does not need to persist features, it would be useful as a read only datastore. Indeed if a netCDF datastore was created then geotools could theoretically (and hopefully soon in practice to) be used to persist it to another datastore, such as shapefile or postgis.
Hide
Permalink
Ed Yu added a comment -
Not trying to beat a dead horse here, but I still don't see how reading numeric arrays from NetCDF files is of any value other than converting them into raster type data sets.
Show
Ed Yu added a comment - Not trying to beat a dead horse here, but I still don't see how reading numeric arrays from NetCDF files is of any value other than converting them into raster type data sets.
Hide
Permalink
Bryce Nordgren added a comment -
Linked this in with the overall netCDF issue.
Show
Bryce Nordgren added a comment - Linked this in with the overall netCDF issue.
Hide
Permalink
Bryce Nordgren added a comment -
One of the "conventions" for storing netCDF data is amenable to feature data (there may be others). It's basically designed to allow a once-per-second record of an air sampling system onboard an aircraft. Mostly, though, it's multidimensional gridded variables (usually more than 2d).

However "conventions" aren't "rules" or "standards". "NetCDF" support may very well degenerate into individual application support, as would be the case with the Weather Research and Forecasting model which does not use any of the conventions.

A list of the conventions can be found on the netCDF website:
http://my.unidata.ucar.edu/content/software/netcdf/conventions.html
Show
Bryce Nordgren added a comment - One of the "conventions" for storing netCDF data is amenable to feature data (there may be others). It's basically designed to allow a once-per-second record of an air sampling system onboard an aircraft. Mostly, though, it's multidimensional gridded variables (usually more than 2d). However "conventions" aren't "rules" or "standards". "NetCDF" support may very well degenerate into individual application support, as would be the case with the Weather Research and Forecasting model which does not use any of the conventions. A list of the conventions can be found on the netCDF website: http://my.unidata.ucar.edu/content/software/netcdf/conventions.html
Hide
Permalink
Bryce Nordgren added a comment -
Edited text to address some of the above comments and give a current review of the netcdf-java library.
Show
Bryce Nordgren added a comment - Edited text to address some of the above comments and give a current review of the netcdf-java library.
Hide
Permalink
Martin Desruisseaux added a comment -
Work started on trunk in modules/unsupported/coverageio-netcdf.
Show
Martin Desruisseaux added a comment - Work started on trunk in modules/unsupported/coverageio-netcdf.

People

  • Assignee:
    Simone Giannecchini
    Reporter:
    James Macgill
Vote (0)
Watch (3)

Dates

  • Created:
    Updated:
    Resolved: