OPH_IMPORTNC¶

Description¶

Type

Data Process.

Behaviour

It imports a NetCDF file into a datacube (both measure and dimensions). WARNING: It imports only mono-dimensional coordinate variables.

Parameters

• container: name of the input container; by default it will be automatically set to file name.
• cwd: absolute path corresponding to the current working directory, used to select the folder where the container is located.
• host_partition: name of I/O host partition used to store data. If default value “auto” is specified, the first host partition available will be used.
• filesystem: type of filesystem used to store data. Possible values are “local”, “global” or “auto” (default). In the last case the first filesystem available will be used.
• ioserver: type of I/O server used to store data. Possible values are: “mysql_table” (default) or “ophidiaio_memory”.
• import_metadata: with “yes” (default), it will import also metadata; with “no”, it will import only data.
• check_compliance: checks if all the metadata registered for reference vocabulary are available. No check is done by default.
• schedule: scheduling algorithm. The only possible value is 0, for a static linear block distribution of resources.
• nhost: number of output hosts. With default value (‘0’), all host available in the host partition are used.
• ndbms: number of the output DBMS per host. With default value (‘0’), all DBMS instance available per host are used.
• ndb: number of output database per host. Default value is ‘1’.
• nfrag: number of fragments per database. With default value (‘0’), the number of fragments will be the ratio of the product of sizes of the n-1 most outer explicit dimensions to the product of the other arguments.
• measure: name of the measure related to the NetCDF file.
• run: if set to ‘no’, the operator simulates the import and computes the fragmentation parameters that would be used else, if set to ‘yes’, the actual import operation is executed.
• src_path: path or OPeNDAP URL of the NetCDF file. Local files have to be stored in folder BASE_SRC_PATH or its sub-folders (see oph_configuration for additional information).
• cdd: absolute path corresponding to the current directory on data repository. It is appended to BASE_SRC_PATH to build the effective path to files (see configuration notes for further details).
• exp_dim: names of explicit dimensions. Multiple-value field: list of dimensions separated by “|” can be provided. It implicitly defines explicit dimension number and level (order). If default value “auto” is specified, then the first n-1 dimension of the measure in the NetCDF file will be used as explicit dimensions.
• imp_dim: names of implicit dimensions. Multiple-value field: list of dimensions separated by “|” can be provided. It implicitly defines implicit dimension number and level (order) and must be the same number of “imp_concept_level”. If default value “auto” is specified, then the last dimension of the measure in the NetCDF file will be used as implicit dimension.
• subset_dims: dimension names used for the subsetting. Multiple-value field: list of dimensions separated by “|” can be provided and must be the same number of “subset_filter”.
• subset_filter: enumeration of comma-separated elementary filters (1 series of filters for each dimension). Multiple-value field: list of filters separated by “|” can be provided and must be the same number of “subset_dims”. Values should be numbers. Example: subset_dims=lat|lon;subset_filter=35:45|15:20. Possible forms are:
  • start_value: single value specifying the start index of the subset
  • start_value:stop_value: select elements from start_index to stop_index.
• subset_type: if set to “index” (default), the subset_filter is considered on dimension index. With “coord”, filter is considered on dimension values. In case of single value, that value is used for all the dimensions.
• time_filter: enable filters using dates for time dimensions; enabled by default.
• offset: it is added to the bounds of subset intervals defined with “subset_filter” in case of “coord” filter type is used.
• exp_concept_level: concept level short name (must be a single char) of explicit dimensions. Default value is “c”. Multiple-value field: list of concept levels separated by “|” can be provided and must be the same number of “exp_dim”.
• imp_concept_level: concept level short name (must be a single char) of implicit dimensions. Default value is “c”. Multiple-value field: list of concept levels separated by “|” can be provided.
• compressed: with “yes”, it will save compressed data; with “no” (default), it will save original data.
• grid: optional argument used to identify the grid of dimensions to be used (if the grid already exists) or the one to be created (if the grid has a new name). If it isn’t specified, no grid will be used.
• hierarchy: concept hierarchy name of the dimensions. Default value is “oph_base”. Multiple-value field: list of concept hierarchies separated by “|” can be provided.
• vocabulary: optional argument used to indicate a vocabulary (name of set of keys) to be used to associate metadata to the container.
• base_time: in case of time hierarchy, it indicates the base time of the dimension. Default value is 1900-01-01.
• units: in case of time hierarchy, it indicates the units of the dimension. Possible values are: s, m, h, 3, 6, d.
• calendar: in case of time hierarchy, it indicates the calendar type. Possible values are:
  • standard (default)
  • gregorian
  • proleptic_gregorian
  • julian
  • 360_day
  • no_leap
  • all_leap
  • user_defined
• month_lengths: in case of time dimension and user-defined calendar, it indicates the sizes of each month in days. There must be 12 positive integers separated by commas. Default is ‘31,28,31,30,31,30,31,31,30,31,30,31’.
• leap_year: in case of time dimension and user-defined calendar, it indicates the first leap year. By default it is set to 0.
• leap_month: in case of time dimension and user-defined calendar, it indicates the leap month. By default it is set to 2 (i.e. February).
• check_grid: optional flag to be enabled in case the values of grid have to be checked (valid only if the grid already exists).
• description: additional description to be associated with the output cube.

System parameters

• exec_mode: operator execution mode. Possible values are async (default) for asynchronous mode, sync for synchronous mode with json-compliant output.
• ncores: number of parallel processes to be used (min. 1).
• sessionid: session identifier used server-side to manage sessions and jobs. Usually, users don’t need to use/modify it, except when it is necessary to create a new session or switch to another one.
• objkey_filter: filter on the output of the operator written to file (default=all => no filter, none => no output, importnc => show operator’s output PID as text).

Examples

Import a NetCDF file excluding metadata into the session directory “session-code1”:

[OPH_TERM] >>  oph_importnc cwd=/session-code1;container=container1;measure=pressure;src_path=/path/of/ncfile.nc;imp_concept_level=d;import_metadata=no;

Arguments¶