The links below give an exhaustive description of all the operators available in the platform. Each manual page describes the operator’s behaviour, its parameters and a simple usage example. A table summarizing the parameters constraints (data type, mandatoriness, admissible and default values) closes each section. To better understand how to submit a request, we recommend you to read the Ophidia Terminal: basic usage guide.
Note
In order to fully use all the Ophidia operators and other features, in dynamic cluster mode, a cluster of I/O server instances must be deployed before running the data operators (e.g. OPH_AGGREGATE, OPH_REDUCE, etc.). A cluster consists of a set of reserved analytics nodes with a single I/O server instance running on each. It is identified by a user-defined host partition name and multiple clusters can be deployed by the same user. After its creation, the user can exploit the computing resources of the cluster by simply specifying the host partition name in the data import operators (OPH_IMPORTNC2, OPH_RANDCUBE2, etc.) and release them at the end of its workflow of operators by undeploying the cluster. Additional information about the usage of cluster management can be found in the I/O server cluster guides.
This group includes the main data processing Ophidia operators. Most of them process an input cube to obtain another cube with different data and/or metadata.
The operators OPH_INTERCUBE, OPH_MERGECUBES and OPH_MERGECUBES2 process two input cubes.
The operator OPH_SCRIPT allows the user to run a generic (pre-registered) bash script.
Data reduction can be applied to dimension values (and the corresponding measures) in two ways:
The former approach is adopted by the operators OPH_AGGREGATE, for tuples, and OPH_REDUCE, for arrays. Use group_size to set the number of elements to be aggregated. The latter approach is adopted by the operators OPH_AGGREGATE2, for explicit dimensions, and OPH_REDUCE2, for implicit dimensions. See Time Management section for more information about concept levels and aggregation.
| NAME | DESCRIPTION |
|---|---|
| OPH_AGGREGATE | It executes an aggregation function on a datacube with respect to explicit dimensions. |
| OPH_AGGREGATE2 | It executes an aggregation operation based on hierarchy on a datacube along an explicit dimension. |
| OPH_APPLY | It executes a query on a datacube. |
| OPH_DRILLDOWN | It performs a drill-down operation on a datacube, i.e. it transforms dimensions from implicit to explicit. |
| OPH_DUPLICATE | It duplicates a datacube creating an exact copy of the input one. |
| OPH_INTERCUBE | It executes an operation between two datacubes with the same fragmentation structure and returns a new datacube as result of the specified operation. |
| OPH_MERGE | It creates a new datacube grouping nmerge input fragments in a new output fragment. |
| OPH_MERGECUBES | It merges the measures of n input datacubes with the same fragmentation structure and creates a new datacube with the union of the n measures. |
| OPH_MERGECUBES2 | It merges the measures of n input datacubes with the same fragmentation structure and creates a new datacube with the union of the n measures. A new implicit dimension will be created. |
| OPH_PERMUTE | It performs a permutation of the dimensions of a datacube. |
| OPH_REDUCE | It performs a reduction operation on a datacube with respect to implicit dimensions. |
| OPH_REDUCE2 | It performs a reduction operation based on hierarchy on a datacube along an implicit dimension. |
| OPH_ROLLUP | It performs a roll-up operation on a datacube, i.e. it transforms dimensions from explicit to implicit. |
| OPH_SCRIPT | It executes a bash script. |
| OPH_SPLIT | It creates a new datacube by splitting input fragments in nsplit output fragments in the same origin database. |
| OPH_SUBSET | It performs a subsetting operation along dimensions of a datacube. |
| OPH_SUBSET2 | It performs a subsetting operation along dimensions of a datacube (DEPRECATED). |
These operators translate the format of a dataset, retrieved from an external source (e.g. an OPeNDAP service) or stored in a local data repository (see configuration notes for futher details), into the format used by Ophidia and vice versa. To get a more advanced understanding about the Ophidia format and how to manage data organization and distribution have a look at the data model section.
The following translations are supported:
| FROM | TO | Operator |
|---|---|---|
| FITS | Ophidia | OPH_IMPORTFITS |
| NetCDF | Ophidia | OPH_IMPORTNC / OPH_IMPORTNC2 |
| NetCDF | JSON | OPH_EXPLORENC |
| Ophidia | JSON | OPH_EXPLORECUBE |
| Ophidia | NetCDF | OPH_EXPORTNC / OPH_EXPORTNC2 |
| Ophidia | HTML | OPH_PUBLISH |
| NAME | DESCRIPTION |
|---|---|
| OPH_B2DROP | It uploads a file onto a B2DROP remote folder. |
| OPH_CONCATNC | It concatenates a NetCDF file to a datacube. |
| OPH_CONCATNC2 | It concatenates a NetCDF file to a datacube (Ophidia I/O Server-based version). |
| OPH_EXPLORECUBE | It prints the data stored into a datacube, possibly subset along its dimensions. |
| OPH_EXPLORENC | It read a NetCDF file (both measure and dimensions). |
| OPH_EXPORTNC | It exports data of a datacube into multiple NetCDF files. |
| OPH_EXPORTNC2 | It exports data of a datacube into a single NetCDF files (parallel version). |
| OPH_IMPORTFITS | It imports a FITS file into a datacube. |
| OPH_IMPORTNC | It imports a NetCDF file into a datacube. |
| OPH_IMPORTNC2 | It imports a NetCDF file into a datacube (Ophidia I/O Server-based version). |
| OPH_PUBLISH | It creates HTML pages with data and other information from a datacube. |
| OPH_RANDCUBE | It creates a new datacube with random data and dimensions. |
| OPH_RANDCUBE2 | It creates a new datacube with random data and dimensions (Ophidia I/O Server-based version). |
| OPH_UNPUBLISH | It removes the HTML pages created by the PUBLISH2 operator. |
Metadata operators are used to print information about datacubes: size, provenance, dimension values, attributes, physical organization of data, etc.
| NAME | DESCRIPTION |
|---|---|
| OPH_CONTAINERSCHEMA | It shows some information about a container: description, vocabulary, dimension list, etc. |
| OPH_CUBEELEMENTS | It computes and displays the number of elements stored in the input datacube. |
| OPH_CUBEIO | It shows the hierarchy of all datacubes used to generate the input datacube and of those derived from it. |
| OPH_CUBESCHEMA | It shows metadata information about a datacube and the dimensions related to it. |
| OPH_CUBESIZE | It computes and displays the size of the input datacube. |
| OPH_METADATA | It provides CRUD operations (Create, Read, Update and Delete) on OphidiaDB metadata. |
| OPH_SEARCH | It provides enhanced searching on metadata. |
| OPH_SHOWGRID | It shows information about one or more grids related to the specified container. |
| OPH_TASKS | It shows information about executed tasks. |
These operators allows the user to browse the Ophidia Virtual File System.
| NAME | DESCRIPTION |
|---|---|
| OPH_CREATECONTAINER | It creates an empty container. |
| OPH_DELETE | It removes a datacube. |
| OPH_DELETECONTAINER | It removes an empty container with related dimensions and grids. |
| OPH_FOLDER | It manages folders of the Ophidia filesystem: creation and removal. |
| OPH_LIST | It shows information about folders, container and datacubes fragmentation (file system). |
| OPH_MOVECONTAINER | It moves/renames a visible container. |
This group includes a number of flow control operators that could be used within an Ophidia workflow to implement complex data processing in batch mode. In particular, they implement several advanced features: setting of run-time variables, iterative and parallel interface, selection interface, interactive workflows, interleaving workflows, etc.
| NAME | DESCRIPTION |
|---|---|
| OPH_ELSE | Start the last sub-block of a selection block “if”. |
| OPH_ELSEIF | Start a new sub-block of a selection block “if”. |
| OPH_ENDFOR | Close a loop “for”. |
| OPH_ENDIF | Close a selection block “if”. |
| OPH_FOR | Implement a loop “for”. |
| OPH_IF | Open a “if” selection block. |
| OPH_INPUT | It sends commands or data to an interactive task. |
| OPH_SET | Set a parameter in the workflow environment. |
| OPH_WAIT | Wait until an event occurs. |
This group consists of operators that allows the user to monitor or stop running workflows (e.g. check for task status), list submitted tasks, get the service parameters to configure automatically the terminal, list or remove sessions (see Session management), grant access permissions to sessions, manager cluster deployment (if enabled), etc.
The operators OPH_LOG_INFO and OPH_SERVICE allows to administrators to read logs and to start/stop Ophidia service.
| NAME | DESCRIPTION |
|---|---|
| OPH_CANCEL | Stop the execution of a running workflow. |
| OPH_CLUSTER | Start, stop and get information about clusters of I/O servers. |
| OPH_GET_CONFIG | Request the configuration parameters. |
| OPH_INSTANCES | It shows information about host partitions, hosts and dbms instances. |
| OPH_LOGGINGBK | It shows info about jobs and sessions (logging bookkeeping). |
| OPH_LOG_INFO | It reads the last lines from the server log or from a specific container log. |
| OPH_MANAGE_SESSION | Request or set session data: session list, session creation date, authorized users, etc. |
| OPH_RESUME | Request the list of the commands submitted within a session or the output of a command. |
| OPH_SERVICE | Request or set the service status. |
| NAME | DESCRIPTION |
|---|---|
| OPH_FS | It browses the real file system: list files, change working directory, etc. |
| OPH_HIERARCHY | It shows the list of the hierarchies or the description of a specified hierarchy. |
| OPH_MAN | It shows a description of the behaviour of an operator/primitive. |
| OPH_OPERATORS_LIST | It shows the list of all active operators. |
| OPH_PRIMITIVES_LIST | It shows a list with info about active Ophidia Primitives loaded into a specific DBMS instance. |
