To read and properly use a dataset, DX has to know how to interpret a set of numbers in a file. Specificly, DX needs to know:

• Whether the data is gridded or scattered.
• The array size/dimension if gridded, or the number of points if scattered.
• The positions of the points in space. If gridded, the positions may be specified by grid origin and grid deltas for each dimension.
• The data format, text or binary.
• Each data type, such as float, integer, double, byte, etc.
• Whether the data is scalar or vector.
• How the data is grouped. If for instance you measure two variables in a volume, perhaps temperature and pressure, you could arrange the data as
  x, y, z, temp, p
  repeated for each x,y,z or as five lists of all the x values, followed by all the y values, etc.
• The order in which the data was written. For instance, C programs tend to write the last index of an array changing fastest, while FORTRAN programs tend to write the first index of an array changing fastest.
• If there are there comments in the file that must be skipped.
• A name for a dataset, for reference in a program.

There are a set of keywords which are used to specify the list above. Very often you only need to specify a few items, since most have a reasonable default. Keywords and their values may be specified by typing them directly into the format field of an import module, but it is a bit easier to read and edit if a Format module is used also. First we will look at three examples then the details of the input specifications.

• Example 1 shows a fragment of DX code which will read a data file with name data.file in which there are 100 scattered values, each located at a 3D point. The data file is assumed to have the form of x,y,z,data repeated for each point. Refering to the image, there are two dialog boxes open in a DX VPE window. The Input module dialog box shows that the first input sets the data filename. The Format dialog box shows six inputs:
  1. Specification to make one string, delimited by commas out of the next five strings.
  2. A keyword which tells DX that a certain class of keywords follows.
  3. Tells DX there are 100 points in the file.
  4. Defines the structure for each variable: location is a 3D vector and the data is a scalar.
  5. Defines names for each variable. "Locations" is a reserved word which makes the variable the positions for any other data in the file.
  6. Indicates that the positions and data are grouped together.
  Given these specifications, DX will default to text format, with data type of float when reading a file.

• The second example we will do by just specifiying the keywords. Assume that you have a 128 by 128 by 64 grid of scalar values stored in a file in binary format and that the data was written in array-default order from a FORTRAN program. (The actual grid positions however are not in the file) Also assume that the grid origin is [0,0,0] and the grid deltas are 1,1,0.3 in the x, y and z directions respectively. The required specifications (which would be typed into the fields of a Format module) are:
  1. %s,%s,%s,%s,%s (concantenates the next strings)
  2. general
  3. grid=128x128x64
  4. majority=column (FORTRAN order--first index changes fastest)
  5. format=binary
  6. positions=0, 1, 0, 1, 0, .3

• The third example will be a data set consisting of an array of 3D valued vectors sampled on a 2D grid of size 10 by 20, written in C default order in a text file. The file also has a 3 line long descriptive header. The actual grid positions are not in the file. The required specifications (which would be typed into the fields of a Format module) are:
  1. %s,%s,%s,%s (concantenates the next strings)
  2. general
  3. grid=10x20
  4. structure=3-vector
  5. header=lines 3

• The first keyword in the format string of the Import module must (for this simplified description) be general .

• The next keyword must be either
  • grid= num_x x num_y x num_z
    for gridded data (The number of entries must match the dimension of the array.)
  • points= n
    for scattered data

• The rest of the specifications can be in any order (except for positions which must be last) and may be omitted if set to the default value. The three tags field, structure, and type form a group. If one is used, then all three should generally be used, with the same number of arguments.

  Note that in the following bullets the "|" symbol means "or".

  • field= name₁, name₂, ...
    This keyword constructs string names for different data items in a file. If, for example, a file had a 3D position, followed by a scalar temperature, and a vector velocity you might use:
    field=locations, temp, vel
    The locations name is actually a reserved word which specifies that the remaining variables (in this case temp and vel) are recorded at the given positions in space.

  • structure= structure₁, structure₂, ...
    This keyword indicates the data form. Each entry may be either scalar, string[n], 2-vector, or 3-vector. For the example in the above bullet the structure would be:
    structure=3-vector, scalar, 3-vector
    The default structure is scalar. The [n] parameter on the string structure indicates the length of the longest string.

  • type=type₁, type₂, ...
    This keyword indicates the data type of each variable. Each entry may be:
    double, float, string, signed byte, unsigned byte, int, unsigned int, short, or unsigned short.
    For the example above the type might be
    type=float,float,float
    The default type is float.

  • format=text | binary | lsb binary | msb binary
    This keyword indicates the file format as either text or binary. The default value is text. If the binary option is used, the msb/lsb indicates whether the machine internal number format is "most significant byte first" or "least significant byte first". The default is whatever system the current machine uses. Usually you can ignore the msb/lsb distinction and just use the binary tag.

  • header= lines n | bytes n | marker "string"
    This keyword indicates any material at the beginning of the file which should be skipped. A certain number of lines or bytes may be specified, or reading may begin after a certain specified string. By default there is no header and reading starts at the beginning of the file.

  • majority= row | column
    This keyword selects whether the data is read with the last dimension changing fastest (row) or with the first dimension changing fastest (column). The default is row.

  • interleaving= field | record | record-vector
    This keyword shows how the data is arranged with in the file. The default is record-vector. As an example assume that there are two different fields in a file: a scalar field and a vector field. The index indicates a spatial location.
    
    • The field tag means that the data would be arranged as:
      t(1), vx(1),vy(1), vz(1)
t(2), vx(2),vy(2), vz(2)
.
.
t(n), vx(n),vy(n), vz(n)

    • The record-vector tag means that the data would be arranged as:
      t(1), t(2),.., t(n)
vx(1),vy(1), vz(1)
vx(2),vy(2), vz(2)
.
.
vx(n),vy(n), vz(n)

    • The record tag means that the data would be arranged as:
      t(1), t(2),.., t(n)
vx(1),vx(2),.., vx(n)
vy(1),vy(2),.., vy(n)
vz(1),vz(2),.., vz(n)


  • positions= origin₁,delta₁, origin₂,delta₂, ...
    This keyword specifies where the data points are. The locations reserved field name (see field above) associates a position with each data point and makes the positions keyword unnecessary. The default (if the grid keyword is used) is a grid starting at the origin, with unit steps along each axis. The default (if the points keyword is used) will be a generally useless 1D array starting at zero and increasing in x. You will almost always want to use the locations field name when inputing scattered data using the points keyword.

• Note that DX has other specification tags. See the IBM Visualization Data ExplorerQuickStart Guide for more details.