GENERAL PROCESSING TIPS
                                                            Gregg Walters
                                                            March 29, 1993

                          DATA ACCESS TIPS

Don't forget to backup your data!

When users complain that data is missing, more often than not
there is a fault in the way that they are testing the identification
(ID) of the grid or observation.

The great majority of these faults involve some variation of one or
more of the following three mistakes.  They are so common that even
experienced programmers make them.  They just happen to recognize
them sooner - usually.

1) When reading a volume which starts near the end of the month
   and ends near the start of another month, code is used which
   is blind to the month change.

   E.g., where IDY is the day unpacked from the ID, and happens
   to be 29, such as in the first data block from a volume
   containing 1991dec29-1992jan04:

      ISDAY = 29       !  start day
      IEDAY = 04       !  end day

      IF (IDY .LT. ISDAY) GO TO 10           ! go to read another
      IF (IDY .GT. IEDAY) STOP 'IEDAY'

   This code will stop without ever falling through to process the
   data the user actually wants.

   We suggest that you take the unpacked values for the year,
   month, day and hour and compute a date, somthing like a Julian
   date, e.g.:

      IDATE = 1000000*IYR + 10000*IMO + 100*IDY + IHR

   and also specify your range of acceptable times as dates:

      ISDAY = 91122900       !  start date/time
      IEDAY = 92010423       !  end date/time

   Then the above IFs will work as expected.  However, we would
   also discourage use of the second IF, as it may STOP before
   reaching desired data that is in bad sort (see gotcha #3 below)


2) When changing the geographic area, or from one format to another,
   the longitude tests become confused.

   E.g., in reading NMC ADP, where XLON is the longitude unpacked from
   the ID, and happens to be 330.0W:

      CLONE = 300.0     !  eastern edge of area
      CLONW = 60.0      !  western edge of area

      IF (XLON.GT.CLONW) GO TO 10          ! go to read another
      IF (XLON.LT.CLONE) GO TO 10          ! go to read another

   This code will not select the desired data - it will not select the
   Greenwich Meridian for one thing.

   First, pay attention to the longitude convention used in the dataset.
   There is no universal standard for specifying longitude.

   The NMC ADP specifies all longitudes as positive, increasing westward
   from the Greenwich Meridian (0 to 360).

   The Navy SPOT specifies western longitudes as negative, decreasing
   westward to the International Date Line (i.e. 0 to -180) and eastern
   longitudes as positive, increasing eastward to the International Date
   Line (i.e. 0 to 180)

   In the example above, it is smarter to nest the test against the
   desired range inside a test on the desired range itself, e.g.

      IF (CLONE.LT.CLONW)  THEN
        IF (XLON.GE.CLONE.AND.XLON.LE.CLONW)  GOTO      ! (want this data)
      ELSE
        IF (XLON.GE.CLONE.OR.XLON.LE.CLONW)  GOTO       ! (want this data)
      ENDIF
      GO TO 10                               ! go to read another

   This particular test of the range works on NMC ADP longitudes, but
   would fail on the Navy SPOT and others.


3) After reading data with an ID that appears near the end of their
   selection criteria, the program stops, without returning some portion
   of the desired data.

   a) E.g. while reading an NMC 65 X 65 analysis volume, a loop is done
      to pull all 500mb heights, then a loop is done to pull all 500mb
      temperatures, but only the heights are retrieved.

   b) E.g. while reading an NMC ADP surface 3-hourly volume, the user
      wants to get both the surface land (ADPSFC) reports and the surface
      marine (SFCSHP) reports, but gets only the ADPSFC.

   Often, users tripped by this "gotcha" have been accustomed to working
   with database software that allows them to "CALL for data".  Neither
   the sort in our archives, nor our software is or will be tailored to
   meet this expectation.

   But our read routines can be tailored to ensure that the data you want
   is retrieved, and then you can put it into a database system if desired.

   Our read routines read the data volumes sequentially.  Successive calls
   return successive blocks of data, together with an ID.  Your selection
   criteria should test this ID to see whether you want this data block,
   and continue reading to the end of the volume.

   In example a), the first loop will be skipping all the temperatures, and
   when the end of the volume is reached, the volume must be rewound and
   read again to get the temperatures.  We would recommend that both
   parameters be pulled at the same time, in the same loop.

   In example b), the MSS archive volumes are in the same sort as received
   from NMC, i.e. all the ADPSFC for the week appears, then all the SFCSHP.
   If the user says to STOP after hitting some date in the ADPSFC, the
   program will probably not read into the SFCSHP.  Don't use a STOP - let
   the program spin through the entire volume.

4) The analysis field or observation report (or field within a report) is
   missing.  This is a "come and go" situation where data that is present
   in recent years was not available in earlier years.  Or it was available
   in earlier years and is no longer.  And other combinations.  What is
   happening is that the operational center changed procedures, something
   that we can not fix.