|
MSS files are stored on an NCAR system that can be directly accessed only by users that have computing accounts at NCAR. Non-NCAR users are encouraged to review the metadata shown in the list and request assistance by contacting Gregg Walters to inquire further and request download access to the files. The data, as it resides on the MSS, is in COS-blocked format. In the descriptions, comments, and NOTEs below, the term 'file' denotes a batch of data, between and including header and trailer blocks, which includes the data for a data subset for a given observation time. No file marks actually appear between files. The 'file time' is the date and time shown in the header of a file. The 'ob time' is the time (without a date) shown in any report. An 'off-hour report' is a report whose ob time differs from the 6-hourly (12-) file times by 3 (6) or more hours. Small counts involve just a few reports whose ob time falls exactly on the limits of the test. Large counts were once suspected to signal a missing header, but checks for 'sort reversals' indicate that off-hour reports appear to duplicate what already appears in the correct file. DSS has NOTEd these. When a file with above normal contents follows one with below normal (or missing) and the above normal file has a lot of off-hour reports, these may actually belong to the below normal file. There are also some cases where the inventories show more than one file for an observation time, but the sequencing of the VSN blocks may suggest that one of these is actually 'mislabeled', as one may fit the file sequencing correctly if a correction is made to the file time. DSS mad many of these corrections, after comparing to other sources. Such fixes when made, are noted as 'repaired.' It is our practice to scrub the label blocks which NMC uses. Occasionally our scrubbing has failed. These may be recognized by their short length (80 characters) which begin with 'VOL1', 'HDR1' or 'EOF1' and should be tossed. DSS has tried to NOTE where these 'unscrubbed label block's appear. There have also been times when it was our practice to scrub NMC's trailer blocks. Where header blocks are missing, it may have been due to an error in this processing. DSS no longer does this, and our latest 'compress' program actually inserts dummy trailer blocks when they are missing, so that tailored data extractions are consistent with NCEP documentation. DSS has tried to NOTE where these have been found. It is our intention to eventually insert headers where they are missing, fix headers that are incorrect, scrub the unscrubbed label blocks, and correct the file sequencing. Other data problems may never lend themselves to economical fixes. NCEP says that each of their dataset types (labeled ADPUPA, AIRCFT, SIRSOB, SATWND, UPABOG and AIRCAR) will involve a certain limited collection of report types. DSS has checked each report against these, and show the strays in our inventories as a count of 'invalid report type'. Some don't match those listed for any dataset type. When these counts run high, which is rare, DSS NOTES them here. Each report will contain one or more sections, each identified by a category number referring to a format The (instrumental) nature of each dataset generally limits the contents of its reports to certain categories, but their is no documented restriction. Access programs (like ours) should be designed to call an independent routine for each category. The order in which these categories appear is not guaranteed. BLOCK SIZES - ALL SUBSETS, except List A (RAOBS & PIBALS): Prior to 1976Sep15,18Z blocks may be 5056, 5104, 5112, 5120, 5136, 6440 or 6432 bytes If you want to copy this early data and don't want to preserve the COS-blocking, then you are urged to use a compress to copy the data, which will rewrite the data in 6440 byte size blocks, and avoid problems with 'direct reads'. Beginning 1976Sep15,18Z blocks are 6440, 6432 or 6480 bytes If you want to copy this later data and don't want to preserve the COS-blocking, then it may be better to use a compress to copy the data, which will rewrite the data in 6440 byte size blocks, and avoid problems with 'direct reads'. Beginning 1979jan07, perhaps earlier, blocks are consistently 6440 bytes. BLOCK SIZES - List A (RAOBS & PIBALS): DSS rewrote the 1973-1977 data into 6440 byte blocks, so that this frequently used subset is easier to handle. CHARACTER SETS: Primaries are ASCII |
