This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
codes:ozoneproduct [2020/09/10 12:28] jlsolano [NDACC AMES files] |
codes:ozoneproduct [2022/06/13 11:59] jlsolano [Level 1.5 REALTIME OBSERVATION] |
||
---|---|---|---|
Line 66: | Line 66: | ||
* //o3ds// and //o3ds_maxstd// values have to be set in [[devel:eubrewnetconfiguration|config]] | * //o3ds// and //o3ds_maxstd// values have to be set in [[devel:eubrewnetconfiguration|config]] | ||
* If not, a default value for std of 2.5 is used | * If not, a default value for std of 2.5 is used | ||
- | - Filtered by ozone slant column / air mass | + | * For other measurements (ZS, FZ, FM, ...) you can set specific thresholds in //Data and Metadata// → //Data Processing// → //Rejection Criteria//. A std of 4 is used by default for ZS measurements. |
- | * //mu_ds// and //mu_ds_value// values have to be set in [[devel:eubrewnetconfiguration|config]], and the model of the Brewer is Mk II or IV and the stray light correction has not been defined | + | - Filtered by ozone slant column / air mass |
- | * If not, a default value of 3.5 is used | + | * For DS measurements, //mu_ds// and //mu_ds_value// values have to be set in [[devel:eubrewnetconfiguration|config]] |
+ | - If the limits are defined, they are used. | ||
+ | - If not, the following default values are used: | ||
+ | * Single (no stray correction): 3.5 | ||
+ | * Single (with stray correction): 6 | ||
+ | * Double: 6 | ||
+ | * For other ozone measurements in which the Stray Light correction is not defined (ZS, FZ, FM, ...), 3.5 is the default value for Single Brewers, and 6 for Doubles. You can also set a specific value in //Data and Metadata// → //Data Processing// → //Rejection Criteria// | ||
- The measures has valid hg (step change lees than 2) before and after (see [[devel:brewerpythonmodule#check_hg|Brewer Python Module (Check HG)]]. | - The measures has valid hg (step change lees than 2) before and after (see [[devel:brewerpythonmodule#check_hg|Brewer Python Module (Check HG)]]. | ||
- Filtered by Minimun Values: lesser Ozone values than Maximun are discarded | - Filtered by Minimun Values: lesser Ozone values than Maximun are discarded | ||
Line 90: | Line 96: | ||
Two Flags have been added to the O<sub>3</sub> and SO<sub>2</sub> for filter a correction representation: | Two Flags have been added to the O<sub>3</sub> and SO<sub>2</sub> for filter a correction representation: | ||
- | ^ Flag ^ Applied ^ Value ^ | + | ^ Flag type ^ Flag name ^ Description ^ Value ^ |
- | | Filter Flag | Ozone standard deviation | 1 | | + | | Data filter | FLAG_OZONE_STD_FILT | Ozone standard deviation | 1 | |
- | | Filter Flag | Max airmass | 2 | | + | | Data filter | FLAG_AIRMASS_FILT | Max airmass | 2 | |
- | | Filter Flag | Valid HG | 4 | | + | | Data filter | FLAG_HG_FILT | Valid HG | 4 | |
- | | Filter Flag | Min Value | 8 | | + | | Data filter | FLAG_MIN_FILT | Min Ozone Value | 8 | |
- | | Filter Flag | Max Value | 16 | | + | | Data filter | FLAG_MAX_FILT | Max Ozone Value | 16 | |
- | | Filter Flag | Exclusion List | 32 | | + | | Data filter | | Exclusion List | 32 | |
- | | Correction Flag | Standard Lamp | 1 | | + | | Correction Flag | | Standard Lamp | 1 | |
- | | Correction Flag | ETC filter dependent | 2 | | + | | Correction Flag | | ETC filter dependent | 2 | |
- | | Correction Flag | Straylight | 4 | | + | | Correction Flag | | Straylight | 4 | |
=== Flag Process === | === Flag Process === | ||
Line 194: | Line 200: | ||
The Level 2 data are produced when the operator activates the verification flag and validates (assign to level 2) the observations data between the current calibration and the previous verified one. | The Level 2 data are produced when the operator activates the verification flag and validates (assign to level 2) the observations data between the current calibration and the previous verified one. | ||
+ | |||
+ | ---- | ||
===== WOUDC extCSV files ===== | ===== WOUDC extCSV files ===== | ||
Line 203: | Line 211: | ||
* ''brewerid'': Brewer serial number. | * ''brewerid'': Brewer serial number. | ||
- | * ''date'': month to process, in the format ''YYYY-MM'' | + | * ''date'': month to process, in the format ''YYYY-MM'', ''last'' for the last month according to today's date. |
- | + | ||
- | * ''level'': EUBREWNET product level to be included in the file, either 1.5 or 2.0 | + | |
For example, use | For example, use | ||
| | ||
- | http://www.eubrewnet.org/eubrewnet/woudc/csv?brewerid=157&date=2019-01&level=1.5 | + | http://www.eubrewnet.org/eubrewnet/woudc/csv?brewerid=157&date=2019-01 |
+ | |||
+ | to get Brewer #157's highest level data (1.5 near real time, 1.6 consolidated NRT, or 2.0 final) for January 2019. | ||
- | to get Brewer #157's L1.5 data for January 2019. | + | Note that EUBREWNET's 1.5, 1.6, and 2.0 data levels correspond to WOUDC's level 2.0 (see https://guide.woudc.org/en/#321-introduction). Files generated by EUBREWNET and submitted to the WOUDC will hence be stored at https://woudc.org/archive/Archive-NewFormat/TotalOzone_2.0_1/ |
The following arguments are optional: | The following arguments are optional: | ||
Line 217: | Line 225: | ||
* Instead of ''brewerid'', you can use one of ''gaw_id'', ''woudc_id'', or ''stationid'' to request the data for a station instead of a Brewer. ''stationid'' corresponds to the internal EUBREWNET ID of the station. Note that the Brewer and station products might be different if more than one Brewer operates at a station. | * Instead of ''brewerid'', you can use one of ''gaw_id'', ''woudc_id'', or ''stationid'' to request the data for a station instead of a Brewer. ''stationid'' corresponds to the internal EUBREWNET ID of the station. Note that the Brewer and station products might be different if more than one Brewer operates at a station. | ||
- | * ''version'': this is EUBREWNET's processing algorithm version. If not specified, version 1.0 is used (note this is the only version available to the public as of April 2020) | + | * ''version'': this is EUBREWNET's processing algorithm version. If not specified, version ''1.0'' is used (note this is the only version available to the public as of April 2020) |
- | * ''file_version'': this is used in the Data Generation Version field of the extCSV file (see https://guide.woudc.org/en/#32-the-woudc-extcsv-data-file-metadata). If not specified, it defaults to EUBREWNET's processing algorithm version, which itself defaults to ''1.0'' (see above) | + | * ''file_version'': this is used in the Data Generation Version field of the extCSV file (see https://guide.woudc.org/en/#32-the-woudc-extcsv-data-file-metadata), and is written as ''major_version.minor_version''. If not specified, the file version is automatically determined depending on whether a file for the same date is already available at the WOUDC: |
+ | * if the file exists, the ''minor_version'' will be increased by ''1'' (e.g., if the existing file is version ''3.2'', the new file will be ''3.3'') | ||
+ | * if no file exists, ''file_version'' defaults to EUBREWNET's processing algorithm version, which itself defaults to ''1.0'' (see above) | ||
Line 249: | Line 259: | ||
* ''file_access'': this is used to fill the ''FILE_ACCESS'' global attribute (see https://evdc.esa.int/documents/1/geoms-1.0.pdf, page 20, section 4.3.3). If more than one data center is specified, please separate their acronyms using only commas, e.g. ''AVDC,NDACC'' (**do not** use semicolons! ''AVDC;NDACC'' will not be interpreted correctly by the server!). If not specified, it defaults to ''NDACC'' | * ''file_access'': this is used to fill the ''FILE_ACCESS'' global attribute (see https://evdc.esa.int/documents/1/geoms-1.0.pdf, page 20, section 4.3.3). If more than one data center is specified, please separate their acronyms using only commas, e.g. ''AVDC,NDACC'' (**do not** use semicolons! ''AVDC;NDACC'' will not be interpreted correctly by the server!). If not specified, it defaults to ''NDACC'' | ||
+ | |||
+ | * ''individual'': write all individual measurements instead of daily means | ||
+ | |||
+ | You can check the GEOMS-HDF files using the AVDC GEOMS QA checker available at https://avdc.gsfc.nasa.gov/index.php?site=1473794367 | ||
===== NDACC AMES files ===== | ===== NDACC AMES files ===== | ||
Line 274: | Line 288: | ||
* Instead of ''brewerid'', you can use one of ''gaw_id'', ''woudc_id'', or ''stationid'' to request the data for a station instead of a Brewer. ''stationid'' corresponds to the internal EUBREWNET ID of the station. Note that the Brewer and station products might be different if more than one Brewer operates at a station. | * Instead of ''brewerid'', you can use one of ''gaw_id'', ''woudc_id'', or ''stationid'' to request the data for a station instead of a Brewer. ''stationid'' corresponds to the internal EUBREWNET ID of the station. Note that the Brewer and station products might be different if more than one Brewer operates at a station. | ||
- | * ''version'': this is EUBREWNET's processing algorithm version. If not specified, version 1.0 is used (note this is the only version available to the public as of April 2020) | + | * ''version'': this is EUBREWNET's processing algorithm version. If not specified, version 1.0 is used (note this is the only version available to the public as of September 2020) |
* ''qualityflag'': this is the 4-bytes value to write at the end of the first line in the NDACC file (see https://www.ndsc.ncep.noaa.gov/data/formats/header/#DFlag). Defaults to ''0001'' | * ''qualityflag'': this is the 4-bytes value to write at the end of the first line in the NDACC file (see https://www.ndsc.ncep.noaa.gov/data/formats/header/#DFlag). Defaults to ''0001'' | ||
* ''individual'': write all individual measurements instead of daily means | * ''individual'': write all individual measurements instead of daily means |