codes:ozoneproduct
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
codes:ozoneproduct [2020/04/16 09:27] jlsolano [WOUDC extCSV files] |
codes:ozoneproduct [2022/10/05 09:14] (current) |
||
|---|---|---|---|
| Line 66: | Line 66: | ||
| * //o3ds// and //o3ds_maxstd// values have to be set in [[devel:eubrewnetconfiguration|config]] | * //o3ds// and //o3ds_maxstd// values have to be set in [[devel:eubrewnetconfiguration|config]] | ||
| * If not, a default value for std of 2.5 is used | * If not, a default value for std of 2.5 is used | ||
| - | - Filtered by ozone slant column / air mass | + | * For other measurements (ZS, FZ, FM, ...) you can set specific thresholds in //Data and Metadata// → //Data Processing// → //Rejection Criteria//. A std of 4 is used by default for ZS measurements. |
| - | * //mu_ds// and //mu_ds_value// values have to be set in [[devel:eubrewnetconfiguration|config]], and the model of the Brewer is Mk II or IV and the stray light correction has not been defined | + | - Filtered by ozone slant column / air mass |
| - | * If not, a default value of 3.5 is used | + | * For DS measurements, //mu_ds// and //mu_ds_value// values have to be set in [[devel:eubrewnetconfiguration|config]] |
| + | - If the limits are defined, they are used. | ||
| + | - If not, the following default values are used: | ||
| + | * Single (no stray correction): 3.5 | ||
| + | * Single (with stray correction): 6 | ||
| + | * Double: 6 | ||
| + | * For other ozone measurements in which the Stray Light correction is not defined (ZS, FZ, FM, ...), 3.5 is the default value for Single Brewers, and 6 for Doubles. You can also set a specific value in //Data and Metadata// → //Data Processing// → //Rejection Criteria// | ||
| - The measures has valid hg (step change lees than 2) before and after (see [[devel:brewerpythonmodule#check_hg|Brewer Python Module (Check HG)]]. | - The measures has valid hg (step change lees than 2) before and after (see [[devel:brewerpythonmodule#check_hg|Brewer Python Module (Check HG)]]. | ||
| - Filtered by Minimun Values: lesser Ozone values than Maximun are discarded | - Filtered by Minimun Values: lesser Ozone values than Maximun are discarded | ||
| Line 90: | Line 96: | ||
| Two Flags have been added to the O<sub>3</sub> and SO<sub>2</sub> for filter a correction representation: | Two Flags have been added to the O<sub>3</sub> and SO<sub>2</sub> for filter a correction representation: | ||
| - | ^ Flag ^ Applied ^ Value ^ | + | ^ Flag type ^ Flag name ^ Description ^ Value ^ |
| - | | Filter Flag | Ozone standard deviation | 1 | | + | | Data filter | FLAG_OZONE_STD_FILT | Ozone standard deviation | 1 | |
| - | | Filter Flag | Max airmass | 2 | | + | | Data filter | FLAG_AIRMASS_FILT | Max airmass | 2 | |
| - | | Filter Flag | Valid HG | 4 | | + | | Data filter | FLAG_HG_FILT | Valid HG | 4 | |
| - | | Filter Flag | Min Value | 8 | | + | | Data filter | FLAG_MIN_FILT | Min Ozone Value | 8 | |
| - | | Filter Flag | Max Value | 16 | | + | | Data filter | FLAG_MAX_FILT | Max Ozone Value | 16 | |
| - | | Filter Flag | Exclusion List | 32 | | + | | Data filter | | Exclusion List | 32 | |
| - | | Correction Flag | Standard Lamp | 1 | | + | | Correction Flag | | Standard Lamp | 1 | |
| - | | Correction Flag | ETC filter dependent | 2 | | + | | Correction Flag | | ETC filter dependent | 2 | |
| - | | Correction Flag | Straylight | 4 | | + | | Correction Flag | | Straylight | 4 | |
| === Flag Process === | === Flag Process === | ||
| Line 194: | Line 200: | ||
| The Level 2 data are produced when the operator activates the verification flag and validates (assign to level 2) the observations data between the current calibration and the previous verified one. | The Level 2 data are produced when the operator activates the verification flag and validates (assign to level 2) the observations data between the current calibration and the previous verified one. | ||
| + | |||
| + | ---- | ||
| ===== WOUDC extCSV files ===== | ===== WOUDC extCSV files ===== | ||
| Line 199: | Line 207: | ||
| EUBREWNET can generate files with monthly data in the extCSV format requested by the WOUDC (see https://guide.woudc.org/en/) | EUBREWNET can generate files with monthly data in the extCSV format requested by the WOUDC (see https://guide.woudc.org/en/) | ||
| - | To generate the WOUDC files, use the URL ''http://www.eubrewnet.org/eubrewnet/woudc/csv'' with the following arguments: | + | To generate the WOUDC files, use the URL ''http://eubrewnet.aemet.es/eubrewnet/woudc/csv'' with the following arguments: |
| + | |||
| + | * ''brewerid'': Brewer serial number. | ||
| + | |||
| + | * ''date'': month to process, in the format ''YYYY-MM'', ''last'' for the last month according to today's date. | ||
| + | |||
| + | For example, use | ||
| + | |||
| + | http://eubrewnet.aemet.es/eubrewnet/woudc/csv?brewerid=157&date=2019-01 | ||
| + | |||
| + | to get Brewer #157's highest level data (1.5 near real time, 1.6 consolidated NRT, or 2.0 final) for January 2019. | ||
| + | |||
| + | Note that EUBREWNET's 1.5, 1.6, and 2.0 data levels correspond to WOUDC's level 2.0 (see https://guide.woudc.org/en/#321-introduction). Files generated by EUBREWNET and submitted to the WOUDC will hence be stored at https://woudc.org/archive/Archive-NewFormat/TotalOzone_2.0_1/ | ||
| + | |||
| + | The following arguments are optional: | ||
| + | |||
| + | * Instead of ''brewerid'', you can use one of ''gaw_id'', ''woudc_id'', or ''stationid'' to request the data for a station instead of a Brewer. ''stationid'' corresponds to the internal EUBREWNET ID of the station. Note that the Brewer and station products might be different if more than one Brewer operates at a station. | ||
| + | |||
| + | * ''version'': this is EUBREWNET's processing algorithm version. If not specified, version ''1.0'' is used (note this is the only version available to the public as of April 2020) | ||
| + | |||
| + | * ''file_version'': this is used in the Data Generation Version field of the extCSV file (see https://guide.woudc.org/en/#32-the-woudc-extcsv-data-file-metadata), and is written as ''major_version.minor_version''. If not specified, the file version is automatically determined depending on whether a file for the same date is already available at the WOUDC: | ||
| + | * if the file exists, the ''minor_version'' will be increased by ''1'' (e.g., if the existing file is version ''3.2'', the new file will be ''3.3'') | ||
| + | * if no file exists, ''file_version'' defaults to EUBREWNET's processing algorithm version, which itself defaults to ''1.0'' (see above) | ||
| + | |||
| + | |||
| + | ===== GEOMS HDF files ===== | ||
| + | |||
| + | It is also possible to generate HDF files with monthly data in the GEOMS format (see https://evdc.esa.int/documents/1/geoms-1.0.pdf) accepted by the NDACC, AVDC, and other data centers. | ||
| + | |||
| + | To generate the GEOMS files, use the URL ''http://eubrewnet.aemet.es/eubrewnet/geoms/hdf'' with the following arguments: | ||
| * ''brewerid'': Brewer serial number. | * ''brewerid'': Brewer serial number. | ||
| Line 209: | Line 246: | ||
| For example, use | For example, use | ||
| | | ||
| - | http://www.eubrewnet.org/eubrewnet/woudc/csv?brewerid=157&date=2019-01&level=1.5 | + | http://eubrewnet.aemet.es/eubrewnet/geoms/hdf?brewerid=157&date=2019-02&level=1.5 |
| - | to get Brewer #157's L1.5 data for January 2019. | + | to get Brewer #157's L1.5 data for February 2019. |
| - | The following arguments are optional and probably not useful for most users: | + | The following arguments are optional: |
| * Instead of ''brewerid'', you can use one of ''gaw_id'', ''woudc_id'', or ''stationid'' to request the data for a station instead of a Brewer. ''stationid'' corresponds to the internal EUBREWNET ID of the station. Note that the Brewer and station products might be different if more than one Brewer operates at a station. | * Instead of ''brewerid'', you can use one of ''gaw_id'', ''woudc_id'', or ''stationid'' to request the data for a station instead of a Brewer. ''stationid'' corresponds to the internal EUBREWNET ID of the station. Note that the Brewer and station products might be different if more than one Brewer operates at a station. | ||
| Line 219: | Line 256: | ||
| * ''version'': this is EUBREWNET's processing algorithm version. If not specified, version 1.0 is used (note this is the only version available to the public as of April 2020) | * ''version'': this is EUBREWNET's processing algorithm version. If not specified, version 1.0 is used (note this is the only version available to the public as of April 2020) | ||
| - | * ''file_version'': this is used in the Data Generation Version field of the extCSV file (see https://guide.woudc.org/en/#32-the-woudc-extcsv-data-file-metadata). If not specified, it defaults to EUBREWNET's processing algorithm version, which itself defaults to ''1.0'' (see above) | + | * ''file_version'': this is used to fill the ''DATA_FILE_VERSION'' global attribute of the HDF file (see https://evdc.esa.int/documents/1/geoms-1.0.pdf, page 17, section 4.2.9). If not specified, it defaults to ''001'' |
| + | * ''file_access'': this is used to fill the ''FILE_ACCESS'' global attribute (see https://evdc.esa.int/documents/1/geoms-1.0.pdf, page 20, section 4.3.3). If more than one data center is specified, please separate their acronyms using only commas, e.g. ''AVDC,NDACC'' (**do not** use semicolons! ''AVDC;NDACC'' will not be interpreted correctly by the server!). If not specified, it defaults to ''NDACC'' | ||
| - | ===== GEOMS HDF files ===== | + | * ''individual'': write all individual measurements instead of daily means |
| + | |||
| + | You can check the GEOMS-HDF files using the AVDC GEOMS QA checker available at https://avdc.gsfc.nasa.gov/index.php?site=1473794367 | ||
| + | |||
| + | ===== NDACC AMES files ===== | ||
| - | It is also possible to generate HDF files with monthly data in the GEOMS format (see https://evdc.esa.int/documents/1/geoms-1.0.pdf) accepted by the NDACC and other data centers | + | EUBREWNET can also generate AMES files with monthly data in the format specified by NDACC (see http://ndaccdemo.org/data/formats) |
| - | To generate the WOUDC files, use the URL ''http://www.eubrewnet.org/eubrewnet/geoms/hdf'' with the following arguments: | + | To generate the AMES files, use the URL ''http://eubrewnet.aemet.es/eubrewnet/ndacc/ames'' with the following arguments: |
| * ''brewerid'': Brewer serial number. | * ''brewerid'': Brewer serial number. | ||
| Line 236: | Line 278: | ||
| For example, use | For example, use | ||
| | | ||
| - | http://www.eubrewnet.org/eubrewnet/geoms/hdf?brewerid=157&date=2019-02&level=1.5 | + | http://eubrewnet.aemet.es/eubrewnet/ndacc/ames?brewerid=157&date=2019-02&level=1.5 |
| - | to get Brewer #157's L1.5 data for January 2019. | + | to get Brewer #157's L1.5 data for February 2019. |
| - | The following arguments are optional and probably not useful for most users: | + | As of September 2020, the error fields have a fixed value corresponding to 1% of the observation values. |
| + | |||
| + | The following arguments are optional: | ||
| * Instead of ''brewerid'', you can use one of ''gaw_id'', ''woudc_id'', or ''stationid'' to request the data for a station instead of a Brewer. ''stationid'' corresponds to the internal EUBREWNET ID of the station. Note that the Brewer and station products might be different if more than one Brewer operates at a station. | * Instead of ''brewerid'', you can use one of ''gaw_id'', ''woudc_id'', or ''stationid'' to request the data for a station instead of a Brewer. ''stationid'' corresponds to the internal EUBREWNET ID of the station. Note that the Brewer and station products might be different if more than one Brewer operates at a station. | ||
| - | * ''version'': this is EUBREWNET's processing algorithm version. If not specified, version 1.0 is used (note this is the only version available to the public as of April 2020) | + | * ''version'': this is EUBREWNET's processing algorithm version. If not specified, version 1.0 is used (note this is the only version available to the public as of September 2020) |
| - | * ''file_version'': this is used to fill the ''DATA_FILE_VERSION'' global attribute of the HDF file (see https://evdc.esa.int/documents/1/geoms-1.0.pdf, page 17, section 4.2.9). If not specified, it defaults to ''001'' | + | * ''qualityflag'': this is the 4-bytes value to write at the end of the first line in the NDACC file (see https://www.ndsc.ncep.noaa.gov/data/formats/header/#DFlag). Defaults to ''0001'' |
| + | |||
| + | * ''individual'': write all individual measurements instead of daily means | ||
| - | * ''file_access'': this is used to fill the ''FILE_ACCESS'' global attribute (see https://evdc.esa.int/documents/1/geoms-1.0.pdf, page 20, section 4.3.3). If more than one data center is specified, please separate their acronyms using only commas, e.g. ''AVDC,NDACC'' (**do not** use semicolons: ''AVDC;NDACC'' will not be interpreted correctly by the server). If not specified, it defaults to ''NDACC'' | ||
codes/ozoneproduct.1587029248.txt.gz · Last modified: 2020/04/16 09:27 by jlsolano
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 3.0 Unported
