This is an old revision of the document!

Ozone Level Definition

Level 0

Are the O₃ and SO₂ comming from the Direct Sun measures from the Brewers. Some considerations:

Standard Algorithm (as related in the BREWER MKIV SPECTROPHOTOMETER OPERATOR'S MANUAL (OM-BA-C231 REV B, August 15, 1999 [Appendix G])
Observations are not processed, are taken directly from B files.
Values for O₃ and SO₂ come from the Direct Sun measures directly from the parsed B files
Constants and values used in the process come from the B file inst section (stored in the Brewer)
- O₃ and SO₂ temperature coefficients, Neutral Density Filters…
- Fixed Values: Interval Scaling, Rayleigh Coefficients…

Level 1

Are the O₃ and SO₂ recalculations with the standard algorithm from the Direct Sun measures and applying a verified by an operator set of constants. Some considerations:

The calulations use Brewer Python Module which implement the Standard Algorithm (the BREWER MKIV SPECTROPHOTOMETER OPERATOR'S MANUAL (OM-BA-C231 REV B, August 15, 1999 [Appendix G]).
Values for O₃ and SO₂ come from the calculation of the Standard Algorithmn using Raw Counts from the Direct Sun measures and the Temperature from the Direct Sun Summaries.
with the exception of location of the measurement (Latitude, Longitude and Pressure) the configuration constants and values used in the process come from the Configurations uploaded and validated to the system by the Operators.
Latitude, Longitude and Pressure are taken from the B inst section of the B file.

The process

Get the DS and DS summaries measures of the day (coming from B file). If there are not available data, stop process.
Get the last available configuration with date equal or earlier. If there is not a available configuration, stop process.
Calulate the O₃ and SO₂ using the Brewer Python Module.
Group the individual measures in groups of fives and calculate standard deviation.
Store them into database.

Database

ozone_product_1_0 Table
Field	Description	Unit	Reference
brewerid	Brewer identification number		db.brewer_t
gmt	UT time of the measure in ISO 8601 format	GMT	ISO 8601
configid	Configuration identification Numer		db.configs_t
n_sum	Index of the daily summary
date_index	Continuous date index (0.0 = 0000-00-00T00:00:00Z) based in Matlab datenum	days	db.b_ds_t.date.toordinal + 366 + db.b_ds_t.mmmm_gmt / SECONDS_IN_A_DAY
sza	Solar zenith angle using time function	Degrees	Brewer Python Module (Solar Zenith Angle)
airmass	Calculated airmass using time function		Brewer Python Module (Brewer Airmass)
temperature	Instrument temperature	Degrees	db.b_ds_summary_t.temp
filt	Applied neutral density filter		db.b_ds_t.nd_filter_position / 64
o3	Calculated Ozone value with Standard algorithm + attenuation filter correction Ozone Level 1	Dobson Units	Brewer Python Module (o3_so2)
std_o3	Standard deviation of the group of measures	Dobson Units	Brewer Operator Manual (Section 9.2.6)
so2	Calculated so2 value with Standard algorithm + attenuation filter correction	Dobson Units	Ozone Level 1
std_so2	Standard deviation of the group of measures	Dobson Units	Brewer Operator Manual (Section 9.2.6)
latitude	Latitude of the Brewer Location	Degrees	db.b_t, db.configs_t
longitude	Longitude of the Brewer Location	Degrees	db.b_t, db.configs_t
press	Medium Pressure of the Brewer Location	Milibars	db.b_t, db.configs_t
ms8	First double ratio		Brewer Python Module (o3_so2)
ms9	Second double ratio		Brewer Python Module (o3_so2)
f1	Corrected Operational-Wavelength counts per second for slit 1 (includes only instrumental corrections, does not include the Rayleigh correction)		Brewer Operator Manual (Appendix G)
f2	Corrected Operational-Wavelength counts per second for slit 2 (includes only instrumental corrections, does not include the Rayleigh correction)		Brewer Operator Manual (Appendix G)
f3	Corrected Operational-Wavelength counts per second for slit 3 (includes only instrumental corrections, does not include the Rayleigh correction)		Brewer Operator Manual (Appendix G)
f4	Corrected Operational-Wavelength counts per second for slit 4 (includes only instrumental corrections, does not include the Rayleigh correction)		Brewer Operator Manual (Appendix G)
f5	Corrected Operational-Wavelength counts per second for slit 5 (includes only instrumental corrections, does not include the Rayleigh correction)		Brewer Operator Manual (Appendix G)
configdate	UT Date of the used configuration in ISO 8601 format	GMT	db.b_t, db.configs_t
configtype	Type of the used configuration: B header(1), ICF (2), Config (3)		db.b_t, db.icf_t db.configs_t
process_date	Date and time when the product was processed	GMT	ISO 8601

Level 1.5 REALTIME OBSERVATION

They are level 1.0 observations with filter and corrections Level 1.0 O₃ and SO₂ observations

The list of config values and where to find them in the upload form can be found in Eubrewnet Configuration.

Filters

Filtered by ozone standard deviation.
- o3ds and o3ds_maxstd values have to be set in config
- If not, a default value for std of 2.5 is used
- For other measurements (ZS, FZ, FM, …) you can set specific thresholds in Data and Metadata → Data Processing → Rejection Criteria. A std of 4 is used by default for ZS measurements.
Filtered by ozone slant column / air mass
- For DS measurements, mu_ds and mu_ds_value values have to be set in config
  1. If the limits are defined, they are used.
  2. If not, the following default values are used:
    - Single (no stray correction): 3.5
    - Single (with stray correction): 6
    - Double: 6
- For other ozone measurements in which the Stray Light correction is not defined (ZS, FZ, FM, …), 3.5 is the default value for Single Brewers, and 6 for Doubles. You can also set a specific value in Data and Metadata → Data Processing → Rejection Criteria
The measures has valid hg (step change lees than 2) before and after (see Brewer Python Module (Check HG).
Filtered by Minimun Values: lesser Ozone values than Maximun are discarded
- Min value is set to 100
Filtered by Maximun Values: greater Ozone values than Maximun are discarded
- Max value is set to 500
Filtered by the Exclusion List: measurements taken at a date and time included in the Exclusion List are discarded.

Corrections

Standard lamp correction
- sl_o3_cor has to be set in config for calculate Standard Lamp Correction (Default)
- If not, Standard Lamp correction is not applied
Filter correction: ETC filter dependent correction
- etc_cor_f_X value has to be provided in config (X filter)
- If not or 0.0 value, ETC filter dependent correction is not applied
Stray Light correction
- stray_light_a and stry_light_b values has to be set in config
- If not provided or both of them are 0.0, straylight correction is not applied

Two Flags have been added to the O₃ and SO₂ for filter a correction representation:

Flag type	Flag name	Description	Value
Data filter	FLAG_OZONE_STD_FILT	Ozone standard deviation	1
Data filter	FLAG_AIRMASS_FILT	Max airmass	2
Data filter	FLAG_HG_FILT	Valid HG	4
Data filter	FLAG_MIN_FILT	Min Ozone Value	8
Data filter	FLAG_MAX_FILT	Max Ozone Value	16
Data filter		Exclusion List	32
Correction Flag		Standard Lamp	1
Correction Flag		ETC filter dependent	2
Correction Flag		Straylight	4

Flag Process

In case of Filter flag, the process of setting it works as related below:

$\begin{displaymath}Flag = 0 \end{displaymath}$
$\begin{displaymath} \text{If } \sigma(Measures) > \text{Standard Deviation Value} \Rightarrow Flag = Flag + 1 \end{displaymath}$
$\begin{displaymath} \text{If } \max(\text{Measure Airmasses}) > \text{Max Airmass} \Rightarrow Flag = Flag + 2 \end{displaymath}$
$\begin{displaymath} \text{If } \text{Measure HG Test Failed} \Rightarrow Flag = Flag + 4 \end{displaymath}$
$\begin{displaymath} \text{If } Ozone < \text{Ozone Min Value} \Rightarrow Flag = Flag + 8 \end{displaymath}$
$\begin{displaymath} \text{If } Ozone > \text{Ozone Max Value} \Rightarrow Flag = Flag + 16 \end{displaymath}$
$\begin{displaymath} \text{If } measurement's\; date\; and\; time \in \text{Exclusion List} \Rightarrow Flag = Flag + 32 \end{displaymath}$
The Group of measures is discarded if at least one of the flags is not 0

In case of Correction flag, the process of setting it works as related below:

$\begin{displaymath}Flag = 0 \end{displaymath}$
$\begin{displaymath} \text{If SL Correction enabled } Flag = Flag + 1 \end{displaymath}$
$\begin{displaymath} (\text{If ETC filter correction (config)} \ne 0) \And (\text{ETC filter Correction} \ne 0.0) \Rightarrow Flag = Flag + 2 \end{displaymath}$
$\begin{displaymath} (\text{If Straylight Parameters} \ne 0) \And (\text{Straylight Correction} \ne 0.0) \Rightarrow Flag = Flag + 4 \end{displaymath}$

The Process

This function computes O₃ and SO₂ amount from Brewer measurements:

$\begin{displaymath} O_3 = O_3_0 + \Delta_{SL} -\Delta_{Filter}-\Delta_{StrayLight} \end{displaymath}$

$\begin{displaymath} O_3_0 = \frac{MS9 - ETC_0}{\mu \alpha } \end{displaymath}$

$\begin{displaymath} \Delta_{SL}=\frac{R6_{ref}- R6}{\mu \alpha } \end{displaymath}$

$\begin{displaymath} \Delta_{Filter}=\frac{ ECT_{N}}{\mu \alpha }, \end{displaymath}$

$\begin{displaymath} \Delta_{StrayLight_{N}}=\frac{ A*(\mu O_3_{N-1})^B}{\mu \alpha }, \end{displaymath}$

The $O_3_0$ is the Level 1 ozone
The SL correction use the R6 reference value and the measure SL is smoothing using a Triangular moving average filter (see Brewer Python Module (Triangular Moving Average))
The ETC filter correction is applied using the ETC correction function, the correction $ETC_N$ only affects to the measurements performed with filter #N
The Stray Light ETC filter correction as depend of he ozone is an iterative process, in most cases only one iteration is needed (OSC<1600 DU) and is enough with two iterations for all the cases (up to OSC 2000DU). On the database two iterations are defined by default.
- The configuration parameters A y B assume that osc ($\mu *O_3$) are in DU/1000.
- We assume that the A parameter is negative in the configuration.

$\begin{displaymath} \Delta_{StrayLight}=\frac{A*(\mu *〖(O_3_0〗+A*\frac{(\mu *O_3_0 )^B}{\alpha \mu}))^B}{\alpha \mu} $ \end{displaymath}$

Database

Ozone Level 1.5

ozone_product_1_5 Table
Field	Description	Unit	Reference
brewerid	Brewer identification number		db.brewer_t
gmt	UT time of the measure in ISO 8601 format	GMT	ISO 8601
configid	Configuration identification Numer		db.configs_t
n_sum	Index of the daily summary
date_index	Continuous date index (1.0 = 0001-01-01T00:00:00Z) based in python date.toordinal	days	db.b_ds_t.date.toordinal + db.b_ds_t.mmmm_gmt / SECONDS_IN_A_DAY
sza	Solar zenith angle using time function	Degrees	Brewer Python Module (Solar Zenith Angle)
airmass	Calculated airmass using time function		Brewer Python Module (Brewer Airmass)
temperature	Instrument temperature	Degrees	db.b_ds_summary_t.temp
filt	Applied neutral density filter		db.b_ds_t.nd_filter_position / 64
o3	Calculated Ozone value with Standard algorithm + attenuation filter correction + config corrections Ozone Level 1.5	Dobson Units	Brewer Python Module (o3_so2_sl)
std_o3	Standard deviation of the group of measures	Dobson Units	Brewer Operator Manual (Section 9.2.6)
so2	Calculated so2 value with Standard algorithm + attenuation filter correction Ozone Level 1.5	Dobson Units	Brewer Python Module (o3_so2_sl)
std_so2	Standard deviation of the group of measures	Dobson Units	Brewer Operator Manual (Section 9.2.6)
latitude	Latitude of the Brewer Location	Degrees	db.b_t, db.configs_t
longitude	Longitude of the Brewer Location	Degrees	db.b_t, db.configs_t
press	Medium Pressure of the Brewer Location	Milibars	db.b_t, db.configs_t
o3_0	Calculated Ozone value with Standard algorithm + attenuation filter correction Ozone Level 1	Dobson Units	Brewer Python Module (o3_so2)
so2_0	Calculated so2 value with Standard algorithm + attenuation filter correction Ozone Level 1	Dobson Units	Brewer Python Module (o3_so2)
r6	Standard Lamp Double Ratio 2		Brewer Python Module (Triangular Moving Average)
r6_ref	Ozone value for Standard Lamp of Reference (Double Ratio 2)		db.configs_t.r6_ref
r5	Standard Lamp Double Ratio 1		Brewer Python Module (Triangular Moving Average)
r5_ref	Ozone value for Standard Lamp of Reference (Double Ratio 1)		db.configs_t.r5_ref
filter_flag	APPLIED FLAGS: HG Filter, Max Airmass, O3 std (1 means True)		Level 1.5 Realtime Observation
correction_flag	APPLIED CORRECTIONS: Straylight, ETC Correction, Standard Lamp correction(1 means True)		Level 1.5 Realtime Observation
configdate	UT Date of the used configuration in ISO 8601 format	GMT	db.b_t, db.configs_t
configtype	Type of the used configuration: B header(1), ICF (2), Config (3)		db.b_t, db.icf_t db.configs_t
process_date	Date and time when the product was processed	GMT

ISO 8601|

Level 2.0

Level 2.0 are 1.5 observations validated with a posterior calibration. The brewer instrument use the measurements of the internal halogen lamp to track the Ozone calibration, but is usually difficult to know if the changes recorded by the SL test are due to changes on the lamp or real changes on the instrumental response. Only with a subsequent calibration/comparison we can validate the SL correction.

The Level 2 data are produced when the operator activates the verification flag and validates (assign to level 2) the observations data between the current calibration and the previous verified one.

WOUDC extCSV files

EUBREWNET can generate files with monthly data in the extCSV format requested by the WOUDC (see https://guide.woudc.org/en/)

To generate the WOUDC files, use the URL http://eubrewnet.aemet.es/eubrewnet/woudc/csv with the following arguments:

brewerid: Brewer serial number.

date: month to process, in the format YYYY-MM, last for the last month according to today's date.

For example, use

http://eubrewnet.aemet.es/eubrewnet/woudc/csv?brewerid=157&date=2019-01

to get Brewer #157's highest level data (1.5 near real time, 1.6 consolidated NRT, or 2.0 final) for January 2019.

Note that EUBREWNET's 1.5, 1.6, and 2.0 data levels correspond to WOUDC's level 1.0 (see https://guide.woudc.org/en/#321-introduction). Files generated by EUBREWNET and submitted to the WOUDC will hence be stored at https://woudc.org/archive/Archive-NewFormat/TotalOzone_1.0_1/ (The WOUDC level in EUBREWNET's files was changed from 2.0 to 1.0 in August 2024)

The following arguments are optional:

Instead of brewerid, you can use one of gaw_id, woudc_id, or stationid to request the data for a station instead of a Brewer. stationid corresponds to the internal EUBREWNET ID of the station. Note that the Brewer and station products might be different if more than one Brewer operates at a station.

version: this is EUBREWNET's processing algorithm version. If not specified, version 1.0 is used (note this is the only version available to the public as of April 2020)

file_version: this is used in the Data Generation Version field of the extCSV file (see https://guide.woudc.org/en/#32-the-woudc-extcsv-data-file-metadata), and is written as major_version.minor_version. If not specified, the file version is automatically determined depending on whether a file for the same date is already available at the WOUDC:
- if the file exists, the minor_version will be increased by 1 (e.g., if the existing file is version 3.2, the new file will be 3.3)
- if no file exists, file_version defaults to EUBREWNET's processing algorithm version, which itself defaults to 1.0 (see above)

GEOMS HDF files

It is also possible to generate HDF files with monthly data in the GEOMS format (see https://evdc.esa.int/documents/1/geoms-1.0.pdf) accepted by the NDACC, AVDC, and other data centers.

To generate the GEOMS files, use the URL http://eubrewnet.aemet.es/eubrewnet/geoms/hdf with the following arguments:

brewerid: Brewer serial number.

date: month to process, in the format YYYY-MM

level: EUBREWNET product level to be included in the file, either 1.5 or 2.0

For example, use

http://eubrewnet.aemet.es/eubrewnet/geoms/hdf?brewerid=157&date=2019-02&level=1.5

to get Brewer #157's L1.5 data for February 2019.

The following arguments are optional:

Instead of brewerid, you can use one of gaw_id, woudc_id, or stationid to request the data for a station instead of a Brewer. stationid corresponds to the internal EUBREWNET ID of the station. Note that the Brewer and station products might be different if more than one Brewer operates at a station.

version: this is EUBREWNET's processing algorithm version. If not specified, version 1.0 is used (note this is the only version available to the public as of April 2020)

file_version: this is used to fill the DATA_FILE_VERSION global attribute of the HDF file (see https://evdc.esa.int/documents/1/geoms-1.0.pdf, page 17, section 4.2.9). If not specified, it defaults to 001

file_access: this is used to fill the FILE_ACCESS global attribute (see https://evdc.esa.int/documents/1/geoms-1.0.pdf, page 20, section 4.3.3). If more than one data center is specified, please separate their acronyms using only commas, e.g. AVDC,NDACC (do not use semicolons! AVDC;NDACC will not be interpreted correctly by the server!). If not specified, it defaults to NDACC

individual: write all individual measurements instead of daily means

You can check the GEOMS-HDF files using the AVDC GEOMS QA checker available at https://avdc.gsfc.nasa.gov/index.php?site=1473794367

NDACC AMES files

EUBREWNET can also generate AMES files with monthly data in the format specified by NDACC (see http://ndaccdemo.org/data/formats)

To generate the AMES files, use the URL http://eubrewnet.aemet.es/eubrewnet/ndacc/ames with the following arguments:

brewerid: Brewer serial number.

date: month to process, in the format YYYY-MM

level: EUBREWNET product level to be included in the file, either 1.5 or 2.0

For example, use

http://eubrewnet.aemet.es/eubrewnet/ndacc/ames?brewerid=157&date=2019-02&level=1.5

to get Brewer #157's L1.5 data for February 2019.

As of September 2020, the error fields have a fixed value corresponding to 1% of the observation values.

The following arguments are optional:

Instead of brewerid, you can use one of gaw_id, woudc_id, or stationid to request the data for a station instead of a Brewer. stationid corresponds to the internal EUBREWNET ID of the station. Note that the Brewer and station products might be different if more than one Brewer operates at a station.

version: this is EUBREWNET's processing algorithm version. If not specified, version 1.0 is used (note this is the only version available to the public as of September 2020)

qualityflag: this is the 4-bytes value to write at the end of the first line in the NDACC file (see https://www.ndsc.ncep.noaa.gov/data/formats/header/#DFlag). Defaults to 0001

individual: write all individual measurements instead of daily means

Eubrewnet Wiki

Table of Contents

Ozone Level Definition

Level 0

Level 1

The process

Database

Level 1.5 REALTIME OBSERVATION

Filters

Corrections

Flag Process

The Process

Database

Ozone Level 1.5

Level 2.0

WOUDC extCSV files

GEOMS HDF files

NDACC AMES files