Eubrewnet Wiki

User Tools

Site Tools

Trace: • dbaccess

Sidebar

Eubrewnet Wiki

Eubrewnet Documentation

  1. Eubrewnet
  2. Eubrewnet Community
  3. Configuration Upload Instructions

Content

  1. Eubrewnet Configuration
  2. Ozone Access Functions
  3. UV Access Functions
  4. AOD Access Functions
  5. Calibration Functions
  6. Error Budget
  7. Code examples
  8. Brewer Python Library
  9. Ozone Product Process
  10. Ozone Products Description
  11. Brewer diagnostic tool
  12. Eubrewnet Report
  13. User Manual

Developers

  1. Wiki content
  2. Eubrewnet Products
  3. Generation of Products
  4. Subversion synchronization (campaigns)
  5. Automatic generation of weekly reports
  6. Eubrewnet Management
  7. Eubrewnet and Iberonesia Backup System
  8. DenyHosts at rbcce.aemet.es
  9. Administrator Manual
  10. Developer Manual
codes:dbaccess

Table of Contents

ACCESS FUNCTIONS

Definition of the access functions that are developed in Eubrewnet

The users with the right permissions have access to some functions to directly get the information stored in the database. The root of the access functions is the same for all http://eubrewnet.aemet.es/eubrewnet/data/. From this point users can select different function. As Example Get DS.

The DS database access function provides the Direct Sun individual measures enhanced by information from their summaries and if they are between correct Mercury Lamp Tests.

Description of the DS Get Function:

  • Function: DS
  • Description: Get the Direct Sun individual measures. Gives extra information from the summaries as the temperature or the double ratios and set a flag for mercury (hg_test) if the measure is between two correct mercury tests with a difference of 1 or less from the reference.
  • Inputs:
    1. brewerid: see Brewerid in common inputs
    2. date: see Date in common inputs
    3. enddate: see Enddate in common inputs
    4. format: see Format
  • Output: Direct Sun individual measures with extra values comming from:
    1. Summary: temp (temperature), zenith_angle, airmass, double_ratio1, double_ratio2
    2. B file: date (in ISO 8601 format)
    3. HG measures: hg_test (0: measures is not between two correct HG tests; 1: measure between two correct HG tests)

There is a Get Function Table with a summary of the available functions to access stored data and their links as example. The extended documentation for each function can be directly access from the table using the links in the column Long Description.

For products, the Eubrewnet provides an interface for processing by demand. The Process Function Table gives a summary of the available process functions. Even the data providers can force a process and database stored data update using the update interface. The Update Function Table sumarizes the available functions. Providers can force only the update of the products of their Brewers.

There are some common inputs for the functions as the parameter format or the list of common inputs (as the brewerid and dates) wich can be used in almost every function. The description can be found in the COMMON SECTION.

Examples of the use of the functions are provided for Matlab 2013A, Matlab 2015A and Python in the EXAMPLE SECTION.

For security purposes user authentication has been added to this tools and their use is registered.

COMMON

Format

The access functions provide four different ways of data access using the format input parameter.

Format input Description Example
jsonM JSON matrix formed by lists of lists (default value). The first list is formed by the value names and the following lists are the query outputs Get DS by default
jsonO JSON object of lists, where each key is the value name and its value is the time sorted list of them Get DS with jsonO format
text Human readable comma separated values where first row is the value names and the following are the query outputs Get DS with text format
csv CSV direct download where first row are the value names and the following are the query outputs Get DS with text format

Common Inputs

Almost all functions accept the following input options:

Parameter Description Example
brewerid Brewer identification number Get function with brewerid input parameter
date Date in YYYY-MM-DD format Get function with date input parameter
enddate Date in YYYY-MM-DD format. If provided, the function will return the query in a date range Get function with enddate input parameter
fields Comma separated list of fields to write in the output. You can use myfields* to select all fields starting with myfields O3L1_5 output with just the ''gmt'' and ''o3'' fields

See also the code examples in matlab and python.

Means

For selected functions, passing the &means=xk variable in the URL will return the mean of all the output fields in the period specified by xk, where x is a number and k is any of min, h (hours), d (days), or M (months). The output includes two extra fields, stats_nobs, with the number of measurements which have been used to calculate the mean, and stats_std_o3 with the standard deviation of the ozone over the specified period. Examples:

  1. Hourly means: http://eubrewnet.aemet.es/eubrewnet/data/get/O3L1_5?brewerid=185&date=2018-05-01&enddate=2018-05-15&format=text&means=1h
  2. Daily means: http://eubrewnet.aemet.es/eubrewnet/data/get/O3L1_5?brewerid=185&date=2018-05-01&enddate=2018-06-01&format=text&means=1d

GET FUNCTIONS FOR LEVEL 0

Function Short Description Long Description Link
DS Returns DS measures DS Get DS
DSS Returns the DS summaries DSS Get DSS
ZS Returns ZS measures ZS Get ZS
ZSS Returns the ZS summaries ZSS Get ZSS
FM Returns FM measures FM Get FM
FMS Returns the FM summaries FMS Get FMS
SL Returns SL measures SL Get SL
SLS Returns the SL summaries SLS Get SLS
ConfigbyDate Returns the available Configuration ConfigbyDate Get Config by Date
HK Returns housekeeping DetailsHK Get HK
HG Returns the mercury lamp tests HG Get HG
HP Returns the grating synchronisation tests HP Get HP
FV Returns the FV tests FV Get FV
BFV Returns the FV tests (from B file) BFV Get BFV
AP Returns the AP tests (from B file) AP Get AP
BHeader Returns the B File Header with INST values BHeader Get BHeader
DTO3 Returns the DTO3 tests (from B file) DTO3 Get DTO3
RSO3 Returns the RSO3 tests (from B file) RSO3 Get RSO3
ActiveBrewers Returns the number of Brewers whith at least one SL test by dates HG Get HG
FileStatus Returns the status of the received files FileStatus Get FileStatus
ActiveBrewers Returns the number of Active Brewers ActiveBrewers Get ActiveBrewers
Umkehr Returns the Umkehr measures Umkehr Get Umkehr
BfilesbyLocation Returns the Available B files for a range of locations BfilesbyLocation Get Bfiles by Location
BrewerLocation Returns the changes in location of a Brewer BrewerLocation Get Brewer Location
AVG Returns the available AVG measures AVG Get AVG
CO Return comments from the B files CO Get comments
bfiletimes Returns the time span included in a bfile bfiletimes Get bfile time span

Descriptions

DS

  • Function: DS
  • Description: Get the Direct Sun individual measures. Gives extra information from the summaries as the temperature or the double ratios and set a flag for mercury (hg_test) if the measure is between two correct mercury tests with a difference of 1 or less from the reference.
  • Inputs:
    1. brewerid: see Brewerid in common inputs
    2. date: see Date in common inputs
    3. enddate: see Enddate in common inputs
    4. format: see Format
  • Output: Direct Sun individual measures with extra values comming from:
    1. Summary: temp (temperature), zenith_angle, airmass, double_ratio1, double_ratio2
    2. B file: date (in ISO 8601 format)
    3. HG measures: hg_test (0: measures is not between two correct HG tests; 1: measure between two correct HG tests)

DSS

  • Function: DSS
  • Description: Get the Direct Sun Summary measures. Set a flag for mercury (hg_test) if the measure is between two correct mercury tests with a difference of 1 or less from the reference.
  • Inputs:
    1. brewerid: see Brewerid in common inputs
    2. date: see Date in common inputs
    3. enddate: see Enddate in common inputs
    4. format: see Format
  • Output: Direct Sun Summary measures with extra value coming fromm:
    1. HG measures: hg_test (0: measures is not between two correct HG tests; 1: measure between two correct HG tests)

ZS

  • Function: ZS
  • Description: Get the Zenith Sky individual measures. Gives extra information from the summaries as the temperature or the double ratios and set a flag for mercury (hg_test) if the measure is between two correct mercury tests with a difference of 1 or less from the reference.
  • Inputs:
    1. brewerid: see Brewerid in common inputs
    2. date: see Date in common inputs
    3. enddate: see Enddate in common inputs
    4. format: see Format
  • Output: Zenith Sky individual measures with extra values comming from:
    1. Summary: temp (temperature), zenith_angle, airmass, double_ratio1, double_ratio2
    2. B file: date (in ISO 8601 format)
    3. HG measures: hg_test (0: measures is not between two correct HG tests; 1: measure between two correct HG tests)

ZSS

  • Function: ZSS
  • Description: Get the Zenith Sky Summary measures. Set a flag for mercury (hg_test) if the measure is between two correct mercury tests with a difference of 1 or less from the reference.
  • Inputs:
    1. brewerid: see Brewerid in common inputs
    2. date: see Date in common inputs
    3. enddate: see Enddate in common inputs
    4. format: see Format
  • Output: Zenith Sky Summary measures with extra value coming fromm:
    1. HG measures: hg_test (0: measures is not between two correct HG tests; 1: measure between two correct HG tests)

FM

  • Function: FM
  • Description: Get the Focused Moon individual measures. Gives extra information from the summaries as the temperature or the double ratios and set a flag for mercury (hg_test) if the measure is between two correct mercury tests with a difference of 1 or less from the reference.
  • Inputs:
    1. brewerid: see Brewerid in common inputs
    2. date: see Date in common inputs
    3. enddate: see Enddate in common inputs
    4. format: see Format
  • Output: Focused Moon individual measures with extra values comming from:
    1. Summary: temp (temperature), zenith_angle, airmass, double_ratio1, double_ratio2
    2. B file: date (in ISO 8601 format)
    3. HG measures: hg_test (0: measures is not between two correct HG tests; 1: measure between two correct HG tests)

FMS

  • Function: FMS
  • Description: Get the Focused Moon Summary measures. Set a flag for mercury (hg_test) if the measure is between two correct mercury tests with a difference of 1 or less from the reference.
  • Inputs:
    1. brewerid: see Brewerid in common inputs
    2. date: see Date in common inputs
    3. enddate: see Enddate in common inputs
    4. format: see Format
  • Output: Focused Moon Summary measures with extra value coming fromm:
    1. HG measures: hg_test (0: measures is not between two correct HG tests; 1: measure between two correct HG tests)

SL

  • Function: SL
  • Description: Get the Standard Lamp individual measures. Gives extra information from the summaries as the temperature or the double ratios.
  • Inputs:
    1. brewerid: see Brewerid in common inputs
    2. date: see Date in common inputs
    3. enddate: see Enddate in common inputs
    4. format: see Format
  • Output: Zenith Sky individual measures with extra values comming from:
    1. Summary: temp (temperature), zenith_angle, airmass, double_ratio1, double_ratio2

SLS

ConfigbyDate

  • Function: ConfigbyDate
  • Description: Returns the latest operative configuration (see configuration info) with earlier date than given as input. If a date range is provided, returns the list of available configuration during the date range.
  • Inputs:
    1. brewerid: see Brewerid in common inputs
    2. date: see Date in common inputs
    3. enddate: see Enddate in common inputs
    4. format: see Format
  • Output: Configuration or list of configurations

HK

  • Function: HK
  • Description: Returns instrumental information to characterize the measuring conditions
  • Inputs:
    1. brewerid: see Brewerid in common inputs
    2. date: see Date in common inputs
    3. enddate: see Enddate in common inputs
    4. format: see Format
  • Output: instrumental parameters: temperature at the PMT, fan temperature, temperature of base plate, relative humidity, absolute humidity, atmospheric pressure and external temperature.

HP

HG

  • Function: HG
  • Description: Returns Mercury Tests with a mercury flag (hg_test) which sets if its a passed mercury test or not
  • Inputs:
    1. brewerid: see Brewerid in common inputs
    2. date: see Date in common inputs
    3. enddate: see Enddate in common inputs
    4. format: see Format
  • Output: Mercury tests with extra value:
    1. hg_test (0: measures is not between two correct HG tests; 1: measure between two correct HG tests)

FV

BFV

AP

BHeader

  • Function: BHeader
  • Description: Returns B File Header. Values like date of B file, latitude, longitude and pressure and the content of the used ICF file
  • Inputs:
    1. brewerid: see Brewerid in common inputs
    2. date: see Date in common inputs
    3. enddate: see Enddate in common inputs
    4. format: see Format
  • Output: Inst section from B file with extra fields. All of them from the B file:
    1. brewerid
    2. date
    3. location
    4. latitude
    5. longitude
    6. temp
    7. press

DTO3

RSO3

FileStatus

  • Function: FileStatus
  • Description: Give the status after parsing of a file. Users can look for files for a Brewer between a range of dates or directly look for the information of a specific file.
  • Inputs:
    1. brewerid: see Brewerid in common inputs
    2. date: see Date in common inputs
    3. enddate: see Enddate in common inputs
    4. format: see Format
    5. file: name of the file to see STATUS
  • Outputs: File status information as:
    1. date: date of the file
    2. status: error status of the Brewer (0: No error)
    3. path: where to find the file
    4. fname: name of the file
    5. process_date, process_time: timestamp of the parsing
    6. kind: kind of the file
    7. text: messages thrown while file parsing

AVG

The list of available AVGS access functions is:

File Table definition Access function
apnavg apnavg definition apnavg access function
apoavg apoavg definition apoavg access function
dtnavg dtnavg definition dtnavg access function
dtoavg dtoavg definition dtoavg access function
fmnavg fmnavg definition fmnavg access function
fmoavg fmoavg definition fmoavg access function
h2oavg h2oavg definition h2oavg access function
hgoavg hgoavg definition hgoavg access function
mioavg mioavg definition mioavg access function
nonavg nonavg definition nonavg access function
opavg opavg definition opavg access function
ozoavg ozoavg definition ozoavg access function
rsnavg rsnavg definition rsnavg access function
rsoavg rsoavg definition rsoavg access function
scoavg scoavg definition scoavg access function
slnavg slnavg definition slnavg access function
sloavg sloavg definition sloavg access function
uvoavg uvoavg definition uvoavg access function
fioavg fioavg definition fioavg access function

CO

bfiletimes

GET FUNCTIONS FOR HIGHER LEVELS

The output of the GET functions is stored in Eubrewnet's database, from where it's retrieved on each function call with little further processing. They are faster than their equivalent PROCESS functions, which process the data in real time, usually starting from the previous level. Whenever possible, GET functions should be used instead of the PROCESS equivalents.

Function Short Description Long Description Link
O3L1 Returns the Level 1 of Ozone O3L1 Get O3L1
O3L1_5 Returns the Level 1.5 of Ozone O3L1_5 Get O3L1_5
O3L2_0 Returns the Level 2.0 of Ozone O3L2_0 Get O3L2_0
O3L1ZS Returns the Level 1 of the Zenith Sky Ozone O3L1ZS Get O3ZS
O3L1_5ZS Returns the Level 1.5 of the Zenith Sky Ozone O3L1_5ZS Get O3L1_5ZS
O3L2_0ZS Returns the Level 2.0 of the Zenith Sky Ozone O3L2_0ZS Get O3L2_0ZS
O3L1byStation, O3L1_5byStation, and O3L2_0byStation For the requested station, returns the L1, L1_5 or L2_0 ozone product O3L1byStation, O3L1_5byStation, and O3L2_0byStation Get O3L1_5byStation
O3L1ALLbyStation, O3L1_5ALLbyStation, and O3L2_0ALLbyStation For the requested station, returns the L1, L1_5 or L2_0 ozone product, with both DS and ZS measurements O3L1ALLbyStation, O3L1_5ALLbyStation, and O3L2_0ALLbyStation Get O3L1_5ALLbyStation
O3L1PRIObyStation, O3L1_5PRIObyStation, and O3L2_0PRIObyStation For the requested station, returns the L1, L1_5 or L2_0 ozone product, with both DS and ZS measurements – the latter only if no DS measurements are available in the day O3L1PRIObyStation, O3L1_5PRIObyStation, and O3L2_0PRIObyStation Get O3L1_5PRIObyStation
UNCL1 and UNCL1_V2 Returns the Uncertainty L1 products for the standard and V2 Ozone algorithms UNCL1 and UNCL1_V2 Get UNCL1, Get UNCL1_V2
UNCL1_5 and UNCL1_5_V2 Returns the Uncertainty L1.5 products for the standard and V2 Ozone algorithms UNCL1_5 and UNCL1_5_V2 Get UNCL1_5, Get UNCL1_5_V2
UNCL1byStation, UNCL1_V2byStation, UNCL1_5byStation and UNCL1_5_V2byStation For the requested station, returns the Uncertainty L1 or L1.5 products for the standard or V2 Ozone algorithms UNCL1byStation, UNCL1_V2byStation, UNCL1_5byStation, and UNCL1_5_V2byStation Get UNCL1_5byStation

Get O3L1

In Products Description Ozone Level 1 there is a summary table with the output of this product

Get O3L1_5

  • Function: O3L1_5
  • Description: Give the processed Ozone calculated using standard Algorithm and Direct Sun Measures. The Standard Lamp corrections, ETC by filter and Straylight corrections. The measures are filtered by standard deviation, max airmass, correct mercury test, maximun and mininum values. The Operators have to upload configurations for calculation. See Ozone Level Definition and Configuration Upload Instructions
  • Inputs:
    1. brewerid: see Brewerid in common inputs
    2. date: see Date in common inputs
    3. enddate: see Enddate in common inputs
    4. format: see Format
    5. means: see means
  • Outputs: Ozone calculated from the DS raw counts using standard algorithm from Brewer Python Library. Measurements have been enhanced with info from:
    1. Filter flag
    2. Correction flag
    3. r5, r6 and their references

In Products Description Ozone Level 1.5 there is a summary table with product content

Get O3L2_0

  • Function: O3L2_0
  • Description: Give the processed Ozone calculated using standard Algorithm and Direct Sun Measures. The Standard Lamp corrections, ETC by filter and Straylight corrections. The measures are filtered by standard deviation, max airmass, correct mercury test, maximun and mininum values. The Operators have to upload configurations for calculation. See Ozone Level Definition and Configuration Upload Instructions. The Level 2.0 is available only for validated Configurations.
  • Inputs:
    1. brewerid: see Brewerid in common inputs
    2. date: see Date in common inputs
    3. enddate: see Enddate in common inputs
    4. format: see Format
    5. debug: change the ouput to see the corrections that are being applied. (True)
    6. rejected: returns the filtered data, too (True)
  • Outputs: Ozone calculated from the DS raw counts using standard algorithm from Brewer Python Library. Measurements have been enhanced with info from:
    1. Filter flag
    2. Correction flag
    3. r5, r6 and their references
  • Outputs in case that the debug input is set, the output will contain:
    1. date index, solar zenith angle, airmass, temperature, filter
    2. o3, o3 sl correction, o3 etc correction by slit, o3 straylight correction,
    3. r6, r6 of reference, filter flag, correction flag

In Products Description Ozone Level 2.0 there is a summary table with the output of this product

Get O3L1ZS, O3L1_5ZS, and O3L2_0ZS

  • Functions: O3L1ZS, O3L1_5ZS, and O3L2_0ZS
  • Description: These functions provide the same output as the O3L1, O3L1_5, and O3L2_0 described above, but for the Zenith Sky measurement. Note that the 9 empirical Zenith Sky constants must be defined in the Brewer configuration.
  • Input: same as the O3L1, O3L1_5, and O3L2_0 functions
  • Output: same as the O3L1, O3L1_5, and O3L2_0 functions

Get O3L1byStation, O3L1_5byStation, and O3L2_0byStation

Get O3L1ALLbyStation, O3L1_5ALLbyStation, and O3L2_0ALLbyStation

Get O3L1PRIObyStation, O3L1_5PRIObyStation, and O3L2_0PRIObyStation

Get brewerStationClimatology

  • Function: brewerStationClimatology
  • Description: This function returns the daily atmospheric parameters corresponding to the location of the Brewer spectrophotometer
  • Example: https://rbcce.aemet.es/eubrewnet/data/get/brewerStationClimatology?brewerid=150&date=2021-01-01&enddate=2021-01-02&format=text
  • Input:
    1. brewerid: this is the ID of the Brewer. This follows the format described in Brewerid in common inputs
    2. date: see Date in common inputs
    3. enddate: see Enddate in common inputs
    4. format: see Format
  • Output:
    1. brewerid: ID of the Brewer
    2. date: date in YYYY-MM-DD format
    3. name: station's name
    4. id: Eubrewnet's internal station ID
    5. latitude: station's latitude in +N degrees
    6. longitude: station's longitude in +W degrees
    7. height: station's altitude in meters
    8. hteff_fun: climatological series used
    9. h_eff: ozone effective altitude in km
    10. e_h_eff: uncertainty of the ozone effective altitude in km
    11. t_eff: effective temperature in K
    12. e_t_eff: uncertainty of the effective temperature in K
    13. press: station's pressure in mbar
    14. e_press: uncertainty of the station's pressure in mbar
    15. heff_r: Rayleigh effective altitude in km
    16. e_heff_r: uncertainty of the Rayleigh effective altitude in km
    17. h_error_m: difference between the ERA5 and TOMS effective altitudes
    18. h_error_std: standard deviation of the ERA5-TOMS difference of effective altitudes
    19. t_error_m: difference between the ERA5 and TOMS effective temperatures
    20. t_error_std: standard deviation of the ERA5-TOMS difference of effective temperatures

Get UNCL1 and UNCL1_V2

Get UNCL1_5 and UNCL1_5_V2

Get UNCL1byStation, UNCL1_V2byStation, UNCL1_5byStation, and UNCL1_5_V2byStation

PROCESS FUNCTIONS FOR HIGHER LEVELS

The main output of the PROCESS functions is the same as their GET equivalents, but here the calculation is done in real time, usually starting from the previous level. This makes the PROCESS functions much slower, so we recommend using the GET equivalents when available.

Function Description Help Example
O3L1 Returns the Level 1 of Ozone O3L1 Process O3L1
O3L1_5 Returns the Level 1.5 of Ozone O3L1_5 Process O3L1_5
O3L2_0 Returns the Level 2.0 of Ozone O3L2_0 Process O3L2_0
UNCL1 and UNCL1_V2 Returns L1 uncertatinty for the standard and V2 Ozone algorithms UNCL1 and UNCL1_V2 Process UNCL1 Process UNCL1_V2
UNCL1_5 and UNCL1_5_V2 Returns L1.5 uncertatinty for the standard and V2 Ozone algorithms UNCL1_5 and UNCL1_5_V2 Process UNCL1_5 Process UNCL1_5_V2
TempCoef Returns SL Temperature analysis TempCoef Process TempCoef

Process O3L1

In Products Description Ozone Level 1 there is a table with a summary of this function's output

Process O3L1_5

  • Function: O3L1_5
  • Description: Process Ozone calculated using standard Algorithm and Direct Sun Measures. The Standard Lamp corrections, ETC by filter and Straylight corrections. The measures are filtered by standard deviation, max airmass, correct mercury test, maximun and mininum values. The Operators have to upload configurations for calculation. See Ozone Level Definition and Configuration Upload Instructions
  • Inputs:
    1. brewerid: see Brewerid in common inputs
    2. date: see Date in common inputs
    3. enddate: see Enddate in common inputs
    4. format: see Format
    5. debug: change the ouput to see the corrections that are being applied. (True)
    6. rejected: returns the filtered data, too (True)
  • Outputs: Ozone calculated from the DS raw counts using standard algorithm from Brewer Python Library. Measurements have been enhanced with info from:
    1. Filter flag
    2. Correction flag
    3. r5, r6 and their references
  • Outputs in case that the debug input is set, the output will contain:
    1. date index, solar zenith angle, airmass, temperature, filter
    2. o3, o3 sl correction, o3 etc correction by slit, o3 straylight correction,
    3. r6, r6 of reference, filter flag, correction flag

In Products Description Ozone Level 1.5 there is a table with a summary of this function's output

Process O3L2_0

  • Function: O3L2_0
  • Description: Process Ozone calculated using standard Algorithm and Direct Sun Measures. The Standard Lamp corrections, ETC by filter and Straylight corrections. The measures are filtered by standard deviation, max airmass, correct mercury test, maximun and mininum values. The Operators have to upload configurations for calculation. See Ozone Level Definition and Configuration Upload Instructions. The Level 2.0 is available only for validated Configurations.
  • Inputs:
    1. brewerid: see Brewerid in common inputs
    2. date: see Date in common inputs
    3. enddate: see Enddate in common inputs
    4. format: see Format
    5. debug: change the ouput to see the corrections that are being applied. (True)
    6. rejected: returns the filtered data, too (True)
  • Outputs: Ozone calculated from the DS raw counts using standard algorithm from Brewer Python Library. Measurements have been enhanced with info from:
    1. Filter flag
    2. Correction flag
    3. r5, r6 and their references
  • Outputs in case that the debug input is set, the output will contain:
    1. date index, solar zenith angle, airmass, temperature, filter
    2. o3, o3 sl correction, o3 etc correction by slit, o3 straylight correction,
    3. r6, r6 of reference, filter flag, correction flag

In Products Description Ozone Level 2.0 there is a table with summary of this function's output

Process UNCL1 and UNCL1_V2

  • Functions: UNCL1 and UNCL1_V2
  • Description: Level 1 uncertainty product for the standard and V2 ozone algorithms.
  • Examples: https://rbcce.aemet.es/eubrewnet/data/process/UNCL1?brewerid=163&date=2019-06-03&format=text, https://rbcce.aemet.es/eubrewnet/data/process/UNCL1_V2?brewerid=163&date=2019-06-03&format=text
  • Input:
    1. brewerid: see Brewerid in common inputs
    2. date: see Date in common inputs
    3. enddate: see Enddate in common inputs
    4. format: see Format
  • Output: O3L1 fields plus uncertainty-specific fields:
    1. combined: total combined uncertainty in DU
    2. systematic: systematic component of the uncertainty in DU
    3. random: random component of the uncertainty in DU
    4. unc_flag: flag describing wether default or Brewer-specific configuration parameters have been used, see this table.
    5. u_dt_f0, u_dt_f1, u_dt_f2, u_dt_f3, u_dt_f4, u_dt_f5 : Uncertainty of the Dead Time Corrected Operational-Wavelength counts per second for slits 0, 1, 2, 3, 4, and 5 (does not include the Rayleigh correction)
    6. u_temp_r6: Uncertainty of the Temperature Corrected Operational counts per second for R6 (does not include the Rayleigh correction) in R6 units
    7. u_a1_xsec_v1 (u_a1_xsec_v2): Uncertainty of the absorption cross section component in (atm-cm)-1 units
    8. u_a1_grad_v1 (u_a1_grad_v2): Uncertainty of the effective temperature gradient component in (atm-cm)-1 units
    9. u_a1_step_v1 (u_a1_step_v2): Uncertainty of the wavelength gradient component in (atm-cm)-1 units
    10. u_a1_v1 (u_a1_v2): Uncertainty of the ozone absorption coefficient in (atm-cm)-1 units
    11. airmass_rayleigh_uncertainty: Uncertainty of the Rayleigh Airmass
    12. airmass_uncertainty (airmass_uncertainty_v2): Uncertainty of the Ozone Airmass
    13. zenith_angle_uncertainty: Uncertainty of the Solar Zenith Angle in degrees
    14. u_ms9_v1 (u_ms9_v2): Second double ratio uncertainty in R6 units
    15. u_b (u_b_v2): Uncertainty of the weighted Rayleigh scattering coefficients in cm-1

Process UNCL1_5 and UNCL1_5_V2

  • Functions: UNCL1_5 and UNCL1_5_V2
  • Description: Level 1.5 uncertainty product calculated from the level 1 uncertainty, standard and V2 ozone algorithms.
  • Examples: https://rbcce.aemet.es/eubrewnet/data/process/UNCL1_5?brewerid=163&date=2019-06-03&format=text, https://rbcce.aemet.es/eubrewnet/data/process/UNCL1_5_V2?brewerid=163&date=2019-06-03&format=text
  • Input:
    1. brewerid: see Brewerid in common inputs
    2. date: see Date in common inputs
    3. enddate: see Enddate in common inputs
    4. format: see Format
  • Output: O3L1_5 fields plus uncertainty-specific fields:
    1. combined: total combined uncertainty in DU
    2. systematic: systematic component of the uncertainty in DU
    3. random: random component of the uncertainty in DU
    4. unc_flag: flag describing wether default or Brewer-specific configuration parameters have been used (flag values up to 2048), and if L1.5 corrections are included (flag values starting at 4096), see this table.
  • Debug output: when called with the debug input argument, the output additionally contains the following variables:
    1. etco3rate: ETC in R6 units
    2. o3o3rate: ozone absorption coefficient (A1) in (atm-cm)-1
    3. ms9: second double ratio in R6 units
    4. c_etc: ETC sensitivity component of the combined uncertainty in DU
    5. c_meas: Measurement sensitivity component of the combined uncertainty in DU
    6. c_b: Rayleigh coefficient sensitivity component of the combined uncertainty in DU
    7. c_pressure: Pressure sensitivity component of the combined uncertainty in DU
    8. c_rayleigh_airmass: Rayleigh airmass sensitivity component of the combined uncertainty in DU
    9. c_a1: Ozone Absorption coefficient sensitivity component of the combined uncertainty in DU
    10. c_airmass: Ozone airmass sensitivity component of the combined uncertainty in DU
    11. u_ms9: Uncertainty of the MS9 double-ratio (corrected by Rayleigh scattering) in R6 units
    12. u_a1: Uncertainty of the ozone absorption coefficient in (atm-cm)-1
    13. airmass_uncertainty: Uncertainty of the Ozone Airmass
    14. u_sl_corr: Uncertainty of the correction associated to the Standard Lamp in DU
    15. u_etc_corr: Uncertainty of the correction associated to the Filter Non-Linearity in DU
    16. u_st_corr: Uncertainty of the correction associated to the Stray Light in DU
    17. e_etco3rate_v1 (e_etco3rate_v2): ETC error from the Brewer configuration in R6 units
  • Rejected output: by default, measurements with more 25 units in the individual (not summaries) values of the combined, systematic and/or random uncertainties are not included in the output. You can get these measurements passing the &rejected option in the url.

UNC Flag

The following table describes the values of the unc_flag field found in the UNCL1 and UNCL1_V2 and UNCL1_5 and UNCL1_5_V2 functions

Correction Component Type Flag Default Value Units Source
Dead Time Measurement Systematic 1 1E-9 sec Fountoulakis et al., AMT 9(4), 1799-1816 (2016)
Temperature Coeff. Measurement Systematic 2 0.093578 counts/s/ºC Campaign maximum
Sun Scan Measurement Systematic 4 0.02262 %MS9/step Campaign maximum
Time (SZA) Measurement Systematic 8 1 sec Software
Ozone Eff. Temperature (V1) Absorption Random 16 3.95 ºC Climatology
Ozone Eff. Temperature (V2) Absorption Random 16 1.56 ºC Climatology
Temperature Grad. Absorption Random 32 0.0149 (atm-cm)-1/K Dispersion
Wavelength Grad. (V1) Absorption Random 64 -0.00102 (atm-cm)-1/step Dispersion
Wavelength Grad. (V2) Absorption Random 64 -0.00097 (atm-cm)-1/step Dispersion
Rayleigh Eff. Altitude Rayleigh airmass Random 128 0.5 km Constant
Ozone Eff. Altitude (V1) Ozone airmass Random 256 0.75 km Climatology
Ozone Eff. Altitude (V2) Ozone airmass Random 256 0.54 km Climatology
Rayleigh Scatt. Coeff. (V1) Scattering Coeff Systematic 512 1 % hard-coded
Rayleigh Scatt. Coeff. (V2) Scattering Coeff Systematic 512 0.002 % Bodhaine et al., Journal of Atmospheric and Oceanic Technology, 16(11), 1854-1861 (1999)
Pressure Pressure Random 1024 15 milibars Climatology
ETC (V1) ETC Systematic 2048 15 R6 Campaing Mean
ETC (V2) ETC Systematic 2048 15 R6 Campaing Mean
Standard Lamp (V1) Standard Lamp Systematic 4096 Yes/No Dobson Units
Standard Lamp (V2) Standard Lamp Systematic 4096 Yes/No Dobson Units
Filter No-Linearity (V1) Filter No-Linearity Systematic 8192 Yes/No Dobson Units
Filter No-Linearity (V2) Filter No-Linearity Systematic 8192 Yes/No Dobson Units
Stray Light (V1) Stray Light Systematic 16384 Yes/No Dobson Units
Stray Light (V2) Stray Light Systematic 16384 Yes/No Dobson Units

UNC Filter Flag

The following table describes the values of the filter_flag field found in the UNCL1_5 and UNCL1_5_V2 functions. Note the values below 1024 correspond to the standard ozone ''filter_flag''

Name Set when Value
FLAG_OZONE_STD_FILT Ozone standard deviation above 2.5 (ds) or 5 (zs), unless another values is set in the configuration 1
FLAG_AIRMASS_FILT above airmass 3.5 (brewers affected by straylight) or 6 (straylight correction provided or MKIII), unless another value is set in the configuration 2
FLAG_HG_FILT no valid HG 4
FLAG_MIN_FILT below min Ozone Value, as set in the configuration 8
FLAG_MAX_FILT above max Ozone Value, as set in the configuration 16
FLAG_BLACKLIST date and time included in the Exclusion List 32
FLAG_U_O3 outside the [0,25] limit for the combined uncertainty of individual measurements 1024
FLAG_U_O3_SYS outside the [0,25] limit for the systematic uncertainty of individual measurements 2048
FLAG_U_O3_RAN outside the [0,25] limit for the random uncertainty of individual measurements 4096

UPDATE FUNCTIONS

These functions use the PROCESS functions to update Eubrewnet's database, so that the data is available to the GET functions. This process is automatically done when a configuration is updated, so users do not need to directly access these functions.

O3L1 Update the Database content for the Level 1 of Ozone O3L1 Update O3L1
O3L1_5 Update the Database content for 1.5 of Ozone O3L1_5 Update O3L1_5
O3L2_0 Update the Database content for 2.0 of Ozone O3L2_0 Update O3L2_0

Update O3L1

Update O3L1_5

  • Function: O3L1_5
  • Description: Process and Updates Ozone calculated using standard Algorithm and Direct Sun Measures. The Standard Lamp corrections, ETC by filter and Straylight corrections. The measures are filtered by standard deviation, max airmass, correct mercury test, maximun and mininum values. The Operators have to upload configurations for calculation. See Ozone Level Definition and Configuration Upload Instructions. Only the providers of the RAW data can update the processed data. After updating, the product will be accesible using Get function.
  • Inputs:
    1. brewerid: see Brewerid in common inputs
    2. date: see Date in common inputs
    3. enddate: see Enddate in common inputs
    4. format: see Format
  • Outputs: The time to process and update the product

Update O3L2_0

  • Function: O3L2_0
  • Description: Process and Updates Ozone calculated using standard Algorithm and Direct Sun Measures. The Standard Lamp corrections, ETC by filter and Straylight corrections. The measures are filtered by standard deviation, max airmass, correct mercury test, maximun and mininum values. The Operators have to upload configurations for calculation. See Ozone Level Definition and Configuration Upload Instructions. Only the providers of the RAW data can update the processed data. After updating, the product will be accesible using Get function. The Level 2.0 is available only for validated Configurations.
  • Inputs:
    1. brewerid: see Brewerid in common inputs
    2. date: see Date in common inputs
    3. enddate: see Enddate in common inputs
    4. format: see Format
  • Outputs: The time to process and update the product

Bypassing data filters

Data with a non-zero filter_flag value will not be included in the O3L1_5 and AODL1_5 output unless &rejected is added included in the URL. For example, the output of

https://eubrewnet.aemet.es/eubrewnet/data/process/O3L1_5?brewerid=185&date=2018-11-01&enddate=2018-11-02&format=text&fields=gmt,filter_flag

only shows data with filter_flag equal to zero, whereas

https://eubrewnet.aemet.es/eubrewnet/data/process/O3L1_5?brewerid=185&date=2018-11-01&enddate=2018-11-02&rejected&format=text&fields=gmt,filter_flag

includes observations with filter_flag values of 1, 2, 3, 6…, which will would have been hidden were it not for the &reject included in the URL

The filter_flag values correspond to the following conditions and variables:

Flag type Flag name Description Value
Ozone data filter FLAG_OZONE_STD_FILT Above the ozone standard deviation threshold 1
Ozone data filter FLAG_AIRMASS_FILT Above max airmass 2
Ozone data filter FLAG_HG_FILT Wrong HG 4
Ozone data filter FLAG_MIN_FILT Below Min Ozone Value 8
Ozone data filter FLAG_MAX_FILT Above Max Ozone Value 16
Ozone data filter FLAG_BLACKLIST Date in Exclusion List 32
Uncertainty data filter FLAG_U_O3 outside the [0,25] limit for combined uncertainty 1024
Uncertainty data filter FLAG_U_O3_SYS outside the [0,25] limit for systematic uncertainty 2048
Uncertainty data filter FLAG_U_O3_RAN outside the [0,25] limit for random uncertainty 4096

You can also omit any of these data filters adding allowfilter=xxx[,yyy] to the URL, where xxx is one of the Flag names in the above table. Note you can pass more than one Flag name, separating them by commas. Capitalization is not taken into account.

Thus for example, if the last URL is changed to

https://eubrewnet.aemet.es/eubrewnet/data/process/O3L1_5?brewerid=185&date=2018-11-01&enddate=2018-11-02&rejected=True&format=text&fields=gmt,filter_flag&allowfilter=FLAG_OZONE_STD_FILT,flag_hg_filt

both the FLAG_OZONE_STD_FILT and FLAG_HG_FILT data filters will be ignored and all the data will have a filter_flag value of 0.

codes/dbaccess.txt · Last modified: 2023/07/07 11:40 by jlsolano

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 3.0 Unported
CC Attribution-Share Alike 3.0 Unported Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki