codes:calibration
Table of Contents
/
Calibration Functions
This page provides a brief overview of the functions avaliable in Eubrewnet to determine Brewer configuration parameters.
Langley
- Function:
langley
- Description: Determine the ETC and the ETC filter corrections applying the Langley-plot method to each half day.
- Input: a complete list of input arguments is provided next, but we strongly recommed to use the web interface at https://eubrewnet.aemet.es/eubrewnet/caldata/langley to access this function
brewerid
: see Brewerid in common inputsdate
: see Date in common inputsenddate
: see Enddate in common inputsformat
: see Formatdata
: input data used, available options areo3l15
(default if the option is not passed in the url),o3l15v2
,o3l1
,sco3
,UNCL15
for the O3L1.5, O3L1.5V2, O3L1, sun scan, and UNCL1.5 measurements, respectively. For thesco3
option, thestep
option with the value of the micrometer step must be also passed in the url; if checking a mic step which does not correspond to the cal step, it's also recommended to pass the ozone absorption coefficient specific to that mic step using theurlconfig
option, see below. Individual observations (not summaries) are used in all cases unlessuse_summaries
is selected, see below. For theuncl15
option, the fit is done using York's method (see this MATLAB and this Python implementations)so2
: calculate the SO2 ETC instead of the (default) O3 onecalc
: calculation to perform, available options areetc
(Langley-plot with all the data, default option),filters
(the Langley-plot fit is done using dummy variables for filters 3, 4, 5, and thus allows to determine the ETC filter correction of those filters with respect to the ETC obtained for the set measurements with filters 0, 1, and 2), andfilters_etc
(this option performs two fits: first, afilters
run to determine the ETC filter corrections, and then anetc
calculation using these ETC filter corrections). Foretc
runs, two different fits are performed:ms9
vsairmass
(brewer method) andms9/airmass
vs1/airmass
(dobson method)minairmass
: airmass minimum inetc
calculations (defaults to 1.70 if the option is not passed in the url)maxairmass
: airmass maximum inetc
calculations (defaults to 3.75 if the option is not passed in the url)minobs
: minimum number of observations (either individual observations or summaries) in the half-day to perform the Langley inetc
calculations (default value: 25). Note by default all thedata
functions are configured to return individual observations, but seeuse_summaries
belowfilters_minairmass
: airmass minimum infilters
calculations (defaults to 1 if the option is not passed in the url)filters_maxairmass
: airmass maximum infilters
calculations (defaults to 5 if the option is not passed in the url)filters_minobs
: minimum number of observations in the half-day to perform the Langley infilters
calculations (default value: 15)maxstdo3
: don't perform the langley if the maximum of the standard deviation of the half-day ozone is above this value (default: 2.5)minrsquared
: don't accept the langley results if the r2 of the fits are below this value (default: 0.999)residuals_filtering
: do each fit twice, using the residuals of the first to remove observations whose residual is larger than 1.5*std(residuals)use_summaries
: if passed as an option, use summaries instead of individual observations for o3l15 and o3l15v2 data. Note the default values forminobs
andfilter_minobs
(25 and 15, respectively) are provided as a guide for calculations with individual observations, the user must set suitable values (eg, 5 forminobs
) ifuse_summaries
is passed.urlconfig
: allows setting the configuration's parameters, overriding the values in Eubrewnet's database. For example, to set a value of 0.4 for the ozone absorption coefficient, add&urlconfig&o3o3rate=0.4
to the url, and to set specific values for the filter corrections add&urlconfig&fc=1,2,3,4,5
debug
: include debug output. As of 2023-06-29, this returns four additional vectorial fields,airmass
,ms9
,filt
, andsolartime
, with the raw data used in the half-day langley fit.
- Output:
date
: date and time of each half-day langley, morning langleys correspond to 00 time, and afternoon ones, to 12etc
: calculated ETCrmse
oru_etc
: root mean squared error or uncertainty in the ETC as calculated in the York method (only for theuncl15
input)tau
: in the case of the determination of the O3 ETC, this corresponds to the ozone optical depth – ie, the ozone multiplied by the ozone absorption coefficient. For the SO2 ETC, this is the sum of the SO2 and O3 optical depths, in the space of the SO2 ratiosrsquared
: r2 of the langley fitnobs
: number of observations used in the fitstdo3
: standard deviation of the half-day ozonemeano3
: mean ozone of the half-dayfilter_corr_3
,filter_corr_4
,filter_corr_5
: ETC filter corrections, either calculated in afilters
run or supplied passing&urlconfig&fc=1,2,3,4,5
in the urlinfo
: a brief description of the calculation performed
ETC filter corrections from ozone differences
- Function:
fcor_o3dif
- Description: this function returns the ETC filter corrections (with respect to the mean of filters 1 and 2) as calculated from the ozone differences of near-simultaneous measurements taken with different filters. Mathematically, for two measurements taken with different filters at almost the same airmass μ and with a ΔO3L1 difference in the (uncorrected) ozone, Δfiltercorr=ΔO3L1∗μ∗abscoeff∗10
- Input:
brewerid
: see Brewerid in common inputsdate
: see Date in common inputs. If not specified, the calculation will be carried out for all the date ranges between the configurations available in Eubrewnetenddate
: see Enddate in common inputs. If not specified, the current date is used.format
: see Formatmaxminutes
: maximum difference in minutes between consecutive measurements, default value: 15airmass
: maximum airmass between consecutive measurements, default value: 0.1
- Output:
config_f1_cor
,config_f2_cor
,config_f3_cor
,config_f4_cor
,config_f5_cor
: ETC filter corrections set in the configuration at Eubrewnetcalc_f1_cor
,calc_f2_cor
,calc_f3_cor
,calc_f4_cor
,calc_f5_cor
: median of all the ETC filter corrections calculated in the period, referred to the mean of filters 1 and 2calc_f1_sd
,calc_f2_sd
,calc_f3_sd
,calc_f4_sd
,calc_f5_sd
: standard deviation of the ETC filter corrections in the periodcalc_f01_nobs
,calc_f12_nobs
,calc_f23_nobs
,calc_f34_nobs
,calc_f45_nobs
: number of near simultaneous observations with the indicated filters
ETC filter corrections from ozone differences -- raw data
- Function:
fcor_o3dif_raw
- Description: this function returns the raw data used in the fcor_o3dif function. That is, it returns Δfiltercorr=ΔO3L1∗μ∗abscoeff∗10 for each pair of measurements, without referring the result to filters 1 and 2 nor calculating any median over a date range.
- Input:
brewerid
: see Brewerid in common inputsdate
: see Date in common inputs. If not specified, the calculation will be carried out for all the date ranges between the configurations available in Eubrewnetenddate
: see Enddate in common inputs. If not specified, the current date is used.format
: see Formatmaxminutes
: maximum difference in minutes between consecutive measurements, default value: 15airmass
: maximum airmass between consecutive measurements, default value: 0.1
- Output:
previous_gmt
: date and time of the first measurement in the pairgmt
: date and time of the second measurementprevious_airmass
: airmass of the first measurementairmass
: airmass of the second measurementprevious_filt
: filter position for the first measurementfilt
: filter position for the second measurementprevious_o3_l1
: level 1 ozone for the first measuremento3_l1
: level 1 ozone for the second measurementconfigdate
: date of the operative configurationo3o3rate
: ozone absorption coefficientfcor
: filter correctionfchange
: a single string with both filter positions
ETC filter correction from FIOAVG data
- Function:
fcor_fioavg
- Description: this function returns the ETC filter corrections (with respect to the mean of filters 1 and 2) as calculated from the results of the Brewer filter tests with the standard lamp, as found in the FIOAVG file.
- Input:
brewerid
: see Brewerid in common inputsdate
: see Date in common inputs. If not specified, the calculation will be carried out for all the date ranges between the configurations available in Eubrewnetenddate
: see Enddate in common inputs. If not specified, the current date is used.format
: see Format
- Output:
config_f1_cor
,config_f2_cor
,config_f3_cor
,config_f4_cor
,config_f5_cor
: ETC filter corrections set in the configuration at Eubrewnetcalc_f1_cor
,calc_f2_cor
,calc_f3_cor
,calc_f4_cor
,calc_f5_cor
: median of all the ETC filter corrections calculated in the period, referred to the mean of filters 1 and 2calc_f1_sd
,calc_f2_sd
,calc_f3_sd
,calc_f4_sd
,calc_f5_sd
: standard deviation of the ETC filter corrections in the periodcalc_nobs
: number of measurements in the FIOAVG file
Temperature Coefficients
- Function:
TempCoef
- Description: This function analyses the temperature sensitivity of SL measurements.
- Input:
brewerid
: see Brewerid in common inputsdate
: see Date in common inputsenddate
: see Enddate in common inputsformat
: see Formattc
: if we pass the tc parameter with no value, the fit is done with the SL measurements corrected using the temperature coefficients from the operational setup on Eubrewnet. If we pass six values separated by commas, the fit is done with the corrected SL measurements using these values as temperature coefficients. If the tc parameter is omitted, no temperature correction is applied to the SL measurements.tmin
: all data below tmin are removed.tmax
: all data above tmax are removed.alpha
: Outliers can be removed by giving the alpha parameter a value greater than zero. In this case, after performing the linear regression, all measures whose residuals are greater than the standard deviation of all residuals multiplied by the alpha factor will be removed, and the linear regression will be repeated. This process is repeated iteratively until all values are below this threshold.step
: optional, if passed uses the data from the JL measurements at the requested mic step. JL measurements are corrected using the linear temperature coefficient and the filter attenuations corresponding to each wavelength, as available in the AOD configuration operative at the beginning of the requested date range (date
)time
: optional, if passed, the time evolution of the lamp intensity is considered in the analysis, including in the linear regression a degree 3 dependence on time.urlconfig
: allows setting the configuration's parameters, overriding the values in Eubrewnet's database.
- Output: the following fields are included in the output:
channel
: Brewer measurement channels in the ozone position plus R5 and R6 (the latter are not included in the JL output)a
: intercept value for the linear regression.sa
: standard error of the intercept of the linear regression.b
: slope value for the linear regression.sb
: standard error of the slope of the linear regression.r2
: R-squared of the linear regression.tmin
: Minimum temperature used in the linear regression.tmax
: Maximum temperature used in the linear regression.- If the parameter
time
is passed, three more coefficients are returned (the time dependence considered is of degree 3) together with their standard errors: c,sc,d,sd,e,se.
Sun scan
- Function:
SC
- Description: Returns SC measurements from B file. It adds the temperature of the last HG test made before measurement
- Inputs:
brewerid
: see Brewerid in common inputsdate
: see Date in common inputsenddate
: see Enddate in common inputsformat
: see Format
- Output: SC measurements with extra field:
temp
: Temperature from the last made HG before each SC measurement
Sun scan with extra information
- Function:
SCO3
- Description: Same as SC above with additional information (O3, ms9, longitude, …). A specific micrometer step can be also selected using
&step=XXX
. Note by default the ozone will be calculated with the ozone absorption coefficient stored in the operative configuration, which corresponds to the cal step. An ozone absorption coefficient specific for the requested mic step can be passed using theurlconfig
method: add e.g.&urlconfig&o3o3rate=0.4
to the url - Inputs:
brewerid
: see Brewerid in common inputsdate
: see Date in common inputsenddate
: see Enddate in common inputsformat
: see Format
- Output: SC measurements with additional fields, including:
temp
: Temperature from the last made HG before each SC measurementO3
: ozonems9
: ms9 double ratio
CI and CJ lamp tests
- Function:
CI
orCJ
- Description: Return the data from the CI (forward scan) or CJ (forward and backward scans) lamp tests. For CJ, the two scans are shown as two measurements with close but different times.
- Input:
brewerid
: see Brewerid in common inputsdate
: see Date in common inputsenddate
: see Enddate in common inputsformat
: see Formatlamp
: optional, allowed values areb1
andb2
, used to select only data with theb1
(Hg) orb2
(Standard) lamp
- Output: standard output plus other fields, including
gmt
: time of the first wavelength measured – the shortest for the CI and CJ forward scan, and the longest for the CJ backward.mmmm_l
: time in minutes of the measurement of wavelengthl
, wherel = 2865, 2870, … 3630
raw_counts_l
: raw counts of wavelengthl
counts_per_second_l
: counts/second for wavelengthl
lamp
: eitherb1
(Hg) orb2
(Standard)
- Debug output: if
&debug
is passed in the url, two extra fields are included in the output:lamp_time
: time at which the lamp was switched onlamp_co
: comment from the B file stating which lamp was switched on. This comment corresponds to the most recent in a search on all the comments containing the keywordsb1
orb2
in the B file in the 20 minutes before the first measurement in the CI/CJ
CZ lamp tests
- Function:
CZ
- Description: Return the data from the CZ lamp tests. Because in this test the wavelengths are defined by the Brewer operator, Eubrewnet's output is different than in the case of the CI/CJ tests. In particular, the
l
in fields such asmmmm_l
is now just a numerical index from 0 to 149, and the wavelengths are shown in the newwavelengths_l
fields - Input: same as for the CI and CJ functions, see above
- Output: the output is quite similar to that of the CI and CJ functions, the main difference being that the
_l
suffix is now a numerical index instead of a wavelength. There are also some new fields:wavelength_l
: wavelength in nm corresponding to the l-th measurementfwpos1
andfwpos2
: filter wheel positionsstartwv
andfinalwv
: initial and final wavelengths
NI responsivity tests
- Function:
NI
- Description: Return the data from the NI responsivity tests.
- Input:
brewerid
: see Brewerid in common inputsdate
: see Date in common inputsenddate
: see Enddate in common inputsformat
: see Format
- Output: besides the usual
brewerid
,gmt
,lat
, … output, some other fields returned includemmmm
: minutes since 00:00wavelength
: wavelength measuredfwpos1
andfwpos2
: filter wheel positionsrawcountssml
: raw counts for the slit maskl
position, wherel=0,1,…,7
HS and HL Mercury lamp tests
- Functions:
HS
andHL
- Description: Return the data from the short (2950 – 2984 Å) and long (3331 – 3351 Å) wavelength-range Mercury lamp tests
- Input:
brewerid
: see Brewerid in common inputsdate
: see Date in common inputsenddate
: see Enddate in common inputsformat
: see Format
- Output: besides the usual
brewerid
,gmt
,lat
, … output, some other fields returned includemmmm
: minutes since 00:00wavelength
: wavelength measuredfwpos1
andfwpos2
: filter wheel positionsrawcounts
: raw countscountspsecond
: counts per second
ERA5 comparison
- Functions:
compareERA5
- Description: Compares the hourly average 1.5 level ozone product for a given brewer with the Total ozone column product from the ERA5 reanalysis provided by ECMWF.
- Input:
brewerid
: see Brewerid in common inputsstationid
: Eubrewnet station id. Ifstationid
is not provided, the station where the Brewer is located indate
is used to select the ERA5 data.date
: see Date in common inputsenddate
: see Enddate in common inputsformat
: see Formatdebug
: In the output, individual measurements are returned instead of hourly averages.
- Output:
gmt
: UTC time.o3_avg
: Mean hourly ozone L1.5 for brewerid.o3_era5
Total ozone column from ERA5 for the station at which the brewer was atdate
.diff
: difference between o3_avg and o3_era5 in Dobson units.reldiff
: relative difference between o3_avg and o3_era5 in %
ERA5
- Functions:
ERA5
- Description: Retrieve the Total ozone column product from the ERA5 reanalysis provided by ECMWF.
- Input:
gaw_id
orstationid
: GAW or Eubrewnet station id.date
: see Date in common inputsenddate
: see Enddate in common inputsformat
: see Format
- Output:
gmt
: UTC time.o3
Total ozone column from ERA5 for the station at which the brewer was atdate
.stationid
: Eubrewnet station id.
compareO3
- Functions:
compareO3
- Description: compare the O3L1.5 measurements of two brewers
- Input:
brewerid
andcompid
: ids of the reference and to-be-compared brewers, respectively.maxminutes
: maximum difference in minutes between the measurements of the two brewers. If not provided, defaults to 2.5 minutesavg
: return the average of the comp measurements corresponding to each one of the reference. If not passed as an argument, each individual measurement is listed in the outputdate
: see Date in common inputsenddate
: see Enddate in common inputsformat
: see Format
- Output:
gmt
andcompgmt
: UTC time of the reference and to-be-compared brewers, respectively.o3
andcompo3
: total ozone column values.airmass
andcompairmass
: airmasses.osc
andcomposc
: OSC=ozone*airmass valuesfilt
andcompfilt
: filter positionsdeltaminutes
: difference in minutes between the measurements- (only if
avg
is passed as option)compn
andcompsdo3
: number of measurements and the standard deviation of the o3, respectively, for all the comp measurements withinmaxminutes
of the time of the reference. Note the other comp fields correspond to averages of the comp measurements.
codes/calibration.txt · Last modified: 2024/10/03 10:16 by ajberjon