User Tools

Site Tools


codes:calibration

Calibration Functions

This page provides a brief overview of the functions avaliable in Eubrewnet to determine Brewer configuration parameters.


Langley

  • Function: langley
  • Description: Determine the ETC and the ETC filter corrections applying the Langley-plot method to each half day.
  • Input: a complete list of input arguments is provided next, but we strongly recommed to use the web interface at https://rbcce.aemet.es/eubrewnet/caldata/langley to access this function
    1. format: see Format
    2. data: input data used, available options are o3l15 (default if the option is not passed in the url), o3l15v2, o3l1, sco3, UNCL15 for the O3L1.5, O3L1.5V2, O3L1, sun scan, and UNCL1.5 measurements, respectively. For the sco3 option, the step option with the value of the micrometer step must be also passed in the url; if checking a mic step which does not correspond to the cal step, it's also recommended to pass the ozone absorption coefficient specific to that mic step using the urlconfig option, see below. Individual observations (not summaries) are used in all cases unless use_summaries is selected, see below. For the uncl15 option, the fit is done using York's method (see this MATLAB and this Python implementations)
    3. so2: calculate the SO2 ETC instead of the (default) O3 one
    4. calc: calculation to perform, available options are etc (Langley-plot with all the data, default option), filters (the Langley-plot fit is done using dummy variables for filters 3, 4, 5, and thus allows to determine the ETC filter correction of those filters with respect to the ETC obtained for the set measurements with filters 0, 1, and 2), and filters_etc (this option performs two fits: first, a filters run to determine the ETC filter corrections, and then an etc calculation using these ETC filter corrections). For etc runs, two different fits are performed: ms9 vs airmass (brewer method) and ms9/airmass vs 1/airmass (dobson method)
    5. minairmass: airmass minimum in etc calculations (defaults to 1.70 if the option is not passed in the url)
    6. maxairmass: airmass maximum in etc calculations (defaults to 3.75 if the option is not passed in the url)
    7. minobs: minimum number of observations (either individual observations or summaries) in the half-day to perform the Langley in etc calculations (default value: 25). Note by default all the data functions are configured to return individual observations, but see use_summaries below
    8. filters_minairmass: airmass minimum in filters calculations (defaults to 1 if the option is not passed in the url)
    9. filters_maxairmass: airmass maximum in filters calculations (defaults to 5 if the option is not passed in the url)
    10. filters_minobs: minimum number of observations in the half-day to perform the Langley in filters calculations (default value: 15)
    11. maxstdo3: don't perform the langley if the maximum of the standard deviation of the half-day ozone is above this value (default: 2.5)
    12. minrsquared: don't accept the langley results if the r2 of the fits are below this value (default: 0.999)
    13. residuals_filtering: do each fit twice, using the residuals of the first to remove observations whose residual is larger than 1.5*std(residuals)
    14. use_summaries: if passed as an option, use summaries instead of individual observations for o3l15 and o3l15v2 data. Note the default values for minobs and filter_minobs (25 and 15, respectively) are provided as a guide for calculations with individual observations, the user must set suitable values (eg, 5 for minobs) if use_summaries is passed.
    15. urlconfig: allows setting the configuration's parameters, overriding the values in Eubrewnet's database. For example, to set a value of 0.4 for the ozone absorption coefficient, add &urlconfig&o3o3rate=0.4 to the url, and to set specific values for the filter corrections add &urlconfig&fc=1,2,3,4,5
    16. debug: include debug output. As of 2023-06-29, this returns four additional vectorial fields, airmass, ms9, filt, and solartime, with the raw data used in the half-day langley fit.
  • Output:
    1. date: date and time of each half-day langley, morning langleys correspond to 00 time, and afternoon ones, to 12
    2. etc: calculated ETC
    3. rmse or u_etc: root mean squared error or uncertainty in the ETC as calculated in the York method (only for the uncl15 input)
    4. tau: in the case of the determination of the O3 ETC, this corresponds to the ozone optical depth – ie, the ozone multiplied by the ozone absorption coefficient. For the SO2 ETC, this is the sum of the SO2 and O3 optical depths, in the space of the SO2 ratios
    5. rsquared: r2 of the langley fit
    6. nobs: number of observations used in the fit
    7. stdo3: standard deviation of the half-day ozone
    8. meano3: mean ozone of the half-day
    9. filter_corr_3,filter_corr_4,filter_corr_5: ETC filter corrections, either calculated in a filters run or supplied passing &urlconfig&fc=1,2,3,4,5 in the url
    10. info: a brief description of the calculation performed

ETC filter corrections from ozone differences

  • Function: fcor_o3dif
  • Description: this function returns the ETC filter corrections (with respect to the mean of filters 1 and 2) as calculated from the ozone differences of near-simultaneous measurements taken with different filters. Mathematically, for two measurements taken with different filters at almost the same airmass μ and with a ΔO3L1 difference in the (uncorrected) ozone, Δfiltercorr=ΔO3L1∗μ∗abscoeff∗10
  • Input:
    1. date: see Date in common inputs. If not specified, the calculation will be carried out for all the date ranges between the configurations available in Eubrewnet
    2. enddate: see Enddate in common inputs. If not specified, the current date is used.
    3. format: see Format
    4. maxminutes: maximum difference in minutes between consecutive measurements, default value: 15
    5. airmass: maximum airmass between consecutive measurements, default value: 0.1
  • Output:
    1. config_f1_cor, config_f2_cor, config_f3_cor, config_f4_cor , config_f5_cor: ETC filter corrections set in the configuration at Eubrewnet
    2. calc_f1_cor, calc_f2_cor, calc_f3_cor, calc_f4_cor, calc_f5_cor: median of all the ETC filter corrections calculated in the period, referred to the mean of filters 1 and 2
    3. calc_f1_sd, calc_f2_sd, calc_f3_sd, calc_f4_sd, calc_f5_sd: standard deviation of the ETC filter corrections in the period
    4. calc_f01_nobs, calc_f12_nobs, calc_f23_nobs, calc_f34_nobs, calc_f45_nobs: number of near simultaneous observations with the indicated filters

ETC filter corrections from ozone differences -- raw data

  • Function: fcor_o3dif_raw
  • Description: this function returns the raw data used in the fcor_o3dif function. That is, it returns Δfiltercorr=ΔO3L1∗μ∗abscoeff∗10 for each pair of measurements, without referring the result to filters 1 and 2 nor calculating any median over a date range.
  • Input:
    1. date: see Date in common inputs. If not specified, the calculation will be carried out for all the date ranges between the configurations available in Eubrewnet
    2. enddate: see Enddate in common inputs. If not specified, the current date is used.
    3. format: see Format
    4. maxminutes: maximum difference in minutes between consecutive measurements, default value: 15
    5. airmass: maximum airmass between consecutive measurements, default value: 0.1
  • Output:
    1. previous_gmt: date and time of the first measurement in the pair
    2. gmt: date and time of the second measurement
    3. previous_airmass: airmass of the first measurement
    4. airmass: airmass of the second measurement
    5. previous_filt: filter position for the first measurement
    6. filt: filter position for the second measurement
    7. previous_o3_l1: level 1 ozone for the first measurement
    8. o3_l1: level 1 ozone for the second measurement
    9. configdate: date of the operative configuration
    10. o3o3rate: ozone absorption coefficient
    11. fcor: filter correction
    12. fchange: a single string with both filter positions

ETC filter correction from FIOAVG data

  • Function: fcor_fioavg
  • Description: this function returns the ETC filter corrections (with respect to the mean of filters 1 and 2) as calculated from the results of the Brewer filter tests with the standard lamp, as found in the FIOAVG file.
  • Input:
    1. date: see Date in common inputs. If not specified, the calculation will be carried out for all the date ranges between the configurations available in Eubrewnet
    2. enddate: see Enddate in common inputs. If not specified, the current date is used.
    3. format: see Format
  • Output:
    1. config_f1_cor, config_f2_cor, config_f3_cor, config_f4_cor , config_f5_cor: ETC filter corrections set in the configuration at Eubrewnet
    2. calc_f1_cor, calc_f2_cor, calc_f3_cor, calc_f4_cor, calc_f5_cor: median of all the ETC filter corrections calculated in the period, referred to the mean of filters 1 and 2
    3. calc_f1_sd, calc_f2_sd, calc_f3_sd, calc_f4_sd, calc_f5_sd: standard deviation of the ETC filter corrections in the period
    4. calc_nobs: number of measurements in the FIOAVG file

Temperature Coefficients

  • Function: TempCoef
  • Description: This function analyses the temperature sensitivity of SL measurements.
  • Input:
    1. format: see Format
    2. tc: if we pass the tc parameter with no value, the fit is done with the SL measurements corrected using the temperature coefficients from the operational setup on Eubrewnet. If we pass six values separated by commas, the fit is done with the corrected SL measurements using these values as temperature coefficients. If the tc parameter is omitted, no temperature correction is applied to the SL measurements.
    3. tmin: all data below tmin are removed.
    4. tmax: all data above tmax are removed.
    5. alpha: Outliers can be removed by giving the alpha parameter a value greater than zero. In this case, after performing the linear regression, all measures whose residuals are greater than the standard deviation of all residuals multiplied by the alpha factor will be removed, and the linear regression will be repeated. This process is repeated iteratively until all values are below this threshold.
    6. step: optional, if passed uses the data from the JL measurements at the requested mic step. JL measurements are corrected using the linear temperature coefficient and the filter attenuations corresponding to each wavelength, as available in the AOD configuration operative at the beginning of the requested date range (date)
    7. time: optional, if passed, the time evolution of the lamp intensity is considered in the analysis, including in the linear regression a degree 3 dependence on time.
    8. urlconfig: allows setting the configuration's parameters, overriding the values in Eubrewnet's database.
  • Output: the following fields are included in the output:
    1. channel: Brewer measurement channels in the ozone position plus R5 and R6 (the latter are not included in the JL output)
    2. a: intercept value for the linear regression.
    3. sa: standard error of the intercept of the linear regression.
    4. b: slope value for the linear regression.
    5. sb: standard error of the slope of the linear regression.
    6. r2: R-squared of the linear regression.
    7. tmin: Minimum temperature used in the linear regression.
    8. tmax: Maximum temperature used in the linear regression.
    9. If the parameter time is passed, three more coefficients are returned (the time dependence considered is of degree 3) together with their standard errors: c,sc,d,sd,e,se.

Sun scan


Sun scan with extra information


CI and CJ lamp tests

  • Function: CI or CJ
  • Description: Return the data from the CI (forward scan) or CJ (forward and backward scans) lamp tests. For CJ, the two scans are shown as two measurements with close but different times.
  • Input:
    1. format: see Format
    2. lamp: optional, allowed values are b1 and b2, used to select only data with the b1 (Hg) or b2 (Standard) lamp
  • Output: standard output plus other fields, including
    1. gmt: time of the first wavelength measured – the shortest for the CI and CJ forward scan, and the longest for the CJ backward.
    2. mmmm_l: time in minutes of the measurement of wavelength l, where l = 2865, 2870, … 3630
    3. raw_counts_l: raw counts of wavelength l
    4. counts_per_second_l: counts/second for wavelength l
    5. lamp: either b1 (Hg) or b2 (Standard)
  • Debug output: if &debug is passed in the url, two extra fields are included in the output:
    1. lamp_time: time at which the lamp was switched on
    2. lamp_co: comment from the B file stating which lamp was switched on. This comment corresponds to the most recent in a search on all the comments containing the keywords b1 or b2 in the B file in the 20 minutes before the first measurement in the CI/CJ

CZ lamp tests

  • Function: CZ
  • Description: Return the data from the CZ lamp tests. Because in this test the wavelengths are defined by the Brewer operator, Eubrewnet's output is different than in the case of the CI/CJ tests. In particular, the l in fields such as mmmm_l is now just a numerical index from 0 to 149, and the wavelengths are shown in the new wavelengths_l fields
  • Input: same as for the CI and CJ functions, see above
  • Output: the output is quite similar to that of the CI and CJ functions, the main difference being that the _l suffix is now a numerical index instead of a wavelength. There are also some new fields:
    1. wavelength_l: wavelength in nm corresponding to the l-th measurement
    2. fwpos1 and fwpos2: filter wheel positions
    3. startwv and finalwv: initial and final wavelengths

NI responsivity tests


HS and HL Mercury lamp tests


ERA5 comparison

.


codes/calibration.txt · Last modified: 2024/04/19 09:48 by jlsolano