Eubrewnet Wiki

User Tools

Site Tools

Trace:
codes:dbaccess

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
codes:dbaccess [2025/10/14 13:34] – [Brewer Summary] ajberjoncodes:dbaccess [2026/09/11 11:45] (current) – [Process O3L1_5] ajberjon
Line 8: Line 8:
   * Optionally, an enddate input parameter can be provided: [[http://eubrewnet.aemet.es/eubrewnet/data/get/DS?brewerid=157&date=2016-01-01&enddate=2016-01-07|Use of get DS with enddate]]   * Optionally, an enddate input parameter can be provided: [[http://eubrewnet.aemet.es/eubrewnet/data/get/DS?brewerid=157&date=2016-01-01&enddate=2016-01-07|Use of get DS with enddate]]
   * The functions accept a format input parameter for formatting the output in different ways: [[http://eubrewnet.aemet.es/eubrewnet/data/get/DS?brewerid=157&date=2016-01-01&format=text|Use of get DS with format]]   * The functions accept a format input parameter for formatting the output in different ways: [[http://eubrewnet.aemet.es/eubrewnet/data/get/DS?brewerid=157&date=2016-01-01&format=text|Use of get DS with format]]
- 
-**Description of the DS Get Function**: 
-  * Function: DS 
-  * Description: Get the Direct Sun individual measures. Gives extra information from the summaries as the temperature or the double ratios and set a flag for mercury (hg_test) if the measure is between two correct mercury tests with a difference of 1 or less from the reference. 
-  * Inputs: 
-      - brewerid: see [[codes:dbaccess#common_inputs|Brewerid in common inputs]] 
-      - date: see [[codes:dbaccess#common_inputs|Date in common inputs]] 
-      - enddate: see [[codes:dbaccess#common_inputs|Enddate in common inputs]] 
-      - format: see [[codes:dbaccess#format|Format]] 
-  * Output: Direct Sun individual measures with extra values comming from: 
-      - Summary: temp (temperature), zenith_angle, airmass, double_ratio1, double_ratio2  
-      - B file: date (in ISO 8601 format) 
-      - HG measures: hg_test (0: measures is not between two correct HG tests; 1: measure between two correct HG tests) 
  
 There is a [[codes:dbaccess#get_functions|Get Function Table]] with a summary of the available functions to access stored data and their links as example. The extended documentation for each function can be directly access from the table using the links in the column //Long Description//. There is a [[codes:dbaccess#get_functions|Get Function Table]] with a summary of the available functions to access stored data and their links as example. The extended documentation for each function can be directly access from the table using the links in the column //Long Description//.
Line 35: Line 22:
  
 ===== COMMON ===== ===== COMMON =====
-==== Format ==== 
-The access functions provide four different ways of data access using the format input parameter. 
-^ Format input ^ Description ^ Example ^ 
-^ jsonM | JSON matrix formed by lists of lists (default value). The first list is formed by the value names and the following lists are the query outputs | [[http://eubrewnet.aemet.es/eubrewnet/data/get/DS?brewerid=157&date=2015-01-01|Get DS by default]] | 
-^ jsonO | JSON object of lists, where each key is the value name and its value is the time sorted list of them| [[http://eubrewnet.aemet.es/eubrewnet/data/get/DS?brewerid=157&date=2015-01-01&format=jsonO|Get DS with jsonO format]] | 
-^ text | Human readable comma separated values where first row is the value names and the following are the query outputs | [[http://eubrewnet.aemet.es/eubrewnet/data/get/DS?brewerid=157&date=2015-01-01&format=text|Get DS with text format]] | 
-^ csv | CSV direct download where first row are the value names and the following are the query outputs | [[http://eubrewnet.aemet.es/eubrewnet/data/get/DS?brewerid=157&date=2015-01-01&format=csv|Get DS with csv format]] | 
- 
 ==== Common Inputs ==== ==== Common Inputs ====
 Almost all functions accept the following input options: Almost all functions accept the following input options:
Line 53: Line 32:
  
 See also the code examples in [[codes:dbaccess#matlab|matlab]] and [[codes:dbaccess#python|python]]. See also the code examples in [[codes:dbaccess#matlab|matlab]] and [[codes:dbaccess#python|python]].
 +
 +==== Format ====
 +The access functions provide four different ways of data access using the format input parameter.
 +^ Format input ^ Description ^ Example ^
 +^ jsonM | JSON matrix formed by lists of lists (default value). The first list is formed by the value names and the following lists are the query outputs | [[http://eubrewnet.aemet.es/eubrewnet/data/get/DS?brewerid=157&date=2015-01-01|Get DS by default]] |
 +^ jsonO | JSON object of lists, where each key is the value name and its value is the time sorted list of them| [[http://eubrewnet.aemet.es/eubrewnet/data/get/DS?brewerid=157&date=2015-01-01&format=jsonO|Get DS with jsonO format]] |
 +^ text | Human readable comma separated values where first row is the value names and the following are the query outputs | [[http://eubrewnet.aemet.es/eubrewnet/data/get/DS?brewerid=157&date=2015-01-01&format=text|Get DS with text format]] |
 +^ csv | CSV direct download where first row are the value names and the following are the query outputs | [[http://eubrewnet.aemet.es/eubrewnet/data/get/DS?brewerid=157&date=2015-01-01&format=csv|Get DS with csv format]] |
 +
 ==== Means ==== ==== Means ====
 For selected functions, passing the ''&means=xk'' variable in the URL will return the mean of all the output fields in the period specified by ''xk'', where ''x'' is a number and ''k'' is any of ''min'', ''h'' (hours), ''d'' (days), or ''M'' (months). The output includes two extra fields, ''stats_nobs'', with the number of measurements which have been used to calculate the mean, and ''stats_std_o3'' with the standard deviation of the ozone over the specified period. Examples: For selected functions, passing the ''&means=xk'' variable in the URL will return the mean of all the output fields in the period specified by ''xk'', where ''x'' is a number and ''k'' is any of ''min'', ''h'' (hours), ''d'' (days), or ''M'' (months). The output includes two extra fields, ''stats_nobs'', with the number of measurements which have been used to calculate the mean, and ''stats_std_o3'' with the standard deviation of the ozone over the specified period. Examples:
Line 100: Line 88:
  
 ==== Descriptions ==== ==== Descriptions ====
 +
 === DS === === DS ===
   * Function: DS   * Function: DS
Line 194: Line 183:
   * Output: Zenith Sky individual measures with extra values comming from:   * Output: Zenith Sky individual measures with extra values comming from:
       - Summary: temp (temperature), zenith_angle, airmass, double_ratio1, double_ratio2        - Summary: temp (temperature), zenith_angle, airmass, double_ratio1, double_ratio2 
 +  * Example: http://eubrewnet.aemet.es/eubrewnet/data/process/SL?brewerid=157&date=2026-01-01&format=text
  
 ---- ----
Line 573: Line 563:
   * Description: Give the processed Ozone calculated using standard Algorithm and Direct Sun Measures. The Operators have to upload configurations for calculation. See [[codes:ozoneproduct|Ozone Level Definition]] and [[http://eubrewnet.aemet.es/cost1207/2015/03/04/configuration-upload-intructions/|Configuration Upload Instructions]]   * Description: Give the processed Ozone calculated using standard Algorithm and Direct Sun Measures. The Operators have to upload configurations for calculation. See [[codes:ozoneproduct|Ozone Level Definition]] and [[http://eubrewnet.aemet.es/cost1207/2015/03/04/configuration-upload-intructions/|Configuration Upload Instructions]]
   * Inputs:   * Inputs:
-      - brewerid: see [[codes:dbaccess#common_inputs|Brewerid in common inputs]]+      - brewerid: see [[codes:dbaccess#common_inputs|Brewerid in common inputs]]. If a station is not selected, data from any station where the selected Brewer has measured will be returned. 
 +      - stationid or gaw_id or woudc_id: Eubrewnet, GAW or WOUDC ID of the station. If a brewer is not selected, data from any brewer that has measured at the selected station will be returned.
       - date: see [[codes:dbaccess#common_inputs|Date in common inputs]]       - date: see [[codes:dbaccess#common_inputs|Date in common inputs]]
       - enddate: see [[codes:dbaccess#common_inputs|Enddate in common inputs]]       - enddate: see [[codes:dbaccess#common_inputs|Enddate in common inputs]]
Line 593: Line 584:
   * Description: Give the processed Ozone calculated using standard Algorithm and Direct Sun Measures. The Standard Lamp corrections, ETC by filter and Straylight corrections. The measures are filtered by standard deviation, max airmass, correct mercury test, maximun and mininum values. The Operators have to upload configurations for calculation. See [[codes:ozoneproduct|Ozone Level Definition]] and [[http://eubrewnet.aemet.es/cost1207/2015/03/04/configuration-upload-intructions/|Configuration Upload Instructions]]   * Description: Give the processed Ozone calculated using standard Algorithm and Direct Sun Measures. The Standard Lamp corrections, ETC by filter and Straylight corrections. The measures are filtered by standard deviation, max airmass, correct mercury test, maximun and mininum values. The Operators have to upload configurations for calculation. See [[codes:ozoneproduct|Ozone Level Definition]] and [[http://eubrewnet.aemet.es/cost1207/2015/03/04/configuration-upload-intructions/|Configuration Upload Instructions]]
   * Inputs:   * Inputs:
-      - brewerid: see [[codes:dbaccess#common_inputs|Brewerid in common inputs]]+      - brewerid: see [[codes:dbaccess#common_inputs|Brewerid in common inputs]]. If a station is not selected, data from any station where the selected Brewer has measured will be returned. 
 +      - stationid or gaw_id or woudc_id: Eubrewnet, GAW or WOUDC ID of the station. If a brewer is not selected, data from any brewer that has measured at the selected station will be returned.
       - date: see [[codes:dbaccess#common_inputs|Date in common inputs]]       - date: see [[codes:dbaccess#common_inputs|Date in common inputs]]
       - enddate: see [[codes:dbaccess#common_inputs|Enddate in common inputs]]       - enddate: see [[codes:dbaccess#common_inputs|Enddate in common inputs]]
Line 609: Line 601:
   * Description: Give the processed Ozone calculated using standard Algorithm and Direct Sun Measures. The Standard Lamp corrections, ETC by filter and Straylight corrections. The measures are filtered by standard deviation, max airmass, correct mercury test, maximun and mininum values. The Operators have to upload configurations for calculation. See [[codes:ozoneproduct|Ozone Level Definition]] and [[http://eubrewnet.aemet.es/cost1207/2015/03/04/configuration-upload-intructions/|Configuration Upload Instructions]]. The Level 2.0 is available only for validated Configurations.   * Description: Give the processed Ozone calculated using standard Algorithm and Direct Sun Measures. The Standard Lamp corrections, ETC by filter and Straylight corrections. The measures are filtered by standard deviation, max airmass, correct mercury test, maximun and mininum values. The Operators have to upload configurations for calculation. See [[codes:ozoneproduct|Ozone Level Definition]] and [[http://eubrewnet.aemet.es/cost1207/2015/03/04/configuration-upload-intructions/|Configuration Upload Instructions]]. The Level 2.0 is available only for validated Configurations.
   * Inputs:   * Inputs:
-      - brewerid: see [[codes:dbaccess#common_inputs|Brewerid in common inputs]]+      - brewerid: see [[codes:dbaccess#common_inputs|Brewerid in common inputs]]. If a station is not selected, data from any station where the selected Brewer has measured will be returned. 
 +      - stationid or gaw_id or woudc_id: Eubrewnet, GAW or WOUDC ID of the station. If a brewer is not selected, data from any brewer that has measured at the selected station will be returned.
       - date: see [[codes:dbaccess#common_inputs|Date in common inputs]]       - date: see [[codes:dbaccess#common_inputs|Date in common inputs]]
       - enddate: see [[codes:dbaccess#common_inputs|Enddate in common inputs]]       - enddate: see [[codes:dbaccess#common_inputs|Enddate in common inputs]]
       - format: see [[codes:dbaccess#format|Format]]       - format: see [[codes:dbaccess#format|Format]]
-      - debug: change the ouput to see the corrections that are being applied. (True) 
-      - rejected: returns the filtered data, too (True) 
   * Outputs: Ozone calculated from the DS raw counts using standard algorithm from [[devel:brewerpythonmodule|Brewer Python Library]]. Measurements have been enhanced with info from:   * Outputs: Ozone calculated from the DS raw counts using standard algorithm from [[devel:brewerpythonmodule|Brewer Python Library]]. Measurements have been enhanced with info from:
       - Filter flag       - Filter flag
Line 719: Line 710:
   * Examples: https://eubrewnet.aemet.es/eubrewnet/data/get/UNCL1?brewerid=163&date=2019-06-03&format=text, https://eubrewnet.aemet.es/eubrewnet/data/get/UNCL1_V2?brewerid=163&date=2019-06-03&format=text   * Examples: https://eubrewnet.aemet.es/eubrewnet/data/get/UNCL1?brewerid=163&date=2019-06-03&format=text, https://eubrewnet.aemet.es/eubrewnet/data/get/UNCL1_V2?brewerid=163&date=2019-06-03&format=text
   * Input:   * Input:
-      - ''brewerid'': see [[codes:dbaccess#common_inputs|Brewerid in common inputs]]+      - ''brewerid'': see [[codes:dbaccess#common_inputs|Brewerid in common inputs]]. If a station is not selected, data from any station where the selected Brewer has measured will be returned. 
 +      - ''stationid'' or ''gaw_id'' or ''woudc_id'': Eubrewnet, GAW or WOUDC ID of the station. If a brewer is not selected, data from any brewer that has measured at the selected station will be returned.
       - ''date'': see [[codes:dbaccess#common_inputs|Date in common inputs]]       - ''date'': see [[codes:dbaccess#common_inputs|Date in common inputs]]
       - ''enddate'': see [[codes:dbaccess#common_inputs|Enddate in common inputs]]       - ''enddate'': see [[codes:dbaccess#common_inputs|Enddate in common inputs]]
Line 735: Line 727:
   * Examples: https://eubrewnet.aemet.es/eubrewnet/data/get/UNCL1_5?brewerid=163&date=2019-06-03&format=text, https://eubrewnet.aemet.es/eubrewnet/data/get/UNCL1_5_V2?brewerid=163&date=2019-06-03&format=text   * Examples: https://eubrewnet.aemet.es/eubrewnet/data/get/UNCL1_5?brewerid=163&date=2019-06-03&format=text, https://eubrewnet.aemet.es/eubrewnet/data/get/UNCL1_5_V2?brewerid=163&date=2019-06-03&format=text
   * Input:   * Input:
-      - ''brewerid'': see [[codes:dbaccess#common_inputs|Brewerid in common inputs]]+      - ''brewerid'': see [[codes:dbaccess#common_inputs|Brewerid in common inputs]]. If a station is not selected, data from any station where the selected Brewer has measured will be returned. 
 +      - ''stationid'' or ''gaw_id'' or ''woudc_id'': Eubrewnet, GAW or WOUDC ID of the station. If a brewer is not selected, data from any brewer that has measured at the selected station will be returned.
       - ''date'': see [[codes:dbaccess#common_inputs|Date in common inputs]]       - ''date'': see [[codes:dbaccess#common_inputs|Date in common inputs]]
       - ''enddate'': see [[codes:dbaccess#common_inputs|Enddate in common inputs]]       - ''enddate'': see [[codes:dbaccess#common_inputs|Enddate in common inputs]]
Line 798: Line 791:
 ==== Process O3L1_5 ==== ==== Process O3L1_5 ====
   * Function: O3L1_5   * Function: O3L1_5
-  * Description: Process Ozone calculated using standard Algorithm and Direct Sun Measures. The Standard Lamp corrections, ETC by filter and Straylight corrections. The measures are filtered by standard deviation, max airmass, correct mercury test, maximun and mininum values. The Operators have to upload configurations for calculation. See [[codes:ozoneproduct|Ozone Level Definition]] and [[http://eubrewnet.aemet.es/cost1207/2015/03/04/configuration-upload-intructions/|Configuration Upload Instructions]]+  * Description: Process Ozone calculated using standard Algorithm and Direct Sun Measures. Summary measurements are returned. The Standard Lamp corrections, ETC by filter and Straylight corrections. The measures are filtered by standard deviation, max airmass, correct mercury test, maximun and mininum values. The Operators have to upload configurations for calculation. See [[codes:ozoneproduct|Ozone Level Definition]] and [[http://eubrewnet.aemet.es/cost1207/2015/03/04/configuration-upload-intructions/|Configuration Upload Instructions]]
   * Inputs:   * Inputs:
       - brewerid: see [[codes:dbaccess#common_inputs|Brewerid in common inputs]]       - brewerid: see [[codes:dbaccess#common_inputs|Brewerid in common inputs]]
Line 804: Line 797:
       - enddate: see [[codes:dbaccess#common_inputs|Enddate in common inputs]]       - enddate: see [[codes:dbaccess#common_inputs|Enddate in common inputs]]
       - format: see [[codes:dbaccess#format|Format]]       - format: see [[codes:dbaccess#format|Format]]
-      - debug: change the ouput to see the corrections that are being applied. (True) +      - individual: returns the individual measurements instead of the summaries. 
-      - rejected: returns the filtered data, too (True)+      - debug: change the ouput to see the corrections that are being applied. 
 +      - rejected: see [[codes:dbaccess#bypassing_data_filters|Bypassing Data Filters]] 
 +      - allowfilter: see [[codes:dbaccess#bypassing_data_filters|Bypassing Data Filters]] 
 +      - allowtag: see [[codes:dbaccess#bypassing_data_filters|Bypassing Data Filters]]
   * Outputs: Ozone calculated from the DS raw counts using standard algorithm from [[devel:brewerpythonmodule|Brewer Python Library]]. Measurements have been enhanced with info from:   * Outputs: Ozone calculated from the DS raw counts using standard algorithm from [[devel:brewerpythonmodule|Brewer Python Library]]. Measurements have been enhanced with info from:
       - Filter flag       - Filter flag
Line 1041: Line 1037:
  
 both the FLAG_OZONE_STD_FILT and FLAG_HG_FILT data filters will be ignored and all the data will have a ''filter_flag'' value of 0. both the FLAG_OZONE_STD_FILT and FLAG_HG_FILT data filters will be ignored and all the data will have a ''filter_flag'' value of 0.
 +
 +Also, ''allowtag=xxx'' can be added to the URL to include periods in the processing that are normally filtered out by the blacklist — for example, when you want to include a specific period whose comment contains the string ''xxx''.
  
 ---- ----
codes/dbaccess.1760448849.txt.gz · Last modified: 2025/10/14 13:34 by ajberjon