Computational Science Community Wiki

Differences between revisions 1 and 3 (spanning 2 versions)
Revision 1 as of 2014-01-24 17:29:24
Size: 5496
Editor: DouglasLowe
Comment:
Revision 3 as of 2014-04-08 10:24:35
Size: 3499
Editor: MichaelBane
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
## page was renamed from WRF-Chem/Inputs/ECMWFMeterology
Line 7: Line 8:
=== 26/02/12: New server, new scripts are needed for batch access === == Website Access for Interim (and other data) and batch scripts for downloading ==
Line 88: Line 89:
== ECMWF Meterology (Old) == == ECMWF data server - use this for accessing Operational Data ==
Line 112: Line 113:

== ECMWF Meterology (New Batch Access) ==

Beta batch access to the ECMWF Interim data fields are now available. The webpage access for this service is here: http://data-portal.ecmwf.int/data/d/interim_daily - here you can register for access to the data, which will give you your own token associated with your email address.

Example python and perl scripts are provided by the website, as well as the communication modules. On hyperbar the perl scripts have been saved in {{{/usr/local/share/perl}}}. One difference with the old method is that we don't know how to extract surface and volume data in the same file. Some modifications will be needed for the WPS scripts (see next page).

Example script:
{{{#!python

#!/usr/bin/perl -I/usr/local/share/perl
use ECMWF::DataServer;
use Date::Calc qw(Add_Delta_Days Delta_Days);
no strict;

my $client = ECMWF::DataServer->new(
    portal => 'http://data-portal.ecmwf.int/data/d/dataserver/',
    token => 'xxxxxxxxxxxxxxxxxxxxxx',
    email => 'xxxxxx@xxxxxxxx',
);

# set dates for data extraction - summer RONOCO
@date = (2011,01,12);
@enddate = (2011,01,25);

while (Delta_Days(@date,@enddate) >= 0){
 $year = @date[0];
 $month = @date[1];
 $day = @date[2];
 $datestr=sprintf("%d-%02d-%02d",$year,$month,$day);
 
 # extract volume data
 $client->retrieve(
  dataset => "interim_full_daily",
  date => "$datestr",
  time => "00/06/12/18",
  step => "0",
  levtype => "pl",
  levelist => "all",
  area => "70/-30/30/20",
  type => "an",
  grid => "128",
  param => "129/130/131/132/157",
  target => "pl_${datestr}.grb",
 );

 # extract surface data
 $client->retrieve(
  dataset => "interim_full_daily",
  date => "$datestr",
  time => "00/06/12/18",
  step => "0",
  levtype => "sfc",
  area => "70/-30/30/20",
  type => "an",
  grid => "128",
  param => "172/134/151/165/166/167/168/235/33/34/31/141/139/170/183/236/39/40/41/42",
  target => "sfc_${datestr}.grb",
 );
 # move to next day
 ($year,$month,$day) = Add_Delta_Days($year,$month,$day,1);
 @date=($year,$month,$day);

}
}}}

ECMWF Meterology, and how to get it.

This page details the methods used to download meterology data from ECMWF, both from their website, and from their data servers.

Website Access for Interim (and other data) and batch scripts for downloading

The new server for batch access has changed the system slightly from the old one. To run, go to: https://software.ecmwf.int/wiki/display/WEBAPI/Accessing+ECMWF+data+servers+in+batch Register to get the token and key. Put this into a file in you home directory called: .ecmwfapirc

 {
    "url"   : "https://api.ecmwf.int/v1",
    "key"   : "###############",
    "email" : "XXXX@YYYY.ac.uk"
 }

Then register for particular database by clicking on relavent link (e.g. for interim data) Set up your environment following instructions on the website e.g.  export PYTHONPATH=/Users/scott_a_n/ecmwf-api-client-python  Run script to download data:

   1 #!/usr/bin/env python
   2 
   3 import time
   4 from datetime import date
   5 
   6 # export PYTHONPATH = /usr/bin/ecmwfapi
   7 from ecmwfapi import ECMWFDataServer
   8  
   9 server = ECMWFDataServer()
  10 
  11 # Set dates for data extraction - SAMBBA campaign period
  12 idate = date(2012,07,01)
  13 edate = date(2012,10,31)
  14 
  15 while (idate <= edate):
  16 
  17     iyear = idate.year
  18     imonth = idate.month
  19     iday = idate.day
  20 
  21     strdate = "%d%02d%02d" % (iyear, imonth, iday)
  22     print strdate   
  23 
  24     # extract 3D data
  25     server.retrieve({
  26         'dataset' : "interim",
  27         'step'    : "0",
  28         'levtype' : "pl",
  29         'levelist': "all",
  30         'date'    : strdate,
  31         'time'    : "00/06/12/18",
  32         'origin'  : "all",
  33         'type'    : "an",
  34         'param'   : "129/130/131/132/157",
  35         'grid'    : "128",
  36         'target'  : "pl_"+strdate+".grib"
  37         })
  38 
  39     # extract surface data
  40     server.retrieve({
  41         'dataset' : "interim",
  42         'step'    : "0",
  43         'levtype' : "sfc",
  44         'date'    : strdate,
  45         'time'    : "00/06/12/18",
  46         'origin'  : "all",
  47         'type'    : "an",
  48         'param'   : "172/134/151/165/166/167/168/235/33/34/31/141/139/170/183/236/39/40/41/42",
  49         'grid'    : "256", # use higher resolution surface data
  50         'target'  : "sfc_"+strdate+".grib"
  51         })
  52 
  53     # move to next day
  54     
  55     idate = idate.replace(day=idate.day + 1)
  56 
  57 print "End."

ECMWF data server - use this for accessing Operational Data

Access

Access to ECMWF met files can be applied for from here: http://www.ecmwf.int/about/computer_access_registration/forms/ Contact at ECMWF is Roddy Sharp (roddy[dot]sharp[at]metoffice[dot]gov[dot]uk)

  • Download the Member State User Registration form. Application should fill in Section A and post the hard-copy to Roddy Sharp at ECMWF. Section B is left blank for ECMWF to complete.
  • The applicant's supervisor (or, project lead, head of department, etc) contacts Roddy to support the application and confirm the applicant's position.
  • On confirmation the ECMWF will post an access token (roughly) within 2 weeks.

Obtaining Data

  • log in to the server:
    • ssh [username]@ecaccess.ecmwf.int

    • the password is generated using your access token
    • select the ecgate host

  • Submit data extraction script using the command:
    • llsubmit [script_name]

Example script:

to be added...