Computational Science Community Wiki

Differences between revisions 5 and 6
Revision 5 as of 2015-08-05 12:23:02
Size: 3552
Editor: DouglasLowe
Comment:
Revision 6 as of 2017-08-08 12:01:42
Size: 4026
Editor: DouglasLowe
Comment:
Deletions are marked like this. Additions are marked like this.
Line 11: Line 11:
The new server for batch access has changed the system slightly from the old one. To run, go to: ECMWF meteorology data is now accessible from their website.

The first time you want to access this data you will need to register here to get a token and key:
Line 13: Line 15:
Register to get the token and key. Put this into a file in you home directory called: .ecmwfapirc
T
hese you should put into a file in you home directory called: .ecmwfapirc
Line 21: Line 24:
Then register for particular database by clicking on relavent link (e.g. for interim data)
Then register for particular database by clicking on relavent link (e.g. for interim data).

Once you have done the above then you can start downloading the meteorology data.

Line 24: Line 32:
Run script to download data:
Copy this script - which will download the data needed by WPS - and change the dates
(idate and edate) to match your period of interest.
Line 86: Line 96:
Finally, run this script to download the data files (on ARCHER, or similar supercomputers, this can be done through a serial batch script).
Line 89: Line 100:
== Below is old information - please use method for Reanalysis data above instead! ==

ECMWF Meterology, and how to get it.

This page details the methods used to download meterology data from ECMWF, both from their website, and from their data servers.

Website Access for Interim (and other data) and batch scripts for downloading

ECMWF meteorology data is now accessible from their website.

The first time you want to access this data you will need to register here to get a token and key: https://software.ecmwf.int/wiki/display/WEBAPI/Access+ECMWF+Public+Datasets

These you should put into a file in you home directory called: .ecmwfapirc

 {
    "url"   : "https://api.ecmwf.int/v1",
    "key"   : "###############",
    "email" : "XXXX@YYYY.ac.uk"
 }

Then register for particular database by clicking on relavent link (e.g. for interim data).

Once you have done the above then you can start downloading the meteorology data.

Set up your environment following instructions on the website e.g.  export PYTHONPATH=/Users/scott_a_n/ecmwf-api-client-python 

Copy this script - which will download the data needed by WPS - and change the dates (idate and edate) to match your period of interest.

   1 #!/usr/bin/env python
   2 
   3 import time
   4 from datetime import date
   5 
   6 # export PYTHONPATH = /usr/bin/ecmwfapi
   7 from ecmwfapi import ECMWFDataServer
   8  
   9 server = ECMWFDataServer()
  10 
  11 # Set dates for data extraction - SAMBBA campaign period
  12 idate = date(2012,07,01)
  13 edate = date(2012,10,31)
  14 
  15 while (idate <= edate):
  16 
  17     iyear = idate.year
  18     imonth = idate.month
  19     iday = idate.day
  20 
  21     strdate = "%d%02d%02d" % (iyear, imonth, iday)
  22     print strdate   
  23 
  24     # extract 3D data
  25     server.retrieve({
  26         'dataset' : "interim",
  27         'step'    : "0",
  28         'levtype' : "pl",
  29         'levelist': "all",
  30         'date'    : strdate,
  31         'time'    : "00/06/12/18",
  32         'origin'  : "all",
  33         'type'    : "an",
  34         'param'   : "129/130/131/132/157",
  35         'grid'    : "128",
  36         'target'  : "pl_"+strdate+".grib"
  37         })
  38 
  39     # extract surface data
  40     server.retrieve({
  41         'dataset' : "interim",
  42         'step'    : "0",
  43         'levtype' : "sfc",
  44         'date'    : strdate,
  45         'time'    : "00/06/12/18",
  46         'origin'  : "all",
  47         'type'    : "an",
  48         'param'   : "172/134/151/165/166/167/168/235/33/34/31/141/139/170/183/236/39/40/41/42",
  49         'grid'    : "256", # use higher resolution surface data
  50         'target'  : "sfc_"+strdate+".grib"
  51         })
  52 
  53     # move to next day
  54     
  55     idate = idate.replace(day=idate.day + 1)
  56 
  57 print "End."

Finally, run this script to download the data files (on ARCHER, or similar supercomputers, this can be done through a serial batch script).

Below is old information - please use method for Reanalysis data above instead!

ECMWF data server - use this for accessing Operational Data

Access

Access to ECMWF met files can be applied for from here: http://www.ecmwf.int/about/computer_access_registration/forms/ Contact at ECMWF is Roddy Sharp (roddy[dot]sharp[at]metoffice[dot]gov[dot]uk)

  • Download the Member State User Registration form. Application should fill in Section A and post the hard-copy to Roddy Sharp at ECMWF. Section B is left blank for ECMWF to complete.
  • The applicant's supervisor (or, project lead, head of department, etc) contacts Roddy to support the application and confirm the applicant's position.
  • On confirmation the ECMWF will post an access token (roughly) within 2 weeks.

Obtaining Data

  • log in to the server:
    • ssh [username]@ecaccess.ecmwf.int

    • the password is generated using your access token
    • select the ecgate host

  • Submit data extraction script using the command:
    • llsubmit [script_name]

Example script:

to be added...