History Reanalysis in WRF EMS 3.2.1 made easy [Guide]

You know something useful, tips&tricks, have some unofficial documentation...? Please share your knowledge with us in this section of EMS forum. * * * Please DON'T open topics with questions in this section! * * *
meteoadriatic
Posts: 1515
Joined: Wed Aug 19, 2009 10:05 am

History Reanalysis in WRF EMS 3.2.1 made easy [Guide]

Post by meteoadriatic » Thu Mar 17, 2011 10:48 am

Until 3.2.1 release of WRF EMS, all reanalysis had to be done manually, by first finding good source of historical data in grib format, then downloading them into your grib/ directory inside runs/<domainname>. After that, user had to rename gribs so they match naming conventions in respective gribinfo.conf file (or change naming convention there to match grib names), and finally run ems_prep script to prepare your data for model run. While not being too hard to do all that, it is little bit tedious to find and set all stuff if you are first time into this.

However, with 3.2.1 beta release, things changed considerably. Among all news announced in this new release, the possibility to start reanalysis with no effort is, IMHO, the greatest one enhancement to WRF EMS 3.2.1.

You can still apply same procedure and end up with the same result, but now, Robert did step forward and enhanced it's scripts so they can be used directly to engage history reanalyse, straight from the shell command line! No need to find history grib download server, no need to register anywhere, no need to download by hand anything or adjust any naming convention... let's see how things work!

Running history reanalysis with WRF EMS 3.2 is very easy. You just have to install WRF EMS (you probably have already?) and create your domain (with or without nests) for simulation. You do that same as you would for any realtime forecast, like explained in official WRF EMS users guide, using domain wizard (dwiz), or, like I always do, using ems_domain.pl script from Robert. That did not changed much from previous version.

Now, when you prepared your domain, all you need is to decide which dataset you want to use. For that purporse, look at this file:

Code: Select all

wrfems/docs/wrfems/strc_datasets.txt
You will find all relevant information there, please read it before you start with your reanalysis.

I will show you an example, how to start reanalysis based on ECMWF Interim Global Reanalysis data(*). This is also one of great news, Robert did included not only the scripts for ECMWF Interim gribs, but also he put whole grib dataset on his download server! Data is available from 1989 up to 2008 year. It is in very good 0.7 degree grid resolution, much better than NCEP/FNL reanalysis (2.5 degrees). That means you can start your simulation with pretty fine mother domain grid, and still achieve very high resolutions with only two or three nest levels (of course, always use ARW core for very fine grids).

After creating domain and setting configuration details like you want, starting reanalysis is a piece of cake. Let say, you want to run simulation on your domain with 4 nest levels, starting at date 21/12/1998, with 00z ECMWF Interim data, and simulation should be run up to 00z, 22/12/1998 (that is, 24 hours). You just change directory into your domain:

Code: Select all

cd $EMS/runs/<domainname>
and type this:

Code: Select all

ems_prep --dset ecmwf --analysis --date 19981221 --cycle 00 --domain 1,2,3,4 --length 24
That is almost all; WRF EMS scripts will download ECMWF data from STRC server for you, ungrib them, and prepare all data for model run. After ems_prep finish, just run model:

Code: Select all

ems_run --domain 1,2,3,4
followed by postprocessing when model finish:

Code: Select all

ems_post --domain 1
ems_post --domain 3
ems_post --domain 4
as you like. That's all folks! Enjoy your simulations!


(*) If you use ECMWF data, and want to publish your work, you must follow some legal requirements. Please read more about that in strc_datasets.txt file.

dwillson88
Posts: 11
Joined: Mon Sep 27, 2010 1:22 pm

Re: History Reanalysis in WRF EMS 3.2.1 made easy [Guide]

Post by dwillson88 » Thu May 19, 2011 12:56 pm

Hi,

That seems like an excellent feature, I'm really after newer datasets such as 2008 and at a quality global resolution.

Just wondering, where is the link to the WRf EMS 3.2.1 Beta, as I cant find it on the SOO/STRC website.

Thanks,

David

meteoadriatic
Posts: 1515
Joined: Wed Aug 19, 2009 10:05 am

Re: History Reanalysis in WRF EMS 3.2.1 made easy [Guide]

Post by meteoadriatic » Thu May 19, 2011 1:39 pm

dwillson88 wrote:Just wondering, where is the link to the WRf EMS 3.2.1 Beta, as I cant find it on the SOO/STRC website.

Thanks,

David
Hello,

here it is, it is still in beta release, that's why it is not officially released yet:
http://soostrc.comet.ucar.edu/wrfems/releases/

You must have new installer for that (don't use installer for v3.1). I can send you new one via e-mail.

smartie
Posts: 94
Joined: Sat May 21, 2011 7:34 am

Re: History Reanalysis in WRF EMS 3.2.1 made easy [Guide]

Post by smartie » Sat May 21, 2011 7:53 am

Hi all,

just to say we've been using ECMWF Interim for a while now- with varying results depending on the quality of analysis for the particular event. Would be happy to offer benefit of our experience if people have problems.

BTW for 3.2 I'm also supplying a stand alone pre-compiled RIP (Read interpolate Plot) package for post-processing. It will not , ATM, be fully integrated, but should work 'out of the box'. I'm also working on an automated shell script for this to do the necessary file processing and plotting.

I have n't seen the beta 3.2 yet and I think Bob's away so I'm not sure of current status of this, if there is a demand I can make RIP available by email or download as a 'no guarantees' package.

Best,
David

surgeon
Posts: 44
Joined: Wed Dec 08, 2010 1:34 pm
Location: Poland
Contact:

Re: History Reanalysis in WRF EMS 3.2.1 made easy [Guide]

Post by surgeon » Mon Jun 06, 2011 11:48 am

Strange. EMS indicates that the files are unavailable. But when you look under the address in the browser you can see that the files are present.
II. Search out requested files for WRF model initialization


* Locating ecmwf files for model initial and boundary conditions

Initiating HTTP connection to soostrc.comet.ucar.edu

-> Checking if available /data/grib/ecmwf/1997/07/1997070300.ecmwf.grb.bz2 - Not currently available
-> Checking if available /data/grib/ecmwf/1997/07/1997070306.ecmwf.grb.bz2 - Not currently available
-> Checking if available /data/grib/ecmwf/1997/07/1997070312.ecmwf.grb.bz2 - Not currently available
-> Checking if available /data/grib/ecmwf/1997/07/1997070318.ecmwf.grb.bz2 - Not currently available
-> Checking if available /data/grib/ecmwf/1997/07/1997070400.ecmwf.grb.bz2 - Not currently available

Antonix
Posts: 256
Joined: Fri Oct 16, 2009 8:53 am

Re: History Reanalysis in WRF EMS 3.2.1 made easy [Guide]

Post by Antonix » Fri Sep 09, 2011 9:34 am

I have the same error

"file non available"

jpb
Posts: 14
Joined: Thu Jan 28, 2010 7:25 am

Re: History Reanalysis in WRF EMS 3.2.1 made easy [Guide]

Post by jpb » Fri Sep 23, 2011 8:47 am

It is a problem of the foreign language you have used during linux install.

You can correct it in strc/ems_prep/Prep_acquire.pm at line number 840 by adding the Length localization like the following for french :

Code: Select all

while (<OF>) {@size = split ' ', $_ if s/Content-Length:|Length:|Longueur://i;}
have nice runs
JPB

denmoko
Posts: 1
Joined: Sun Nov 13, 2011 7:10 am

Re: History Reanalysis in WRF EMS 3.2.1 made easy [Guide]

Post by denmoko » Sun Nov 13, 2011 7:23 am

I have the same trouble too :cry:

1. I can't find the strc/ems_prep/Prep_acquire.pm file at line 840
2. Before find this guide, already download ECMWF at http://strc.comet.ucar.edu/data/soostrc_archive.htm , how to running the grib data from local folder..
3. how to know the version of my WRF EMS, maybe i run the different version.?

thanks,
moko

pattim
Posts: 157
Joined: Sun Jun 24, 2012 8:42 pm
Location: Los Angeles, CA, USA

Re: History Reanalysis in WRF EMS 3.2.1 made easy [Guide]

Post by pattim » Sun Jul 01, 2012 5:00 pm

Wow! That does it - I'm baking Robert and his family some cookies right today! This is especially helpful for those poor folks who grew up in the GUI generation and never learnt vi. :lol:

j1d1w1
Posts: 32
Joined: Wed Mar 13, 2013 2:02 am

Re: History Reanalysis in WRF EMS 3.2.1 made easy [Guide]

Post by j1d1w1 » Wed Mar 20, 2013 10:31 pm

I am attempting a simulation over a small area of southern New Zealand (about 200 x 200 km), for 3-4 July 1996. If I use ems_prep with --dset nnrp things are ok, but reading your post I thought I might be able to get a higher-resolution initialization. But

Code: Select all

ems_prep --dset ecmwf --analysis --date 19960703 --cycle 18 --domain 1 --length 06
results in
Problems acquiring ecmwf files for model initialization. EMS ERROR : Some of the initialization or boundary condition files were unavailable. WRF EMS Program ems_prep failure
More specifically,
-> Copying /data/ecmwf/1996/07/1996070318.ecmwf.grb.bz2 - Not Available
-> Copying /data/ecmwf/1996/07/1996070400.ecmwf.grb.bz2 - Not Available
Am I doing something wrong here? Thanks John

Post Reply

Who is online

Users browsing this forum: Google [Bot] and 1 guest