SOLUTION for gfsptiles/gfs not being available or too slow

Compiling NMM/ARW code for EMS system, upgrading WRF cores on your own, changing EMS scripts to suit your particular needs, and other modifications to original EMS distribution goes in this forum. These are officially unsupported actions.
meteo60
Posts: 96
Joined: Tue Apr 17, 2012 4:50 pm

Re: SOLUTION for gfsptiles/gfs not being available or too slow

Post by meteo60 » Tue Aug 20, 2013 10:30 am

OK.
So i tested it and it doesn't work:

Search out requested files for WRF model initialization

* Locating gfs-nomads files for model initial and boundary conditions

Initiating HTTP connection to nomads.ncep.noaa.gov

-> Checking if available /pub/data/nccf/com/gfs/prod/gfs.2013082006/gfs.t06z.pgrb2f00 - Not currently available


However in my browser http://nomads.ncep.noaa.gov/pub/data/nc ... 013082006/ is ok so :?:

meteoadriatic
Posts: 1512
Joined: Wed Aug 19, 2009 10:05 am

Re: SOLUTION for gfsptiles/gfs not being available or too slow

Post by meteoadriatic » Tue Aug 20, 2013 10:39 am

I see, that's strange.

Antonix
Posts: 256
Joined: Fri Oct 16, 2009 8:53 am

Re: SOLUTION for gfsptiles/gfs not being available or too slow

Post by Antonix » Tue Aug 20, 2013 4:41 pm

same problem that has meteo60

meteoadriatic
Posts: 1512
Joined: Wed Aug 19, 2009 10:05 am

Re: SOLUTION for gfsptiles/gfs not being available or too slow

Post by meteoadriatic » Wed Aug 21, 2013 10:20 am

It works here on newest EMS without problem (click on image to zoom):

Image

I guess you didn't do something right but I can't guess what?

Ben Lankamp
Posts: 9
Joined: Thu Jun 17, 2010 1:18 pm

Re: SOLUTION for gfsptiles/gfs not being available or too slow

Post by Ben Lankamp » Thu Aug 22, 2013 6:09 pm

We use the get_gfs.pl Perl script from NCEP for downloading GFS boundary and initialization data from NCEP (bypassing nomads). We have two copies of the script: one for the GFS A-files and one for the B-files (with extended levels and variables). The $URL variable in the Perl scripts is set to

Code: Select all

http://www.ftp.ncep.noaa.gov/data/nccf/com/gfs/prod/gfs.$YYYY$MM$DD$HH/gfs.t${HH}z.pgrb2f${FHR}
...for the A-files and to...

Code: Select all

http://www.ftp.ncep.noaa.gov/data/nccf/com/gfs/prod/gfs.$YYYY$MM$DD$HH/gfs.t${HH}z.pgrb2bf${FHR}
for the B-files.

Then we combine the two files using WGRIB2 by using a foreach-loop and the -append option. In pseudo-code for a CSH script that takes the run as parameter (e.g. 'runmodel.csh 00' for 00 UTC run):

Code: Select all

set dt=`date +%Y%m%d`
set run=$1

perl get_gfs.pl data ${dt}${run} 6 78 3 all all /wrfems/runs/yourrun/grib
perl get_gfs_b.pl data ${dt}${run} 6 78 3 all all /wrfems/runs/yourrun/grib

cd /wrfems/runs/yourrun/grib

# Combine GRIB2 files
foreach file (/wrfems/runs/yourrun/grib/*pgrb2f*)
        set file = `basename $file`
        set tau = `echo $file | sed "s;gfs.t${run}z.pgrb2f;;g"`
        cp $file gfs.t${run}z.$tau.grb2
end

foreach file (/wrfems/runs/yourrun/grib/*pgrb2bf*)
        set file = `basename $file`
        set tau = `echo $file | sed "s;gfs.t${run}z.pgrb2bf;;g"`
        wgrib2 $file -append -grib gfs.t${run}z.$tau.grb2
end
This will get you gfs.tXXz.YY.grb2 files in the grib folder for your run, which should be around 90 MB if you download all elements and variables like us. You have to modify the gfs_gribinfo.conf file in /wrfems/conf/grib_info to get ems_prep to read these files:

Code: Select all

LOCFIL = gfs.tCCz.FF.grb2
Perhaps this will help someone. Like said, we download the entire GFS 0.5° output, just to be sure that the model gets everything it needs. You could use the optional parameters of the get_gfs Perl scripts to limit the variables you are downloading. With regard to speed: naturally it depends on the NCEP server, but the download usually takes 3-4 minutes for fetching all time steps necessary for a 72 hour forecast run, with GFS files every 3 hours. Combining the A- and B-files takes another minute or so.

Regards, Ben

meteoadriatic
Posts: 1512
Joined: Wed Aug 19, 2009 10:05 am

Re: SOLUTION for gfsptiles/gfs not being available or too slow

Post by meteoadriatic » Fri Aug 23, 2013 9:43 am

Hello,

thanks for sharing!

For WRF, there is no need to download B set, everything that WPS use, exist in A set (and much much more).

This is what WPS use to initialize WRF from GFS:

cat /usr1/wrfems/data/tables/vtables/Vtable.GFS
GRIB1| Level| From | To | metgrid | metgrid | metgrid |GRIB2|GRIB2|GRIB2|GRIB2|
Param| Type |Level1|Level2| Name | Units | Description |Discp|Catgy|Param|Level|
-----+------+------+------+----------+---------+-----------------------------------------+-----------------------+
11 | 100 | * | | TT | K | Temperature | 0 | 0 | 0 | 100 |
33 | 100 | * | | UU | m s-1 | U | 0 | 2 | 2 | 100 |
34 | 100 | * | | VV | m s-1 | V | 0 | 2 | 3 | 100 |
52 | 100 | * | | RH | % | Relative Humidity | 0 | 1 | 1 | 100 |
7 | 100 | * | | HGT | m | Height | 0 | 3 | 5 | 100 |
11 | 105 | 2 | | TT | K | Temperature at 2 m | 0 | 0 | 0 | 103 |
52 | 105 | 2 | | RH | % | Relative Humidity at 2 m | 0 | 1 | 1 | 103 |
33 | 105 | 10 | | UU | m s-1 | U at 10 m | 0 | 2 | 2 | 103 |
34 | 105 | 10 | | VV | m s-1 | V at 10 m | 0 | 2 | 3 | 103 |
1 | 1 | 0 | | PSFC | Pa | Surface Pressure | 0 | 3 | 0 | 1 |
2 | 102 | 0 | | PMSL | Pa | Sea-level Pressure | 0 | 3 | 1 | 101 |
144 | 112 | 0 | 10 | SM000010 | fraction| Soil Moist 0-10 cm below grn layer (Up) | 2 | 0 | 192 | 106 |
144 | 112 | 10 | 40 | SM010040 | fraction| Soil Moist 10-40 cm below grn layer | 2 | 0 | 192 | 106 |
144 | 112 | 40 | 100 | SM040100 | fraction| Soil Moist 40-100 cm below grn layer | 2 | 0 | 192 | 106 |
144 | 112 | 100 | 200 | SM100200 | fraction| Soil Moist 100-200 cm below gr layer | 2 | 0 | 192 | 106 |
144 | 112 | 10 | 200 | SM010200 | fraction| Soil Moist 10-200 cm below gr layer | 2 | 0 | 192 | 106 |
11 | 112 | 0 | 10 | ST000010 | K | T 0-10 cm below ground layer (Upper) | 0 | 0 | 0 | 106 |
11 | 112 | 10 | 40 | ST010040 | K | T 10-40 cm below ground layer (Upper) | 0 | 0 | 0 | 106 |
11 | 112 | 40 | 100 | ST040100 | K | T 40-100 cm below ground layer (Upper) | 0 | 0 | 0 | 106 |
11 | 112 | 100 | 200 | ST100200 | K | T 100-200 cm below ground layer (Bottom)| 0 | 0 | 0 | 106 |
11 | 112 | 10 | 200 | ST010200 | K | T 10-200 cm below ground layer (Bottom) | 0 | 0 | 0 | 106 |
91 | 1 | 0 | | SEAICE | proprtn | Ice flag | 10 | 2 | 0 | 1 |
81 | 1 | 0 | | LANDSEA | proprtn | Land/Sea flag (1=land, 0 or 2=sea) | 2 | 0 | 0 | 1 |
7 | 1 | 0 | | SOILHGT | m | Terrain field of source analysis | 0 | 3 | 5 | 1 |
11 | 1 | 0 | | SKINTEMP | K | Skin temperature (can use for SST also) | 0 | 0 | 0 | 1 |
65 | 1 | 0 | | SNOW | kg m-2 | Water equivalent snow depth | 0 | 1 | 13 | 1 |
| 1 | 0 | | SNOWH | m | Physical Snow Depth | 0 | 1 | | 1 |
-----+------+------+------+----------+---------+-----------------------------------------+-----------------------+

Ben Lankamp
Posts: 9
Joined: Thu Jun 17, 2010 1:18 pm

Re: SOLUTION for gfsptiles/gfs not being available or too slow

Post by Ben Lankamp » Fri Aug 23, 2013 1:08 pm

You are right about the variables available for running WPS/WRF, however the B-files contain additional levels of temperature, etc. (at x25 and x75 mb) that improve the initialization and forecast, when you specify 40+ levels in the model. We found a significant reduction in error of surface winds, precipitation and cloud cover.

Regards, Ben

meteoadriatic
Posts: 1512
Joined: Wed Aug 19, 2009 10:05 am

Re: SOLUTION for gfsptiles/gfs not being available or too slow

Post by meteoadriatic » Fri Aug 23, 2013 1:28 pm

Interesting! Then it would be great if inclusion of those fields is made for gfsptiles also. Maybe it would be good to suggest that change to Robert.

Also this is great for separate topic here. I will split your guide to the new topic.

meteoadriatic
Posts: 1512
Joined: Wed Aug 19, 2009 10:05 am

Re: SOLUTION for gfsptiles/gfs not being available or too slow

Post by meteoadriatic » Sat Sep 07, 2013 10:23 am

Ben,

it looks to me that all this downloading two sets and combining into one can be avoided by just using master files:
http://www.ftp.ncep.noaa.gov/data/nccf/ ... 00/master/

Is this the same as A+B set combined?

surgeon
Posts: 44
Joined: Wed Dec 08, 2010 1:34 pm
Location: Poland
Contact:

Re: SOLUTION for gfsptiles/gfs not being available or too slow

Post by surgeon » Mon Sep 09, 2013 10:37 am

meteoadriatic wrote:Ben,

it looks to me that all this downloading two sets and combining into one can be avoided by just using master files:
http://www.ftp.ncep.noaa.gov/data/nccf/ ... 00/master/

Is this the same as A+B set combined?
In mastergrib are 47 (wrf want num_metgrid_levels = 48 (101300?) levels, instead of 27, and processing them takes a little longer.

zdef 47 levels 100000 97500 95000 92500 90000 87500 85000 82500 80000 77500 75000 72500 70000 67500 65000 62500 60000 57500 55000 52500 50000 47500 45000 42500 40000 37500 35000 32500 30000 27500 25000 22500 20000 17500 15000 12500 10000 7000 5000 3000 2000 1000 700 500 300 200 100

Post Reply

Who is online

Users browsing this forum: No registered users and 4 guests