gribinfo.conf for NOMADS Grib Filter

Questions and other topics related to UEMS 15.
Post Reply
norulz
Posts: 92
Joined: Thu Mar 12, 2015 1:43 pm

gribinfo.conf for NOMADS Grib Filter

Post by norulz » Tue Mar 07, 2017 11:39 am

Hi all,

Is it possible to set up a gribinfo.conf file to use the NOMADS grib filter system? This would need to be set to a preset lon/lat and the required variables.

I would like to use such a possibility as a fallback on the ptiles system.

Thanks,
David

windyweek
Posts: 28
Joined: Thu Aug 21, 2014 8:46 am

Re: gribinfo.conf for NOMADS Grib Filter

Post by windyweek » Tue Mar 07, 2017 4:08 pm

It is truly possible. I actually use the NOMADS Grib filter instead of the EMS Ptile service all the time.

In gribinfo.conf, the most significant part is:

Code: Select all

SERVER-HTTP = NOMADS:/cgi-bin/filter_gfs_0p25.pl?file=gfs.tCCz.pgrb2.0p25.fFFF&all_lev=on&all_var=on&subregion=&dir=%2Fgfs.YYYYMMDDCC
You can even play around a bit and trim down the needed variables/levels even more. But even if you stay with the full levels and variables, for regional grids, the size of the file being downloaded is fairly modest (for example: 2.3 MB for 250x260 grid points @ 6 km resolution).

You also have to modify strc/EMSprep/Ptiles.pm a bit and add a simple if statement to deal with the custom ptile:

Code: Select all

    if ($ptile eq 'nomadsab') {

        $ptnav{nxmax} = 1440;  #  Number of NX grid points
        $ptnav{nymax} = 721;  #  Number of NY grid points

        $ptnav{bzpts} = 3;    #  Number of additional boundary points
        $ptnav{globl} = 1;    #  Is it a global data set (1|0)

        $ptnav{dsid}  = '0.25 degree NOMADS GFS';   #  ID used for information
        $ptnav{dset}  = 'nomadsab';   #  Data set name

        %{$ptnav{grnav}} = &Emaproj::map_set('LL',-90.,0.0,1.0,1.0,0.25,0.25,0.25,0.25,$ptnav{nxmax},$ptnav{nymax});

        $ptnav{mproj} = $pmap{LL};
    }
HTH,
Ivan

norulz
Posts: 92
Joined: Thu Mar 12, 2015 1:43 pm

Re: gribinfo.conf for NOMADS Grib Filter

Post by norulz » Tue Mar 07, 2017 4:46 pm

Hi Ivan,

Thanks for the details. I'll give it a try ASAP.

David

norulz
Posts: 92
Joined: Thu Mar 12, 2015 1:43 pm

Re: gribinfo.conf for NOMADS Grib Filter

Post by norulz » Wed Mar 08, 2017 9:50 am

Hi Ivan,

I have not been able to make it work. The Prep routines add a 'dset' parameter to the end of the URL and the NOMADS server rejects this with any value as an invalid parameter.

Did you make any other tweaks to the Ptiles.pm script?

windyweek
Posts: 28
Joined: Thu Aug 21, 2014 8:46 am

Re: gribinfo.conf for NOMADS Grib Filter

Post by windyweek » Wed Mar 08, 2017 12:11 pm

Hi David,

It looks like I had one more change in Ptiles.pm. Sorry I didn't notice that earlier.

Just after the line:

Code: Select all

my $request = "\"$https$script?$flist&$domain&$dset\"";
Put:

Code: Select all

if ($mod eq 'nomadsab') {
    $request = $domain;
}
So for the custom ptile ('nomadsab' in this case) I override the request variable leaving only the domain part.

Then further, you also need to customize Pacquire.pm. This is needed because EMS Ptile service uses a two-phase approach for acquiring ptiles: the needed files are only requested first (unique files/urls created on the EMS server) and in the second pass the actual data file downloaded. This of course, won't work with the NOMADS Grib filter and hence custom logic is needed. Fortunately it is not that difficult to implement that:

In Pacquire.pm locate the following if statement:

Code: Select all

if ($ptile) {

        my $request = &Ptiles::PTreqst($ptile,$host,%gfiles);

        return -3 unless $request;

        &Estyle::emsprint(0,12,96,1,2,"Initiating HTTP connection to $host");
and within its scope put the following snippet:

Code: Select all

if ($ptile eq 'nomadsab') {
	    for my $lfile (sort keys %gfiles) {
		my $script = $gfiles{$lfile};
		my $url_command = "\"http://$host$script&$request\"";
        	my $text = sprintf ("Attempting to acquire %s \n",$url_command);
        	&Estyle::emsprint(5,14,256,0,0,$text);

		# Retry downloading the file in case of 404 Not found.
		my $attempts = 0;
		while ($attempts < 10) {
			$attempts++;
			my $wget_status = system "wget -a $dlog -L --waitretry=5 --retry-connrefused --connect-timeout=30 --read-timeout=15 --tries=inf -O $lfile $url_command";
			# wget might return non-zero status in case of 404 Not found. Sleep for a while and retry.
			if ($wget_status != 0) {
			   sleep 180;
			   next;
			} else {
			   last;			
			}
		}
	   };
	   return 0;
	}
I have put that code together a while ago and at first I didn't grasp all the changes needed. Sorry for any confusion this may have caused. Let me know if you need further information.

Cheers,
Ivan
Last edited by windyweek on Wed Mar 08, 2017 4:02 pm, edited 1 time in total.

norulz
Posts: 92
Joined: Thu Mar 12, 2015 1:43 pm

Re: gribinfo.conf for NOMADS Grib Filter

Post by norulz » Wed Mar 08, 2017 12:44 pm

....mmmmmm

I'm giving it a try

Thanks

norulz
Posts: 92
Joined: Thu Mar 12, 2015 1:43 pm

Re: gribinfo.conf for NOMADS Grib Filter

Post by norulz » Wed Mar 08, 2017 1:11 pm

Ivan thanks a million!
It's working well.
David

Post Reply