EMS Servers

Questions and other topics related to UEMS 18.
rojo79
Posts: 9
Joined: Fri Mar 27, 2020 7:46 am

EMS Servers

Post by rojo79 »

Hi

Is anyone else having issues with pulling data from the EMS servers? I cannot get any gribs downloaded using ems_prep today, yet browsing the EMS servers through a web browser shows the data is there...all download attempts just fail. Even running the install script uems_install.pl complains that there are no releases on the server. Confused.
nemzamp
Posts: 48
Joined: Thu Jul 19, 2012 11:42 am

Re: EMS Servers

Post by nemzamp »

equal to me ... solutions? the links have changed?
rojo79
Posts: 9
Joined: Fri Mar 27, 2020 7:46 am

Re: EMS Servers

Post by rojo79 »

I don't know for sure, perhaps there is something wrong with the EMS personal tile server? I can use gfsp25 or gfsp50 dsets without too much trouble. I did have to modify these slightly to reflect the path changes on the servers though.
rojo79
Posts: 9
Joined: Fri Mar 27, 2020 7:46 am

Re: EMS Servers

Post by rojo79 »

Well that didn't work. Still not working this morning :(
norulz
Posts: 111
Joined: Thu Mar 12, 2015 1:43 pm

Re: EMS Servers

Post by norulz »

I communicated with Robert on this yesterday. They have changed from http to https and apparently curl has a problem with this.
Robert sent me a patch that works with Ubuntu and UEMS version 19.8.1.

HOWEVER, although this fix worked for version 19.8.1, I'm having problems with version 21.1.2 on CentOS 8 machines and the patch does not help in this case.
norulz
Posts: 111
Joined: Thu Mar 12, 2015 1:43 pm

Re: EMS Servers

Post by norulz »

I've worked out a fix for version 21.1.2 based on Robert's fix for version 19.8 that is now working for me on 21.1.2.

Edit the file uems/strc/Uprep/Pacqure.pm after backing it up.

change line 608 by adding the -L switch for curl (follow redirects):

Code: Select all

        my $cmd = $curl ? "curl -f -L -o $logout -k --connect-timeout 3 -sI http://$host$remfile"
                        : "wget -o $logout --no-check-certificate -T 3 -t 1 --spider http://$host$remfile";
do the same for lines 841 and line 1307.

Then edit the file uems/strc/Uprep/Pconf.pm after backing it up.
Change the conditional starting on line 302 to look like this:

Code: Select all

        #----------------------------------------------------------------------------------
        #  If this is a personal tile dataset set HTTP method to PTILE and delete the
        #  HTTP hash entries.
        #----------------------------------------------------------------------------------
        #
        if ($gribinfo{ptile}) {
            %{$gribinfo{sources}{HTTP}}  = () unless defined $gribinfo{sources}{HTTP};
            %{$gribinfo{sources}{HTTPS}} = () unless defined $gribinfo{sources}{HTTPS};
            %{$gribinfo{sources}{PTILE}} = (%{$gribinfo{sources}{HTTP}},%{$gribinfo{sources}{HTTPS}});
            %{$gribinfo{sources}{HTTP}}  = ();
            %{$gribinfo{sources}{HTTPS}} = ();
        }
I hope it helps out.
David
nemzamp
Posts: 48
Joined: Thu Jul 19, 2012 11:42 am

Re: EMS Servers

Post by nemzamp »

It works... thanks
norulz
Posts: 111
Joined: Thu Mar 12, 2015 1:43 pm

Re: EMS Servers

Post by norulz »

I found that more calls to curl need to be modified in Pacqure:

Code: Select all

	
	Line 608:         my $cmd = $curl ? "curl -f -L -o $logout -k --connect-timeout 3 -sI http://$host$remfile"
	Line 688:         $cmd = $curl ? "curl -s -L -f -k --connect-timeout 30 --max-time 1200  http://$host$remfile -o $gfile"
	Line 841:         my $cmd = $curl ? "curl -f -L -o $logout -k --connect-timeout 3 -sI https://$host$remfile"
	Line 921:         $cmd = $curl ? "curl -s -L -f -k --connect-timeout 30 --max-time 1200  https://$host$remfile -o $gfile"
	Line 1069:     my $cmd = $curl ? "curl -s -L -f --no-buffer --connect-timeout 30 --max-time 1200 $pturl -o $logout"
	Line 1159:             my $cmd = $curl ? "curl -s -L -f --no-buffer --connect-timeout 30 --max-time 1200  $url/$file -o $lfile"
	Line 1307:         my $cmd = $curl ? "curl -f -L -o $logout --connect-timeout 3 -sI ftp://$host$remfile" 
	Line 1386:         $cmd = $curl ? "curl -s -L -f --connect-timeout 30 --max-time 1200  ftp://$host$remfile -o $gfile"
	Line 1647:                          : $command."curl -s -L -f --connect-timeout 30 --max-time 1200  http://$host$remfile -o $gfile >> $dlog1 2>&1\n";
	Line 1880:                          : $command."curl -s -L -f -k --connect-timeout 30 --max-time 1200  https://$host$remfile -o $gfile >> $dlog1 2>&1\n";
	Line 2101:     my $rcmd = $curl ? "curl -s -L -f --no-buffer --connect-timeout 30 --max-time 1200  $pturl > $rlog1 2>&1"
	Line 2231:                           : $dcmd."curl -s -L -f --no-buffer --connect-timeout 30 --max-time 1200  $url/$file -o $lfile  > $dlog1 2>&1\n";
	Line 2444:         $command = $curl ? $command."curl -s -L -f --connect-timeout 30 --max-time 1200  ftp://$host$remfile -o $gfile >> $dlog1 2>&1\n"
rojo79
Posts: 9
Joined: Fri Mar 27, 2020 7:46 am

Re: EMS Servers

Post by rojo79 »

Working on my CentOS 7.9 and Uems 21.1.2 install :)

Thank you
dominic
Posts: 347
Joined: Thu Jun 14, 2012 7:19 am

Re: EMS Servers

Post by dominic »

Hi all, this is the fix to solve this problem. Unzip and load it into the strc folder. Everything works

https://soostrc.comet.ucar.edu/strc/uem ... tpsfix.tgz
Post Reply