personal tile datasets - can't resolve proxy

Questions and other topics related to UEMS 18.
Post Reply
pattim
Posts: 199
Joined: Sun Jun 24, 2012 8:42 pm
Location: Los Angeles, CA, USA

personal tile datasets - can't resolve proxy

Post by pattim » Fri Mar 13, 2020 9:59 pm

I keep getting "unable to resolve proxy" errors whenever I try to use any of the personal tile datasets. Is this a known problem? uEMS works just fine on non-personal-tile datasets (v19.9) Perhaps there's some setup I missed, or the server DNS entries have changed?

Thank You Very Much,
Patricia

EDIT: Looking closer, it looks like the STRC servers (ems1/ems2/ems3) aren't fully online? Can anyone say anything about their status?

Code: Select all

   II.  Collecting the initialization datasets for your simulation

           *  Searching for GFS files to use as model initial and boundary conditions

              Initiating HTTP connection to ems3.comet.ucar.edu

                -> Checking if available : /data/grib/gfsp50/202003/20200314/grib.t00z/20031400.gfs.t00z.0p50.pgrb2f000 - Not Available (Could not resolve proxy)
                -> Checking if available : /data/grib/gfsp50/202003/20200314/grib.t00z/20031400.gfs.t00z.0p50.pgrb2f003 - Not Available (Could not resolve proxy)

              Initiating HTTP connection to ems1.comet.ucar.edu

                -> Checking if available : /data/grib/gfsp50/202003/20200314/grib.t00z/20031400.gfs.t00z.0p50.pgrb2f000 - Not Available (Could not resolve proxy)
                -> Checking if available : /data/grib/gfsp50/202003/20200314/grib.t00z/20031400.gfs.t00z.0p50.pgrb2f003 - Not Available (Could not resolve proxy)

              Initiating HTTP connection to ems2.comet.ucar.edu

                -> Checking if available : /data/grib/gfsp50/202003/20200314/grib.t00z/20031400.gfs.t00z.0p50.pgrb2f000 - Not Available (Could not resolve proxy)
                -> Checking if available : /data/grib/gfsp50/202003/20200314/grib.t00z/20031400.gfs.t00z.0p50.pgrb2f003 - Not Available (Could not resolve proxy)

              Initiating FTP connection to ftp.ncep.noaa.gov

                -> Acquiring /pub/data/nccf/com/gfs/prod/gfs.20200314/00/gfs.t00z.pgrb2.0p50.f000 - Success (9.15 MB/s)

                -> Acquiring /pub/data/nccf/com/gfs/prod/gfs.20200314/00/gfs.t00z.pgrb2.0p50.f003 - Success (9.60 MB/s)


         Excellent - Your master plan is working!

pattim
Posts: 199
Joined: Sun Jun 24, 2012 8:42 pm
Location: Los Angeles, CA, USA

Re: personal tile datasets - can't resolve proxy

Post by pattim » Wed Mar 25, 2020 12:39 am

Anyone?

Can someone at least verify for me that the ems1/ems2/ems3 servers are working? (either with ems_install.pl or by downloading grib data from there)

I found out can't install/update using ems_install.pl either. I don't have any proxy on my end and I know my internet connection is working. (At this point I feel it's an unusual bug somehow in the Perl scripts.)

Thank You,
Patricia

michaelwx
Posts: 10
Joined: Mon Nov 18, 2019 5:29 am

Re: personal tile datasets - can't resolve proxy

Post by michaelwx » Tue Mar 31, 2020 3:21 am

"Interesting" issue, but not much fun. Have you read through the perl scripts regarding downloading the required files? Is there some "inner-secret-hidden" switch in your network or your router?

I admit it's a space shot...

Michael
Michael Allen
Industrial Weather

michaelwx
Posts: 10
Joined: Mon Nov 18, 2019 5:29 am

Re: personal tile datasets - can't resolve proxy

Post by michaelwx » Tue Mar 31, 2020 3:23 am

Another avenue: have you pinged the server(s) to find if they're online or if the router path to them is blocked in some strange way between your machine and the server?

Michael
Michael Allen
Industrial Weather

pattim
Posts: 199
Joined: Sun Jun 24, 2012 8:42 pm
Location: Los Angeles, CA, USA

Re: personal tile datasets - can't resolve proxy

Post by pattim » Wed Apr 08, 2020 10:25 pm

Thank you very much for the reply! I take it that you *are* able to use the install script and access the ems1/ems2/ems3 data machines? If that is so, can you make sure of these let me know? And, if so, it must definitely be a problem on my end. (It's odd because I haven't changed anything, but I've learned with linux there are things that can get borked if you're not careful.)

So let me know? Sorry it took so long to get back to you - I'd sort of given up waiting for a reply.

Best, and looking forward to your reply
PattiM :) :) :)

PS: I think there could still be a problem if the EMS servers are behind a firewall oddly setup somehow (I mean, that error is from the EMS script).

meteosciez
Posts: 55
Joined: Sat Sep 05, 2009 6:36 am

Re: personal tile datasets - can't resolve proxy

Post by meteosciez » Thu Apr 09, 2020 10:28 am

I had a few months ago a similar error in a new virtual machine running UEMS.
It appeared that an empty HTTP_PROXY environment variable was created, and therefore the http_proxy(lowercase) environment variable was set to "http://". This was causing the proxy error.
In the file uems/etc/EMS.profile there is, at line 433 a section concerning the HTTP_PROXY. I commented out the lines as shown below and the proxy error disappeared.

Code: Select all

#   Additions to HTTP_PROXY
#
#   if [ -n $HTTP_PROXY ] ; then
#       http_proxy=http://$HTTP_PROXY ; export http_proxy
#       unset HTTP_PROXY
#   fi

In case you are using the csh/tcsh shell, you have to modify the file uems/etc/EMS.cshrc near the line 460 :

Code: Select all

#   Additions to HTTP_PROXY
#
#    if ($?HTTP_PROXY) then
#        set tmp = `echo $HTTP_PROXY | wc -w`
#        if ($tmp) then
#            setenv http_proxy http://$HTTP_PROXY
#        endif
#        unsetenv  HTTP_PROXY
#    endif
meteo-sciez.fr

pattim
Posts: 199
Joined: Sun Jun 24, 2012 8:42 pm
Location: Los Angeles, CA, USA

Re: personal tile datasets - can't resolve proxy

Post by pattim » Thu Apr 09, 2020 8:09 pm

Thanks, Meteo!!! I will give that a try. I did check my linux control panel and there was no HTTP_PROXY defined, but I didn't check deeply into linux (/etc and other places).

It didn't work - I am using bash and commented out the indicated lines
Connection to ems1.comet.ucar.edu failed (failed to resolve proxy)

I'm also confused that this ONLY occurs (1) when EMS tries to use the ems1/ems2/ems3 servers for forecast datasets, and (2) when the ems_install script tries to access info for installing ems. It accesses all the other forecast datasets correctly. I am racking my brain - why is there a "proxy" issue involved only with ems1/ems2/ems3. I am thinking the actual issue is unrelated to a proxy problem, but is an error mistakenly thrown by some other problem (a proxy is strictly on my LAN, and thus should affect all forecast data sources used by EMS script structure).

Can someone verify that they can properly access ems1/ems2/ems3 (this can be done by running the install script or trying a run with a "personal tile" dataset)?

Thank You!

pattim
Posts: 199
Joined: Sun Jun 24, 2012 8:42 pm
Location: Los Angeles, CA, USA

Re: personal tile datasets - can't resolve proxy

Post by pattim » Thu Apr 09, 2020 9:00 pm

Hmmmm... is that code for proxy permanently set? I found this on
a different machine - one I don't run EMS upon often:

Code: Select all

~> echo $HTTP_PROXY
~> echo $HTTPS_PROXY
~> echo $FTP_PROXY
~> echo $ALL_PROXY
~> echo $all_proxy
~> echo $http_proxy
http://
~> echo $https_proxy
~> echo $ftp_proxy
EDIT: OK, that fixed it (tested with just the EMS installer). Maybe I misunderstood your instructions. Just after where the EMS.profile is sourced in my .bashrc, I added the line export http_proxy="" So many ideas were tumbling around in my head on how I might approach this - it was your info about EMS creating a system variable that really opened the door to this one.

THANK YOU SO MUCH!!!

Post Reply