EMS Servers

Questions and other topics related to UEMS 18.
lpajurek
Posts: 5
Joined: Tue Dec 01, 2015 8:30 am

Re: EMS Servers

Post by lpajurek »

Could you please upload the fix again? it looks like its been deleted from the webpage.
norulz
Posts: 111
Joined: Thu Mar 12, 2015 1:43 pm

Re: EMS Servers

Post by norulz »

The fix has been incorporated into the latest version. 21.3.1.
pattim
Posts: 204
Joined: Sun Jun 24, 2012 8:42 pm
Location: Los Angeles, CA, USA

Re: EMS Servers

Post by pattim »

Does anyone know if that fixes the "personal tile" services? I'm running v21.1.2 and I can use NAM, RAP, GFS*, etc., but not the personal tile equivalent locations (NAMPT, etc.). I know that the ems servers are running, but ems_run returns "Internal Server Error" when trying to use personal tiles.

It looks like I need to try upgrading... I'll let you know...

EDIT: OK, now NAMPT is working again!

Thanks!!
lpajurek
Posts: 5
Joined: Tue Dec 01, 2015 8:30 am

Re: EMS Servers

Post by lpajurek »

IT looks strange, but it doesn't work for me - I have a proxy with login in my system.
The best part is that after adding alias for curl with proxy address, I am able to go to the https://ems1.comet.ucar.edu but uems ems_prep application cannot load the data.

This is what I have while trying to get gfsp25pt:

II. Collecting the initialization datasets for your simulation

* Searching for GFSP25PT personal tiles to use as model initial and boundary conditions

* Areal coverage of your 0.25 degree GFS personal tile

Corner Lat-Lon points of the domain:

55.75, 12.25 55.75, 26.00
* *

* 52.00, 19.14

* *
48.25, 12.25 48.25, 26.00


* Sending personal tile request to https://ems1.comet.ucar.edu - Failed (Network Failure or Timeout (4))


Any ideas what is wrong?
I have made an update to latest version of uems from quite old installation.
norulz
Posts: 111
Joined: Thu Mar 12, 2015 1:43 pm

Re: EMS Servers

Post by norulz »

I had no problems with the ptile server today on multiple runs with several machines.

I can suggest checking that uems/strc/Uprep/Paquire.pm is dated 2021-03-19.

Your description suggests that it may be an earlier version that is actually making the call to the ptile server.

David
norulz
Posts: 111
Joined: Thu Mar 12, 2015 1:43 pm

Re: EMS Servers

Post by norulz »

Another possibility for problems is a very old version of 'curl' ( ~2003 ) where there were problems following redirects with the -L option.
lpajurek
Posts: 5
Joined: Tue Dec 01, 2015 8:30 am

Re: EMS Servers

Post by lpajurek »

PAcquire looks fine, the date is ok.
I updated curl

curl -V
curl 7.64.0 (x86_64-pc-linux-gnu) libcurl/7.64.0 OpenSSL/1.1.1d zlib/1.2.11 libidn2/2.0.5 libpsl/0.20.2 (+libidn2/2.0.5) libssh2/1.8.0 nghttp2/1.36.0 librtmp/2.3
Release-Date: 2019-02-06

But ems_prep still cannot see the data:


ems_prep --dset gfsp50 --length 06 --cycle 00


Starting UEMS program ems_prep (V21.3.1) on ultron at Tue Mar 23 09:27:47 2021 UTC


I. Initialization Summary

* Initialization dataset information:
----------------------------------------------------------------------------------
Simulation Start Time | Tue Mar 23 00:00:00 2021 UTC
Simulation End Time | Tue Mar 23 06:00:00 2021 UTC
Simulation Length | 6 Hours

Initial Condition Dataset | (Forecast dataset)
GFSP50 Cycle Date | Tue Mar 23 00:00:00 2021 UTC
GFSP50 Verif Date | Tue Mar 23 00:00:00 2021 UTC (0 Hour Forecast)

Boundary Condition Frequency | 180 Minutes

Additional Surface Datasets | None
Alt Lake Temp Initialization | No (Simulation Length < 24 hours)

Additional Land Surface Data | None

Aerosol Climatology Dataset | QNWFA_QNIFA_SIGMA_MONTHLY.dat



II. Collecting the initialization datasets for your simulation

* Searching for GFSP50 files to use as model initial and boundary conditions

Initiating FTP connection to ftp.ncep.noaa.gov

-> Checking if available : /data/nccf/com/gfs/prod/gfs.20210323/00/atmos/gfs.t00z.pgrb2.0p50.f000 - Not Available
-> Checking if available : /data/nccf/com/gfs/prod/gfs.20210323/00/atmos/gfs.t00z.pgrb2.0p50.f003 - Not Available
-> Checking if available : /data/nccf/com/gfs/prod/gfs.20210323/00/atmos/gfs.t00z.pgrb2.0p50.f006 - Not Available

Initiating HTTPS connection to https://ems3.comet.ucar.edu

-> Checking if available : /data/grib/gfsp50/202103/20210323/grib.t00z/21032300.gfs.t00z.0p50.pgrb2f000 - Not Available
-> Checking if available : /data/grib/gfsp50/202103/20210323/grib.t00z/21032300.gfs.t00z.0p50.pgrb2f003 - Not Available
-> Checking if available : /data/grib/gfsp50/202103/20210323/grib.t00z/21032300.gfs.t00z.0p50.pgrb2f006 - Not Available

Initiating HTTPS connection to https://ems2.comet.ucar.edu

-> Checking if available : /data/grib/gfsp50/202103/20210323/grib.t00z/21032300.gfs.t00z.0p50.pgrb2f000 - Not Available
-> Checking if available : /data/grib/gfsp50/202103/20210323/grib.t00z/21032300.gfs.t00z.0p50.pgrb2f003 - Not Available
-> Checking if available : /data/grib/gfsp50/202103/20210323/grib.t00z/21032300.gfs.t00z.0p50.pgrb2f006 - Not Available

Initiating HTTPS connection to https://ems1.comet.ucar.edu

-> Checking if available : /data/grib/gfsp50/202103/20210323/grib.t00z/21032300.gfs.t00z.0p50.pgrb2f000 - Not Available
-> Checking if available : /data/grib/gfsp50/202103/20210323/grib.t00z/21032300.gfs.t00z.0p50.pgrb2f003 - Not Available
-> Checking if available : /data/grib/gfsp50/202103/20210323/grib.t00z/21032300.gfs.t00z.0p50.pgrb2f006 - Not Available

Initiating HTTPS connection to nomads.ncep.noaa.gov

-> Checking if available : /pub/data/nccf/com/gfs/prod/gfs.20210323/00/atmos/gfs.t00z.pgrb2.0p50.f000 - Not Available
-> Checking if available : /pub/data/nccf/com/gfs/prod/gfs.20210323/00/atmos/gfs.t00z.pgrb2.0p50.f003 - Not Available
-> Checking if available : /pub/data/nccf/com/gfs/prod/gfs.20210323/00/atmos/gfs.t00z.pgrb2.0p50.f006 - Not Available

Initiating HTTPS connection to ftpprd.ncep.noaa.gov

-> Checking if available : /data/nccf/com/gfs/prod/gfs.20210323/00/atmos/gfs.t00z.pgrb2.0p50.f000 - Not Available
-> Checking if available : /data/nccf/com/gfs/prod/gfs.20210323/00/atmos/gfs.t00z.pgrb2.0p50.f003 - Not Available
-> Checking if available : /data/nccf/com/gfs/prod/gfs.20210323/00/atmos/gfs.t00z.pgrb2.0p50.f006 - Not Available

Sorry your Royal Awesomeness, but none of the initialization files are available.


Your UEMS Pre-processing rave was busted at Tue Mar 23 09:27:51 2021 UTC - Ya know, 'cause stuff just happens

As the UEMS Genie says: "Think Globally, Model Locally!"


I tried with personal tiles:

ms_prep --dset gfsp50pt --length 06 --cycle 00


Starting UEMS program ems_prep (V21.3.1) on ultron at Tue Mar 23 09:35:20 2021 UTC


I. Initialization Summary

* Initialization dataset information:
----------------------------------------------------------------------------------
Simulation Start Time | Tue Mar 23 00:00:00 2021 UTC
Simulation End Time | Tue Mar 23 06:00:00 2021 UTC
Simulation Length | 6 Hours

Initial Condition Dataset | GFSP50PT Personal Tile Dataset (Forecast dataset)
GFSP50PT Cycle Date | Tue Mar 23 00:00:00 2021 UTC
GFSP50PT Verif Date | Tue Mar 23 00:00:00 2021 UTC (0 Hour Forecast)

Boundary Condition Frequency | 180 Minutes

Additional Surface Datasets | None
Alt Lake Temp Initialization | No (Simulation Length < 24 hours)

Additional Land Surface Data | None

Aerosol Climatology Dataset | QNWFA_QNIFA_SIGMA_MONTHLY.dat



II. Collecting the initialization datasets for your simulation

* Searching for GFSP50PT personal tiles to use as model initial and boundary conditions

* Areal coverage of your 0.5 degree GFS personal tile

Corner Lat-Lon points of the domain:

56.00, 12.00 56.00, 26.50
* *

* 52.00, 19.14

* *
48.00, 12.00 48.00, 26.50


* Sending personal tile request to https://ems3.comet.ucar.edu - Failed (SSL handshaking failed (35))

* Areal coverage of your 0.5 degree GFS personal tile

Corner Lat-Lon points of the domain:

56.00, 12.00 56.00, 26.50
* *

* 52.00, 19.14

* *
48.00, 12.00 48.00, 26.50


* Sending personal tile request to https://ems2.comet.ucar.edu - Failed (SSL handshaking failed (35))

* Areal coverage of your 0.5 degree GFS personal tile

Corner Lat-Lon points of the domain:

56.00, 12.00 56.00, 26.50
* *

* 52.00, 19.14

* *
48.00, 12.00 48.00, 26.50


* Sending personal tile request to https://ems1.comet.ucar.edu - Failed (SSL handshaking failed (35))

Unfortunately, some of your initialization files are unavailable, but at least you still look good!


Your UEMS Pre-processing rave was busted at Tue Mar 23 09:35:21 2021 UTC - Ya know, 'cause stuff just happens

As the UEMS Genie says: "Think Globally, Model Locally!"



I tried to dwonload it directly with curl:

curl https://ems3.comet.ucar.edu/data/grib/g ... .pgrb2f000 -o test
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
7 143M 7 11.0M 0 0 413k 0 0:05:54 0:00:27 0:05:27 395k^C


and it is possible. I will check if my network is fine too.
norulz
Posts: 111
Joined: Thu Mar 12, 2015 1:43 pm

Re: EMS Servers

Post by norulz »

It appears that the communication link to the ptile and other servers is OK

I might have suggested slow population of the servers today but I ran several runs at 0430Z today and found all 00Z files I needed.
I tried your command line request at this time for --cycle 00 and got the files OK
dominic
Posts: 347
Joined: Thu Jun 14, 2012 7:19 am

Re: EMS Servers

Post by dominic »

lpajurek wrote: Mon Mar 22, 2021 12:11 pm Could you please upload the fix again? it looks like its been deleted from the webpage.
https://www.mediafire.com/file/mxtlfjyd ... x.tgz/file
lpajurek
Posts: 5
Joined: Tue Dec 01, 2015 8:30 am

Re: EMS Servers

Post by lpajurek »

Thank you for the link.
I checked everything and it looks like now curl allows download with proxy but there is something else - gribinfo configuration.
I run uems with gfs data downloaded by my own script and it is fine.
But I looked into the gribinfo and I compared it with messages from uems when running with it with ems_prep for personal tiles and etc.
I found that my gribinfo configuration has different names of the grib files on the servers and I really don't know why :)
Now this is my final issue which I think I resolve fast but I could use some help from properly configured gribinfo files just to check/compare.
Post Reply