cloud infrastructure

Looking for new hardware to run WRF? Intel or AMD? Check this forum.
Post Reply
julien1188
Posts: 36
Joined: Fri Mar 09, 2012 1:02 pm

cloud infrastructure

Post by julien1188 » Thu May 15, 2014 9:06 am

Hello,

I would like to have your opinion about cloud infrastructure.

I tested OVH cloud a few month ago but the results are very slow. I think it is due to low speed between nodes and NFS. OVH support confirmed me my point of view... OVH suggest me to switch to HPC cluster but it does not suit for weather forecast every 6h. (Finally, I use SoYouStart server with SSD)

I looked by accident offers from "Amazon web service" (AWS) and "google cloud compute engine". I think it is expensive but which qualities? Someone he already tested this service?

I am convinced that physical server is always better but, sometimes, physical servers are limited with large zone and high resolution.

meteoadriatic
Posts: 1510
Joined: Wed Aug 19, 2009 10:05 am

Re: cloud infrastructure

Post by meteoadriatic » Thu May 15, 2014 2:58 pm

I might be wrong but I think that you won't ba happy with any network based cluster system today. It was good when desktop CPUs were one core, slow, when computers had low amount of RAM. Only way to get reasonable performance was to connect them to cluster with gigabit network. Network bandwidth and latency were acceptable for such slow computing nodes. Now when they do much more amount of dataflow, network became serious bottleneck. More powerful hardware = more loss due to network congestion.

How big domains you need to run (how much points)?

julien1188
Posts: 36
Joined: Fri Mar 09, 2012 1:02 pm

Re: cloud infrastructure

Post by julien1188 » Fri May 16, 2014 9:58 am

my d03 are 115 * 115 (1km) with ARW

meteoadriatic
Posts: 1510
Joined: Wed Aug 19, 2009 10:05 am

Re: cloud infrastructure

Post by meteoadriatic » Fri May 16, 2014 12:13 pm

And other two domains are similar size? This should not be problem for single workstation. Of course, it depends on how fast you require it to finish...

julien1188
Posts: 36
Joined: Fri Mar 09, 2012 1:02 pm

Re: cloud infrastructure

Post by julien1188 » Fri May 16, 2014 12:20 pm

d01 : 30*30
d02 : 58*58
d03 : 110*110

max time calculation 50min... Currently, I use 4 CPUs 3.4Ghz from SoYouStart (OVH- server) with SSD

meteoadriatic
Posts: 1510
Joined: Wed Aug 19, 2009 10:05 am

Re: cloud infrastructure

Post by meteoadriatic » Fri May 16, 2014 12:42 pm

SSD gains you nothing, try to find computer with more CPU power and especially more memory bandwidth but with HDD instead to save some money.

julien1188
Posts: 36
Joined: Fri Mar 09, 2012 1:02 pm

Re: cloud infrastructure

Post by julien1188 » Fri May 16, 2014 12:49 pm

Following my test I have a gain with SSD when wrfout file are large. (5€ of difference between SSD and HDD...thanks OVH)

I will probably test big HG with 20 cores (2.5 GHz), 500 MBps

meteoadriatic
Posts: 1510
Joined: Wed Aug 19, 2009 10:05 am

Re: cloud infrastructure

Post by meteoadriatic » Fri May 16, 2014 1:22 pm

julien1188 wrote:Following my test I have a gain with SSD when wrfout file are large. (5€ of difference between SSD and HDD...thanks OVH)
Yes of course the difference equals total difference of write time to disk.. for me this is too little to be important but if there is only 5€ difference in cost then might be worth of using, especially if you do lot of postprocessing...

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest