cloud infrastructure

[julien1188]

Hello,

I would like to have your opinion about cloud infrastructure.

I tested OVH cloud a few month ago but the results are very slow. I think it is due to low speed between nodes and NFS. OVH support confirmed me my point of view... OVH suggest me to switch to HPC cluster but it does not suit for weather forecast every 6h. (Finally, I use SoYouStart server with SSD)

I looked by accident offers from "Amazon web service" (AWS) and "google cloud compute engine". I think it is expensive but which qualities? Someone he already tested this service?

I am convinced that physical server is always better but, sometimes, physical servers are limited with large zone and high resolution.

[meteoadriatic]

I might be wrong but I think that you won't ba happy with any network based cluster system today. It was good when desktop CPUs were one core, slow, when computers had low amount of RAM. Only way to get reasonable performance was to connect them to cluster with gigabit network. Network bandwidth and latency were acceptable for such slow computing nodes. Now when they do much more amount of dataflow, network became serious bottleneck. More powerful hardware = more loss due to network congestion.

How big domains you need to run (how much points)?

[julien1188]

my d03 are 115 * 115 (1km) with ARW

[meteoadriatic]

And other two domains are similar size? This should not be problem for single workstation. Of course, it depends on how fast you require it to finish...

[julien1188]

d01 : 30*30
d02 : 58*58
d03 : 110*110

max time calculation 50min... Currently, I use 4 CPUs 3.4Ghz from SoYouStart (OVH- server) with SSD

[meteoadriatic]

SSD gains you nothing, try to find computer with more CPU power and especially more memory bandwidth but with HDD instead to save some money.

[julien1188]

Following my test I have a gain with SSD when wrfout file are large. (5€ of difference between SSD and HDD...thanks OVH)

I will probably test big HG with 20 cores (2.5 GHz), 500 MBps

[meteoadriatic]

Following my test I have a gain with SSD when wrfout file are large. (5€ of difference between SSD and HDD...thanks OVH)

Yes of course the difference equals total difference of write time to disk.. for me this is too little to be important but if there is only 5€ difference in cost then might be worth of using, especially if you do lot of postprocessing...