I am going to build a new UEMS station.
I would like to ask you all for a help in making a decision which of the following CPUs would be more efficient.
Is it better to have less Cores with higher freq, or more cores would be better?
My experience is that the first priority is the number of physical cores 10 is much better than 6 even with a somewhat different cpu speed.
Second speed issue is often related to the RAM slot population and bios settings. The CPU to memory speed is important. The CPU caches also come into play here. For example in a HP DL360 G6 server there are 3 memory channels for each CPU (6 channels total). There is a dramatic increase in speed if only one memory card is placed in each channel. 6 x 2 2GB cards giving a total of 12GB is much faster than 8x2 giving 16GB as one channel has to deal with 2 cards.
Also the bios memory settings regarding Node Interlace should be disabled. It has a big affect on the run speed.
I found that CPU clock rating had less of an affect than the above issues.
After the pure hardware issues the main "hardware-to-software" adjustments that affect speed will be in run_ncpus.conf where the NODECPUS, DECOMP (in relation to domain size) and (less) NUMTILES all affect speed.
I'm using 12GB RAM in each box and find that this is enough. 3 x 2GB for each CPU i.e 1 memory card for each channel. I had a reduction in speed when I used 16GB as it required 2 cards in one of the memory channels which slowed things down.
Sure, let's do a comparison. We can use the benchmark case (27april2011). When I run --domains 1 --length 6h, I get 2 minutes and 54 seconds. This is for a cluster of two boxes and a total of 24 cpu physical cores.
david@hp03:~/uems/util/benchmark/27april2011$ ems_run --domains 1 --length 6h [109/109]
Starting UEMS Program ems_run (V15.89.3) on hp03 at Thu Feb 23 21:31:56 2017 UTC
I. Preparing your EMS Run experience
* You are running the WRF ARW core 18-19 August 2008 benchmark case - Always a wise decision
* Simulation start and end times:
Domain Start End
1 2011-04-27_06:00:00 2011-04-27_12:00:00
* Simulation length will be 6 hours
* A large timestep of 90 seconds will be used for this simulation
II. Creating the initial and boundary condition files for the user domain(s)
* The WRF REAL program shall be run on the following systems and processors:
12 processors on hp03 (10.0.0.4) with 2 tiles per processor
12 processors on hp04 (10.0.0.5) with 2 tiles per processor
* Testing viability of system for MPI execution - Success!
* MPICH to select the network interface on hp03 to be used for communication
* Creating WRF initial and boundary condition files
* Initial and boundary conditions created in 10 seconds
Moving on to bigger and better delusions of grandeur
III. Running ARW WRF while thinking happy thoughts
[66/109]
* The WRF ARW core shall be run on the following systems and processors:
12 processors on hp03 (10.0.0.4) with 2 tiles per processor
12 processors on hp04 (10.0.0.5) with 2 tiles per processor
* Run Output Frequency Primary wrfout Aux File 1
---------------------------------------------------
Domain 01 : 3 hours Off
* Testing viability of system for MPI execution - Success!
* MPICH to select the network interface on hp03 to be used for communication
* Runnning your simulation with enthusiasm!
You can sing along to the progress of the simulation while watching:
% tail -f /home/david/uems/util/benchmark/27april2011/rsl.out.0000
Unless you have something better to do with your time
* Hey, hey! Your simulation appears to have been successful, just like you!
* The simulation output files have been moved to the wrfprd directory
Basic System Information for hp03
System Date : Thu Feb 23 21:35:26 2017 UTC
System Hostname : hp03
System Address : 10.0.0.4
System OS : Linux
Linux Distribution : Ubuntu 16.04.2 LTS
OS Kernel : 4.4.0-59-generic
Kernel Type : x86_64
Processor and Memory Information for hp03
CPU Name : Intel(R) Xeon(R) CPU X5650 @ 2.67GHz
CPU Instructions : nehalem
CPU Type : 64-bit
CPU Speed : 1600 MHz
EMS Determined Processor Count
Physical CPUs : 2
Cores per CPU : 6
Total Processors : 12
Hyper-Threading : Off
System Memory : 11.7 Gbytes
EMS Release Information for hp03
EMS Release : 15.89.3,WRF3.7.1
EMS Binaries : x64
Basic System Information for hp04
System Date : Thu Feb 23 21:35:28 2017 UTC
System Hostname : hp04
System Address : 10.0.0.5
System OS : Linux
Linux Distribution : Ubuntu 16.04.2 LTS
OS Kernel : 4.4.0-59-generic
Kernel Type : x86_64
Processor and Memory Information for hp04
CPU Name : Intel(R) Xeon(R) CPU X5650 @ 2.67GHz
CPU Instructions : nehalem
CPU Type : 64-bit
CPU Speed : 1621.32 MHz
EMS Determined Processor Count
Physical CPUs : 2
Cores per CPU : 6
Total Processors : 12
Hyper-Threading : Off
System Memory : 11.7 Gbytes
EMS Release Information for hp04
EMS Release : 15.89.3,WRF3.7.1
EMS Binaries : x64
EMS ARW core benchmark simulation completed in 2 minutes 54 seconds
* Benchmark information is available in static/ems_benchmark.info
Simulation Accomplished! Now go interrogate the results.
Your awesome EMS Run party is complete - Thu Feb 23 21:35:28 2017 UTC
The very first message detected by SETI will be: "Think Globally, Model Locally!"