FP64-capable nodes for simulation, genomics and computational chemistry.
Consumer cards are effectively useless for FP64 work - the RTX 4090 runs double precision at 1/64 of its single-precision rate. The datacentre cards do not have that restriction, which makes the A100 and H100 the only sensible choices for anything that needs numerical accuracy over throughput.
MPI over InfiniBand is available on all SXM nodes. The image ships OpenMPI 4.1 with UCX configured for the fabric, so a standard mpirun invocation works without tuning.