So, you've bought a new accelerator (maybe one of the dev XPhi on promo in Q4 2014?) or co-processor, or commissioned a cluster.
What's the first thing you want to do? Yes, benchmark every aspect to check it all works as you expect.
You would probably use the Intel MPI Benchmark suite to see how well your MPI implementation is working. There's also the HPCC Challenge benchmarks to determine how typical applications might run.