HPCC Application Note

Step 1 – Overview

This guide is intended to help current HPCC users get better benchmark performance by utilizing Intel® Math Kernel Library (Intel® MKL).

HPCC stands for High Performance Computing Challenge benchmark and is actually a suite of benchmarks that measure performance of the CPU, memory subsystem and interconnect. It consists of 7 benchmark tests – HPL (High Performance LINPACK), DGEMM (Double-precision GEneral Matrix-Matrix multiply), STREAM, PTRANS (Parallel TRANSpose, Random Access, FFT (Fast Fourier Tranform and communication bandwidth/latency.

Please find more information on HPCC from: http://icl.cs.utk.edu/hpcc/* .

Version Information

This application note was created to help users who benchmark clusters using HPCC to make use of the latest versions of Intel MKL on Linux platforms on Xeon systems. Specifically we'll address Intel MKL version 9.1.

Step 2 – Downloading HPCC Source Code

The HPCC source code can be downloaded from: http://icl.cs.utk.edu/hpcc/software/index.html*.

Prerequisites

1. Intel MKL contains highly optimized FFT and also the wrappers for FFTW, which can be obtained from the following options:
   • Download a FREE evaluation version of the Intel MKL product.
   • Download the FREE non-commercial* version of the Intel MKL product.
   All of these can be obtained at: http://www.intel.com/software/products/mkl.

2. Intel MPI can be obtained from http://www.intel.com/software/products/cluster.
   Open source MPI (MPICH2) can be obtained from http://www-unix.mcs.anl.gov/mpi/mpich/*.

3. Download modified MPI FFTW wrapper interfaces with 64bit long for the MKL DFT.
   

   We have changed a few lines from 32-bit to 64 bit parameters in these modified wrappers, since MKL match the FFTW interfaces completely, given some 32-bit parameters in FFTW, the wrappers without modification will not work for the HPCC-FFT component.

Step 3 - Configuration

Use the following commands to extract the HPCC tar files from the downloaded hpcc-x.x.x.tar.gz and fftw2x_cdft.tar.gz files:

$gunzip hpcc-x.x.x.tar.gz
$tar –xvf hpcc-x.x.x.tar This will create a directory hpcc-x.x.x Extract fftw2x_cdft.tar.gz$gunzip fftw2x_cdft.tar.gz
$tar –xvf fftw2x_cdft.tar

Make sure that MPI, C++ and FORTRAN compilers are installed and they are in PATH. Also set LD_LIBRARY_PATH to your compiler (C++ and FORTRAN), MPI, and MKL libraries.

Step 4 – Building HPCC

1. Build MPI MKL FFTW library.

   From the fftw2x_cdft directory, run the following command:

   $make libem64t mpi=intel3 comp=intel PRECISION=DOUBLE

   Here we are building for EM64t architecture with Intel MPI version 3.0, with Intel compilers and DOUBLE precision. This will create the MKL FFTW interface library libfftw2x_cdft_DOUBLE.a in lib/em64t directory.

   Note: Please execute $make to see the different options.

2. Build FFTW C wrapper library

   Change the directory to <your mkl installation>/interfaces/fftw2xc, and run the command as below

   $make libem64t PRECISION=MKL_DOUBLE

   This will create libfftw2xc_intel.a library in <your mkl installation>/lib/em64t directory

3. Build HPCC

   Change directory to hpcc-x.x.x/hpl

   Create a Makefile from the existing one, for e.g. Make.mkl. You can reuse one from the hpl/setup directory.

   Edit Make.mkl as follows: modify the LAdir, LAlib lines as below to point to MKL libraries. Assuming you have the double precision MPI fftw2x_cdft wrapper library built in $HOME/fftw2x_cdft/lib/em64t directory and you have installed 9.1.021 MKL version.

   LAdir = /opt/intel/cmkl/9.1.021

   LAlib = $(LAdir)/libmkl_em64t.a $(HOME)/fftw2x_cdft/lib/libfftw2x_cdft_DOUBLE.a $(LAdir)/libfftw2xc_intel.a $(LAdir)/libmkl_blacs.a $(LAdir)/libmkl_cdft.a $(LAdir)/libguide.a –lpthread –lm

   Build HPCC by using

   $make all arch=mkl

   This will create an executable with name hpcc in the hpcc-x.x.x directory and a file _hpccinf.txt which is a template input file for hpcc. Rename the file to hpccinf.txt.

Step 5 - Running HPCC

Modify the configuration parameters in hpccinf.txt file.

Run hpcc by executing the following command.

$mpirun –np 4 hpcc

hpccinf.txt is the same as standard hpl input file with a few additional lines. Please refer our HPL application note on tuning parameters in the configuration file.

Appendix A - Performance Results

Below are the hpcc benchmark results of Intel Atlantis cluster which can also be found in hpcc website*.

HPC Challenge Benchmark Record

Appendix B - Known Issues and Limitations

Appendix C – References

• Intel® Cluster Tools
• Intel Xeon Processor- and Itanium 2-based Servers Homepage

This applies to:

Intel® Math Kernel Library (Intel® MKL) Intel® Math Kernel Library (Intel® MKL) for Linux*