Build and install instructions

Introduction

Likwid is build using GNU make and Perl. Besides the Linux kernel and the standard C library, all required dependencies are shipped with the archive (Lua and hwloc). It should build on any Linux distribution with a recent GCC compiler or CLANG compiler and 2.6 or newer kernel without any changes.

There is one generic top level Makefile and one .mk configuration file for each compiler (at the moment GCC, CLANG and ICC). Please note that we test LIKWID only with GCC. CLANG and ICC is only tested for basic functionality.

There is one exception: If you want to use LIKWID on a Intel Xeon Phi card you have to choose the MIC as compiler in config.mk, which is based on Intel ICC compiler.

Directory structure

All source files are in the src/ directory. All header files are located in src/includes/ . Lua application source files are in src/applications/. All external tools, namely HWLOC and Lua, are located in ext/. The bench/ folder contains all files of the benchmarking suite of LIKWID.

All build products are generated in the directory ./TAG, where TAG is the compiler configuration, default ./GCC.

Configuration

Usually the only thing you have to configure is the PREFIX install path in the build config file config.mk in the top directory.

Changing color of likwid-pin output

Depending on the background of your terminal window you can choose a color for likwid-pin output.

Usage of the access daemon likwid-accessD

Usually on your own system, you can use LIKWID with direct access to the MSR files. If you install LIKWID on a shared system as a HPC compute cluster you may consider to use the access daemon. This is a proxy application which was implemented with security in mind and performs address checks for allowed access. Using the access daemon, the measurements involve more overhead, especially if you use likwid-perfctr in timeline mode or with the marker API.

To enable using the access daemon, configure in config.mk:

• Set BUILDDAEMON to true
• Configure the path to the accessDaemon binary at ACCESSDAEMON
• Set the ACCESSMODE to accessdaemon

ACCESSMODE can be direct, accessdaemon and sysdaemon (not yet officially supported). You can overwrite the default setting on the command line using the -M switch.

If you want to access Uncore performance counters that are located in the PCI memory range, like they are implemented in Intel SandyBridge EP and IvyBridge EP, you have to use the access daemon or have root privileges because access to the PCI space is only permitted for highly privileged users.

Usage of frequency daemon likwid-setFreq

The application likwid-setFrequencies uses another daemon to modify the frequency of CPUs. The daemon is build and later installed if BUILDFREQ is set to true in config.mk.

Build Likwid as shared library

Per default the LIKWID library is build as a shared library. You need the library if you want to use the Marker API. You can also use the LIKWID modules like perfmon directly. This is still not officially supported at the moment. In some settings it is necessary to build LIKWID as a shared library. To do so set SHARED_LIBRARY to true.

Instrument likwid-bench for usage with likwid-perfctr

likwid-bench is instrumented for use with likwid-perfctr. This allows you to measure various metrics of your likwid-bench kernels. Enable instrumentation by setting INSTRUMENT_BENCH to true in config.mk.

Enabling Fortran interface for marker API

If you want to use the Marker API in Fortran programs LIKWID offers a native Fortran90 interface. To enable it set FORTRAN_INTERFACE to true in config.mk.

Build targets

You have to edit config.mk to configure your build and install path.

The following make targets are available:

• make - Build everything
• make likwid-bench - Build likwid-bench
• make likwid-accessD - Build likwid-accessD
• make likwid-setFreq - Build likwid-setFreq
• make docs - Create HTML documentation using doxygen
• make clean - Remove the object file directory *./GCC*, keep the executables
• make distclean - Remove all generated files
• make local - Adjust paths in Lua scripts to work from the build directory. Requires the daemons and the pinning library to be already installed. Mainly used for testing.

The build system has a working dependency tracking, therefore make clean is only needed if you change the Makefile configuration.

Installing

NOTE: The pinning functionality and the daemons only work if configured in config.mk and installed with make install. If you do not use the pinning functionality the tools can be used without installation.

• make install - Installs the executables, libraries, man pages and headers to the path you configured in config.mk (PREFIX).
• make uninstall - Delete all installed files under PREFIX.
  • make move - Copy the executables, libraries, man pages and headers from PREFIX in config.mk to INSTALLED_PREFIX.
• make uninstall_moved - Delete all installed files under INSTALLED_PREFIX.

Usage of the access daemon likwid-accessD

Hardware performance monitoring on x86 is enabled using model-specific registers (MSR). MSR registers are special registers not part of the instruction set architecture. To read and write to these registers the x86 ISA provides special instructions. These instructions can only be executed in protected mode or in other words only kernel code can execute these instructions. Fortunately, any Linux kernel 2.6 or newer provides access to these registers via a set of device files. This allows to implement all of the functionality in user space. Still it does not allow to use those more advanced features of hardware performance monitoring which require to setup interrupt service routines or kernel located memory.

Per default only root has read/write access to these msr device files. In order to use the LIKWID tools, which need access to these files (likwid-perfctr, likwid-powermeter and likwid-agent) as standard user, you need to setup access rights to these files.

likwid-perfctr, likwid-powermeter and likwid-features require the Linux msr kernel module. This module is part of most standard distro kernels. You have to be root to do the initial setup.

- Check if the <CODE>msr</CODE> module is loaded with <CODE>lsmod | grep msr</CODE>. There should be an output.
- It the module is not loaded, load it with <CODE>modprobe msr</CODE>. For automatic loading at startup consult your distros documentation how to do so.
- Adopt access rights on the MSR device files for normal user. To grant access to anyone, you can use <CODE>chmod o+rw /dev/cpu/*/msr</CODE>. This is only recommended on single user desktop systems.

As in general access to MSRs is not desired on security sensitive systems, you can either implement a more sophisticated access rights settings with e.g. setgid. A common solution used on many other device files, e.g. for audio, is to introduce a group and make a chown on the msr device files to that group. Now if you execute likwid-perfctr with setgid on that group, the executing user can use the tool but cannot directly write or read the MSR device files.

Some distributions backported the capabilities check for the msr device to older kernels. If there are problems with accessing the msr device for older kernels with file system permissions set to read&write, please check your kernel code (arch/x86/kernel/msr.c) for the backport and set the MSR capabilities in case.

A secure solution is to use the access daemon likwid-accessD, which encapsulates the access to the MSR device files and performs a address check for allowed registers.

Some newer kernels implement the so-called capabilities, a fine-grained permission system that can allow access to the MSR files for common users. On the downside it may be not enough anymore to set the suid-root flag for the access daemon, the executable must be registerd at the libcap.

sudo setcap cap_sys_rawio+ep EXECUTABLE

This is only possible on local file systems. A feasible way is to use the likwid-accessD for all accesses and just enable the capabilities for this one binary. This will enable the usage for all LIKWID tools and also for all instrumented binaries. If likwid-perfctr utility should only be used in wrapper mode, it is suitable to set the capabilities for likwid-perfctr only. Please remember to set the file permission of the MSR device files to read/write for all users, even if capabilites are configured correctly.

Dependencies

Although we tried to minimize the external dependencies of LIKWID, some advanced tools or only specific tool options require external packages.
likwid-perfscope uses the Perl script feedGnuplot to forward the real-time data to gnuplot. feedGnuplot is included into LIKWID, but gnuplot itself is not.
likwid-agent provided multiple backends to output the periodically measured data. The syslog backend requires the shell tool logger to be installed. The RRD backend requires rrdtool and the GMetric backend the gmetric tool, part of the Ganglia Monitoring System.
In order to create the HTML documentation of LIKWID, the tool Doxygen is required. */

/*!