How-to: Build VPP FD.IO with Mellanox DPDK PMD on top CentOS 7.7 with inbox drivers.

https://docs.mellanox.com/pages/releaseview.action?pageId=15053848

1. Install CentOS 7.7

2. Set up an account to use the Linux Foundation resources. To set up the account, refer to The Linux Foundation website: https://identity.linuxfoundation.org.

3. Set up a Gerrit account and apply an ssh public key. Follow the instructions on the FD.IO wiki web page: https://wiki.fd.io/view/DEV/Setting_up_Gerrit#Log_into_Gerrit

4. Install the following prerequisites packages:

Cli command

yum install -y git cpp gcc rpm-build openssl-devel libmnl-devel numactl-devel epel-release net-tools rdma-core-devel nasm

Installation, Compilation and Configuration

1. Download VPP main master branch.

Cli commands

git clone ssh://<USERNAME>@gerrit.fd.io:29418/vpp.git

git checkout origin/stable/<"Release_version_number">

  • Use your Linux Foundation user account name (see Prerequisites above) instead of <USERNAME> as shows in above example.
  • To find desired <"Release_version_number"> do: "git -r branch" form VPP root git directory.

2. For VPP 18.07 ONLY: Enable the mlx5 PMD driver compilation as follow:

Cli commands

cd /vpp

sed -i '/vpp_uses_dpdk_mlx5_pmd/s/^# //g' build-data/platforms/vpp.mk

3. Compile and install VPP with DLopen linkage

Cli commands

cd /vpp

make install-dep

make dpdk-install-dev DPDK_MLX5_PMD=y DPDK_MLX5_PMD_DLOPEN_DEPS=y

## Copy manually newly compiled shared module: ##

cp /opt/vpp/external/x86_64/lib/librte_pmd_mlx5_glue.so* /usr/lib64/

## For VPP 18.07 ONLY, check vpp-dpdk-devel package is installed: ##

rpm -qa | grep vpp-dpdk-devel

## Compile new binary: ##

make build-release vpp_uses_dpdk_mlx5_pmd=yes DPDK_MLX5_PMD_DLOPEN_DEPS=y

## Building RPM packages: ##

make pkg-rpm vpp_uses_dpdk_mlx5_pmd=yes DPDK_MLX5_PMD_DLOPEN_DEPS=y

## Install VPP RPMs: ##

rpm -ivh build-root/vpp-sel*.rpm

rpm -ivh build-root/vpp-lib*.rpm

rpm -ivh build-root/vpp-19*.rpm

rpm -ivh build-root/vpp-plug*.rpm

4. Compile and install without DLopen linkage

Cli commands

cd /vpp

make install-dep

make dpdk-install-dev DPDK_MLX5_PMD=y DPDK_MLX5_PMD_DLOPEN_DEPS=n

## Compile new binary: ##

make build-release DPDK_MLX5_PMD=y DPDK_MLX5_PMD_DLOPEN_DEPS=n

## Building RPM packages: ##

make pkg-rpm vpp_uses_dpdk_mlx5_pmd=yes DPDK_MLX5_PMD_DLOPEN_DEPS=n

## Install VPP RPMs: ##

rpm -ivh build-root/vpp-sel*.rpm

rpm -ivh build-root/vpp-lib*.rpm

rpm -ivh build-root/vpp-19*.rpm

rpm -ivh build-root/vpp-plug*.rpm

5. Edit the /etc/vpp/startup.conf start file to include the following:

Important: Uncomment no-mulit-seg parameter. It disables jumbo packets handling and improves performances significantly.

File

unix{

nodaemon

log /tmp/vpp.log

full-coredump

}

dpdk{

dev 0000:07:00.0

dev 0000:07:00.1

no-multi-seg

dev default{

num-rx-queues2

}

}

Insert PCI slot numbers of ConnectX-4/5 interfaces with correct IDs at dpdk { dev } section, as above example.

To verify the correct IDs use:

Cli command

lspci -nn | grep -i mel

07:00.0 Ethernet controller [0200]: Mellanox Technologies MT27620 Family [15b3:1013]

07:00.1 Ethernet controller [0200]: Mellanox Technologies MT27620 Family [15b3:1013]

6. Start VPP.

Cli command

vpp -c /etc/vpp/startup.conf

7. Enter the VPP Command Line Interface.

Cli command

vppctl

8. Configure L3 interfaces as shown in the example below:

VPP Cli

_______ _ _ _____ ___

__/ __/ _ \ (_)__ | | / / _ \/ _ \

_/ _///// / / _ \ | |/ / ___/ ___/

/_//____(_)_/\___/ |___/_//_/

vpp# set int ip address HundredGigabitEthernet7/0/0 1.1.1.1/24

vpp# set interface state HundredGigabitEthernet7/0/0 up

vpp# set int ip address HundredGigabitEthernet7/0/1 2.2.2.2/24

vpp# set interface state HundredGigabitEthernet7/0/1 up

vpp# show int

Name Idx State Counter Count

HundredGigabitEthernet7/0/01 up

HundredGigabitEthernet7/0/12 up

vpp# show int address

HundredGigabitEthernet7/0/0(up):

1.1.1.1/24

HundredGigabitEthernet7/0/1(up):

2.2.2.2/24

local0 (dn):