docker: ubuntu-ci 18.04 -> 22.04, topotest parallel run examples

Update the READMEs, and give example of a full parallel topotest run with
result extraction for analyze.py

Signed-off-by: Christian Hopps <chopps@labn.net>
This commit is contained in:
Christian Hopps 2023-06-08 13:47:01 -04:00
parent 1a60c3e9a4
commit 3c43eaa7d4
9 changed files with 245 additions and 236 deletions

View file

@ -16,8 +16,8 @@ The following platform images are used to support Travis CI and can also
be used to reproduce topotest failures when the docker host is Ubuntu
(tested on 18.04 and 20.04):
* Ubuntu 18.04
* Ubuntu 20.04
* Ubuntu 22.04
The following platform images may also be built, but these simply install a
binary package from an existing repository and do not perform source builds:
@ -130,44 +130,25 @@ No script, multi-arch (ex. amd64, arm64)::
Building Ubuntu 18.04 Image
---------------------------
Build image (from project root directory)::
docker build -t frr-ubuntu18:latest -f docker/ubuntu18-ci/Dockerfile .
Start the container::
docker run -d --privileged --name frr-ubuntu18 --mount type=bind,source=/lib/modules,target=/lib/modules frr-ubuntu18:latest
Running a topotest (when the docker host is Ubuntu)::
docker exec frr-ubuntu18 bash -c 'cd ~/frr/tests/topotests/ospf-topo1 ; sudo pytest test_ospf_topo1.py'
Starting an interactive bash session::
docker exec -it frr-ubuntu18 bash
Stopping an removing a container::
docker stop frr-ubuntu18 ; docker rm frr-ubuntu18
Removing the built image::
docker rmi frr-ubuntu18:latest
Building Ubuntu 20.04 Image
---------------------------
Build image (from project root directory)::
docker build -t frr-ubuntu20:latest -f docker/ubuntu20-ci/Dockerfile .
docker build -t frr-ubuntu20:latest --build-arg=UBUNTU_VERSION=20.04 -f docker/ubuntu-ci/Dockerfile .
Running Full Topotest::
docker run --init -it --privileged --name frr -v /lib/modules:/lib/modules \
frr-ubuntu20:latest bash -c 'cd ~/frr/tests/topotests ; sudo pytest -nauto --dist=loadfile'
Extract results from the above run into `run-results` dir and analyze::
tests/topotest/analyze.py -C frr -Ar run-results
Start the container::
docker run -d --privileged --name frr-ubuntu20 --mount type=bind,source=/lib/modules,target=/lib/modules frr-ubuntu20:latest
docker run -d --init --privileged --name frr-ubuntu20 --mount type=bind,source=/lib/modules,target=/lib/modules frr-ubuntu20:latest
Running a topotest (when the docker host is Ubuntu)::
@ -184,3 +165,40 @@ Stopping an removing a container::
Removing the built image::
docker rmi frr-ubuntu20:latest
Building Ubuntu 22.04 Image
---------------------------
Build image (from project root directory)::
docker build -t frr-ubuntu22:latest -f docker/ubuntu-ci/Dockerfile .
Running Full Topotest::
docker run --init -it --privileged --name frr -v /lib/modules:/lib/modules \
frr-ubuntu22:latest bash -c 'cd ~/frr/tests/topotests ; sudo pytest -nauto --dist=loadfile'
Extract results from the above run into `run-results` dir and analyze::
tests/topotest/analyze.py -C frr -Ar run-results
Start the container::
docker run -d --init --privileged --name frr-ubuntu22 --mount type=bind,source=/lib/modules,target=/lib/modules frr-ubuntu22:latest
Running a topotest (when the docker host is Ubuntu)::
docker exec frr-ubuntu22 bash -c 'cd ~/frr/tests/topotests/ospf-topo1 ; sudo pytest test_ospf_topo1.py'
Starting an interactive bash session::
docker exec -it frr-ubuntu22 bash
Stopping an removing a container::
docker stop frr-ubuntu22 ; docker rm frr-ubuntu22
Removing the built image::
docker rmi frr-ubuntu22:latest

125
docker/ubuntu-ci/Dockerfile Normal file
View file

@ -0,0 +1,125 @@
ARG UBUNTU_VERSION=22.04
FROM ubuntu:$UBUNTU_VERSION
ARG DEBIAN_FRONTEND=noninteractive
ENV APT_KEY_DONT_WARN_ON_DANGEROUS_USAGE=DontWarn
# Update and install build requirements.
RUN apt update && apt upgrade -y && \
# Basic build requirements from documentation
apt-get install -y \
autoconf \
automake \
bison \
build-essential \
flex \
git \
install-info \
libc-ares-dev \
libcap-dev \
libelf-dev \
libjson-c-dev \
libpam0g-dev \
libreadline-dev \
libsnmp-dev \
libtool \
make \
perl \
pkg-config \
python3-dev \
python3-sphinx \
texinfo \
&& \
# Protobuf build requirements
apt-get install -y \
libprotobuf-c-dev \
protobuf-c-compiler \
&& \
# Libyang2 extra build requirements
apt-get install -y \
cmake \
libpcre2-dev \
&& \
# Runtime/triage/testing requirements
apt-get install -y \
curl \
gdb \
iproute2 \
iputils-ping \
liblua5.3-dev \
libssl-dev \
lua5.3 \
net-tools \
python2 \
python3-pip \
snmp \
snmp-mibs-downloader \
snmpd \
sudo \
time \
tshark \
valgrind \
yodl \
&& \
download-mibs && \
wget https://raw.githubusercontent.com/FRRouting/frr-mibs/main/iana/IANA-IPPM-METRICS-REGISTRY-MIB -O /usr/share/snmp/mibs/iana/IANA-IPPM-METRICS-REGISTRY-MIB && \
wget https://raw.githubusercontent.com/FRRouting/frr-mibs/main/ietf/SNMPv2-PDU -O /usr/share/snmp/mibs/ietf/SNMPv2-PDU && \
wget https://raw.githubusercontent.com/FRRouting/frr-mibs/main/ietf/IPATM-IPMC-MIB -O /usr/share/snmp/mibs/ietf/IPATM-IPMC-MIB && \
curl https://bootstrap.pypa.io/pip/2.7/get-pip.py --output /tmp/get-pip.py && \
python2 /tmp/get-pip.py && \
rm -f /tmp/get-pip.py && \
python3 -m pip install wheel && \
python3 -m pip install pytest && \
python3 -m pip install pytest-sugar && \
python3 -m pip install pytest-xdist && \
python3 -m pip install "scapy>=2.4.2" && \
python3 -m pip install xmltodict && \
python3 -m pip install grpcio grpcio-tools && \
python2 -m pip install 'exabgp<4.0.0'
RUN groupadd -r -g 92 frr && \
groupadd -r -g 85 frrvty && \
adduser --system --ingroup frr --home /home/frr \
--gecos "FRR suite" --shell /bin/bash frr && \
usermod -a -G frrvty frr && \
useradd -d /var/run/exabgp/ -s /bin/false exabgp && \
echo 'frr ALL = NOPASSWD: ALL' | tee /etc/sudoers.d/frr && \
mkdir -p /home/frr && chown frr.frr /home/frr
USER frr:frr
# build and install libyang2
RUN cd && pwd && ls -al && \
git clone https://github.com/CESNET/libyang.git && \
cd libyang && \
git checkout v2.1.80 && \
mkdir build; cd build && \
cmake -DCMAKE_INSTALL_PREFIX:PATH=/usr \
-DCMAKE_BUILD_TYPE:String="Release" .. && \
make -j $(nproc) && \
sudo make install
COPY --chown=frr:frr . /home/frr/frr/
RUN cd ~/frr && \
./bootstrap.sh && \
./configure \
--prefix=/usr \
--localstatedir=/var/run/frr \
--sbindir=/usr/lib/frr \
--sysconfdir=/etc/frr \
--enable-sharpd \
--enable-multipath=64 \
--enable-user=frr \
--enable-group=frr \
--enable-vty-group=frrvty \
--enable-snmp=agentx \
--enable-scripting \
--with-pkg-extra-version=-my-manual-build && \
make -j $(nproc) && \
sudo make install
RUN cd ~/frr && make check || true
COPY docker/ubuntu-ci/docker-start /usr/sbin/docker-start
CMD ["/usr/sbin/docker-start"]

View file

@ -1,73 +0,0 @@
FROM ubuntu:18.04
ARG DEBIAN_FRONTEND=noninteractive
ENV APT_KEY_DONT_WARN_ON_DANGEROUS_USAGE=DontWarn
# Update Ubuntu Software repository
RUN apt update && \
apt-get install -y \
git autoconf automake libtool make libreadline-dev texinfo \
pkg-config libpam0g-dev libjson-c-dev bison flex python3-pip \
libc-ares-dev python3-dev python3-sphinx \
install-info build-essential libsnmp-dev perl libcap-dev \
libelf-dev libprotobuf-c-dev protobuf-c-compiler \
sudo gdb iputils-ping time \
python-pip net-tools iproute2 && \
python3 -m pip install wheel && \
python3 -m pip install pytest && \
python3 -m pip install pytest-xdist && \
python3 -m pip install "scapy>=2.4.2" && \
python3 -m pip install xmltodict && \
python2 -m pip install 'exabgp<4.0.0'
RUN groupadd -r -g 92 frr && \
groupadd -r -g 85 frrvty && \
adduser --system --ingroup frr --home /home/frr \
--gecos "FRR suite" --shell /bin/bash frr && \
usermod -a -G frrvty frr && \
useradd -d /var/run/exabgp/ -s /bin/false exabgp && \
echo 'frr ALL = NOPASSWD: ALL' | tee /etc/sudoers.d/frr && \
mkdir -p /home/frr && chown frr.frr /home/frr
#for libyang 2
RUN apt-get install -y cmake libpcre2-dev
USER frr:frr
# build and install libyang2
RUN cd && pwd && ls -al && \
git clone https://github.com/CESNET/libyang.git && \
cd libyang && \
git checkout v2.0.0 && \
mkdir build; cd build && \
cmake -DCMAKE_INSTALL_PREFIX:PATH=/usr \
-DCMAKE_BUILD_TYPE:String="Release" .. && \
make -j $(nproc) && \
sudo make install
COPY --chown=frr:frr . /home/frr/frr/
RUN cd && ls -al && ls -al frr
RUN cd ~/frr && \
./bootstrap.sh && \
./configure \
--prefix=/usr \
--localstatedir=/var/run/frr \
--sbindir=/usr/lib/frr \
--sysconfdir=/etc/frr \
--enable-vtysh \
--enable-pimd \
--enable-sharpd \
--enable-multipath=64 \
--enable-user=frr \
--enable-group=frr \
--enable-vty-group=frrvty \
--enable-snmp=agentx \
--with-pkg-extra-version=-my-manual-build && \
make -j $(nproc) && \
sudo make install
RUN cd ~/frr && make check || true
COPY docker/ubuntu18-ci/docker-start /usr/sbin/docker-start
CMD ["/usr/sbin/docker-start"]

View file

@ -1,44 +0,0 @@
# Ubuntu 18.04
This builds an ubuntu 18.04 container for dev / test
# Build
```
docker build -t frr-ubuntu18:latest -f docker/ubuntu18-ci/Dockerfile .
```
# Running
```
docker run -d --privileged --name frr-ubuntu18 --mount type=bind,source=/lib/modules,target=/lib/modules frr-ubuntu18:latest
```
# make check
```
docker exec frr-ubuntu18 bash -c 'cd ~/frr ; make check'
```
# interactive bash
```
docker exec -it frr-ubuntu18 bash
```
# topotest -- when Host O/S is Ubuntu only
```
docker exec frr-ubuntu18 bash -c 'cd ~/frr/tests/topotests/ospf-topo1 ; sudo pytest test_ospf_topo1.py'
```
# stop & remove container
```
docker stop frr-ubuntu18 ; docker rm frr-ubuntu18
```
# remove image
```
docker rmi frr-ubuntu18:latest
```

View file

@ -1,78 +0,0 @@
FROM ubuntu:20.04
ARG DEBIAN_FRONTEND=noninteractive
ENV APT_KEY_DONT_WARN_ON_DANGEROUS_USAGE=DontWarn
# Update Ubuntu Software repository
RUN apt update && \
apt-get install -y \
git autoconf automake libtool make libreadline-dev texinfo \
pkg-config libpam0g-dev libjson-c-dev bison flex python3-pip \
libc-ares-dev python3-dev python3-sphinx \
install-info build-essential libsnmp-dev perl \
libcap-dev python2 libelf-dev libprotobuf-c-dev protobuf-c-compiler \
sudo gdb curl iputils-ping time \
lua5.3 liblua5.3-dev \
net-tools iproute2 && \
curl https://bootstrap.pypa.io/pip/2.7/get-pip.py --output /tmp/get-pip.py && \
python2 /tmp/get-pip.py && \
rm -f /tmp/get-pip.py && \
python3 -m pip install wheel && \
python3 -m pip install pytest && \
python3 -m pip install pytest-xdist && \
python3 -m pip install "scapy>=2.4.2" && \
python3 -m pip install xmltodict && \
python2 -m pip install 'exabgp<4.0.0'
RUN groupadd -r -g 92 frr && \
groupadd -r -g 85 frrvty && \
adduser --system --ingroup frr --home /home/frr \
--gecos "FRR suite" --shell /bin/bash frr && \
usermod -a -G frrvty frr && \
useradd -d /var/run/exabgp/ -s /bin/false exabgp && \
echo 'frr ALL = NOPASSWD: ALL' | tee /etc/sudoers.d/frr && \
mkdir -p /home/frr && chown frr.frr /home/frr
#for libyang 2
RUN apt-get install -y cmake libpcre2-dev
USER frr:frr
# build and install libyang2
RUN cd && pwd && ls -al && \
git clone https://github.com/CESNET/libyang.git && \
cd libyang && \
git checkout v2.0.0 && \
mkdir build; cd build && \
cmake -DCMAKE_INSTALL_PREFIX:PATH=/usr \
-DCMAKE_BUILD_TYPE:String="Release" .. && \
make -j $(nproc) && \
sudo make install
COPY --chown=frr:frr . /home/frr/frr/
RUN cd && ls -al && ls -al frr
RUN cd ~/frr && \
./bootstrap.sh && \
./configure \
--prefix=/usr \
--localstatedir=/var/run/frr \
--sbindir=/usr/lib/frr \
--sysconfdir=/etc/frr \
--enable-vtysh \
--enable-pimd \
--enable-sharpd \
--enable-multipath=64 \
--enable-user=frr \
--enable-group=frr \
--enable-vty-group=frrvty \
--enable-snmp=agentx \
--enable-scripting \
--with-pkg-extra-version=-my-manual-build && \
make -j $(nproc) && \
sudo make install
RUN cd ~/frr && make check || true
COPY docker/ubuntu20-ci/docker-start /usr/sbin/docker-start
CMD ["/usr/sbin/docker-start"]

View file

@ -5,13 +5,25 @@ This builds an ubuntu 20.04 container for dev / test
# Build
```
docker build -t frr-ubuntu20:latest -f docker/ubuntu20-ci/Dockerfile .
docker build -t frr-ubuntu20:latest --build-arg=UBUNTU_VERSION=20.04 -f docker/ubuntu-ci/Dockerfile .
```
# Running Full Topotest
```
docker run --init -it --privileged --name frr -v /lib/modules:/lib/modules frr-ubuntu22:latest bash -c 'cd ~/frr/tests/topotests ; sudo pytest -nauto --dist=loadfile'
```
# Extract results from the above run into `run-results` dir and analyze
```
tests/topotest/analyze.py -C frr -Ar run-results
```
# Running
```
docker run -d --privileged --name frr-ubuntu20 --mount type=bind,source=/lib/modules,target=/lib/modules frr-ubuntu20:latest
docker run -d --init --privileged --name frr-ubuntu20 --mount type=bind,source=/lib/modules,target=/lib/modules frr-ubuntu20:latest
```
# make check

View file

@ -1,8 +0,0 @@
#!/bin/bash
if [ $(uname -a | grep -ci Ubuntu) -ge 1 ]; then
#for topotests under ubuntu host
sudo modprobe mpls-router mpls-iptunnel
sudo /etc/init.d/openvswitch-switch start
fi
while true ; do sleep 365d ; done

View file

@ -0,0 +1,57 @@
# Ubuntu 22.04
This builds an ubuntu 22.04 container for dev / test
# Build
```
docker build -t frr-ubuntu22:latest -f docker/ubuntu-ci/Dockerfile .
```
# Running Full Topotest
```
docker run --init -it --privileged --name frr -v /lib/modules:/lib/modules frr-ubuntu22:latest bash -c 'cd ~/frr/tests/topotests ; sudo pytest -nauto --dist=loadfile'
```
# Extract results from the above run into `run-results` dir and analyze
```
tests/topotest/analyze.py -C frr -Ar run-results
```
# Running
```
docker run -d --init --privileged --name frr --mount type=bind,source=/lib/modules,target=/lib/modules frr-ubuntu22:latest
```
# make check
```
docker exec frr bash -c 'cd ~/frr ; make check'
```
# interactive bash
```
docker exec -it frr bash
```
# topotest -- when Host O/S is Ubuntu only
```
docker exec frr bash -c 'cd ~/frr/tests/topotests/ospf-topo1 ; sudo pytest test_ospf_topo1.py'
```
# stop & remove container
```
docker stop frr ; docker rm frr
```
# remove image
```
docker rmi frr-ubuntu22:latest
```