Apparently logger.warn is being deprecated. So let's
switch over to logger.warning. Clearly it's better
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
This allows retrying functions to possibly change their logging level
for diagnostics.
In order to maintain backward compatibility with this longstanding
function we catch the specific exception of it not being handled by the
retrying function and call again w/o the argument.
Signed-off-by: Christian Hopps <chopps@labn.net>
The Support bundle generation was/is failing in both
our upstream ci and locally. This cleans up the failures
that I am seeing such that tests now continue to run
instead of aborting the test run.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
A bunch of tests rely on pre-generated config from
json files. These tests were not putting `exit` stanzas
and a bunch of the tests as a result are silently failing
to configure properly at all, as commands were being sent
to the wrong daemons.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
For multicast pimv6 join and traffic, socat is
used, which was not cleaned up post tests executions,
enhanced kill_socat() API to kill socat join and
traffic specific PIDs during teardown module.
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
Multicast pim6 static RP tests are failing
when run in parallel using micronet. There
are APIs to clean mcast traffic before
starting new test but these cleanups
are not needed when socat is used.
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
Earlier daemon parameter was passed to
start_topology(), which is not needed now,
as new code is implemented to start
feature specific daemons.
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
Currently topotests starts all daemons by default,
made changes to f/w so only needed daemons can
be started, daemons which are needed to tests
particular test suite.
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
Enhanced few exsiting PIM APIs to support both
IPv4 and IPv6 configuration. Added few new APIs
for PIMv6. Tested all existing tests with new
API changes.
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
Issue was reported by Donald, we were hitting
with key not found error and execution was
stopped, which is fixed by this PR.
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
Verifying and making sure PIM neighbors are
up before sending BSM packet using Scapy.
Verifying static routes are installed before
proceeding fruther.
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
When you have a static route with multiple different admin
distances there exists a chance that route will have been
installed multiple times due to system load when inserted
at about the same time. If this is the case then the
verify_rib function can and will select the wrong route
that happens to have a nexthop group that is still installed.
Modify verify_rib to ensure that the route that is going to
be looked at for nexthop correctness is the actual installed
route, not a previous version of it.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
As of now we are logging only JSON output of CLIs
in topotests(topojson) executions and same o/p is
getting printed twice, which is of no use.
Enhanced code to show both plain and JSON output
of CLIs and remove duplicate logging.
It will help in reducing execution logs and in
verification, if sometimes there is mis-match
in CLI plain and JSON outputs.
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
Under heavy load I am seeing verify_rib failing after 12 seconds
but succeeding after 17:
2022-05-19 18:52:54,374 DEBUG: topolog: Exiting lib API: verify_rib
2022-05-19 18:52:54,374 DEBUG: topolog: Function returned True
2022-05-19 18:52:54,374 WARNING: topolog: RETRY DIAGNOSTIC: SUCCEED after FAILED with requested timeout of 12.0s; however, succeeded in 14.7s, investigate timeout timing
There is no reason to not have the test wait a bit longer for very very
heavily loaded systems. Change the time to 40 seconds.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
Lots of tests call verify_rib that takes a list of routes that
need to be verified in some fashion. This verify_rib functionality
will try up to 12 seconds before failing the check that zebra
has the route and has installed it.
Unfortunately the verify_rib code was not looking to see if
the route was queued for installation and was then allowing
tests to immediately do subsuquent steps that depended on
that route actually being installed sometimes causing tests
to fail.
Write a bit of additional code that looks at the queued
status and allows the test to wait a bit longer for zebra
to finish processing before allowing the test to move on
to the next bit.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
VRF name should not be printed in the config since 574445ec. The update
was done for NB config output but I missed it for regular vty output.
Signed-off-by: Igor Ryzhov <iryzhov@nfware.com>
Under heavy system load, we are sometimes seeing this
output for addKernelRoute:
2021-11-28 16:17:27,604 INFO: topolog: [DUT: b1]: Running command: [ip route add 224.0.0.13 dev b1-f1-eth0]
2021-11-28 16:17:27,604 DEBUG: topolog.b1: LinuxNamespace(b1): cmd_status("['/bin/bash', '-c', 'ip route add 224.0.0.13 dev b1-f1-eth0']", kwargs: {'encoding': 'utf-8', 'stdout': -1, 'stderr': -2, 'shell': False, 'stdin': None})
2021-11-28 16:17:27,967 DEBUG: topolog.b1: LinuxNamespace(b1): cmd_status("['/bin/bash', '-c', 'ip route']", kwargs: {'encoding': 'utf-8', 'stdout': -1, 'stderr': -2, 'shell': False, 'stdin': None})
2021-11-28 16:17:28,243 DEBUG: topolog: ip route
70.0.0.0/24 dev b1-f1-eth0 proto kernel scope link src 70.0.0.1
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
This tells us that the ip route add succeeded but when looking for it
the system failed to immediately find it. Why is this happening?
Probably we are under heavy system load and the two different
commands, 'ip route add..' and 'ip route show' are being executed
on different cpu's and the data has not been copied to the different
cpu yet in the kernel. This is not necessarily something normally
seen but entirely possible. Giving the system a few extra seconds
for the kernel to execute/work the memory barrier system seems
prudent for long term success of our programming.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
Several tests used the route_map_create functionality
with `metric-type` but never bothered to add the
backend code to ensure it works correctly.
Add it in so it can be used.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
Modernize the test a bit, generate expected results rather than load from
file, and add a general json_cmp with retry function and use it.
Signed-off-by: Christian Hopps <chopps@labn.net>
When looking for a implied host route it is not necessary
to add the `/32` to an ip route add. As such masks
will not be set in this case. Set the value of masks
to a known good value so that when the route installation
fails the test for it actually being there will tell you
that the route is not there -vs- complaining about mask
being uninited.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
Refactor the bgp_auth test to create common_config code to allow
non-json based tests to reset routers and load configs in parallel.
Signed-off-by: Christian Hopps <chopps@labn.net>
