When a daemon wants to know about its routes, make it possible to have
that work for dst-src routes.
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
The Linux kernel doesn't support dst-src routes with NHGs as nexthop,
for some (rather dubious) caching reasons.
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
Implement the necessary data structures and code changes to support sending
table-direct routes to protocols running in different VRFs.
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
Return error if IPv6 address or prefix is passed as an argument
to "show ip route" command.
UT:
r1# show ip route 2::3/128
% Cannot specify IPv6 address/prefix for IPv4 table
r1#
r1# show ip route 2::3
% Cannot specify IPv6 address/prefix for IPv4 table
r1#
Signed-off-by: Pooja Jagadeesh Doijode <pdoijode@nvidia.com>
The dg_update_list access is controlled by the dg_mutex in all
other locations. Let's just add a mutex usage around the initialization
of the dg_update_list even if it's part of the startup, just to keep
things consistent.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
Current -n option is only for zebra and mgmtd. All other daemons receive
the VRF backend configuration from zebra upon connection to it. This
leads to a potential race condition - daemons need to know the backend
before they start reading their config, but they can be not connected to
zebra yet at this point. As the VRF backend cannot change during runtime,
let's introduce a new global -w option for setting netns backend, to
make sure that all daemons know their VRF backend immediately after
start.
The reason for introducing a new option instead of making -n global is
that ospfd already uses -n for another purposes.
Signed-off-by: Igor Ryzhov <idryzhov@gmail.com>
vrf->ns_ctxt is only ever used in zebra, so move its initialization to
zebra's callback. Ideally this pointer shouldn't even be a part of
library's vrf struct, and moved to zebra-specific struct, but this is
the first step.
Signed-off-by: Igor Ryzhov <idryzhov@gmail.com>
The backend type cannot be unknown. It is configured to VRF_LITE by
default in zebra anyway, so just init to VRF_LITE in the lib and remove
the UNKNOWN type.
Signed-off-by: Igor Ryzhov <idryzhov@gmail.com>
Currently FRR when installing a nexthop group, the installation can fail.
The assumption with the code was that the current nexthop group was
not already installed. This leaves a problem state where if the
users of the nexthop group are removed, the nexthop group will be
removed possibly leaving a orphaned nexthop group in the data plane.
FRR on a nexthop group installation does not actually know the status
of the nexthop group in the kernel. It's possible that a earlier
version of the nexthop group is left in play. It's possible that
there is no nexthop group in the kernel at all. Leaving the
Installed flag alone allows upon Zebra removing the nexthop
group when it is removed from zebra.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
Currently if you have an interface down event, Zebra
sets the nexthop(s) as !ACTIVE that use it. On
interface up events the singleton nexthops are not being
set as ACTIVE. Due to timing events it is sometimes
possible to end up with a route that is using a singleton
Change singleton nexthops to set the nexthop to ACTIVE.
This will allow the nexthop to be reinstalled appropriately
as well.
I was able to easily reproduce this using sharpd since
it does not attempt to reinstall the routes when a interface
goes up/down.
Before:
D>* 10.0.0.0/32 [150/0] via 192.168.102.34, dummy2, weight 1, 00:00:01
sharpd@eva ~/frr5 (master)> sudo ip link set dummy2 down ; sudo ip link set dummy2 up
D> 10.0.0.0/32 [150/0] (350) via 192.168.102.34, dummy2 inactive, weight 1, 00:00:10
After code change:
D>* 10.0.0.0/32 [150/0] (73) via 192.168.102.34, dummy2, weight 1, 00:00:14
sharpd@eva ~/frr5 (master)> sudo ip link set dummy2 down ; sudo ip link set dummy2 up
D>* 10.0.0.0/32 [150/0] (73) via 192.168.102.34, dummy2, weight 1, 00:00:21
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
In some cases, the old_re nhe and the newnhe is same and there is no
point in comparing them both since they are the same. Skip comparing in
such cases.
Ex:
2025/01/09 23:49:27.489020 ZEBRA: [W4Z4R-NTSMD] zebra_nhg_rib_find_nhe: => nhe 0x555f611d30c0 (44[38/39/45])
2025/01/09 23:49:27.489021 ZEBRA: [ZH3FQ-TE9NV] zebra_nhg_rib_compare_old_nhe: 0.0.0.0/0 new id: 44 old id: 44
2025/01/09 23:49:27.489021 ZEBRA: [YB8HE-Z86GN] zebra_nhg_rib_compare_old_nhe: 0.0.0.0/0 NEW 0x555f611d30c0 (44[38/39/45])
2025/01/09 23:49:27.489023 ZEBRA: [ZSB1Z-XM2V3] 0.0.0.0/0: NH 20.1.1.9[0] vrf default(0) wgt 1, with flags
2025/01/09 23:49:27.489024 ZEBRA: [ZSB1Z-XM2V3] 0.0.0.0/0: NH 30.1.2.9[0] vrf default(0) wgt 1, with flags
2025/01/09 23:49:27.489025 ZEBRA: [ZSB1Z-XM2V3] 0.0.0.0/0: NH 20.1.1.2[4] vrf default(0) wgt 1, with flags ACTIVE
2025/01/09 23:49:27.489026 ZEBRA: [ZM3BX-HPETZ] zebra_nhg_rib_compare_old_nhe: 0.0.0.0/0 OLD 0x555f611d30c0 (44[38/39/45])
2025/01/09 23:49:27.489027 ZEBRA: [ZSB1Z-XM2V3] 0.0.0.0/0: NH 20.1.1.9[0] vrf default(0) wgt 1, with flags
2025/01/09 23:49:27.489028 ZEBRA: [ZSB1Z-XM2V3] 0.0.0.0/0: NH 30.1.2.9[0] vrf default(0) wgt 1, with flags
2025/01/09 23:49:27.489028 ZEBRA: [ZSB1Z-XM2V3] 0.0.0.0/0: NH 20.1.1.2[4] vrf default(0) wgt 1, with flags ACTIVE
Signed-off-by: Rajasekar Raja <rajasekarr@nvidia.com>
If you have this series of events:
a) Decision to install a NHG is made in zebra, enqueue to DPLANE
b) Changes to NHG are made and we remove it in the master pthread
Since this NHG is not marked as installed it is not removed
but the NHG data structure is deleted
c) DPLANE installs the NHG
In the end the NHG stays installed but ZEBRA has lost track of it.
Modify the removal code to check to see if the NHG is queued.
There are 2 cases:
a) NHG is kept around for a bit before being deleted. In this case
just see that the NHG is Queued and keep it around too.
b) NHG is not kept around and we are just removing it. In this case
check to see if it is queued and send another deletion event.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
During route processing in zebra, Zebra will create a nexthop
group that matches the nexthops passed down from the routing
protocol. Then Zebra will look to see if it can re-use a
nhe from a previous version of the route entry( say a interface
goes down ). If Zebra decides to re-use an nhe it was just dropping
the route entry created. Which led to nexthop group's that had
a refcount of 0 and in some cases these nexthop groups were installed
into the kernel.
Add a bit of code to see if the returned entry is not being used
and it has no reference count and if so, properly dispose of it.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
"ip/ipv6 protocol any route-map <route map>" cli is setting
wrong route type ( ZEBRA_ROUTE_MAX ), It should set route type
ZEBRA_ROUTE_ALL.
Ticket: #4101560
Signed-off-by: Sougata Barik <sougatab@nvidia.com>
Fixing compilation error in a switch statement case
Fixes :aa4786642c9a65c282d0fd5247a35b0f14fa1c3c
Signed-off-by: Rajasekar Raja <rajasekarr@nvidia.com>
If Duplicate Address Detection action is freeze
(permanent or definite time means not warn only mode)
then locally duplicate detected MAC delete notification
is not require to inform,
instead ask BGP to sync previous remote MAC entry.
In freeze case local MAC event is not known to BGP,
instead BGP is pointing to remote VTEP for the MAC.
Ticket: #3652383
Issue: 3652383
Signed-off-by: Chirag Shah <chirag@nvidia.com>
Upon if_down, we don't reset the valid flag for dependents
and unset the INSTALLED flag.
So when its time for the NHG to be deleted (routes dereferenced),
zebra deletes it since refcnt goes to 0, but stale NHG remains in kernel.
Ticket :#4200788
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
Signed-off-by: Rajasekar Raja <rajasekarr@nvidia.com>
Just like `link down`, check all kernel routes when interface become up.
And, they maybe will be selected as the best one by zebra.
Signed-off-by: anlan_cs <anlan_cs@126.com>
After the nexthop check is fixed, zebra will wrongly uninstall the kernel routes
with inactive nexthop.
This commit would skip the uninstallation for kernel routes.
Signed-off-by: anlan_cs <anlan_cs@126.com>
The kernel routes are wrongly selected even the nexthop interface is linkdown.
Use `ip link set dev <interface> down` on the other box to set the box's
nexthop interface linkdown. The kernel routes will be kept as `linkdown`,
but are still with active nexthop in `zebra`.
Add three changes/commits for kernel routes in this PR:
1) The active nexthop should be the operative interface.
2) Don't uninstall the kernel routes from `zebra` even no active nexthops.
(It doesn't affect the kernel routes' deletion from kernel netlink messages.)
3) Update the kernel routes when the nexthop interface becomes up.
Before: (during nexthop interface is linkdown)
```
K>* 3.3.3.3/32 [0/0] via 88.88.88.1, enp2s0, weight 1, 00:00:14
```
After: (during nexthop interface is linkdown, with all three changes)
```
K 3.3.3.3/32 [0/0] via 88.88.88.1, enp2s0 inactive, weight 1, 00:00:07
```
This commit is 1st change:
Improve the judgment for "active" nexthop to be more accurate, the active
nexthop should be the operative interface.
Signed-off-by: anlan_cs <anlan_cs@126.com>
When debugging a crash I noticed that sometimes we talked about
a zclient connection in relation to the fd associated with it
and sometimes we did not. Let's just always give the data
associated with the fd. It will make it a bit easier for me
to follow the transitions.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
Add `mrib` flag to existing "show ip route" commands which then use
the multicast safi rather than the unicast safi. Updated the vty output
to include the AFI and SAFI string when printing the table.
Deprecate `show ip rpf` command, aliased to `show ip route mrib`.
Removed `show ip rpf A.B.C.D`.
Signed-off-by: Nathan Bahr <nbahr@atcorp.com>
Multicast mode belongs in PIM, so removing it completely from zebra.
Modified `show (ip|ipv6) rpf ADDRESS` to always lookup from SAFI_MULTICAST.
This means this command is now specific to the multicast table and does
not necessarily reflect the PIM RPF lookup, but that should be implemented
in PIM instead.
Signed-off-by: Nathan Bahr <nbahr@atcorp.com>
Modified ZEBRA_NEXTHOP_LOOKUP_MRIB to include the SAFI from which to do the lookup.
This generalizes the API away from MRIB specifically and allows the user to decide how it should do lookups.
Rename ZEBRA_NEXTHOP_LOOKUP_MRIB to ZEBRA_NEXTHOP_LOOKUP now that it is more generalized.
This change is in preperation to remove multicast lookup mode completely from zebra.
Signed-off-by: Nathan Bahr <nbahr@atcorp.com>
Effectively When bgp would send a route update down
to zebra and immediately after that a asic update
from the kernel was read. Zebra would choose the
asic update and drop the bgp update leaving us in
a state where bgp was not used as the true source.
Modify the code so that in rib_multipath_nhe
we notice that we have an unprocessed route update
from bgp. And if so just drop this kernel update
about an older version of the route since it is
no longer needed.
Ticket: 2722533
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
For those packets that we are not sending 16k of data, but something
far less than 256 bytes. Reduce those stream sizes we allocate
to something much more reasonable.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
When a MAC gets deleted but associated neighbors remain, the MAC is
kept in the zebra MAC database as an internal ("auto") entry. When
this happens, reset the MAC's remote sequence number. This ensures that
when the host with the MAC later comes up behind a remote VTEP, the
local switch accepts the MAC and installs it into the bridge FDB and
we don't end up in a situation where remote MACs are not installed
into the bridge FDB.
This fix is a corollary of CM-22753 and is this time done for local
MACs upon delete.
Note: Commit is marked Cumulus-only because I need to evalute more
comprehensive changes before upstreaming it.
Ticket: CM-29581
Reviewed By: As above
Testing Done:
1. Multiple rounds of manual testing
2. Two rounds of evpn-smoke, 1 round of precommit
Signed-off-by: Vivek Venkatraman <vivek@cumulusnetworks.com>
Acked-by: Chirag Shah <chirag@cumulusnetworks.com>
Acked-by: Anuradha Karuppiah <anuradhak@cumulusnetworks.com>
The netlink route message encode function accessed and used
zebra vrfs to produce debug output. That's not thread-safe, and
that encode code needs to run in (at least) the dplane pthread
and the FPM plugin pthread.
Signed-off-by: Mark Stapp <mjs@cisco.com>
Separate core netlink route message parsing into a new api that
uses a dplane ctx to hold the parsed attribute data. Use the new
api in two paths: the normal netlink update message parsing path,
and in the FPM plugin, which also uses netlink encoding. The FPM
route-notificatin code runs in its own pthread, and only needs
a subset of the route info that zebra ordinarily develops. This
change stops that pthread from accessing zebra's internal
data, such as vrfs and ifps, that are not thread-safe.
Signed-off-by: Mark Stapp <mjs@cisco.com>
Separate zebra's ZAPI server socket handling into two phases:
an early phase that opens the socket, and a later phase that
starts listening for client connections.
Signed-off-by: Mark Stapp <mjs@cisco.com>
During zebra shutdown, the main pthread and the FPM pthread can
deadlock if the FPM pthread is in fpm_reconnect(). Each pthread
tries to use event_cancel_async() to cancel tasks that may be
scheduled for the other pthread - this leads to a deadlock as
neither thread can progress.
This adds an atomic boolean that's managed as each pthread
enters and leaves the cleanup code in question, preventing the
two threads from running into the deadlock.
Signed-off-by: Mark Stapp <mjs@cisco.com>
The functions:
if_get_flags
if_flags_update
if_flags_mangle
are never invoked from a linux netlink build. Put a #ifdef
around those functions so that they are not included on the
linux build as that they are not needed there.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
For interface config:
shutdown
mpls
multicast
These states were never being shown in output, let's show it.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
When VLAN-VNI mapping is updated, do not set the L2VNI up event
if the associated VXLAN device is not up.
This may result in bgp synced remote routes to skip installing
in Zebra and onwards (Kernel).
Ticket: #4139506
Signed-off-by: Chirag Shah <chirag@nvidia.com>
In zebra_mpls.c it has a usage of MTYPE_NH_LABEL which is
defined in both lib/nexthop.c and zebra/zebra_mpls.c. The
usage in zebra_mpls.c is a realloc. This leads to a crash:
(gdb) bt
0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=126487246404032) at ./nptl/pthread_kill.c:44
1 __pthread_kill_internal (signo=6, threadid=126487246404032) at ./nptl/pthread_kill.c:78
2 __GI___pthread_kill (threadid=126487246404032, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
3 0x0000730a1b442476 in __GI_raise (sig=6) at ../sysdeps/posix/raise.c:26
4 0x0000730a1b94fb18 in core_handler (signo=6, siginfo=0x7ffeed1e07b0, context=0x7ffeed1e0680) at lib/sigevent.c:268
5 <signal handler called>
6 __pthread_kill_implementation (no_tid=0, signo=6, threadid=126487246404032) at ./nptl/pthread_kill.c:44
7 __pthread_kill_internal (signo=6, threadid=126487246404032) at ./nptl/pthread_kill.c:78
8 __GI___pthread_kill (threadid=126487246404032, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
9 0x0000730a1b442476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
10 0x0000730a1b4287f3 in __GI_abort () at ./stdlib/abort.c:79
11 0x0000730a1b9984f5 in _zlog_assert_failed (xref=0x730a1ba59480 <_xref.16>, extra=0x0) at lib/zlog.c:789
12 0x0000730a1b8f8908 in mt_count_free (mt=0x576e0edda520 <MTYPE_NH_LABEL>, ptr=0x576e36617b80) at lib/memory.c:74
13 0x0000730a1b8f8a59 in qrealloc (mt=0x576e0edda520 <MTYPE_NH_LABEL>, ptr=0x576e36617b80, size=16) at lib/memory.c:112
14 0x0000576e0ec85e2e in nhlfe_out_label_update (nhlfe=0x576e368895f0, nh_label=0x576e3660e9b0) at zebra/zebra_mpls.c:1462
15 0x0000576e0ec833ff in lsp_install (zvrf=0x576e3655fb50, label=17, rn=0x576e366197c0, re=0x576e3660a590) at zebra/zebra_mpls.c:224
16 0x0000576e0ec87c34 in zebra_mpls_lsp_install (zvrf=0x576e3655fb50, rn=0x576e366197c0, re=0x576e3660a590) at zebra/zebra_mpls.c:2215
17 0x0000576e0ecbb427 in rib_process_update_fib (zvrf=0x576e3655fb50, rn=0x576e366197c0, old=0x576e36619660, new=0x576e3660a590) at zebra/zebra_rib.c:1084
18 0x0000576e0ecbc230 in rib_process (rn=0x576e366197c0) at zebra/zebra_rib.c:1480
19 0x0000576e0ecbee04 in process_subq_route (lnode=0x576e368e0270, qindex=8 '\b') at zebra/zebra_rib.c:2661
20 0x0000576e0ecc0711 in process_subq (subq=0x576e3653fc80, qindex=META_QUEUE_BGP) at zebra/zebra_rib.c:3226
21 0x0000576e0ecc07f9 in meta_queue_process (dummy=0x576e3653fae0, data=0x576e3653fb80) at zebra/zebra_rib.c:3265
22 0x0000730a1b97d2a9 in work_queue_run (thread=0x7ffeed1e3f30) at lib/workqueue.c:282
23 0x0000730a1b96b039 in event_call (thread=0x7ffeed1e3f30) at lib/event.c:1996
24 0x0000730a1b8e4d2d in frr_run (master=0x576e36277e10) at lib/libfrr.c:1232
25 0x0000576e0ec35ca9 in main (argc=7, argv=0x7ffeed1e4208) at zebra/main.c:536
Clearly replacing a label stack is an operation that should be owned by
lib/nexthop.c. So lets move this function into there and have
zebra_mpls.c just call the function to replace the label stack.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
Currently when doing a `show ip route table XXXX`, zebra is displaying
the current default vrf as the vrf we are in. We are displaying a
table not a vrf. This is only true if you are not using namespace
based vrf's, so modify the output to display accordingly.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
`ZEBRA_DEBUG_SRV6` is not the correct macro to evaluate if SRv6 debug is enabled or not.
The correct macro is `IS_ZEBRA_DEBUG_SRV6`.
Fix this by replacing `ZEBRA_DEBUG_SRV6` with `IS_ZEBRA_DEBUG_SRV6`.
Signed-off-by: Carmine Scarpitta <cscarpit@cisco.com>
`ZEBRA_DEBUG_SRV6` is not the correct macro to evaluate if SRv6 debug is enabled or not.
The correct macro is `IS_ZEBRA_DEBUG_SRV6`.
Fix this by replacing `ZEBRA_DEBUG_SRV6` with `IS_ZEBRA_DEBUG_SRV6`.
Signed-off-by: Carmine Scarpitta <cscarpit@cisco.com>
`ZEBRA_DEBUG_SRV6` is not the correct macro to evaluate if SRv6 debug is enabled or not.
The correct macro is `IS_ZEBRA_DEBUG_SRV6`.
Fix this by replacing `ZEBRA_DEBUG_SRV6` with `IS_ZEBRA_DEBUG_SRV6`.
Signed-off-by: Carmine Scarpitta <cscarpit@cisco.com>
`ZEBRA_DEBUG_SRV6` is not the correct macro to evaluate if SRv6 debug is enabled or not.
The correct macro is `IS_ZEBRA_DEBUG_SRV6`.
Fix this by replacing `ZEBRA_DEBUG_SRV6` with `IS_ZEBRA_DEBUG_SRV6`.
Signed-off-by: Carmine Scarpitta <cscarpit@cisco.com>
`ZEBRA_DEBUG_SRV6` is not the correct macro to evaluate if SRv6 debug is enabled or not.
The correct macro is `IS_ZEBRA_DEBUG_SRV6`.
Fix this by replacing `ZEBRA_DEBUG_SRV6` with `IS_ZEBRA_DEBUG_SRV6`.
Signed-off-by: Carmine Scarpitta <cscarpit@cisco.com>
`ZEBRA_DEBUG_SRV6` is not the correct macro to evaluate if SRv6 debug is enabled or not.
The correct macro is `IS_ZEBRA_DEBUG_SRV6`.
Fix this by replacing `ZEBRA_DEBUG_SRV6` with `IS_ZEBRA_DEBUG_SRV6`.
Signed-off-by: Carmine Scarpitta <cscarpit@cisco.com>
`ZEBRA_DEBUG_SRV6` is not the correct macro to evaluate if SRv6 debug is enabled or not.
The correct macro is `IS_ZEBRA_DEBUG_SRV6`.
Fix this by replacing `ZEBRA_DEBUG_SRV6` with `IS_ZEBRA_DEBUG_SRV6`.
Signed-off-by: Carmine Scarpitta <cscarpit@cisco.com>
`ZEBRA_DEBUG_SRV6` is not the correct macro to evaluate if SRv6 debug is enabled or not.
The correct macro is `IS_ZEBRA_DEBUG_SRV6`.
Fix this by replacing `ZEBRA_DEBUG_SRV6` with `IS_ZEBRA_DEBUG_SRV6`.
Signed-off-by: Carmine Scarpitta <cscarpit@cisco.com>
Currently zebra starts the graceful restart timer as well as
allows connections from clients before all data is read in
from the kernel as well as the possiblity of allowing client
connections before this happens as well.
Let's move the graceful restart timer start till after this is
done as well as not allowing client connections till then as well.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
The `rib_update_handle_kernel_route_down_possibility()` didn't consider
the kernel routes ( blackhole ) without interface. When some other
interfaces are down, these kernel routes will be wrongly removed.
Signed-off-by: anlan_cs <anlan_cs@126.com>
Expanded the cli command to include an mrib flag for importing to
the main table MRIB instead of the main table URIB.
Piped through specifying the safi through the import table functions
rather than hardcoding to SAFI_UNICAST.
Import still only import routes from the URIB subtable, only added the
ability to import into the main table MRIB.
Signed-off-by: Nathan Bahr <nbahr@atcorp.com>
Currently the mroute code was not allowing the mroute
to be sent to the dataplane. This leaves us with a
situation where the routes being installed where never
being set as installed and additionally nht against
the mrib would not work if the route came into existence
after the nexthop tracking was asked for.
Turns out all the pieces where there to let this work.
Modify the code to pass it to the dplane and to send
it back up as having worked.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
Add new per-NS interface typerb tree, using external linkage, to
replace the use of the if_table table.
Add apis to iterate the per-NS collection - which is not public.
Signed-off-by: Mark Stapp <mjs@cisco.com>
Add a specific debug command to handle srv6 troubleshooting.
Move the srv6 traces that initially were under 'debug zebra packet'
debug.
Signed-off-by: Philippe Guibert <philippe.guibert@6wind.com>
0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=130719886083648) at ./nptl/pthread_kill.c:44
1 __pthread_kill_internal (signo=6, threadid=130719886083648) at ./nptl/pthread_kill.c:78
2 __GI___pthread_kill (threadid=130719886083648, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
3 0x000076e399e42476 in __GI_raise (sig=6) at ../sysdeps/posix/raise.c:26
4 0x000076e39a34f950 in core_handler (signo=6, siginfo=0x76e3985fca30, context=0x76e3985fc900) at lib/sigevent.c:258
5 <signal handler called>
6 __pthread_kill_implementation (no_tid=0, signo=6, threadid=130719886083648) at ./nptl/pthread_kill.c:44
7 __pthread_kill_internal (signo=6, threadid=130719886083648) at ./nptl/pthread_kill.c:78
8 __GI___pthread_kill (threadid=130719886083648, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
9 0x000076e399e42476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
10 0x000076e399e287f3 in __GI_abort () at ./stdlib/abort.c:79
11 0x000076e39a39874b in _zlog_assert_failed (xref=0x76e39a46cca0 <_xref.27>, extra=0x0) at lib/zlog.c:789
12 0x000076e39a369dde in cancel_event_helper (m=0x5eda32df5e40, arg=0x5eda33afeed0, flags=1) at lib/event.c:1428
13 0x000076e39a369ef6 in event_cancel_event_ready (m=0x5eda32df5e40, arg=0x5eda33afeed0) at lib/event.c:1470
14 0x00005eda0a94a5b3 in bgp_stop (connection=0x5eda33afeed0) at bgpd/bgp_fsm.c:1355
15 0x00005eda0a94b4ae in bgp_stop_with_notify (connection=0x5eda33afeed0, code=8 '\b', sub_code=0 '\000') at bgpd/bgp_fsm.c:1610
16 0x00005eda0a979498 in bgp_packet_add (connection=0x5eda33afeed0, peer=0x5eda33b11800, s=0x76e3880daf90) at bgpd/bgp_packet.c:152
17 0x00005eda0a97a80f in bgp_keepalive_send (peer=0x5eda33b11800) at bgpd/bgp_packet.c:639
18 0x00005eda0a9511fd in peer_process (hb=0x5eda33c9ab80, arg=0x76e3985ffaf0) at bgpd/bgp_keepalives.c:111
19 0x000076e39a2cd8e6 in hash_iterate (hash=0x76e388000be0, func=0x5eda0a95105e <peer_process>, arg=0x76e3985ffaf0) at lib/hash.c:252
20 0x00005eda0a951679 in bgp_keepalives_start (arg=0x5eda3306af80) at bgpd/bgp_keepalives.c:214
21 0x000076e39a2c9932 in frr_pthread_inner (arg=0x5eda3306af80) at lib/frr_pthread.c:180
22 0x000076e399e94ac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
23 0x000076e399f26850 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
(gdb) f 12
12 0x000076e39a369dde in cancel_event_helper (m=0x5eda32df5e40, arg=0x5eda33afeed0, flags=1) at lib/event.c:1428
1428 assert(m->owner == pthread_self());
In this decode the attempt to cancel the connection's events from
the wrong thread is causing the crash. Modify the code to create an
event on the bm->master to cancel the events for the connection.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
NEWLINK is only registered by the dplane thread, the main thread
doesn't care about it. So remove the real process of `netlink_link_change()`
for NEWLINK event in main thread.
And move NEWLINK/DELLINK event to the block where the dplane messages
are kept together.
Signed-off-by: anlan_cs <anlan_cs@126.com>
For some reasons the Linux kernel associates the ipv6 blackhole of non
default table the lo interface.
> root@r1# ip -6 route show table 100
> root@r1# ip -6 route add unreachable default metric 4278198272 table 100
> root@r1# ip -6 route show table 100
> unreachable default dev lo metric 4278198272 pref medium
As a consequence, the VRF default that owns the lo interface is shown as
the nexthop VRF:
> r1# show ipv6 route table 20
> Table 20:
> K>* ::/0 [255/8192] unreachable (ICMP unreachable) (vrf default), 00:18:12
Do not display the nexthop VRF of a blackhole. It does not make sense
for a blackhole and it was not displayed in the past.
Signed-off-by: Louis Scalbert <louis.scalbert@6wind.com>
`dirfd()` can theoretically return an error. Call it once and check the
result.
clang-SA: technically correct™. Ain't that the best kind of correct?
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
`errno` has its own semantics. Sometimes it is correct to write to it.
This is not one of those cases - just use a separate `nl_errno`.
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
`FILE *` objects are theoretically in an invalid state if you try to use
them past their reporting EOF. Adjust the code to make it correct.
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
In these cases the value assigned by the switch block is used directly
rather than returned. Mark the initial/default value as used so
clang-SA doesn't complain about it.
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
clang-19's SA complains about unused initializers for this kind of
"switch (enum) { return string }" kind of code. Use direct string
return values to avoid the issue.
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
I got asked today what was going on in the rnh code. I
had to take time off of what I was doing and rewrap my
head around this code, since it's been a long time.
As that this question may come up again in the future
I am trying to document this better so that someone
coming behind us will be able to read this and get
a better idea of what the algorithm is attempting
to do.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
Currently FRR needs to send a uint16_t value for the number
of nexthops as well it needs the ability to properly decode
all of this. Find and handle all the places that this happens.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
There exists a series of events where a kernel route is learned
first( that happens to be exactly what a connected route should be )
and FRR ends up with both a kernel route and a connected route,
leaving us in a very strange spot. This code change just mirrors
the existing code of if there is a connected route drop the kernel
route. Here we just do the reverse, if we have a kernel route
already and a connected should be created, remove the kernel and
keep the connected.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
When the fpm_process_queue has run out of space
but has written to the fpm output buffer, schedule
it to wake up immediately, as that the write will go out
pretty much immediately, since it was scheduled first.
If the fpm_process_queue has not written to the output
buffer then delay the processing by 10 milliseconds to
allow a possibly backed up write processing to have a
chance to complete it's work.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
The fpm_nl_process function was getting the count
of the total number of ctx's processed. This leads
to after having processed 1 context to always signal
the dataplane that there is work to do. Change the
code to only notify the dplane worker when a context
was actually added to the outgoing context queue.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
The following ASAN issue has been observed:
> ERROR: AddressSanitizer: heap-use-after-free on address 0x6160000acba4 at pc 0x55910c5694d0 bp 0x7ffe3a8ac850 sp 0x7ffe3a8ac840
> READ of size 4 at 0x6160000acba4 thread T0
> #0 0x55910c5694cf in ctx_info_from_zns zebra/zebra_dplane.c:3315
> #1 0x55910c569696 in dplane_ctx_ns_init zebra/zebra_dplane.c:3331
> #2 0x55910c56bf61 in dplane_ctx_nexthop_init zebra/zebra_dplane.c:3680
> #3 0x55910c5711ca in dplane_nexthop_update_internal zebra/zebra_dplane.c:4490
> #4 0x55910c571c5c in dplane_nexthop_delete zebra/zebra_dplane.c:4717
> #5 0x55910c61e90e in zebra_nhg_uninstall_kernel zebra/zebra_nhg.c:3413
> #6 0x55910c615d8a in zebra_nhg_decrement_ref zebra/zebra_nhg.c:1919
> #7 0x55910c6404db in route_entry_update_nhe zebra/zebra_rib.c:454
> #8 0x55910c64c904 in rib_re_nhg_free zebra/zebra_rib.c:2822
> #9 0x55910c655be2 in rib_unlink zebra/zebra_rib.c:4212
> #10 0x55910c6430f9 in zebra_rtable_node_cleanup zebra/zebra_rib.c:968
> #11 0x7f26f275b8a9 in route_node_free lib/table.c:75
> #12 0x7f26f275bae4 in route_table_free lib/table.c:111
> #13 0x7f26f275b749 in route_table_finish lib/table.c:46
> #14 0x55910c65db17 in zebra_router_free_table zebra/zebra_router.c:191
> #15 0x55910c65dfb5 in zebra_router_terminate zebra/zebra_router.c:244
> #16 0x55910c4f40db in zebra_finalize zebra/main.c:249
> #17 0x7f26f2777108 in event_call lib/event.c:2011
> #18 0x7f26f264180e in frr_run lib/libfrr.c:1212
> #19 0x55910c4f49cb in main zebra/main.c:531
> #20 0x7f26f2029d8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
> #21 0x7f26f2029e3f in __libc_start_main_impl ../csu/libc-start.c:392
> #22 0x55910c4b0114 in _start (/usr/lib/frr/zebra+0x1ae114)
It happens with FRR using the kernel. During shutdown, the
namespace identifier is attempted to be obtained by zebra, in an
attempt to prepare zebra dataplane nexthop messages.
Fix this by accessing the ns structure.
Signed-off-by: Philippe Guibert <philippe.guibert@6wind.com>
Currently FRR is limiting the nexthop count to a uint8_t not a
uint16_t. This leads to issues when the nexthop count is 256
which results in the count to overflow to 0 causing problems
in the code.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
Currently the zebra pw code has setup a retry to install the
pw after 30 seconds when it is decided that reachability to
the pw is gone. This causes a failure mode where the
pw code just goes and re-installs the pw after 30 seconds
in the non-reachability case. Instead it should just be
reinstalling after reachability is restored.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
Currently the pw code sets the status of the pw for install
and uninstall immediately when notifying the dplane. This
is incorrect in that we do not actually know the status at
this point in time. When we get the result is when to set
the status.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
fpm_enqueue_rmac_table expects an fpm_rmac_arg* as its argument.
The issue can be reproduced by dropping the TCP session using:
ss -K dst 127.0.0.1 dport = 2620
I used Fedora 40 and frr 9.1.2 and I got the gdb backtrace:
(gdb) bt
0 0x00007fdd7d6997ea in fpm_enqueue_rmac_table (bucket=0x2134dd0, arg=0x2132b60) at zebra/dplane_fpm_nl.c:1217
1 0x00007fdd7dd1560d in hash_iterate (hash=0x21335f0, func=0x7fdd7d6997a0 <fpm_enqueue_rmac_table>, arg=0x2132b60) at lib/hash.c:252
2 0x00007fdd7dd1560d in hash_iterate (hash=0x1e5bf10, func=func@entry=0x7fdd7d698900 <fpm_enqueue_l3vni_table>,
arg=arg@entry=0x7ffed983bef0) at lib/hash.c:252
3 0x00007fdd7d698b5c in fpm_rmac_send (t=<optimized out>) at zebra/dplane_fpm_nl.c:1262
4 0x00007fdd7dd6ce22 in event_call (thread=thread@entry=0x7ffed983c010) at lib/event.c:1970
5 0x00007fdd7dd20758 in frr_run (master=0x1d27f10) at lib/libfrr.c:1213
6 0x0000000000425588 in main (argc=10, argv=0x7ffed983c2e8) at zebra/main.c:492
Signed-off-by: Igor Zhukov <fsb4000@yandex.ru>
Trigger: Zebra core seen when we convert l2vni to l3vni and back
BackTrace:
/usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(_zlog_assert_failed+0xe9) [0x7f4af96989d9]
/usr/lib/frr/zebra(zebra_vxlan_if_vni_up+0x250) [0x5561022ae030]
/usr/lib/frr/zebra(netlink_vlan_change+0x2f4) [0x5561021fd354]
/usr/lib/frr/zebra(netlink_parse_info+0xff) [0x55610220d37f]
/usr/lib/frr/zebra(+0xc264a) [0x55610220d64a]
/usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(thread_call+0x7d) [0x7f4af967e96d]
/usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(frr_run+0xe8) [0x7f4af9637588]
/usr/lib/frr/zebra(main+0x402) [0x5561021f4d32]
/lib/x86_64-linux-gnu/libc.so.6(+0x2724a) [0x7f4af932624a]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x85) [0x7f4af9326305]
/usr/lib/frr/zebra(_start+0x21) [0x5561021f72f1]
Root Cause:
In working case,
- We get a RTM_NEWLINK whose ctx is enqueued by zebra dplane and
dequeued by zebra main and processed i.e.
(102000 is deleted from vxlan99) before we handle RTM_NEWVLAN.
- So in handling of NEWVLAN (vxlan99) we bail out since find with
vlan id 703 does not exist.
root@leaf2:mgmt:/var/log/frr# cat ~/raja_logs/working/nocras.log | grep "RTM_NEWLINK\|QUEUED\|vxlan99\|in thread"
2024/07/18 23:09:33.741105 ZEBRA: [KMXEB-K771Y] netlink_parse_info: netlink-dp-in (NS 0) type RTM_NEWLINK(16), len=616, seq=0, pid=0
2024/07/18 23:09:33.744061 ZEBRA: [K8FXY-V65ZJ] Intf dplane ctx 0x7f2244000cf0, op INTF_INSTALL, ifindex (65), result QUEUED
2024/07/18 23:09:33.767240 ZEBRA: [KMXEB-K771Y] netlink_parse_info: netlink-dp-in (NS 0) type RTM_NEWLINK(16), len=508, seq=0, pid=0
2024/07/18 23:09:33.767380 ZEBRA: [K8FXY-V65ZJ] Intf dplane ctx 0x7f2244000cf0, op INTF_INSTALL, ifindex (73), result QUEUED
2024/07/18 23:09:33.767389 ZEBRA: [NVFT0-HS1EX] INTF_INSTALL for vxlan99(73)
2024/07/18 23:09:33.767404 ZEBRA: [TQR2A-H2RFY] Vlan-Vni(1186:1186-6000002:6000002) update for VxLAN IF vxlan99(73)
2024/07/18 23:09:33.767422 ZEBRA: [TP4VP-XZ627] Del L2-VNI 102000 intf vxlan99(73)
2024/07/18 23:09:33.767858 ZEBRA: [QYXB9-6RNNK] RTM_NEWVLAN bridge IF vxlan99 NS 0
2024/07/18 23:09:33.767866 ZEBRA: [KKZGZ-8PCDW] Cannot find VNI for VID (703) IF vxlan99 for vlan state update >>>>BAIL OUT
In failure case,
- The NEWVLAN is received first even before processing RTM_NEWLINK.
- Since the vxlan id 102000 is not removed from the vxlan99,
the find with vlan id 703 returns the 102000 one and we invoke
zebra_vxlan_if_vni_up where the interfaces don't match and assert.
root@leaf2:mgmt:/var/log/frr# cat ~/raja_logs/noworking/crash.log | grep "RTM_NEWLINK\|QUEUED\|vxlan99\|in thread"
2024/07/18 22:26:43.829370 ZEBRA: [KMXEB-K771Y] netlink_parse_info: netlink-dp-in (NS 0) type RTM_NEWLINK(16), len=616, seq=0, pid=0
2024/07/18 22:26:43.829646 ZEBRA: [K8FXY-V65ZJ] Intf dplane ctx 0x7fe07c026d30, op INTF_INSTALL, ifindex (65), result QUEUED
2024/07/18 22:26:43.853930 ZEBRA: [QYXB9-6RNNK] RTM_NEWVLAN bridge IF vxlan99 NS 0
2024/07/18 22:26:43.853949 ZEBRA: [K61WJ-XQQ3X] Intf vxlan99(73) L2-VNI 102000 is UP >>> VLAN PROCESSED BEFORE INTF EVENT
2024/07/18 22:26:43.853951 ZEBRA: [SPV50-BX2RP] RAJA zevpn_vxlanif vxlan48 and ifp vxlan99
2024/07/18 22:26:43.854005 ZEBRA: [KMXEB-K771Y] netlink_parse_info: netlink-dp-in (NS 0) type RTM_NEWLINK(16), len=508, seq=0, pid=0
2024/07/18 22:26:43.854241 ZEBRA: [KMXEB-K771Y] netlink_parse_info: netlink-dp-in (NS 0) type RTM_NEWLINK(16), len=516, seq=0, pid=0
2024/07/18 22:26:43.854251 ZEBRA: [KMXEB-K771Y] netlink_parse_info: netlink-dp-in (NS 0) type RTM_NEWLINK(16), len=544, seq=0, pid=0
ZEBRA: in thread kernel_read scheduled from zebra/kernel_netlink.c:505 kernel_read()
Fix:
Similar to #13396, where link change
handling was offloaded to dplane, same is being done for vlan events.
Note: Prior to this change, zebra main thread was interested in the
RTNLGRP_BRVLAN. So all the kernel events pertaining to vlan was
handled by zebra main.
With this change change as well the handling of vlan events is still
with Zebra main. However we offload it via Dplane thread.
Ticket :#3878175
Signed-off-by: Rajasekar Raja <rajasekarr@nvidia.com>
Relocating functions used by vlan in if_netlink into zebra vxlan
Note: Static variable to the functions will be added back in the next
commit.
Ticket :#3878175
Signed-off-by: Rajasekar Raja <rajasekarr@nvidia.com>
