[release-4.22] USHIFT-7267: fix conformance networking issues#7056
Conversation
The upstream OVN-K binary removed the --nb-address/--sb-address flags, changing the connection model from TCP to unix sockets. With TCP, each connection attempt was fresh and a missing SBDB caused a clean retry. With unix sockets on a shared hostPath, the socket file persists between container restarts. If ovn-controller connects to a leftover socket from a previous SBDB instance, it caches a stale raft commit index. When the new SBDB starts fresh, ovn-controller sees the lower index as "stale data" and enters an infinite reconnect loop, blocking all pod networking. Fix this with a two-part approach: - SBDB/NBDB containers remove stale sockets before starting ovsdb-server, which then creates fresh ones. - ovn-controller waits for the fresh SBDB socket to appear before connecting, ensuring it never grabs a leftover socket. Applied to both single-node and multi-node DaemonSets.
The northd container used template-rendered OVN_NB_DB_LIST and OVN_SB_DB_LIST with the node IP baked into the DaemonSet spec. In multinode, each node rendered the DaemonSet with its own IP, causing spec changes on every reconcile and triggering rolling updates that restarted the SBDB — the root cause of the stale data issue. Replace the baked-in template variables with the runtime env var K8S_NODE_IP, which is resolved per-pod from status.hostIP. This makes the DaemonSet spec identical regardless of which node renders it, eliminating spurious rolling updates. Remove the now-unused Controlplane field from MultiNodeConfig and the OVN_NB_DB_LIST/OVN_SB_DB_LIST render parameters.
|
@openshift-cherrypick-robot: Ignoring requests to cherry-pick non-bug issues: USHIFT-7267 DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository YAML (base), Central YAML (inherited) Review profile: CHILL Plan: Enterprise Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
/test all |
|
/retest |
pacevedom
left a comment
pacevedom
left a comment
There was a problem hiding this comment.
/lgtm
/label backport-risk-assessed
/label jira/valid-bug
/verified by CI
openshift-ci-robot
added
the
verified
|
@pacevedom: This PR has been marked as verified by DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
openshift-ci
Bot
added
backport-risk-assessed
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: openshift-cherrypick-robot, pacevedom The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci
Bot
added
the
approved
|
/retest |
|
@openshift-cherrypick-robot: all tests passed! Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
2e1753f
into
openshift:release-4.22
4 participants
This is an automated cherry-pick of #7052
/assign pacevedom