[RLC-8] Rebase Custom Changes to rlc-8/4.18.0-553.132.1.el8_10 and cve-2025-10263#1334
[RLC-8] Rebase Custom Changes to rlc-8/4.18.0-553.132.1.el8_10 and cve-2025-10263#1334PlaidCat wants to merge 71 commits into
Conversation
Using the kernel crypto API, the SHA3-256 algorithm is used as
conditioning element to replace the LFSR in the Jitter RNG. All other
parts of the Jitter RNG are unchanged.
The application and use of the SHA-3 conditioning operation is identical
to the user space Jitter RNG 3.4.0 by applying the following concept:
- the Jitter RNG initializes a SHA-3 state which acts as the "entropy
pool" when the Jitter RNG is allocated.
- When a new time delta is obtained, it is inserted into the "entropy
pool" with a SHA-3 update operation. Note, this operation in most of
the cases is a simple memcpy() onto the SHA-3 stack.
- To cause a true SHA-3 operation for each time delta operation, a
second SHA-3 operation is performed hashing Jitter RNG status
information. The final message digest is also inserted into the
"entropy pool" with a SHA-3 update operation. Yet, this data is not
considered to provide any entropy, but it shall stir the entropy pool.
- To generate a random number, a SHA-3 final operation is performed to
calculate a message digest followed by an immediate SHA-3 init to
re-initialize the "entropy pool". The obtained message digest is one
block of the Jitter RNG that is returned to the caller.
Mathematically speaking, the random number generated by the Jitter RNG
is:
aux_t = SHA-3(Jitter RNG state data)
Jitter RNG block = SHA-3(time_i || aux_i || time_(i-1) || aux_(i-1) ||
... || time_(i-255) || aux_(i-255))
when assuming that the OSR = 1, i.e. the default value.
This operation implies that the Jitter RNG has an output-blocksize of
256 bits instead of the 64 bits of the LFSR-based Jitter RNG that is
replaced with this patch.
The patch also replaces the varying number of invocations of the
conditioning function with one fixed number of invocations. The use
of the conditioning function consistent with the userspace Jitter RNG
library version 3.4.0.
The code is tested with a system that exhibited the least amount of
entropy generated by the Jitter RNG: the SiFive Unmatched RISC-V
system. The measured entropy rate is well above the heuristically
implied entropy value of 1 bit of entropy per time delta. On all other
tested systems, the measured entropy rate is even higher by orders
of magnitude. The measurement was performed using updated tooling
provided with the user space Jitter RNG library test framework.
The performance of the Jitter RNG with this patch is about en par
with the performance of the Jitter RNG without the patch.
Signed-off-by: Stephan Mueller <smueller@chronox.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Back-port of commit bb897c5
Author: Stephan Müller <smueller@chronox.de>
Date: Fri Apr 21 08:08:04 2023 +0200
Signed-off-by: Jeremy Allison <jallison@ciq.com>
I.G 9.7.B for FIPS 140-3 specifies that variables temporarily holding
cryptographic information should be zeroized once they are no longer
needed. Accomplish this by using kfree_sensitive for buffers that
previously held the private key.
Signed-off-by: Hailey Mothershead <hailmo@amazon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Back-ported from commit 23e4099
Author: Hailey Mothershead <hailmo@amazon.com>
Date: Mon Apr 15 22:19:15 2024 +0000
Signed-off-by: Jeremy Allison <jallison@ciq.com>
Signed-off-by: Jeremy Allison <jallison@ciq.com>
Signed-off-by: Jeremy Allison <jallison@ciq.com>
The output n bits can receive more than n bits of min entropy, of course,
but the fixed output of the conditioning function can only asymptotically
approach the output size bits of min entropy, not attain that bound.
Random maps will tend to have output collisions, which reduces the
creditable output entropy (that is what SP 800-90B Section 3.1.5.1.2
attempts to bound).
The value "64" is justified in Appendix A.4 of the current 90C draft,
and aligns with NIST's in "epsilon" definition in this document, which is
that a string can be considered "full entropy" if you can bound the min
entropy in each bit of output to at least 1-epsilon, where epsilon is
required to be <= 2^(-32).
Note, this patch causes the Jitter RNG to cut its performance in half in
FIPS mode because the conditioning function of the LFSR produces 64 bits
of entropy in one block. The oversampling requires that additionally 64
bits of entropy are sampled from the noise source. If the conditioner is
changed, such as using SHA-256, the impact of the oversampling is only
one fourth, because for the 256 bit block of the conditioner, only 64
additional bits from the noise source must be sampled.
This patch is derived from the user space jitterentropy-library.
Signed-off-by: Stephan Mueller <smueller@chronox.de>
Reviewed-by: Simo Sorce <simo@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Back-port of upstream commit 908dffa.
Signed-off-by: Jeremy Allison <jallison@ciq.com>
private_key is overwritten with the key parameter passed in by the caller (if present), or alternatively a newly generated private key. However, it is possible that the caller provides a key (or the newly generated key) which is shorter than the previous key. In that scenario, some key material from the previous key would not be overwritten. The easiest solution is to explicitly zeroize the entire private_key array first. Note that this patch slightly changes the behavior of this function: previously, if the ecc_gen_privkey failed, the old private_key would remain. Now, the private_key is always zeroized. This behavior is consistent with the case where params.key is set and ecc_is_key_valid fails. Signed-off-by: Joachim Vandersmissen <git@jvdsn.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Back-port of upstream commit: 73e5984 Signed-off-by: Jeremy Allison <jallison@ciq.com>
key might contain private part of the key, so better use
kfree_sensitive to free it
Signed-off-by: Mahmoud Adam <mngyadam@amazon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Back-port of upstream commit: 9f3fa6b
Signed-off-by: Jeremy Allison <jallison@ciq.com>
…ey() to zeroize keys on exit. Signed-off-by: Jeremy Allison <jallison@ciq.com>
Add workflows for pushes and pull requests. Signed-off-by: Greg Rose <g.v.rose@ciq.com>
LE-2786 Sync kernel-x86_64.config with el86-fips-compliant-8 branch from internal dist-git. Same as shipped src.rpm. Signed-off-by: Jonathan Maple <jmaple@ciq.com>
Signed-off-by: Jonathan Maple <jmaple@ciq.com>
We run build checks on pull requests now instead of push Signed-off-by: Jonathan Maple <jmaple@ciq.com>
LE-3770 This github action checks the PR commits for references to upstream linux commits (lines starting with "commit <hash>") and does two things: 1. Checks that this hash exists in the upstream linux kernel history 2. Checks if there are any Fixes: references for the referenced commit in the upstream linux kernel history If either of those are found to be true a comment is added to the PR with the pertinent information. The logic for the check is provided by the check_upstream_commits.py script from kernel-src-tree-tools Signed-off-by: Jonathan Maple <jmaple@ciq.com>
LE-3799 After the build check, perform a kabi check Signed-off-by: Jonathan Maple <jmaple@ciq.com>
The upstream commit check workflow was failing for pull requests originating from forked repositories. The previous implementation incorrectly assumed the pull request branch existed on the base repository. This commit corrects the workflow to ensure the pull request branch is checked out from the correct source repository, while the base branch is fetched from the target repository. Signed-off-by: Jonathan Maple <jmaple@ciq.com>
The process-pull-request workflow was failing for pull requests originating from forked repositories. The previous implementation incorrectly assumed the pull request branch existed on the base repository. This commit corrects the workflow to ensure the pull request branch is checked out from the correct source repository, while the base branch is fetched from the target repository. Signed-off-by: Jonathan Maple <jmaple@ciq.com>
There will be a new PR checker inbound soon this one is just broken so removing it. Signed-off-by: Roxana Nicolescu <rnicolescu@ciq.com>
Simplifies the workflow to use the reusable workflow defined in main branch. This reduces duplication and makes the workflow easier to maintain across multiple branches. The workflow was renamed because it now includes validation over and above just checking for upstream fixes Signed-off-by: Roxana Nicolescu <rnicolescu@ciq.com>
Signed-off-by: Roxana Nicolescu <rnicolescu@ciq.com>
Signed-off-by: Roxana Nicolescu <rnicolescu@ciq.com>
This reverts commit f933b95.
cve CVE-2026-43284 commit-author Kuan-Ting Chen <h3xrabbit@gmail.com> commit f4c50a4 upstream-diff | While this kernel lacks MSG_SPLICE_PAGES, it still has udp_sendpage(). Upstream didn't patch udp_sendpage() because sendpage was removed entirely in favor of MSG_SPLICE_PAGES. In this kernel, splice() can still attach a page cache page to an IPv4 datagram via sendpage. Therefore, udp_sendpage() must be patched instead. IPv6 does not have MSG_SPLICE_PAGES but the __ip_addend_data() path is exercisable in ipv6/esp6.c so we're including that block. Due to missing 06b4feb ("net: group skb_shinfo zerocopy related bits together.") SKBFL_SHARED_FRAG was replaced with SKBTX_SHARED_FRAG. Also replaced flags with tx_flags. MSG_SPLICE_PAGES can attach pages from a pipe directly to an skb. TCP marks such skbs with SKBFL_SHARED_FRAG after skb_splice_from_iter(), so later paths that may modify packet data can first make a private copy. The IPv4/IPv6 datagram append paths did not set this flag when splicing pages into UDP skbs. That leaves an ESP-in-UDP packet made from shared pipe pages looking like an ordinary uncloned nonlinear skb. ESP input then takes the no-COW fast path for uncloned skbs without a frag_list and decrypts in place over data that is not owned privately by the skb. Mark IPv4/IPv6 datagram splice frags with SKBFL_SHARED_FRAG, matching TCP. Also make ESP input fall back to skb_cow_data() when the flag is present, so ESP does not decrypt externally backed frags in place. Private nonlinear skb frags still use the existing fast path. This intentionally does not change ESP output. In esp_output_head(), the path that appends the ESP trailer to existing skb tailroom without calling skb_cow_data() is not reachable for nonlinear skbs: skb_tailroom() returns zero when skb->data_len is nonzero, while ESP tailen is positive. Thus ESP output will either use the separate destination-frag path or fall back to skb_cow_data(). Fixes: cac2661 ("esp4: Avoid skb_cow_data whenever possible") Fixes: 03e2a30 ("esp6: Avoid skb_cow_data whenever possible") Fixes: 7da0dde ("ip, udp: Support MSG_SPLICE_PAGES") Fixes: 6d8192b ("ip6, udp6: Support MSG_SPLICE_PAGES") Reported-by: Hyunwoo Kim <imv4bel@gmail.com> Reported-by: Kuan-Ting Chen <h3xrabbit@gmail.com> Tested-by: Hyunwoo Kim <imv4bel@gmail.com> Cc: stable@vger.kernel.org Signed-off-by: Kuan-Ting Chen <h3xrabbit@gmail.com> Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com> (cherry picked from commit f4c50a4) Signed-off-by: Shreeya Patel <spatel@ciq.com> Signed-off-by: Roxana Nicolescu <rnicolescu@ciq.com>
… packets jira LE-1733 bugfix geneve_fixes commit 791b408 Move the vxlan_features_check() call to after we verified the packet is a tunneled VXLAN packet. Without this, tunneled UDP non-VXLAN packets (for ex. GENENVE) might wrongly not get offloaded. In some cases, it worked by chance as GENEVE header is the same size as VXLAN, but it is obviously incorrect. Fixes: e3cfc7e ("net/mlx5e: TX, Add geneve tunnel stateless offload support") Signed-off-by: Gal Pressman <gal@nvidia.com> Reviewed-by: Dragos Tatulea <dtatulea@nvidia.com> Signed-off-by: Tariq Toukan <tariqt@nvidia.com> Reviewed-by: Wojciech Drewek <wojciech.drewek@intel.com> Signed-off-by: David S. Miller <davem@davemloft.net> (cherry picked from commit 791b408) Signed-off-by: Jonathan Maple <jmaple@ciq.com>
jira VULN-12931 cve CVE-2024-56642 commit-author Kuniyuki Iwashima <kuniyu@amazon.com> commit 6a2fa13 syzkaller reported a use-after-free of UDP kernel socket in cleanup_bearer() without repro. [0][1] When bearer_disable() calls tipc_udp_disable(), cleanup of the UDP kernel socket is deferred by work calling cleanup_bearer(). tipc_net_stop() waits for such works to finish by checking tipc_net(net)->wq_count. However, the work decrements the count too early before releasing the kernel socket, unblocking cleanup_net() and resulting in use-after-free. Let's move the decrement after releasing the socket in cleanup_bearer(). [0]: ref_tracker: net notrefcnt@000000009b3d1faf has 1/1 users at sk_alloc+0x438/0x608 inet_create+0x4c8/0xcb0 __sock_create+0x350/0x6b8 sock_create_kern+0x58/0x78 udp_sock_create4+0x68/0x398 udp_sock_create+0x88/0xc8 tipc_udp_enable+0x5e8/0x848 __tipc_nl_bearer_enable+0x84c/0xed8 tipc_nl_bearer_enable+0x38/0x60 genl_family_rcv_msg_doit+0x170/0x248 genl_rcv_msg+0x400/0x5b0 netlink_rcv_skb+0x1dc/0x398 genl_rcv+0x44/0x68 netlink_unicast+0x678/0x8b0 netlink_sendmsg+0x5e4/0x898 ____sys_sendmsg+0x500/0x830 [1]: BUG: KMSAN: use-after-free in udp_hashslot include/net/udp.h:85 [inline] BUG: KMSAN: use-after-free in udp_lib_unhash+0x3b8/0x930 net/ipv4/udp.c:1979 udp_hashslot include/net/udp.h:85 [inline] udp_lib_unhash+0x3b8/0x930 net/ipv4/udp.c:1979 sk_common_release+0xaf/0x3f0 net/core/sock.c:3820 inet_release+0x1e0/0x260 net/ipv4/af_inet.c:437 inet6_release+0x6f/0xd0 net/ipv6/af_inet6.c:489 __sock_release net/socket.c:658 [inline] sock_release+0xa0/0x210 net/socket.c:686 cleanup_bearer+0x42d/0x4c0 net/tipc/udp_media.c:819 process_one_work kernel/workqueue.c:3229 [inline] process_scheduled_works+0xcaf/0x1c90 kernel/workqueue.c:3310 worker_thread+0xf6c/0x1510 kernel/workqueue.c:3391 kthread+0x531/0x6b0 kernel/kthread.c:389 ret_from_fork+0x60/0x80 arch/x86/kernel/process.c:147 ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:244 Uninit was created at: slab_free_hook mm/slub.c:2269 [inline] slab_free mm/slub.c:4580 [inline] kmem_cache_free+0x207/0xc40 mm/slub.c:4682 net_free net/core/net_namespace.c:454 [inline] cleanup_net+0x16f2/0x19d0 net/core/net_namespace.c:647 process_one_work kernel/workqueue.c:3229 [inline] process_scheduled_works+0xcaf/0x1c90 kernel/workqueue.c:3310 worker_thread+0xf6c/0x1510 kernel/workqueue.c:3391 kthread+0x531/0x6b0 kernel/kthread.c:389 ret_from_fork+0x60/0x80 arch/x86/kernel/process.c:147 ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:244 CPU: 0 UID: 0 PID: 54 Comm: kworker/0:2 Not tainted 6.12.0-rc1-00131-gf66ebf37d69c #7 91723d6f74857f70725e1583cba3cf4adc716cfa Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 Workqueue: events cleanup_bearer Fixes: 26abe14 ("net: Modify sk_alloc to not reference count the netns of kernel sockets.") Reported-by: syzkaller <syzkaller@googlegroups.com> Signed-off-by: Kuniyuki Iwashima <kuniyu@amazon.com> Link: https://patch.msgid.link/20241127050512.28438-1-kuniyu@amazon.com Signed-off-by: Paolo Abeni <pabeni@redhat.com> (cherry picked from commit 6a2fa13) Signed-off-by: David Gomez <dgomez@ciq.com>
jira VULN-56026 cve CVE-2025-21927 commit-author Maurizio Lombardi <mlombard@redhat.com> commit ad95bab upstream-diff Removed `nvme_tcp_c2h_term' case from `nvme_tcp_recv_pdu_supported' for the sake of consistency of `nvme_tcp_recv_pdu''s behavior relative to the upstream version, between the cases of proper and improper header. (What could be considered as "`c2h_term' type support" started with 84e0090 commit, not included in `ciqlts9_2''s history, so `nvme_tcp_recv_pdu_supported' in `ciqlts9_2' shouldn't report the `nvme_tcp_c2h_term' type as supported.) nvme_tcp_recv_pdu() doesn't check the validity of the header length. When header digests are enabled, a target might send a packet with an invalid header length (e.g. 255), causing nvme_tcp_verify_hdgst() to access memory outside the allocated area and cause memory corruptions by overwriting it with the calculated digest. Fix this by rejecting packets with an unexpected header length. Fixes: 3f2304f ("nvme-tcp: add NVMe over TCP host driver") Signed-off-by: Maurizio Lombardi <mlombard@redhat.com> Reviewed-by: Sagi Grimberg <sagi@grimberg.me> Signed-off-by: Keith Busch <kbusch@kernel.org> (cherry picked from commit ad95bab) Signed-off-by: Brett Mastbergen <bmastbergen@ciq.com>
jira VULN-65790 cve CVE-2022-49803 commit-author Wang Yufen <wangyufen@huawei.com> commit 064bc73 kmemleak reports this issue: unreferenced object 0xffff8881bac872d0 (size 8): comm "sh", pid 58603, jiffies 4481524462 (age 68.065s) hex dump (first 8 bytes): 04 00 00 00 de ad be ef ........ backtrace: [<00000000c80b8577>] __kmalloc+0x49/0x150 [<000000005292b8c6>] nsim_dev_trap_fa_cookie_write+0xc1/0x210 [netdevsim] [<0000000093d78e77>] full_proxy_write+0xf3/0x180 [<000000005a662c16>] vfs_write+0x1c5/0xaf0 [<000000007aabf84a>] ksys_write+0xed/0x1c0 [<000000005f1d2e47>] do_syscall_64+0x3b/0x90 [<000000006001c6ec>] entry_SYSCALL_64_after_hwframe+0x63/0xcd The issue occurs in the following scenarios: nsim_dev_trap_fa_cookie_write() kmalloc() fa_cookie nsim_dev->fa_cookie = fa_cookie .. nsim_drv_remove() The fa_cookie allocked in nsim_dev_trap_fa_cookie_write() is not freed. To fix, add kfree(nsim_dev->fa_cookie) to nsim_drv_remove(). Fixes: d3cbb90 ("netdevsim: add ACL trap reporting cookie as a metadata") Signed-off-by: Wang Yufen <wangyufen@huawei.com> Cc: Jiri Pirko <jiri@mellanox.com> Link: https://lore.kernel.org/r/1668504625-14698-1-git-send-email-wangyufen@huawei.com Signed-off-by: Jakub Kicinski <kuba@kernel.org> (cherry picked from commit 064bc73) Signed-off-by: Brett Mastbergen <bmastbergen@ciq.com>
jira VULN-45766 jira VULN-45767 cve cve-2024-49978 commit-author Willem de Bruijn <willemb@google.com> commit a1e40ac upstream-diff contextual diff is off due to massive reworks. In addition __udpv6_gso_segment_list_csum definition is not included. This was included via "net/gro.h" via 75082e7 which is a bug fix to 4721031 "net: move gro definitions to include/net/gro.h". Since we also do not have that we're just directly including net/ip6_checksum.h to this file. Detect gso fraglist skbs with corrupted geometry (see below) and pass these to skb_segment instead of skb_segment_list, as the first can segment them correctly. Valid SKB_GSO_FRAGLIST skbs - consist of two or more segments - the head_skb holds the protocol headers plus first gso_size - one or more frag_list skbs hold exactly one segment - all but the last must be gso_size Optional datapath hooks such as NAT and BPF (bpf_skb_pull_data) can modify these skbs, breaking these invariants. In extreme cases they pull all data into skb linear. For UDP, this causes a NULL ptr deref in __udpv4_gso_segment_list_csum at udp_hdr(seg->next)->dest. Detect invalid geometry due to pull, by checking head_skb size. Don't just drop, as this may blackhole a destination. Convert to be able to pass to regular skb_segment. Link: https://lore.kernel.org/netdev/20240428142913.18666-1-shiming.cheng@mediatek.com/ Fixes: 9fd1ff5 ("udp: Support UDP fraglist GRO/GSO.") Signed-off-by: Willem de Bruijn <willemb@google.com> Cc: stable@vger.kernel.org Link: https://patch.msgid.link/20241001171752.107580-1-willemdebruijn.kernel@gmail.com Signed-off-by: Jakub Kicinski <kuba@kernel.org> (cherry picked from commit a1e40ac) Signed-off-by: Jonathan Maple <jmaple@ciq.com>
jira VULN-38750 jira VULN-38751 cve CVE-2024-42281 commit-author Fred Li <dracodingfly@gmail.com> commit fa5ef65 Linearize the skb when downgrading gso_size because it may trigger a BUG_ON() later when the skb is segmented as described in [1,2]. Fixes: 2be7e21 ("bpf: add bpf_skb_adjust_room helper") Signed-off-by: Fred Li <dracodingfly@gmail.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Reviewed-by: Willem de Bruijn <willemb@google.com> Acked-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/all/20240626065555.35460-2-dracodingfly@gmail.com [1] Link: https://lore.kernel.org/all/668d5cf1ec330_1c18c32947@willemb.c.googlers.com.notmuch [2] Link: https://lore.kernel.org/bpf/20240719024653.77006-1-dracodingfly@gmail.com (cherry picked from commit fa5ef65) Signed-off-by: Jonathan Maple <jmaple@ciq.com>
jira VULN-156444 jira VULN-156445 cve CVE-2025-38124 commit-author Shiming Cheng <shiming.cheng@mediatek.com> commit 3382a1e Commit a1e40ac ("net: gso: fix udp gso fraglist segmentation after pull from frag_list") detected invalid geometry in frag_list skbs and redirects them from skb_segment_list to more robust skb_segment. But some packets with modified geometry can also hit bugs in that code. We don't know how many such cases exist. Addressing each one by one also requires touching the complex skb_segment code, which risks introducing bugs for other types of skbs. Instead, linearize all these packets that fail the basic invariants on gso fraglist skbs. That is more robust. If only part of the fraglist payload is pulled into head_skb, it will always cause exception when splitting skbs by skb_segment. For detailed call stack information, see below. Valid SKB_GSO_FRAGLIST skbs - consist of two or more segments - the head_skb holds the protocol headers plus first gso_size - one or more frag_list skbs hold exactly one segment - all but the last must be gso_size Optional datapath hooks such as NAT and BPF (bpf_skb_pull_data) can modify fraglist skbs, breaking these invariants. In extreme cases they pull one part of data into skb linear. For UDP, this causes three payloads with lengths of (11,11,10) bytes were pulled tail to become (12,10,10) bytes. The skbs no longer meets the above SKB_GSO_FRAGLIST conditions because payload was pulled into head_skb, it needs to be linearized before pass to regular skb_segment. skb_segment+0xcd0/0xd14 __udp_gso_segment+0x334/0x5f4 udp4_ufo_fragment+0x118/0x15c inet_gso_segment+0x164/0x338 skb_mac_gso_segment+0xc4/0x13c __skb_gso_segment+0xc4/0x124 validate_xmit_skb+0x9c/0x2c0 validate_xmit_skb_list+0x4c/0x80 sch_direct_xmit+0x70/0x404 __dev_queue_xmit+0x64c/0xe5c neigh_resolve_output+0x178/0x1c4 ip_finish_output2+0x37c/0x47c __ip_finish_output+0x194/0x240 ip_finish_output+0x20/0xf4 ip_output+0x100/0x1a0 NF_HOOK+0xc4/0x16c ip_forward+0x314/0x32c ip_rcv+0x90/0x118 __netif_receive_skb+0x74/0x124 process_backlog+0xe8/0x1a4 __napi_poll+0x5c/0x1f8 net_rx_action+0x154/0x314 handle_softirqs+0x154/0x4b8 [118.376811] [C201134] rxq0_pus: [name:bug&]kernel BUG at net/core/skbuff.c:4278! [118.376829] [C201134] rxq0_pus: [name:traps&]Internal error: Oops - BUG: 00000000f2000800 [#1] PREEMPT SMP [118.470774] [C201134] rxq0_pus: [name:mrdump&]Kernel Offset: 0x178cc00000 from 0xffffffc008000000 [118.470810] [C201134] rxq0_pus: [name:mrdump&]PHYS_OFFSET: 0x40000000 [118.470827] [C201134] rxq0_pus: [name:mrdump&]pstate: 60400005 (nZCv daif +PAN -UAO) [118.470848] [C201134] rxq0_pus: [name:mrdump&]pc : [0xffffffd79598aefc] skb_segment+0xcd0/0xd14 [118.470900] [C201134] rxq0_pus: [name:mrdump&]lr : [0xffffffd79598a5e8] skb_segment+0x3bc/0xd14 [118.470928] [C201134] rxq0_pus: [name:mrdump&]sp : ffffffc008013770 Fixes: a1e40ac ("gso: fix udp gso fraglist segmentation after pull from frag_list") Signed-off-by: Shiming Cheng <shiming.cheng@mediatek.com> Reviewed-by: Willem de Bruijn <willemb@google.com> Signed-off-by: David S. Miller <davem@davemloft.net> (cherry picked from commit 3382a1e) Signed-off-by: Jonathan Maple <jmaple@ciq.com>
…tead of a two-phase approach jira roc-2673 commit fbf6449 Instead of setting x86_virt_bits to a possibly-correct value and then correcting it later, do all the necessary checks before setting it. At this point, the #VC handler references boot_cpu_data.x86_virt_bits, and in the previous version, it would be triggered by the CPUIDs between the point at which it is set to 48 and when it is set to the correct value. Suggested-by: Dave Hansen <dave.hansen@linux.intel.com> Signed-off-by: Adam Dunlap <acdunlap@google.com> Signed-off-by: Ingo Molnar <mingo@kernel.org> Tested-by: Jacob Xu <jacobhxu@google.com> Link: https://lore.kernel.org/r/20230912002703.3924521-3-acdunlap@google.com Signed-off-by: Ronnie Sahlberg <rsahlberg@ciq.com>
cve cve-2025-10263 commit-author Mark Rutland <mark.rutland@arm.com> commit 0ce85db Add cputype definitions for Neoverse-V3. These will be used for errata detection in subsequent patches. These values can be found in Table B-249 ("MIDR_EL1 bit descriptions") in issue 0001-04 of the Neoverse-V3 TRM, which can be found at: https://developer.arm.com/documentation/107734/0001/?lang=en Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: James Morse <james.morse@arm.com> Cc: Will Deacon <will@kernel.org> Link: https://lore.kernel.org/r/20240508081400.235362-4-mark.rutland@arm.com Signed-off-by: Will Deacon <will@kernel.org> (cherry picked from commit 0ce85db) Signed-off-by: Jonathan Maple <jmaple@ciq.com>
cve cve-2025-10263 commit-author Mark Rutland <mark.rutland@arm.com> commit be5a6f2 Add cputype definitions for Cortex-X3. These will be used for errata detection in subsequent patches. These values can be found in Table A-263 ("MIDR_EL1 bit descriptions") in issue 07 of the Cortex-X3 TRM, which can be found at: https://developer.arm.com/documentation/101593/0102/?lang=en Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: James Morse <james.morse@arm.com> Cc: Will Deacon <will@kernel.org> Link: https://lore.kernel.org/r/20240603111812.1514101-2-mark.rutland@arm.com Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> (cherry picked from commit be5a6f2) Signed-off-by: Jonathan Maple <jmaple@ciq.com>
cve cve-2025-10263 commit-author Mark Rutland <mark.rutland@arm.com> commit fd2ff5f Add cputype definitions for Cortex-X925. These will be used for errata detection in subsequent patches. These values can be found in Table A-285 ("MIDR_EL1 bit descriptions") in issue 0001-05 of the Cortex-X925 TRM, which can be found at: https://developer.arm.com/documentation/102807/0001/?lang=en Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: James Morse <james.morse@arm.com> Cc: Will Deacon <will@kernel.org> Link: https://lore.kernel.org/r/20240603111812.1514101-4-mark.rutland@arm.com Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> (cherry picked from commit fd2ff5f) Signed-off-by: Jonathan Maple <jmaple@ciq.com>
cve cve-2025-10263 commit-author Mark Rutland <mark.rutland@arm.com> commit 58d245e Add cputype definitions for Cortex-X1C. These will be used for errata detection in subsequent patches. These values can be found in the Cortex-X1C TRM: https://developer.arm.com/documentation/101968/0002/ ... in section B2.107 ("MIDR_EL1, Main ID Register, EL1"). Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: James Morse <james.morse@arm.com> Cc: Will Deacon <will@kernel.org> Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com> Link: https://lore.kernel.org/r/20240801101803.1982459-2-mark.rutland@arm.com Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> (cherry picked from commit 58d245e) Signed-off-by: Jonathan Maple <jmaple@ciq.com>
cve cve-2025-10263 commit-author Douglas Anderson <dianders@chromium.org> commit a9b5bd8 >From the TRM, MIDR_CORTEX_A76AE has a partnum of 0xDOE and an implementor of 0x41 (ARM). Add the values. Cc: stable@vger.kernel.org # dependency of the next fix in the series Signed-off-by: Douglas Anderson <dianders@chromium.org> Link: https://lore.kernel.org/r/20250107120555.v4.4.I151f3b7ee323bcc3082179b8c60c3cd03308aa94@changeid Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> (cherry picked from commit a9b5bd8) Signed-off-by: Jonathan Maple <jmaple@ciq.com>
cve cve-2025-10263 commit-author Mark Rutland <mark.rutland@arm.com> commit 3bbf004 Add cputype definitions for Neoverse-V3AE. These will be used for errata detection in subsequent patches. These values can be found in the Neoverse-V3AE TRM: https://developer.arm.com/documentation/SDEN-2615521/9-0/ ... in section A.6.1 ("MIDR_EL1, Main ID Register"). Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: James Morse <james.morse@arm.com> Cc: Will Deacon <will@kernel.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Signed-off-by: Ryan Roberts <ryan.roberts@arm.com> Signed-off-by: Will Deacon <will@kernel.org> (cherry picked from commit 3bbf004) Signed-off-by: Jonathan Maple <jmaple@ciq.com>
cve cve-2025-10263 commit-author Mark Rutland <mark.rutland@arm.com> commit - commit-source-sha 60349e64a6c65f9f0aa118af711b3c7e137f07ff commit-source arm65 upstream-diff | Context due to missing fd2ff5f - arm64: cputype: Add Cortex-X925 definitions 9ef54a3 - arm64: cputype: Add Cortex-A725 definitions f38c2c3 - arm64: cputype: Add Cortex-A720AE definitions 9247257 - arm64: cputype: Add Neoverse-N3 definitions 2c99561 - arm64: cputype: Add C1-Pro definitions Add cputype definitions for C1-Ultra. These will be used for errata detection in subsequent patches. These values can be found in the C1-Ultra TRM: https://developer.arm.com/documentation/108014/0100/ ... in section A.5.1 ("MIDR_EL1, Main ID Register"). Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org> (cherry picked from commit 60349e64a6c65f9f0aa118af711b3c7e137f07ff) Signed-off-by: Jonathan Maple <jmaple@ciq.com>
cve cve-2025-10263 commit-author Mark Rutland <mark.rutland@arm.com> commit - commit-source-sha d28413bfc5a255957241f1df5d7fd0c2cd74fe18 commit-source arm64 upstream-diff | context differences due to: 9247257 - arm64: cputype: Add Neoverse-N3 definitions 2c99561 - arm64: cputype: Add C1-Pro definitions Add cputype definitions for C1-Premium. These will be used for errata detection in subsequent patches. These values can be found in the C1-Premium TRM: https://developer.arm.com/documentation/109416/0100/ ... in section A.5.1 ("MIDR_EL1, Main ID Register"). Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org> (cherry picked from commit d28413bfc5a255957241f1df5d7fd0c2cd74fe18) Signed-off-by: Jonathan Maple <jmaple@ciq.com>
cve cve-2025-10263 commit-author Shanker Donthineni <sdonthineni@nvidia.com> commit - commit-source-sha e185c8a commit-source arm64 Add cpu part and model macro definitions for NVIDIA Olympus core. Signed-off-by: Shanker Donthineni <sdonthineni@nvidia.com> Signed-off-by: Will Deacon <will@kernel.org> (cherry picked from commit e185c8a) Signed-off-by: Jonathan Maple <jmaple@ciq.com>
cve-pre CVE-2025-10263 commit-author Easwar Hariharan <eahariha@linux.microsoft.com> commit fb091ff upstream-diff Only the MIDR definition is backported. The N2 errata subscriptions (TRBE overwrite fill mode, TSB flush failure, TRBE write out of range) are omitted because those errata and their workaround infrastructure do not exist in this kernel version. Add the MIDR value of Microsoft Azure Cobalt 100, which is a Microsoft implemented CPU based on r0p0 of the ARM Neoverse N2 CPU, and therefore suffers from all the same errata. CC: stable@vger.kernel.org # 5.15+ Signed-off-by: Easwar Hariharan <eahariha@linux.microsoft.com> Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com> Acked-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Oliver Upton <oliver.upton@linux.dev> Link: https://lore.kernel.org/r/20240214175522.2457857-1-eahariha@linux.microsoft.com Signed-off-by: Will Deacon <will@kernel.org> (cherry picked from commit fb091ff) Signed-off-by: Brett Mastbergen <bmastbergen@ciq.com> (cherry picked from commit 63fa0c3) Signed-off-by: Jonathan Maple <jmaple@ciq.com>
cve cve-2025-10263 commit-author Mark Rutland <mark.rutland@arm.com> commit - commit-source-sha cfd391e74134db664feb499d43af286380b10ba8 commit-source arm64 upstream-diff silicon-errata.rst at different path (Documentation/arm64/ vs Documentation/arch/arm64/) and required manual conflict resolution due to condensed table formatting and missing entries in our branch. Content is identical. On 4.18, arm64_repeat_tlbi_cpus[] is a flat struct midr_range array, not struct arm64_cpu_capabilities[]. Upstream uses a nested ERRATA_MIDR_RANGE_LIST() with inline array initialization, which sets .type, .matches, and .midr_range_list fields that do not exist on struct midr_range. Flattened to direct MIDR_ALL_VERSIONS() entries in the array. A number of CPUs developed by Arm suffer from errata whereby a broadcast TLBI;DSB sequence may complete before the global observation of writes which are translated by an affected TLB entry. These errata ONLY affect the completion of memory accesses which have been translated by an invalidated TLB entry, and these errata DO NOT affect the actual invalidation of TLB entries. TLB entries are removed correctly. This issue has been assigned CVE ID CVE-2025-10263. To mitigate this issue, Arm recommends that software follows any affected TLBI;DSB sequence with an additional TLBI;DSB, which will ensure that all memory write effects affected by the first TLBI have been globally observed. The additional TLBI can use any operation that is broadcast to affected CPUs, and the additional DSB can use any option that is sufficient to complete the additional TLBI. The ARM64_WORKAROUND_REPEAT_TLBI workaround is sufficient to mitigate the issue. Enable this workaround for affected CPUs, and update the silicon errata documentation accordingly. Note that due to the manner in which Arm develops IP and tracks errata, some CPUs share a common erratum number. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org> (cherry picked from commit cfd391e74134db664feb499d43af286380b10ba8) Signed-off-by: Jonathan Maple <jmaple@ciq.com> (cherry picked from commit a96d5f4) Signed-off-by: Jonathan Maple <jmaple@ciq.com>
|
🤖 Validation Checks In Progress Workflow run: https://github.com/ctrliq/kernel-src-tree/actions/runs/27383376095 |
🔍 Upstream Linux Kernel Commit Check
This is an automated message from the kernel commit checker workflow. |
🔍 Interdiff Analysis
================================================================================
* DELTA DIFFERENCES - code changes that differ between the patches *
================================================================================
--- b/net/ipv4/ip_output.c
+++ b/net/ipv4/ip_output.c
@@ -1347,9 +1347,6 @@
goto error;
}
- if (!(flags & MSG_NO_SHARED_FRAGS))
- skb_shinfo(skb)->tx_flags |= SKBTX_SHARED_FRAG;
-
if (skb->ip_summed == CHECKSUM_NONE) {
__wsum csum;
csum = csum_page(page, offset, len);
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/net/ipv4/ip_output.c
+++ b/net/ipv4/ip_output.c
@@ -1233,6 +1233,8 @@
if (err < 0)
goto error;
copy = err;
+ if (!(flags & MSG_NO_SHARED_FRAGS))
+ skb_shinfo(skb)->flags |= SKBFL_SHARED_FRAG;
wmem_alloc_delta += copy;
} else if (!zc) {
int i = skb_shinfo(skb)->nr_frags;
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/net/ipv4/ip_output.c
+++ b/net/ipv4/ip_output.c
@@ -1230,6 +1230,6 @@
- goto error;
- }
-
- if (skb->ip_summed == CHECKSUM_NONE) {
- __wsum csum;
- csum = csum_page(page, offset, len);
+ if (err < 0)
+ goto error;
+ copy = err;
+ wmem_alloc_delta += copy;
+ } else if (!zc) {
+ int i = skb_shinfo(skb)->nr_frags;
================================================================================
* ONLY IN PATCH2 - files not modified by patch1 *
================================================================================
--- a/net/ipv6/esp6.c
+++ b/net/ipv6/esp6.c
@@ -915,7 +915,8 @@ static int esp6_input(struct xfrm_state *x, struct sk_buff *skb)
nfrags = 1;
goto skip_cow;
- } else if (!skb_has_frag_list(skb)) {
+ } else if (!skb_has_frag_list(skb) &&
+ !skb_has_shared_frag(skb)) {
nfrags = skb_shinfo(skb)->nr_frags;
nfrags++;
--- a/net/ipv6/ip6_output.c
+++ b/net/ipv6/ip6_output.c
@@ -1794,6 +1794,8 @@ alloc_new_skb:
if (err < 0)
goto error;
copy = err;
+ if (!(flags & MSG_NO_SHARED_FRAGS))
+ skb_shinfo(skb)->flags |= SKBFL_SHARED_FRAG;
wmem_alloc_delta += copy;
} else if (!zc) {
int i = skb_shinfo(skb)->nr_frags;
================================================================================
* DELTA DIFFERENCES - code changes that differ between the patches *
================================================================================
--- b/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -152,18 +152,6 @@
return queue - queue->ctrl->queues;
}
-static inline bool nvme_tcp_recv_pdu_supported(enum nvme_tcp_pdu_type type)
-{
- switch (type) {
- case nvme_tcp_c2h_data:
- case nvme_tcp_r2t:
- case nvme_tcp_rsp:
- return true;
- default:
- return false;
- }
-}
-
static inline struct blk_mq_tags *nvme_tcp_tagset(struct nvme_tcp_queue *queue)
{
u32 queue_idx = nvme_tcp_queue_id(queue);
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -217,6 +217,19 @@
return queue - queue->ctrl->queues;
}
+static inline bool nvme_tcp_recv_pdu_supported(enum nvme_tcp_pdu_type type)
+{
+ switch (type) {
+ case nvme_tcp_c2h_term:
+ case nvme_tcp_c2h_data:
+ case nvme_tcp_r2t:
+ case nvme_tcp_rsp:
+ return true;
+ default:
+ return false;
+ }
+}
+
/*
* Check if the queue is TLS encrypted
*/
@@ -818,6 +831,16 @@
return 0;
hdr = queue->pdu;
+ if (unlikely(hdr->hlen != sizeof(struct nvme_tcp_rsp_pdu))) {
+ if (!nvme_tcp_recv_pdu_supported(hdr->type))
+ goto unsupported_pdu;
+
+ dev_err(queue->ctrl->ctrl.device,
+ "pdu type %d has unexpected header length (%d)\n",
+ hdr->type, hdr->hlen);
+ return -EPROTO;
+ }
+
if (unlikely(hdr->type == nvme_tcp_c2h_term)) {
/*
* C2HTermReq never includes Header or Data digests.
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -149,6 +149,6 @@
return queue - queue->ctrl->queues;
}
-static inline struct blk_mq_tags *nvme_tcp_tagset(struct nvme_tcp_queue *queue)
-{
- u32 queue_idx = nvme_tcp_queue_id(queue);
+/*
+ * Check if the queue is TLS encrypted
+ */
@@ -674,6 +818,6 @@
return 0;
hdr = queue->pdu;
- if (queue->hdr_digest) {
- ret = nvme_tcp_verify_hdgst(queue, queue->pdu, hdr->hlen);
- if (unlikely(ret))
+ if (unlikely(hdr->type == nvme_tcp_c2h_term)) {
+ /*
+ * C2HTermReq never includes Header or Data digests.
================================================================================
* DELTA DIFFERENCES - code changes that differ between the patches *
================================================================================
--- b/net/ipv4/udp_offload.c
+++ b/net/ipv4/udp_offload.c
@@ -11,7 +11,6 @@
*/
#include <linux/skbuff.h>
-#include <net/ip6_checksum.h>
#include <net/udp.h>
#include <net/protocol.h>
#include <net/inet_common.h>
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/net/ipv4/udp_offload.c
+++ b/net/ipv4/udp_offload.c
@@ -273,6 +269,6 @@
if (skb_shinfo(gso_skb)->gso_type & SKB_GSO_FRAGLIST)
return __udp_gso_segment_list(gso_skb, features, is_ipv6);
- mss = skb_shinfo(gso_skb)->gso_size;
- if (gso_skb->len <= sizeof(*uh) + mss)
+ skb_pull(gso_skb, sizeof(*uh));
+
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/net/ipv4/udp_offload.c
+++ b/net/ipv4/udp_offload.c
@@ -273,6 +273,6 @@
bool copy_dtor;
__sum16 check;
__be16 newlen;
- if (skb_shinfo(gso_skb)->gso_type & SKB_GSO_FRAGLIST) {
- /* Detect modified geometry and pass those to skb_segment. */
+ mss = skb_shinfo(gso_skb)->gso_size;
+ if (gso_skb->len <= sizeof(*uh) + mss)
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -1607,7 +1613,6 @@
cpu_detect(c);
get_cpu_vendor(c);
get_cpu_cap(c);
- get_cpu_address_sizes(c);
setup_force_cpu_cap(X86_FEATURE_CPUID);
cpu_parse_early_param();
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -1502,6 +1597,5 @@
get_cpu_vendor(c);
get_cpu_cap(c);
- get_model_name(c); /* RHEL8: get model name for unsupported check */
get_cpu_address_sizes(c);
setup_force_cpu_cap(X86_FEATURE_CPUID);
cpu_parse_early_param();
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -1508,5 +1509,4 @@
get_cpu_cap(c);
- get_model_name(c); /* RHEL8: get model name for unsupported check */
setup_force_cpu_cap(X86_FEATURE_CPUID);
cpu_parse_early_param();
@@ -1520,5 +1599,4 @@
} else {
- identify_cpu_without_cpuid(c);
setup_clear_cpu_cap(X86_FEATURE_CPUID);
}
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/drivers/net/ethernet/microsoft/Kconfig
+++ b/drivers/net/ethernet/microsoft/Kconfig
@@ -20,4 +20,4 @@
depends on PCI_MSI && X86_64
depends on PCI_HYPERV
select AUXILIARY_BUS
- help
+ select PAGE_POOL
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/drivers/net/ethernet/microsoft/Kconfig
+++ b/drivers/net/ethernet/microsoft/Kconfig
@@ -21,4 +21,4 @@
depends on X86_64 || (ARM64 && !CPU_BIG_ENDIAN && ARM64_4K_PAGES)
depends on PCI_HYPERV
select AUXILIARY_BUS
- help
+ select PAGE_POOL
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -217,17 +256,6 @@
struct pci_bus_region region, inverted_region;
const char *res_name = pci_resource_name(dev, res - dev->resource);
- mask = type ? PCI_ROM_ADDRESS_MASK : ~0;
-
- /* No printks while decoding is disabled! */
- if (!dev->mmio_always_on) {
- pci_read_config_word(dev, PCI_COMMAND, &orig_cmd);
- if (orig_cmd & PCI_COMMAND_DECODE_ENABLE) {
- pci_write_config_word(dev, PCI_COMMAND,
- orig_cmd & ~PCI_COMMAND_DECODE_ENABLE);
- }
- }
-
res->name = pci_name(dev);
pci_read_config_dword(dev, pos, &l);
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/drivers/pci/iov.c
+++ b/drivers/pci/iov.c
@@ -740,4 +739,5 @@
struct resource *res;
+ const char *res_name;
struct pci_dev *pdev;
pci_read_config_word(dev, pos + PCI_SRIOV_CTRL, &ctrl);
--- b/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -186,6 +180,7 @@
u64 l64, sz64, mask64;
u16 orig_cmd;
struct pci_bus_region region, inverted_region;
+ const char *res_name = pci_resource_name(dev, res - dev->resource);
mask = type ? PCI_ROM_ADDRESS_MASK : ~0;
================================================================================
* DELTA DIFFERENCES - code changes that differ between the patches *
================================================================================
--- b/drivers/net/ethernet/intel/idpf/idpf_singleq_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_singleq_txrx.c
@@ -1,8 +1,6 @@
// SPDX-License-Identifier: GPL-2.0-only
/* Copyright (C) 2023 Intel Corporation */
-#include <net/libeth/tx.h>
-
#include "idpf.h"
/**
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -1,8 +1,6 @@
// SPDX-License-Identifier: GPL-2.0-only
/* Copyright (C) 2023 Intel Corporation */
-#include <net/libeth/tx.h>
-
#include "idpf.h"
#include "idpf_virtchnl.h"
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/drivers/net/ethernet/intel/idpf/idpf_singleq_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_singleq_txrx.c
@@ -2,6 +2,7 @@
/* Copyright (C) 2023 Intel Corporation */
#include <net/libeth/rx.h>
+#include <net/libeth/tx.h>
#include "idpf.h"
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -2,6 +2,7 @@
/* Copyright (C) 2023 Intel Corporation */
#include <net/libeth/rx.h>
+#include <net/libeth/tx.h>
#include "idpf.h"
#include "idpf_virtchnl.h"
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/drivers/net/ethernet/intel/idpf/idpf_singleq_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_singleq_txrx.c
@@ -2,4 +1,8 @@
/* Copyright (C) 2023 Intel Corporation */
+#include <net/libeth/rx.h>
+
+#include <net/libeth/tx.h>
+
#include "idpf.h"
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -2,5 +1,9 @@
/* Copyright (C) 2023 Intel Corporation */
+#include <net/libeth/rx.h>
+
+#include <net/libeth/tx.h>
+
#include "idpf.h"
#include "idpf_virtchnl.h"
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
@@ -785,7 +785,7 @@
u32 next_to_use;
u32 next_to_clean;
- u32 num_completions;
+ aligned_u64 num_completions;
__cacheline_group_end_aligned(read_write);
__cacheline_group_begin_aligned(cold);
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
@@ -971,4 +920,4 @@
u32 num_completions_pending;
};
-/**
+static inline int idpf_q_vector_to_mem(const struct idpf_q_vector *q_vector)
================================================================================
* DELTA DIFFERENCES - code changes that differ between the patches *
================================================================================
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -247,7 +247,6 @@
struct device *dev = tx_q->dev;
struct idpf_sw_queue *refillq;
int err;
- unsigned int i = 0;
err = idpf_tx_buf_alloc_all(tx_q);
if (err)
@@ -282,7 +281,7 @@
goto err_alloc;
}
- for (i = 0; i < refillq->desc_count; i++)
+ for (unsigned int i = 0; i < refillq->desc_count; i++)
refillq->ring[i] =
FIELD_PREP(IDPF_RFL_BI_BUFID_M, i) |
FIELD_PREP(IDPF_RFL_BI_GEN_M,
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -3551,7 +3630,7 @@
skip_data:
rx_buf->netmem = 0;
- idpf_rx_post_buf_refill(refillq, buf_id);
+ idpf_post_buf_refill(refillq, buf_id);
IDPF_RX_BUMP_NTC(rxq, ntc);
/* skip if it is non EOP desc */
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
@@ -622,6 +622,7 @@
* @cleaned_pkts: Number of packets cleaned for the above said case
* @tx_max_bufs: Max buffers that can be transmitted with scatter-gather
* @stash: Tx buffer stash for Flow-based scheduling mode
+ * @refillq: Pointer to refill queue
* @compl_tag_bufid_m: Completion tag buffer id mask
* @compl_tag_cur_gen: Used to keep track of current completion tag generation
* @compl_tag_gen_max: To determine when compl_tag_cur_gen should be reset
@@ -671,6 +672,7 @@
u16 tx_max_bufs;
struct idpf_txq_stash *stash;
+ struct idpf_sw_queue *refillq;
u16 compl_tag_bufid_m;
u16 compl_tag_cur_gen;
@@ -692,7 +694,7 @@
__cacheline_group_end_aligned(cold);
};
libeth_cacheline_set_assert(struct idpf_tx_queue, 64,
- 112 + sizeof(struct u64_stats_sync),
+ 120 + sizeof(struct u64_stats_sync),
24);
/**
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -244,4 +246,5 @@
struct device *dev = tx_q->dev;
int err;
+ unsigned int i = 0;
err = idpf_tx_buf_alloc_all(tx_q);
@@ -3358,4 +3417,4 @@
idpf_rx_post_buf_refill(refillq, buf_id);
-
IDPF_RX_BUMP_NTC(rxq, ntc);
+
/* skip if it is non EOP desc */
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
@@ -624,6 +619,6 @@
* @cleaned_pkts: Number of packets cleaned for the above said case
* @tx_max_bufs: Max buffers that can be transmitted with scatter-gather
- * @tx_min_pkt_len: Min supported packet length
+ * @stash: Tx buffer stash for Flow-based scheduling mode
* @compl_tag_bufid_m: Completion tag buffer id mask
- * @compl_tag_gen_s: Completion tag generation bit
- * The format of the completion tag will change based on the TXQ
+ * @compl_tag_cur_gen: Used to keep track of current completion tag generation
+ * @compl_tag_gen_max: To determine when compl_tag_cur_gen should be reset
@@ -743,6 +692,2 @@
- u16 tx_max_bufs;
- u16 tx_min_pkt_len;
-
- u16 compl_tag_bufid_m;
- u16 compl_tag_gen_s;
+/**
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
@@ -610,6 +610,8 @@
* @netdev: &net_device corresponding to this queue
* @next_to_use: Next descriptor to use
* @next_to_clean: Next descriptor to clean
+ * @last_re: last descriptor index that RE bit was set
+ * @tx_max_bufs: Max buffers that can be transmitted with scatter-gather
* @cleaned_bytes: Splitq only, TXQ only: When a TX completion is received on
* the TX completion queue, it can be for any TXQ associated
* with that completion queue. This means we can clean up to
@@ -620,7 +622,6 @@
* only once at the end of the cleaning routine.
* @clean_budget: singleq only, queue cleaning budget
* @cleaned_pkts: Number of packets cleaned for the above said case
- * @tx_max_bufs: Max buffers that can be transmitted with scatter-gather
* @stash: Tx buffer stash for Flow-based scheduling mode
* @refillq: Pointer to refill queue
* @compl_tag_bufid_m: Completion tag buffer id mask
@@ -672,7 +675,6 @@
};
u16 cleaned_pkts;
- u16 tx_max_bufs;
struct idpf_txq_stash *stash;
struct idpf_sw_queue *refillq;
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
@@ -607,6 +607,5 @@
- * @desc_count: Number of descriptors
+ * @netdev: &net_device corresponding to this queue
* @next_to_use: Next descriptor to use
* @next_to_clean: Next descriptor to clean
- * @netdev: &net_device corresponding to this queue
* @cleaned_bytes: Splitq only, TXQ only: When a TX completion is received on
* the TX completion queue, it can be for any TXQ associated
@@ -685,5 +622,5 @@
* @cleaned_pkts: Number of packets cleaned for the above said case
* @tx_max_bufs: Max buffers that can be transmitted with scatter-gather
- * @tx_min_pkt_len: Min supported packet length
+ * @stash: Tx buffer stash for Flow-based scheduling mode
* @refillq: Pointer to refill queue
* @compl_tag_bufid_m: Completion tag buffer id mask
@@ -723,4 +660,4 @@
- u16 desc_count;
+ __cacheline_group_begin_aligned(read_write);
u16 next_to_use;
u16 next_to_clean;
@@ -731,5 +668,5 @@
u16 tx_max_bufs;
- u16 tx_min_pkt_len;
+ struct idpf_txq_stash *stash;
struct idpf_sw_queue *refillq;
================================================================================
* DELTA DIFFERENCES - code changes that differ between the patches *
================================================================================
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -2287,4 +2287,55 @@
/**
+ * idpf_tx_dma_map_error - handle TX DMA map errors
+ * @txq: queue to send buffer on
+ * @skb: send buffer
+ * @first: original first buffer info buffer for packet
+ * @idx: starting point on ring to unwind
+ */
+void idpf_tx_dma_map_error(struct idpf_tx_queue *txq, struct sk_buff *skb,
+ struct idpf_tx_buf *first, u16 idx)
+{
+ struct libeth_sq_napi_stats ss = { };
+ struct libeth_cq_pp cp = {
+ .dev = txq->dev,
+ .ss = &ss,
+ };
+
+ u64_stats_update_begin(&txq->stats_sync);
+ u64_stats_inc(&txq->q_stats.dma_map_errs);
+ u64_stats_update_end(&txq->stats_sync);
+
+ /* clear dma mappings for failed tx_buf map */
+ for (;;) {
+ struct idpf_tx_buf *tx_buf;
+
+ tx_buf = &txq->tx_buf[idx];
+ libeth_tx_complete(tx_buf, &cp);
+ if (tx_buf == first)
+ break;
+ if (idx == 0)
+ idx = txq->desc_count;
+ idx--;
+ }
+
+ if (skb_is_gso(skb)) {
+ union idpf_tx_flex_desc *tx_desc;
+
+ /* If we failed a DMA mapping for a TSO packet, we will have
+ * used one additional descriptor for a context
+ * descriptor. Reset that here.
+ */
+ tx_desc = &txq->flex_tx[idx];
+ memset(tx_desc, 0, sizeof(struct idpf_flex_tx_ctx_desc));
+ if (idx == 0)
+ idx = txq->desc_count;
+ idx--;
+ }
+
+ /* Update tail in case netdev_xmit_more was previously true */
+ idpf_tx_buf_hw_update(txq, idx, false);
+}
+
+/**
* idpf_tx_splitq_bump_ntu - adjust NTU and generation
* @txq: the tx ring to wrap
@@ -2335,35 +2386,4 @@
/**
- * idpf_tx_splitq_pkt_err_unmap - Unmap buffers and bump tail in case of error
- * @txq: Tx queue to unwind
- * @params: pointer to splitq params struct
- * @first: starting buffer for packet to unmap
- */
-static void idpf_tx_splitq_pkt_err_unmap(struct idpf_tx_queue *txq,
- struct idpf_tx_splitq_params *params,
- struct idpf_tx_buf *first)
-{
- struct libeth_sq_napi_stats ss = { };
- struct idpf_tx_buf *tx_buf = first;
- struct libeth_cq_pp cp = {
- .dev = txq->dev,
- .ss = &ss,
- };
- u32 idx = 0;
-
- u64_stats_update_begin(&txq->stats_sync);
- u64_stats_inc(&txq->q_stats.dma_map_errs);
- u64_stats_update_end(&txq->stats_sync);
-
- do {
- libeth_tx_complete(tx_buf, &cp);
- idpf_tx_clean_buf_ring_bump_ntc(txq, idx, tx_buf);
- } while (idpf_tx_buf_compl_tag(tx_buf) == params->compl_tag);
-
- /* Update tail in case netdev_xmit_more was previously true. */
- idpf_tx_buf_hw_update(txq, params->prev_ntu, false);
-}
-
-/**
* idpf_tx_splitq_map - Build the Tx flex descriptor
* @tx_q: queue to send buffer on
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -2339,57 +2339,6 @@
return count;
}
-/**
- * idpf_tx_dma_map_error - handle TX DMA map errors
- * @txq: queue to send buffer on
- * @skb: send buffer
- * @first: original first buffer info buffer for packet
- * @idx: starting point on ring to unwind
- */
-void idpf_tx_dma_map_error(struct idpf_tx_queue *txq, struct sk_buff *skb,
- struct idpf_tx_buf *first, u16 idx)
-{
- struct libeth_sq_napi_stats ss = { };
- struct libeth_cq_pp cp = {
- .dev = txq->dev,
- .ss = &ss,
- };
-
- u64_stats_update_begin(&txq->stats_sync);
- u64_stats_inc(&txq->q_stats.dma_map_errs);
- u64_stats_update_end(&txq->stats_sync);
-
- /* clear dma mappings for failed tx_buf map */
- for (;;) {
- struct idpf_tx_buf *tx_buf;
-
- tx_buf = &txq->tx_buf[idx];
- libeth_tx_complete(tx_buf, &cp);
- if (tx_buf == first)
- break;
- if (idx == 0)
- idx = txq->desc_count;
- idx--;
- }
-
- if (skb_is_gso(skb)) {
- union idpf_tx_flex_desc *tx_desc;
-
- /* If we failed a DMA mapping for a TSO packet, we will have
- * used one additional descriptor for a context
- * descriptor. Reset that here.
- */
- tx_desc = &txq->flex_tx[idx];
- memset(tx_desc, 0, sizeof(*tx_desc));
- if (idx == 0)
- idx = txq->desc_count;
- idx--;
- }
-
- /* Update tail in case netdev_xmit_more was previously true */
- idpf_tx_buf_hw_update(txq, idx, false);
-}
-
/**
* idpf_tx_splitq_bump_ntu - adjust NTU and generation
* @txq: the tx ring to wrap
@@ -2438,6 +2387,37 @@
return true;
}
+/**
+ * idpf_tx_splitq_pkt_err_unmap - Unmap buffers and bump tail in case of error
+ * @txq: Tx queue to unwind
+ * @params: pointer to splitq params struct
+ * @first: starting buffer for packet to unmap
+ */
+static void idpf_tx_splitq_pkt_err_unmap(struct idpf_tx_queue *txq,
+ struct idpf_tx_splitq_params *params,
+ struct idpf_tx_buf *first)
+{
+ struct libeth_sq_napi_stats ss = { };
+ struct idpf_tx_buf *tx_buf = first;
+ struct libeth_cq_pp cp = {
+ .dev = txq->dev,
+ .ss = &ss,
+ };
+ u32 idx = 0;
+
+ u64_stats_update_begin(&txq->stats_sync);
+ u64_stats_inc(&txq->q_stats.dma_map_errs);
+ u64_stats_update_end(&txq->stats_sync);
+
+ do {
+ libeth_tx_complete(tx_buf, &cp);
+ idpf_tx_clean_buf_ring_bump_ntc(txq, idx, tx_buf);
+ } while (idpf_tx_buf_compl_tag(tx_buf) == params->compl_tag);
+
+ /* Update tail in case netdev_xmit_more was previously true. */
+ idpf_tx_buf_hw_update(txq, params->prev_ntu, false);
+}
+
/**
* idpf_tx_splitq_map - Build the Tx flex descriptor
* @tx_q: queue to send buffer on
@@ -2482,8 +2462,9 @@
for (frag = &skb_shinfo(skb)->frags[0];; frag++) {
unsigned int max_data = IDPF_TX_MAX_DESC_DATA_ALIGNED;
- if (dma_mapping_error(tx_q->dev, dma))
- return idpf_tx_dma_map_error(tx_q, skb, first, i);
+ if (unlikely(dma_mapping_error(tx_q->dev, dma)))
+ return idpf_tx_splitq_pkt_err_unmap(tx_q, params,
+ first);
first->nr_frags++;
idpf_tx_buf_compl_tag(tx_buf) = params->compl_tag;
@@ -2939,7 +2920,9 @@
static netdev_tx_t idpf_tx_splitq_frame(struct sk_buff *skb,
struct idpf_tx_queue *tx_q)
{
- struct idpf_tx_splitq_params tx_params = { };
+ struct idpf_tx_splitq_params tx_params = {
+ .prev_ntu = tx_q->next_to_use,
+ };
union idpf_flex_tx_ctx_desc *ctx_desc;
struct idpf_tx_buf *first;
unsigned int count;
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -2326,7 +2380,7 @@
* descriptor. Reset that here.
*/
tx_desc = &txq->flex_tx[idx];
- memset(tx_desc, 0, sizeof(struct idpf_flex_tx_ctx_desc));
+ memset(tx_desc, 0, sizeof(*tx_desc));
if (idx == 0)
idx = txq->desc_count;
idx--;
@@ -2817,4 +2871,5 @@
{
struct idpf_tx_splitq_params tx_params = { };
+ union idpf_flex_tx_ctx_desc *ctx_desc;
struct idpf_tx_buf *first;
unsigned int count;
================================================================================
* DELTA DIFFERENCES - code changes that differ between the patches *
================================================================================
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -1915,11 +1915,8 @@
.napi = budget,
};
- tx_buf = &txq->tx_buf[buf_id];
- if (tx_buf->type == LIBETH_SQE_SKB) {
+ if (tx_buf->type == LIBETH_SQE_SKB)
libeth_tx_complete(tx_buf, &cp);
- idpf_post_buf_refill(txq->refillq, buf_id);
- }
while (idpf_tx_buf_next(tx_buf) != IDPF_TXBUF_NULL) {
buf_id = idpf_tx_buf_next(tx_buf);
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -1962,6 +1962,7 @@
idpf_tx_buf_compl_tag(tx_buf) != compl_tag))
return false;
+ tx_buf = &txq->tx_buf[buf_id];
if (tx_buf->type == LIBETH_SQE_SKB) {
if (skb_shinfo(tx_buf->skb)->tx_flags & SKBTX_IN_PROGRESS)
idpf_tx_read_tstamp(txq, tx_buf->skb);
@@ -1965,6 +1966,7 @@
idpf_tx_read_tstamp(txq, tx_buf->skb);
libeth_tx_complete(tx_buf, &cp);
+ idpf_post_buf_refill(txq->refillq, buf_id);
}
idpf_tx_clean_buf_ring_bump_ntc(txq, idx, tx_buf);
@@ -2892,6 +2859,7 @@
struct idpf_tx_buf *first;
unsigned int count;
int tso, idx;
+ u32 buf_id;
count = idpf_tx_desc_count_required(tx_q, skb);
if (unlikely(!count))
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
@@ -703,6 +710,7 @@
dma_addr_t dma;
struct idpf_q_vector *q_vector;
+ u32 buf_pool_size;
__cacheline_group_end_aligned(cold);
};
libeth_cacheline_set_assert(struct idpf_tx_queue, 64,
@@ -706,7 +714,7 @@
};
libeth_cacheline_set_assert(struct idpf_tx_queue, 64,
120 + sizeof(struct u64_stats_sync),
- 24);
+ 32);
/**
* struct idpf_buf_queue - software structure representing a buffer queue
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -1915,8 +1965,12 @@
idpf_tx_buf_compl_tag(tx_buf) != compl_tag))
return false;
- if (tx_buf->type == LIBETH_SQE_SKB)
+ if (tx_buf->type == LIBETH_SQE_SKB) {
+ if (skb_shinfo(tx_buf->skb)->tx_flags & SKBTX_IN_PROGRESS)
+ idpf_tx_read_tstamp(txq, tx_buf->skb);
+
libeth_tx_complete(tx_buf, &cp);
+ }
idpf_tx_clean_buf_ring_bump_ntc(txq, idx, tx_buf);
@@ -2744,4 +2798,4 @@
- struct idpf_flex_tx_ctx_desc *desc;
+ union idpf_flex_tx_ctx_desc *desc;
int i = txq->next_to_use;
txq->tx_buf[i].type = LIBETH_SQE_CTX;
@@ -2802,6 +2856,6 @@
struct idpf_tx_buf *first;
unsigned int count;
- int tso;
+ int tso, idx;
count = idpf_tx_desc_count_required(tx_q, skb);
if (unlikely(!count))
@@ -2840,4 +2962,4 @@
- u64_stats_update_end(&tx_q->stats_sync);
+ idpf_tx_set_tstamp_desc(ctx_desc, idx);
}
/* record the location of the first descriptor for this packet */
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
@@ -692,5 +694,9 @@
struct idpf_q_vector *q_vector;
-} ____cacheline_aligned;
+ __cacheline_group_end_aligned(cold);
+};
+libeth_cacheline_set_assert(struct idpf_tx_queue, 64,
+ 120 + sizeof(struct u64_stats_sync),
+ 24);
/**
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -2909,7 +2926,7 @@
};
union idpf_flex_tx_ctx_desc *ctx_desc;
struct idpf_tx_buf *first;
- unsigned int count;
+ u32 count, buf_count = 1;
int tso, idx;
u32 buf_id;
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -2768,7 +2821,8 @@
};
+ union idpf_flex_tx_ctx_desc *ctx_desc;
struct idpf_tx_buf *first;
unsigned int count;
- int tso;
+ int tso, idx;
u32 buf_id;
count = idpf_tx_desc_count_required(tx_q, skb);
================================================================================
* DELTA DIFFERENCES - code changes that differ between the patches *
================================================================================
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -1557,6 +1557,82 @@
wake_up(&vport->sw_marker_wq);
}
+/**
+ * idpf_tx_clean_stashed_bufs - clean bufs that were stored for
+ * out of order completions
+ * @txq: queue to clean
+ * @compl_tag: completion tag of packet to clean (from completion descriptor)
+ * @cleaned: pointer to stats struct to track cleaned packets/bytes
+ * @budget: Used to determine if we are in netpoll
+ */
+static void idpf_tx_clean_stashed_bufs(struct idpf_tx_queue *txq,
+ u16 compl_tag,
+ struct libeth_sq_napi_stats *cleaned,
+ int budget)
+{
+ struct idpf_tx_stash *stash;
+ struct hlist_node *tmp_buf;
+ struct libeth_cq_pp cp = {
+ .dev = txq->dev,
+ .ss = cleaned,
+ .napi = budget,
+ };
+
+ /* Buffer completion */
+ hash_for_each_possible_safe(txq->stash->sched_buf_hash, stash, tmp_buf,
+ hlist, compl_tag) {
+ if (unlikely(idpf_tx_buf_compl_tag(&stash->buf) != compl_tag))
+ continue;
+
+ hash_del(&stash->hlist);
+ libeth_tx_complete(&stash->buf, &cp);
+
+ /* Push shadow buf back onto stack */
+ idpf_buf_lifo_push(&txq->stash->buf_stack, stash);
+ }
+}
+
+/**
+ * idpf_stash_flow_sch_buffers - store buffer parameters info to be freed at a
+ * later time (only relevant for flow scheduling mode)
+ * @txq: Tx queue to clean
+ * @tx_buf: buffer to store
+ */
+static int idpf_stash_flow_sch_buffers(struct idpf_tx_queue *txq,
+ struct idpf_tx_buf *tx_buf)
+{
+ struct idpf_tx_stash *stash;
+
+ if (unlikely(tx_buf->type <= LIBETH_SQE_CTX))
+ return 0;
+
+ stash = idpf_buf_lifo_pop(&txq->stash->buf_stack);
+ if (unlikely(!stash)) {
+ net_err_ratelimited("%s: No out-of-order TX buffers left!\n",
+ netdev_name(txq->netdev));
+
+ return -ENOMEM;
+ }
+
+ /* Store buffer params in shadow buffer */
+ stash->buf.skb = tx_buf->skb;
+ stash->buf.bytes = tx_buf->bytes;
+ stash->buf.packets = tx_buf->packets;
+ stash->buf.type = tx_buf->type;
+ stash->buf.nr_frags = tx_buf->nr_frags;
+ dma_unmap_addr_set(&stash->buf, dma, dma_unmap_addr(tx_buf, dma));
+ dma_unmap_len_set(&stash->buf, len, dma_unmap_len(tx_buf, len));
+ idpf_tx_buf_compl_tag(&stash->buf) = idpf_tx_buf_compl_tag(tx_buf);
+
+ /* Add buffer to buf_hash table to be freed later */
+ hash_add(txq->stash->sched_buf_hash, &stash->hlist,
+ idpf_tx_buf_compl_tag(&stash->buf));
+
+ tx_buf->type = LIBETH_SQE_EMPTY;
+
+ return 0;
+}
+
#define idpf_tx_splitq_clean_bump_ntc(txq, ntc, desc, buf) \
do { \
if (unlikely(++(ntc) == (txq)->desc_count)) { \
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
@@ -663,6 +663,7 @@
* @cleaned_pkts: Number of packets cleaned for the above said case
* @tx_min_pkt_len: Min supported packet length
* @refillq: Pointer to refill queue
+ * @compl_tag_bufid_m: Completion tag buffer id mask
* @compl_tag_gen_s: Completion tag generation bit
* The format of the completion tag will change based on the TXQ
* descriptor ring size so that we can maintain roughly the same level
@@ -683,6 +684,9 @@
* --------------------------------
*
* This gives us 8*8160 = 65280 possible unique values.
+ * @compl_tag_cur_gen: Used to keep track of current completion tag generation
+ * @compl_tag_gen_max: To determine when compl_tag_cur_gen should be reset
+ * @stash: Tx buffer stash for Flow-based scheduling mode
* @stats_sync: See struct u64_stats_sync
* @q_stats: See union idpf_tx_queue_stats
* @q_id: Queue id
@@ -724,6 +728,14 @@
u16 tx_min_pkt_len;
struct idpf_sw_queue *refillq;
+ u16 compl_tag_bufid_m;
+ u16 compl_tag_gen_s;
+
+ u16 compl_tag_cur_gen;
+ u16 compl_tag_gen_max;
+
+ struct idpf_txq_stash *stash;
+
struct u64_stats_sync stats_sync;
struct idpf_tx_queue_stats q_stats;
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -1602,87 +1462,6 @@
spin_unlock_bh(&tx_tstamp_caps->status_lock);
}
-/**
- * idpf_tx_clean_stashed_bufs - clean bufs that were stored for
- * out of order completions
- * @txq: queue to clean
- * @compl_tag: completion tag of packet to clean (from completion descriptor)
- * @cleaned: pointer to stats struct to track cleaned packets/bytes
- * @budget: Used to determine if we are in netpoll
- */
-static void idpf_tx_clean_stashed_bufs(struct idpf_tx_queue *txq,
- u16 compl_tag,
- struct libeth_sq_napi_stats *cleaned,
- int budget)
-{
- struct idpf_tx_stash *stash;
- struct hlist_node *tmp_buf;
- struct libeth_cq_pp cp = {
- .dev = txq->dev,
- .ss = cleaned,
- .napi = budget,
- };
-
- /* Buffer completion */
- hash_for_each_possible_safe(txq->stash->sched_buf_hash, stash, tmp_buf,
- hlist, compl_tag) {
- if (unlikely(idpf_tx_buf_compl_tag(&stash->buf) != compl_tag))
- continue;
-
- hash_del(&stash->hlist);
-
- if (stash->buf.type == LIBETH_SQE_SKB &&
- (skb_shinfo(stash->buf.skb)->tx_flags & SKBTX_IN_PROGRESS))
- idpf_tx_read_tstamp(txq, stash->buf.skb);
-
- libeth_tx_complete(&stash->buf, &cp);
-
- /* Push shadow buf back onto stack */
- idpf_buf_lifo_push(&txq->stash->buf_stack, stash);
- }
-}
-
-/**
- * idpf_stash_flow_sch_buffers - store buffer parameters info to be freed at a
- * later time (only relevant for flow scheduling mode)
- * @txq: Tx queue to clean
- * @tx_buf: buffer to store
- */
-static int idpf_stash_flow_sch_buffers(struct idpf_tx_queue *txq,
- struct idpf_tx_buf *tx_buf)
-{
- struct idpf_tx_stash *stash;
-
- if (unlikely(tx_buf->type <= LIBETH_SQE_CTX))
- return 0;
-
- stash = idpf_buf_lifo_pop(&txq->stash->buf_stack);
- if (unlikely(!stash)) {
- net_err_ratelimited("%s: No out-of-order TX buffers left!\n",
- netdev_name(txq->netdev));
-
- return -ENOMEM;
- }
-
- /* Store buffer params in shadow buffer */
- stash->buf.skb = tx_buf->skb;
- stash->buf.bytes = tx_buf->bytes;
- stash->buf.packets = tx_buf->packets;
- stash->buf.type = tx_buf->type;
- stash->buf.nr_frags = tx_buf->nr_frags;
- dma_unmap_addr_set(&stash->buf, dma, dma_unmap_addr(tx_buf, dma));
- dma_unmap_len_set(&stash->buf, len, dma_unmap_len(tx_buf, len));
- idpf_tx_buf_compl_tag(&stash->buf) = idpf_tx_buf_compl_tag(tx_buf);
-
- /* Add buffer to buf_hash table to be freed later */
- hash_add(txq->stash->sched_buf_hash, &stash->hlist,
- idpf_tx_buf_compl_tag(&stash->buf));
-
- tx_buf->type = LIBETH_SQE_EMPTY;
-
- return 0;
-}
-
#define idpf_tx_splitq_clean_bump_ntc(txq, ntc, desc, buf) \
do { \
if (unlikely(++(ntc) == (txq)->desc_count)) { \
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
@@ -598,7 +565,6 @@
* only once at the end of the cleaning routine.
* @clean_budget: singleq only, queue cleaning budget
* @cleaned_pkts: Number of packets cleaned for the above said case
- * @stash: Tx buffer stash for Flow-based scheduling mode
* @refillq: Pointer to refill queue
* @compl_tag_bufid_m: Completion tag buffer id mask
* @compl_tag_cur_gen: Used to keep track of current completion tag generation
@@ -600,9 +566,6 @@
* @cleaned_pkts: Number of packets cleaned for the above said case
* @stash: Tx buffer stash for Flow-based scheduling mode
* @refillq: Pointer to refill queue
- * @compl_tag_bufid_m: Completion tag buffer id mask
- * @compl_tag_cur_gen: Used to keep track of current completion tag generation
- * @compl_tag_gen_max: To determine when compl_tag_cur_gen should be reset
* @cached_tstamp_caps: Tx timestamp capabilities negotiated with the CP
* @tstamp_task: Work that handles Tx timestamp read
* @stats_sync: See struct u64_stats_sync
@@ -633,7 +596,6 @@
u16 desc_count;
u16 tx_min_pkt_len;
- u16 compl_tag_gen_s;
struct net_device *netdev;
__cacheline_group_end_aligned(read_mostly);
@@ -650,7 +612,6 @@
};
u16 cleaned_pkts;
- struct idpf_txq_stash *stash;
struct idpf_sw_queue *refillq;
u16 compl_tag_bufid_m;
@@ -653,10 +614,6 @@
struct idpf_txq_stash *stash;
struct idpf_sw_queue *refillq;
- u16 compl_tag_bufid_m;
- u16 compl_tag_cur_gen;
- u16 compl_tag_gen_max;
-
struct idpf_ptp_vport_tx_tstamp_caps *cached_tstamp_caps;
struct work_struct *tstamp_task;
@@ -674,7 +631,7 @@
__cacheline_group_end_aligned(cold);
};
libeth_cacheline_set_assert(struct idpf_tx_queue, 64,
- 120 + sizeof(struct u64_stats_sync),
+ 104 + sizeof(struct u64_stats_sync),
32);
/**
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -3,4 +3,4 @@
-#include "idpf.h"
+#include "idpf_ptp.h"
#include "idpf_virtchnl.h"
struct idpf_tx_stash {
@@ -1725,6 +1770,11 @@
continue;
hash_del(&stash->hlist);
+
+ if (stash->buf.type == LIBETH_SQE_SKB &&
+ (skb_shinfo(stash->buf.skb)->tx_flags & SKBTX_IN_PROGRESS))
+ idpf_tx_read_tstamp(txq, stash->buf.skb);
+
libeth_tx_complete(&stash->buf, &cp);
/* Push shadow buf back onto stack */
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
@@ -713,9 +663,7 @@
+ u16 desc_count;
+
+ u16 tx_min_pkt_len;
+ u16 compl_tag_gen_s;
+
+ struct net_device *netdev;
+ __cacheline_group_end_aligned(read_mostly);
- * --------------------------------
- *
- * This gives us 8*8160 = 65280 possible unique values.
- * @compl_tag_cur_gen: Used to keep track of current completion tag generation
- * @compl_tag_gen_max: To determine when compl_tag_cur_gen should be reset
- * @stash: Tx buffer stash for Flow-based scheduling mode
- * @stats_sync: See struct u64_stats_sync
- * @q_stats: See union idpf_tx_queue_stats
- * @q_id: Queue id
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/kernel/exit.c
+++ b/kernel/exit.c
@@ -475,6 +475,18 @@
BUG_ON(mm != current->active_mm);
/* more a memory barrier than a real lock */
task_lock(current);
+ /*
+ * When a thread stops operating on an address space, the loop
+ * in membarrier_private_expedited() may not observe that
+ * tsk->mm, and the loop in membarrier_global_expedited() may
+ * not observe a MEMBARRIER_STATE_GLOBAL_EXPEDITED
+ * rq->membarrier_state, so those would not issue an IPI.
+ * Membarrier requires a memory barrier after accessing
+ * user-space memory, before clearing tsk->mm or the
+ * rq->membarrier_state.
+ */
+ smp_mb__after_spinlock();
+ local_irq_disable();
current->mm = NULL;
mmap_read_unlock(mm);
enter_lazy_tlb(mm, current);
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/include/linux/sched/mm.h
+++ b/include/linux/sched/mm.h
@@ -395,4 +363,4 @@
}
#endif
-#ifdef CONFIG_IOMMU_SVA
+#endif /* _LINUX_SCHED_MM_H */
--- b/kernel/exit.c
+++ b/kernel/exit.c
@@ -472,7 +472,6 @@
BUG_ON(mm != current->active_mm);
/* more a memory barrier than a real lock */
task_lock(current);
- current->user_dumpable = (get_dumpable(mm) == SUID_DUMP_USER);
current->mm = NULL;
mmap_read_unlock(mm);
enter_lazy_tlb(mm, current);
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -136,2 +133,3 @@
#define MIDR_CORTEX_X1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X1)
+#define MIDR_CORTEX_A510 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A510)
#define MIDR_CORTEX_A710 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A710)
================================================================================
* ONLY IN PATCH1 - files not modified by patch2 *
================================================================================
--- b/arch/arm64/kernel/proton-pack.c
+++ /dev/null
@@ -1,1157 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * Handle detection, reporting and mitigation of Spectre v1, v2, v3a and v4, as
- * detailed at:
- *
- * https://developer.arm.com/support/arm-security-updates/speculative-processor-vulnerability
- *
- * This code was originally written hastily under an awful lot of stress and so
- * aspects of it are somewhat hacky. Unfortunately, changing anything in here
- * instantly makes me feel ill. Thanks, Jann. Thann.
- *
- * Copyright (C) 2018 ARM Ltd, All Rights Reserved.
- * Copyright (C) 2020 Google LLC
- *
- * "If there's something strange in your neighbourhood, who you gonna call?"
- *
- * Authors: Will Deacon <will@kernel.org> and Marc Zyngier <maz@kernel.org>
- */
-
-#include <linux/arm-smccc.h>
-#include <linux/bpf.h>
-#include <linux/cpu.h>
-#include <linux/device.h>
-#include <linux/nospec.h>
-#include <linux/prctl.h>
-#include <linux/sched/task_stack.h>
-
-#include <asm/debug-monitors.h>
-#include <asm/insn.h>
-#include <asm/spectre.h>
-#include <asm/traps.h>
-#include <asm/vectors.h>
-#include <asm/virt.h>
-
-/*
- * We try to ensure that the mitigation state can never change as the result of
- * onlining a late CPU.
- */
-static void update_mitigation_state(enum mitigation_state *oldp,
- enum mitigation_state new)
-{
- enum mitigation_state state;
-
- do {
- state = READ_ONCE(*oldp);
- if (new <= state)
- break;
-
- /* Userspace almost certainly can't deal with this. */
- if (WARN_ON(system_capabilities_finalized()))
- break;
- } while (cmpxchg_relaxed(oldp, state, new) != state);
-}
-
-/*
- * Spectre v1.
- *
- * The kernel can't protect userspace for this one: it's each person for
- * themselves. Advertise what we're doing and be done with it.
- */
-ssize_t cpu_show_spectre_v1(struct device *dev, struct device_attribute *attr,
- char *buf)
-{
- return sprintf(buf, "Mitigation: __user pointer sanitization\n");
-}
-
-/*
- * Spectre v2.
- *
- * This one sucks. A CPU is either:
- *
- * - Mitigated in hardware and advertised by ID_AA64PFR0_EL1.CSV2.
- * - Mitigated in hardware and listed in our "safe list".
- * - Mitigated in software by firmware.
- * - Mitigated in software by a CPU-specific dance in the kernel and a
- * firmware call at EL2.
- * - Vulnerable.
- *
- * It's not unlikely for different CPUs in a big.LITTLE system to fall into
- * different camps.
- */
-static enum mitigation_state spectre_v2_state;
-
-static bool __read_mostly __nospectre_v2;
-static int __init parse_spectre_v2_param(char *str)
-{
- __nospectre_v2 = true;
- return 0;
-}
-early_param("nospectre_v2", parse_spectre_v2_param);
-
-static bool spectre_v2_mitigations_off(void)
-{
- bool ret = __nospectre_v2 || cpu_mitigations_off();
-
- if (ret)
- pr_info_once("spectre-v2 mitigation disabled by command line option\n");
-
- return ret;
-}
-
-static const char *get_bhb_affected_string(enum mitigation_state bhb_state)
-{
- switch (bhb_state) {
- case SPECTRE_UNAFFECTED:
- return "";
- default:
- case SPECTRE_VULNERABLE:
- return ", but not BHB";
- case SPECTRE_MITIGATED:
- return ", BHB";
- }
-}
-
-static bool _unprivileged_ebpf_enabled(void)
-{
-#ifdef CONFIG_BPF_SYSCALL
- return !sysctl_unprivileged_bpf_disabled;
-#else
- return false;
-#endif
-}
-
-ssize_t cpu_show_spectre_v2(struct device *dev, struct device_attribute *attr,
- char *buf)
-{
- enum mitigation_state bhb_state = arm64_get_spectre_bhb_state();
- const char *bhb_str = get_bhb_affected_string(bhb_state);
- const char *v2_str = "Branch predictor hardening";
-
- switch (spectre_v2_state) {
- case SPECTRE_UNAFFECTED:
- if (bhb_state == SPECTRE_UNAFFECTED)
- return sprintf(buf, "Not affected\n");
-
- /*
- * Platforms affected by Spectre-BHB can't report
- * "Not affected" for Spectre-v2.
- */
- v2_str = "CSV2";
- fallthrough;
- case SPECTRE_MITIGATED:
- if (bhb_state == SPECTRE_MITIGATED && _unprivileged_ebpf_enabled())
- return sprintf(buf, "Vulnerable: Unprivileged eBPF enabled\n");
-
- return sprintf(buf, "Mitigation: %s%s\n", v2_str, bhb_str);
- case SPECTRE_VULNERABLE:
- fallthrough;
- default:
- return sprintf(buf, "Vulnerable\n");
- }
-}
-
-static enum mitigation_state spectre_v2_get_cpu_hw_mitigation_state(void)
-{
- u64 pfr0;
- static const struct midr_range spectre_v2_safe_list[] = {
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A35),
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A53),
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A55),
- MIDR_ALL_VERSIONS(MIDR_BRAHMA_B53),
- MIDR_ALL_VERSIONS(MIDR_HISI_TSV110),
- MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_2XX_SILVER),
- MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_3XX_SILVER),
- MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_4XX_SILVER),
- { /* sentinel */ }
- };
-
- /* If the CPU has CSV2 set, we're safe */
- pfr0 = read_cpuid(ID_AA64PFR0_EL1);
- if (cpuid_feature_extract_unsigned_field(pfr0, ID_AA64PFR0_CSV2_SHIFT))
- return SPECTRE_UNAFFECTED;
-
- /* Alternatively, we have a list of unaffected CPUs */
- if (is_midr_in_range_list(read_cpuid_id(), spectre_v2_safe_list))
- return SPECTRE_UNAFFECTED;
-
- return SPECTRE_VULNERABLE;
-}
-
-static enum mitigation_state spectre_v2_get_cpu_fw_mitigation_state(void)
-{
- int ret;
- struct arm_smccc_res res;
-
- arm_smccc_1_1_invoke(ARM_SMCCC_ARCH_FEATURES_FUNC_ID,
- ARM_SMCCC_ARCH_WORKAROUND_1, &res);
-
- ret = res.a0;
- switch (ret) {
- case SMCCC_RET_SUCCESS:
- return SPECTRE_MITIGATED;
- case SMCCC_ARCH_WORKAROUND_RET_UNAFFECTED:
- return SPECTRE_UNAFFECTED;
- default:
- fallthrough;
- case SMCCC_RET_NOT_SUPPORTED:
- return SPECTRE_VULNERABLE;
- }
-}
-
-bool has_spectre_v2(const struct arm64_cpu_capabilities *entry, int scope)
-{
- WARN_ON(scope != SCOPE_LOCAL_CPU || preemptible());
-
- if (spectre_v2_get_cpu_hw_mitigation_state() == SPECTRE_UNAFFECTED)
- return false;
-
- if (spectre_v2_get_cpu_fw_mitigation_state() == SPECTRE_UNAFFECTED)
- return false;
-
- return true;
-}
-
-enum mitigation_state arm64_get_spectre_v2_state(void)
-{
- return spectre_v2_state;
-}
-
-DEFINE_PER_CPU_READ_MOSTLY(struct bp_hardening_data, bp_hardening_data);
-
-static void install_bp_hardening_cb(bp_hardening_cb_t fn)
-{
- __this_cpu_write(bp_hardening_data.fn, fn);
-
- /*
- * Vinz Clortho takes the hyp_vecs start/end "keys" at
- * the door when we're a guest. Skip the hyp-vectors work.
- */
- if (!is_hyp_mode_available())
- return;
-
- __this_cpu_write(bp_hardening_data.slot, HYP_VECTOR_SPECTRE_DIRECT);
-}
-
-/* Called during entry so must be noinstr */
-static noinstr void call_smc_arch_workaround_1(void)
-{
- arm_smccc_1_1_smc(ARM_SMCCC_ARCH_WORKAROUND_1, NULL);
-}
-
-/* Called during entry so must be noinstr */
-static noinstr void call_hvc_arch_workaround_1(void)
-{
- arm_smccc_1_1_hvc(ARM_SMCCC_ARCH_WORKAROUND_1, NULL);
-}
-
-/* Called during entry so must be noinstr */
-static noinstr void qcom_link_stack_sanitisation(void)
-{
- u64 tmp;
-
- asm volatile("mov %0, x30 \n"
- ".rept 16 \n"
- "bl . + 4 \n"
- ".endr \n"
- "mov x30, %0 \n"
- : "=&r" (tmp));
-}
-
-static bp_hardening_cb_t spectre_v2_get_sw_mitigation_cb(void)
-{
- u32 midr = read_cpuid_id();
- if (((midr & MIDR_CPU_MODEL_MASK) != MIDR_QCOM_FALKOR) &&
- ((midr & MIDR_CPU_MODEL_MASK) != MIDR_QCOM_FALKOR_V1))
- return NULL;
-
- return qcom_link_stack_sanitisation;
-}
-
-static enum mitigation_state spectre_v2_enable_fw_mitigation(void)
-{
- bp_hardening_cb_t cb;
- enum mitigation_state state;
-
- state = spectre_v2_get_cpu_fw_mitigation_state();
- if (state != SPECTRE_MITIGATED)
- return state;
-
- if (spectre_v2_mitigations_off())
- return SPECTRE_VULNERABLE;
-
- switch (arm_smccc_1_1_get_conduit()) {
- case SMCCC_CONDUIT_HVC:
- cb = call_hvc_arch_workaround_1;
- break;
-
- case SMCCC_CONDUIT_SMC:
- cb = call_smc_arch_workaround_1;
- break;
-
- default:
- return SPECTRE_VULNERABLE;
- }
-
- /*
- * Prefer a CPU-specific workaround if it exists. Note that we
- * still rely on firmware for the mitigation at EL2.
- */
- cb = spectre_v2_get_sw_mitigation_cb() ?: cb;
- install_bp_hardening_cb(cb);
- return SPECTRE_MITIGATED;
-}
-
-void spectre_v2_enable_mitigation(const struct arm64_cpu_capabilities *__unused)
-{
- enum mitigation_state state;
-
- WARN_ON(preemptible());
-
- state = spectre_v2_get_cpu_hw_mitigation_state();
- if (state == SPECTRE_VULNERABLE)
- state = spectre_v2_enable_fw_mitigation();
-
- update_mitigation_state(&spectre_v2_state, state);
-}
-
-/*
- * Spectre-v3a.
- *
- * Phew, there's not an awful lot to do here! We just instruct EL2 to use
- * an indirect trampoline for the hyp vectors so that guests can't read
- * VBAR_EL2 to defeat randomisation of the hypervisor VA layout.
- */
-bool has_spectre_v3a(const struct arm64_cpu_capabilities *entry, int scope)
-{
- static const struct midr_range spectre_v3a_unsafe_list[] = {
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A57),
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
- {},
- };
-
- WARN_ON(scope != SCOPE_LOCAL_CPU || preemptible());
- return is_midr_in_range_list(read_cpuid_id(), spectre_v3a_unsafe_list);
-}
-
-void spectre_v3a_enable_mitigation(const struct arm64_cpu_capabilities *__unused)
-{
- struct bp_hardening_data *data = this_cpu_ptr(&bp_hardening_data);
-
- if (this_cpu_has_cap(ARM64_SPECTRE_V3A))
- data->slot += HYP_VECTOR_INDIRECT;
-}
-
-/*
- * Spectre v4.
- *
- * If you thought Spectre v2 was nasty, wait until you see this mess. A CPU is
- * either:
- *
- * - Mitigated in hardware and listed in our "safe list".
- * - Mitigated in hardware via PSTATE.SSBS.
- * - Mitigated in software by firmware (sometimes referred to as SSBD).
- *
- * Wait, that doesn't sound so bad, does it? Keep reading...
- *
- * A major source of headaches is that the software mitigation is enabled both
- * on a per-task basis, but can also be forced on for the kernel, necessitating
- * both context-switch *and* entry/exit hooks. To make it even worse, some CPUs
- * allow EL0 to toggle SSBS directly, which can end up with the prctl() state
- * being stale when re-entering the kernel. The usual big.LITTLE caveats apply,
- * so you can have systems that have both firmware and SSBS mitigations. This
- * means we actually have to reject late onlining of CPUs with mitigations if
- * all of the currently onlined CPUs are safelisted, as the mitigation tends to
- * be opt-in for userspace. Yes, really, the cure is worse than the disease.
- *
- * The only good part is that if the firmware mitigation is present, then it is
- * present for all CPUs, meaning we don't have to worry about late onlining of a
- * vulnerable CPU if one of the boot CPUs is using the firmware mitigation.
- *
- * Give me a VAX-11/780 any day of the week...
- */
-static enum mitigation_state spectre_v4_state;
-
-/* This is the per-cpu state tracking whether we need to talk to firmware */
-DEFINE_PER_CPU_READ_MOSTLY(u64, arm64_ssbd_callback_required);
-
-enum spectre_v4_policy {
- SPECTRE_V4_POLICY_MITIGATION_DYNAMIC,
- SPECTRE_V4_POLICY_MITIGATION_ENABLED,
- SPECTRE_V4_POLICY_MITIGATION_DISABLED,
-};
-
-static enum spectre_v4_policy __read_mostly __spectre_v4_policy;
-
-static const struct spectre_v4_param {
- const char *str;
- enum spectre_v4_policy policy;
-} spectre_v4_params[] = {
- { "force-on", SPECTRE_V4_POLICY_MITIGATION_ENABLED, },
- { "force-off", SPECTRE_V4_POLICY_MITIGATION_DISABLED, },
- { "kernel", SPECTRE_V4_POLICY_MITIGATION_DYNAMIC, },
-};
-static int __init parse_spectre_v4_param(char *str)
-{
- int i;
-
- if (!str || !str[0])
- return -EINVAL;
-
- for (i = 0; i < ARRAY_SIZE(spectre_v4_params); i++) {
- const struct spectre_v4_param *param = &spectre_v4_params[i];
-
- if (strncmp(str, param->str, strlen(param->str)))
- continue;
-
- __spectre_v4_policy = param->policy;
- return 0;
- }
-
- return -EINVAL;
-}
-early_param("ssbd", parse_spectre_v4_param);
-
-/*
- * Because this was all written in a rush by people working in different silos,
- * we've ended up with multiple command line options to control the same thing.
- * Wrap these up in some helpers, which prefer disabling the mitigation if faced
- * with contradictory parameters. The mitigation is always either "off",
- * "dynamic" or "on".
- */
-static bool spectre_v4_mitigations_off(void)
-{
- bool ret = cpu_mitigations_off() ||
- __spectre_v4_policy == SPECTRE_V4_POLICY_MITIGATION_DISABLED;
-
- if (ret)
- pr_info_once("spectre-v4 mitigation disabled by command-line option\n");
-
- return ret;
-}
-
-/* Do we need to toggle the mitigation state on entry to/exit from the kernel? */
-static bool spectre_v4_mitigations_dynamic(void)
-{
- return !spectre_v4_mitigations_off() &&
- __spectre_v4_policy == SPECTRE_V4_POLICY_MITIGATION_DYNAMIC;
-}
-
-static bool spectre_v4_mitigations_on(void)
-{
- return !spectre_v4_mitigations_off() &&
- __spectre_v4_policy == SPECTRE_V4_POLICY_MITIGATION_ENABLED;
-}
-
-ssize_t cpu_show_spec_store_bypass(struct device *dev,
- struct device_attribute *attr, char *buf)
-{
- switch (spectre_v4_state) {
- case SPECTRE_UNAFFECTED:
- return sprintf(buf, "Not affected\n");
- case SPECTRE_MITIGATED:
- return sprintf(buf, "Mitigation: Speculative Store Bypass disabled via prctl\n");
- case SPECTRE_VULNERABLE:
- fallthrough;
- default:
- return sprintf(buf, "Vulnerable\n");
- }
-}
-
-enum mitigation_state arm64_get_spectre_v4_state(void)
-{
- return spectre_v4_state;
-}
-
-static enum mitigation_state spectre_v4_get_cpu_hw_mitigation_state(void)
-{
- static const struct midr_range spectre_v4_safe_list[] = {
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A35),
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A53),
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A55),
- MIDR_ALL_VERSIONS(MIDR_BRAHMA_B53),
- MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_3XX_SILVER),
- MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_4XX_SILVER),
- { /* sentinel */ },
- };
-
- if (is_midr_in_range_list(read_cpuid_id(), spectre_v4_safe_list))
- return SPECTRE_UNAFFECTED;
-
- /* CPU features are detected first */
- if (this_cpu_has_cap(ARM64_SSBS))
- return SPECTRE_MITIGATED;
-
- return SPECTRE_VULNERABLE;
-}
-
-static enum mitigation_state spectre_v4_get_cpu_fw_mitigation_state(void)
-{
- int ret;
- struct arm_smccc_res res;
-
- arm_smccc_1_1_invoke(ARM_SMCCC_ARCH_FEATURES_FUNC_ID,
- ARM_SMCCC_ARCH_WORKAROUND_2, &res);
-
- ret = res.a0;
- switch (ret) {
- case SMCCC_RET_SUCCESS:
- return SPECTRE_MITIGATED;
- case SMCCC_ARCH_WORKAROUND_RET_UNAFFECTED:
- fallthrough;
- case SMCCC_RET_NOT_REQUIRED:
- return SPECTRE_UNAFFECTED;
- default:
- fallthrough;
- case SMCCC_RET_NOT_SUPPORTED:
- return SPECTRE_VULNERABLE;
- }
-}
-
-bool has_spectre_v4(const struct arm64_cpu_capabilities *cap, int scope)
-{
- enum mitigation_state state;
-
- WARN_ON(scope != SCOPE_LOCAL_CPU || preemptible());
-
- state = spectre_v4_get_cpu_hw_mitigation_state();
- if (state == SPECTRE_VULNERABLE)
- state = spectre_v4_get_cpu_fw_mitigation_state();
-
- return state != SPECTRE_UNAFFECTED;
-}
-
-static int ssbs_emulation_handler(struct pt_regs *regs, u32 instr)
-{
- if (user_mode(regs))
- return 1;
-
- if (instr & BIT(PSTATE_Imm_shift))
- regs->pstate |= PSR_SSBS_BIT;
- else
- regs->pstate &= ~PSR_SSBS_BIT;
-
- arm64_skip_faulting_instruction(regs, 4);
- return 0;
-}
-
-static struct undef_hook ssbs_emulation_hook = {
- .instr_mask = ~(1U << PSTATE_Imm_shift),
- .instr_val = 0xd500401f | PSTATE_SSBS,
- .fn = ssbs_emulation_handler,
-};
-
-static enum mitigation_state spectre_v4_enable_hw_mitigation(void)
-{
- static bool undef_hook_registered = false;
- static DEFINE_RAW_SPINLOCK(hook_lock);
- enum mitigation_state state;
-
- /*
- * If the system is mitigated but this CPU doesn't have SSBS, then
- * we must be on the safelist and there's nothing more to do.
- */
- state = spectre_v4_get_cpu_hw_mitigation_state();
- if (state != SPECTRE_MITIGATED || !this_cpu_has_cap(ARM64_SSBS))
- return state;
-
- raw_spin_lock(&hook_lock);
- if (!undef_hook_registered) {
- register_undef_hook(&ssbs_emulation_hook);
- undef_hook_registered = true;
- }
- raw_spin_unlock(&hook_lock);
-
- if (spectre_v4_mitigations_off()) {
- sysreg_clear_set(sctlr_el1, 0, SCTLR_ELx_DSSBS);
- set_pstate_ssbs(1);
- return SPECTRE_VULNERABLE;
- }
-
- /* SCTLR_EL1.DSSBS was initialised to 0 during boot */
- set_pstate_ssbs(0);
- return SPECTRE_MITIGATED;
-}
-
-/*
- * Patch a branch over the Spectre-v4 mitigation code with a NOP so that
- * we fallthrough and check whether firmware needs to be called on this CPU.
- */
-void __init spectre_v4_patch_fw_mitigation_enable(struct alt_instr *alt,
- __le32 *origptr,
- __le32 *updptr, int nr_inst)
-{
- BUG_ON(nr_inst != 1); /* Branch -> NOP */
-
- if (spectre_v4_mitigations_off())
- return;
-
- if (cpus_have_final_cap(ARM64_SSBS))
- return;
-
- if (spectre_v4_mitigations_dynamic())
- *updptr = cpu_to_le32(aarch64_insn_gen_nop());
-}
-
-/*
- * Patch a NOP in the Spectre-v4 mitigation code with an SMC/HVC instruction
- * to call into firmware to adjust the mitigation state.
- */
-void __init smccc_patch_fw_mitigation_conduit(struct alt_instr *alt,
- __le32 *origptr,
- __le32 *updptr, int nr_inst)
-{
- u32 insn;
-
- BUG_ON(nr_inst != 1); /* NOP -> HVC/SMC */
-
- switch (arm_smccc_1_1_get_conduit()) {
- case SMCCC_CONDUIT_HVC:
- insn = aarch64_insn_get_hvc_value();
- break;
- case SMCCC_CONDUIT_SMC:
- insn = aarch64_insn_get_smc_value();
- break;
- default:
- return;
- }
-
- *updptr = cpu_to_le32(insn);
-}
-
-static enum mitigation_state spectre_v4_enable_fw_mitigation(void)
-{
- enum mitigation_state state;
-
- state = spectre_v4_get_cpu_fw_mitigation_state();
- if (state != SPECTRE_MITIGATED)
- return state;
-
- if (spectre_v4_mitigations_off()) {
- arm_smccc_1_1_invoke(ARM_SMCCC_ARCH_WORKAROUND_2, false, NULL);
- return SPECTRE_VULNERABLE;
- }
-
- arm_smccc_1_1_invoke(ARM_SMCCC_ARCH_WORKAROUND_2, true, NULL);
-
- if (spectre_v4_mitigations_dynamic())
- __this_cpu_write(arm64_ssbd_callback_required, 1);
-
- return SPECTRE_MITIGATED;
-}
-
-void spectre_v4_enable_mitigation(const struct arm64_cpu_capabilities *__unused)
-{
- enum mitigation_state state;
-
- WARN_ON(preemptible());
-
- state = spectre_v4_enable_hw_mitigation();
- if (state == SPECTRE_VULNERABLE)
- state = spectre_v4_enable_fw_mitigation();
-
- update_mitigation_state(&spectre_v4_state, state);
-}
-
-static void __update_pstate_ssbs(struct pt_regs *regs, bool state)
-{
- u64 bit = compat_user_mode(regs) ? PSR_AA32_SSBS_BIT : PSR_SSBS_BIT;
-
- if (state)
- regs->pstate |= bit;
- else
- regs->pstate &= ~bit;
-}
-
-void spectre_v4_enable_task_mitigation(struct task_struct *tsk)
-{
- struct pt_regs *regs = task_pt_regs(tsk);
- bool ssbs = false, kthread = tsk->flags & PF_KTHREAD;
-
- if (spectre_v4_mitigations_off())
- ssbs = true;
- else if (spectre_v4_mitigations_dynamic() && !kthread)
- ssbs = !test_tsk_thread_flag(tsk, TIF_SSBD);
-
- __update_pstate_ssbs(regs, ssbs);
-}
-
-/*
- * The Spectre-v4 mitigation can be controlled via a prctl() from userspace.
- * This is interesting because the "speculation disabled" behaviour can be
- * configured so that it is preserved across exec(), which means that the
- * prctl() may be necessary even when PSTATE.SSBS can be toggled directly
- * from userspace.
- */
-static void ssbd_prctl_enable_mitigation(struct task_struct *task)
-{
- task_clear_spec_ssb_noexec(task);
- task_set_spec_ssb_disable(task);
- set_tsk_thread_flag(task, TIF_SSBD);
-}
-
-static void ssbd_prctl_disable_mitigation(struct task_struct *task)
-{
- task_clear_spec_ssb_noexec(task);
- task_clear_spec_ssb_disable(task);
- clear_tsk_thread_flag(task, TIF_SSBD);
-}
-
-static int ssbd_prctl_set(struct task_struct *task, unsigned long ctrl)
-{
- switch (ctrl) {
- case PR_SPEC_ENABLE:
- /* Enable speculation: disable mitigation */
- /*
- * Force disabled speculation prevents it from being
- * re-enabled.
- */
- if (task_spec_ssb_force_disable(task))
- return -EPERM;
-
- /*
- * If the mitigation is forced on, then speculation is forced
- * off and we again prevent it from being re-enabled.
- */
- if (spectre_v4_mitigations_on())
- return -EPERM;
-
- ssbd_prctl_disable_mitigation(task);
- break;
- case PR_SPEC_FORCE_DISABLE:
- /* Force disable speculation: force enable mitigation */
- /*
- * If the mitigation is forced off, then speculation is forced
- * on and we prevent it from being disabled.
- */
- if (spectre_v4_mitigations_off())
- return -EPERM;
-
- task_set_spec_ssb_force_disable(task);
- fallthrough;
- case PR_SPEC_DISABLE:
- /* Disable speculation: enable mitigation */
- /* Same as PR_SPEC_FORCE_DISABLE */
- if (spectre_v4_mitigations_off())
- return -EPERM;
-
- ssbd_prctl_enable_mitigation(task);
- break;
- case PR_SPEC_DISABLE_NOEXEC:
- /* Disable speculation until execve(): enable mitigation */
- /*
- * If the mitigation state is forced one way or the other, then
- * we must fail now before we try to toggle it on execve().
- */
- if (task_spec_ssb_force_disable(task) ||
- spectre_v4_mitigations_off() ||
- spectre_v4_mitigations_on()) {
- return -EPERM;
- }
-
- ssbd_prctl_enable_mitigation(task);
- task_set_spec_ssb_noexec(task);
- break;
- default:
- return -ERANGE;
- }
-
- spectre_v4_enable_task_mitigation(task);
- return 0;
-}
-
-int arch_prctl_spec_ctrl_set(struct task_struct *task, unsigned long which,
- unsigned long ctrl)
-{
- switch (which) {
- case PR_SPEC_STORE_BYPASS:
- return ssbd_prctl_set(task, ctrl);
- default:
- return -ENODEV;
- }
-}
-
-static int ssbd_prctl_get(struct task_struct *task)
-{
- switch (spectre_v4_state) {
- case SPECTRE_UNAFFECTED:
- return PR_SPEC_NOT_AFFECTED;
- case SPECTRE_MITIGATED:
- if (spectre_v4_mitigations_on())
- return PR_SPEC_NOT_AFFECTED;
-
- if (spectre_v4_mitigations_dynamic())
- break;
-
- /* Mitigations are disabled, so we're vulnerable. */
- fallthrough;
- case SPECTRE_VULNERABLE:
- fallthrough;
- default:
- return PR_SPEC_ENABLE;
- }
-
- /* Check the mitigation state for this task */
- if (task_spec_ssb_force_disable(task))
- return PR_SPEC_PRCTL | PR_SPEC_FORCE_DISABLE;
-
- if (task_spec_ssb_noexec(task))
- return PR_SPEC_PRCTL | PR_SPEC_DISABLE_NOEXEC;
-
- if (task_spec_ssb_disable(task))
- return PR_SPEC_PRCTL | PR_SPEC_DISABLE;
-
- return PR_SPEC_PRCTL | PR_SPEC_ENABLE;
-}
-
-int arch_prctl_spec_ctrl_get(struct task_struct *task, unsigned long which)
-{
- switch (which) {
- case PR_SPEC_STORE_BYPASS:
- return ssbd_prctl_get(task);
- default:
- return -ENODEV;
- }
-}
-
-/*
- * Spectre BHB.
- *
- * A CPU is either:
- * - Mitigated by a branchy loop a CPU specific number of times, and listed
- * in our "loop mitigated list".
- * - Mitigated in software by the firmware Spectre v2 call.
- * - Has the ClearBHB instruction to perform the mitigation.
- * - Has the 'Exception Clears Branch History Buffer' (ECBHB) feature, so no
- * software mitigation in the vectors is needed.
- * - Has CSV2.3, so is unaffected.
- */
-static enum mitigation_state spectre_bhb_state;
-
-enum mitigation_state arm64_get_spectre_bhb_state(void)
-{
- return spectre_bhb_state;
-}
-
-enum bhb_mitigation_bits {
- BHB_LOOP,
- BHB_FW,
- BHB_HW,
- BHB_INSN,
-};
-static unsigned long system_bhb_mitigations;
-
-/*
- * This must be called with SCOPE_LOCAL_CPU for each type of CPU, before any
- * SCOPE_SYSTEM call will give the right answer.
- */
-u8 spectre_bhb_loop_affected(int scope)
-{
- u8 k = 0;
- static u8 max_bhb_k;
-
- if (scope == SCOPE_LOCAL_CPU) {
- static const struct midr_range spectre_bhb_k32_list[] = {
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A78),
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A78AE),
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A78C),
- MIDR_ALL_VERSIONS(MIDR_CORTEX_X1),
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A710),
- MIDR_ALL_VERSIONS(MIDR_CORTEX_X2),
- MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N2),
- MIDR_ALL_VERSIONS(MIDR_NEOVERSE_V1),
- {},
- };
- static const struct midr_range spectre_bhb_k24_list[] = {
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A76),
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A77),
- MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N1),
- {},
- };
- static const struct midr_range spectre_bhb_k8_list[] = {
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A57),
- {},
- };
-
- if (is_midr_in_range_list(read_cpuid_id(), spectre_bhb_k32_list))
- k = 32;
- else if (is_midr_in_range_list(read_cpuid_id(), spectre_bhb_k24_list))
- k = 24;
- else if (is_midr_in_range_list(read_cpuid_id(), spectre_bhb_k8_list))
- k = 8;
-
- max_bhb_k = max(max_bhb_k, k);
- } else {
- k = max_bhb_k;
- }
-
- return k;
-}
-
-static enum mitigation_state spectre_bhb_get_cpu_fw_mitigation_state(void)
-{
- int ret;
- struct arm_smccc_res res;
-
- arm_smccc_1_1_invoke(ARM_SMCCC_ARCH_FEATURES_FUNC_ID,
- ARM_SMCCC_ARCH_WORKAROUND_3, &res);
-
- ret = res.a0;
- switch (ret) {
- case SMCCC_RET_SUCCESS:
- return SPECTRE_MITIGATED;
- case SMCCC_ARCH_WORKAROUND_RET_UNAFFECTED:
- return SPECTRE_UNAFFECTED;
- default:
- fallthrough;
- case SMCCC_RET_NOT_SUPPORTED:
- return SPECTRE_VULNERABLE;
- }
-}
-
-static bool is_spectre_bhb_fw_affected(int scope)
-{
- static bool system_affected;
- enum mitigation_state fw_state;
- bool has_smccc = arm_smccc_1_1_get_conduit() != SMCCC_CONDUIT_NONE;
- static const struct midr_range spectre_bhb_firmware_mitigated_list[] = {
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
- {},
- };
- bool cpu_in_list = is_midr_in_range_list(read_cpuid_id(),
- spectre_bhb_firmware_mitigated_list);
-
- if (scope != SCOPE_LOCAL_CPU)
- return system_affected;
-
- fw_state = spectre_bhb_get_cpu_fw_mitigation_state();
- if (cpu_in_list || (has_smccc && fw_state == SPECTRE_MITIGATED)) {
- system_affected = true;
- return true;
- }
-
- return false;
-}
-
-static bool supports_ecbhb(int scope)
-{
- u64 mmfr1;
-
- if (scope == SCOPE_LOCAL_CPU)
- mmfr1 = read_sysreg_s(SYS_ID_AA64MMFR1_EL1);
- else
- mmfr1 = read_sanitised_ftr_reg(SYS_ID_AA64MMFR1_EL1);
-
- return cpuid_feature_extract_unsigned_field(mmfr1,
- ID_AA64MMFR1_ECBHB_SHIFT);
-}
-
-bool is_spectre_bhb_affected(const struct arm64_cpu_capabilities *entry,
- int scope)
-{
- WARN_ON(scope != SCOPE_LOCAL_CPU || preemptible());
-
- if (supports_csv2p3(scope))
- return false;
-
- if (supports_clearbhb(scope))
- return true;
-
- if (spectre_bhb_loop_affected(scope))
- return true;
-
- if (is_spectre_bhb_fw_affected(scope))
- return true;
-
- return false;
-}
-
-static void this_cpu_set_vectors(enum arm64_bp_harden_el1_vectors slot)
-{
- const char *v = arm64_get_bp_hardening_vector(slot);
-
- if (slot < 0)
- return;
-
- __this_cpu_write(this_cpu_vector, v);
-
- /*
- * When KPTI is in use, the vectors are switched when exiting to
- * user-space.
- */
- if (arm64_kernel_unmapped_at_el0())
- return;
-
- write_sysreg(v, vbar_el1);
- isb();
-}
-
-void spectre_bhb_enable_mitigation(const struct arm64_cpu_capabilities *entry)
-{
- bp_hardening_cb_t cpu_cb;
- enum mitigation_state fw_state, state = SPECTRE_VULNERABLE;
- struct bp_hardening_data *data = this_cpu_ptr(&bp_hardening_data);
-
- if (!is_spectre_bhb_affected(entry, SCOPE_LOCAL_CPU))
- return;
-
- if (arm64_get_spectre_v2_state() == SPECTRE_VULNERABLE) {
- /* No point mitigating Spectre-BHB alone. */
- } else if (!IS_ENABLED(CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY)) {
- pr_info_once("spectre-bhb mitigation disabled by compile time option\n");
- } else if (cpu_mitigations_off()) {
- pr_info_once("spectre-bhb mitigation disabled by command line option\n");
- } else if (supports_ecbhb(SCOPE_LOCAL_CPU)) {
- state = SPECTRE_MITIGATED;
- set_bit(BHB_HW, &system_bhb_mitigations);
- } else if (supports_clearbhb(SCOPE_LOCAL_CPU)) {
- /*
- * Ensure KVM uses the indirect vector which will have ClearBHB
- * added.
- */
- if (!data->slot)
- data->slot = HYP_VECTOR_INDIRECT;
-
- this_cpu_set_vectors(EL1_VECTOR_BHB_CLEAR_INSN);
- state = SPECTRE_MITIGATED;
- set_bit(BHB_INSN, &system_bhb_mitigations);
- } else if (spectre_bhb_loop_affected(SCOPE_LOCAL_CPU)) {
- /*
- * Ensure KVM uses the indirect vector which will have the
- * branchy-loop added. A57/A72-r0 will already have selected
- * the spectre-indirect vector, which is sufficient for BHB
- * too.
- */
- if (!data->slot)
- data->slot = HYP_VECTOR_INDIRECT;
-
- this_cpu_set_vectors(EL1_VECTOR_BHB_LOOP);
- state = SPECTRE_MITIGATED;
- set_bit(BHB_LOOP, &system_bhb_mitigations);
- } else if (is_spectre_bhb_fw_affected(SCOPE_LOCAL_CPU)) {
- fw_state = spectre_bhb_get_cpu_fw_mitigation_state();
- if (fw_state == SPECTRE_MITIGATED) {
- /*
- * Ensure KVM uses one of the spectre bp_hardening
- * vectors. The indirect vector doesn't include the EL3
- * call, so needs upgrading to
- * HYP_VECTOR_SPECTRE_INDIRECT.
- */
- if (!data->slot || data->slot == HYP_VECTOR_INDIRECT)
- data->slot += 1;
-
- this_cpu_set_vectors(EL1_VECTOR_BHB_FW);
-
- /*
- * The WA3 call in the vectors supersedes the WA1 call
- * made during context-switch. Uninstall any firmware
- * bp_hardening callback.
- */
- cpu_cb = spectre_v2_get_sw_mitigation_cb();
- if (__this_cpu_read(bp_hardening_data.fn) != cpu_cb)
- __this_cpu_write(bp_hardening_data.fn, NULL);
-
- state = SPECTRE_MITIGATED;
- set_bit(BHB_FW, &system_bhb_mitigations);
- }
- }
-
- update_mitigation_state(&spectre_bhb_state, state);
-}
-
-/* Patched to NOP when enabled */
-void noinstr spectre_bhb_patch_loop_mitigation_enable(struct alt_instr *alt,
- __le32 *origptr,
- __le32 *updptr, int nr_inst)
-{
- BUG_ON(nr_inst != 1);
-
- if (test_bit(BHB_LOOP, &system_bhb_mitigations))
- *updptr++ = cpu_to_le32(aarch64_insn_gen_nop());
-}
-
-/* Patched to NOP when enabled */
-void noinstr spectre_bhb_patch_fw_mitigation_enabled(struct alt_instr *alt,
- __le32 *origptr,
- __le32 *updptr, int nr_inst)
-{
- BUG_ON(nr_inst != 1);
-
- if (test_bit(BHB_FW, &system_bhb_mitigations))
- *updptr++ = cpu_to_le32(aarch64_insn_gen_nop());
-}
-
-/* Patched to correct the immediate */
-void noinstr spectre_bhb_patch_loop_iter(struct alt_instr *alt,
- __le32 *origptr, __le32 *updptr, int nr_inst)
-{
- u8 rd;
- u32 insn;
- u16 loop_count = spectre_bhb_loop_affected(SCOPE_SYSTEM);
-
- BUG_ON(nr_inst != 1); /* MOV -> MOV */
-
- if (!IS_ENABLED(CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY))
- return;
-
- insn = le32_to_cpu(*origptr);
- rd = aarch64_insn_decode_register(AARCH64_INSN_REGTYPE_RD, insn);
- insn = aarch64_insn_gen_movewide(rd, loop_count, 0,
- AARCH64_INSN_VARIANT_64BIT,
- AARCH64_INSN_MOVEWIDE_ZERO);
- *updptr++ = cpu_to_le32(insn);
-}
-
-/* Patched to mov WA3 when supported */
-void noinstr spectre_bhb_patch_wa3(struct alt_instr *alt,
- __le32 *origptr, __le32 *updptr, int nr_inst)
-{
- u8 rd;
- u32 insn;
-
- BUG_ON(nr_inst != 1); /* MOV -> MOV */
-
- if (!IS_ENABLED(CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY) ||
- !test_bit(BHB_FW, &system_bhb_mitigations))
- return;
-
- insn = le32_to_cpu(*origptr);
- rd = aarch64_insn_decode_register(AARCH64_INSN_REGTYPE_RD, insn);
-
- insn = aarch64_insn_gen_logical_immediate(AARCH64_INSN_LOGIC_ORR,
- AARCH64_INSN_VARIANT_32BIT,
- AARCH64_INSN_REG_ZR, rd,
- ARM_SMCCC_ARCH_WORKAROUND_3);
- if (WARN_ON_ONCE(insn == AARCH64_BREAK_FAULT))
- return;
-
- *updptr++ = cpu_to_le32(insn);
-}
-
-/* Patched to NOP when not supported */
-void __init spectre_bhb_patch_clearbhb(struct alt_instr *alt,
- __le32 *origptr, __le32 *updptr, int nr_inst)
-{
- BUG_ON(nr_inst != 2);
-
- if (test_bit(BHB_INSN, &system_bhb_mitigations))
- return;
-
- *updptr++ = cpu_to_le32(aarch64_insn_gen_nop());
- *updptr++ = cpu_to_le32(aarch64_insn_gen_nop());
-}
-
-#ifdef CONFIG_BPF_SYSCALL
-#define EBPF_WARN "Unprivileged eBPF is enabled, data leaks possible via Spectre v2 BHB attacks!\n"
-void unpriv_ebpf_notify(int new_state)
-{
- if (spectre_v2_state == SPECTRE_VULNERABLE ||
- spectre_bhb_state != SPECTRE_MITIGATED)
- return;
-
- if (!new_state)
- pr_err("WARNING: %s", EBPF_WARN);
-}
-#endif
================================================================================
* ONLY IN PATCH2 - files not modified by patch1 *
================================================================================
--- a/arch/arm64/kernel/proton-pack.c
+++ b/arch/arm64/kernel/proton-pack.c
@@ -853,6 +853,7 @@ u8 spectre_bhb_loop_affected(int scope)
if (scope == SCOPE_LOCAL_CPU) {
static const struct midr_range spectre_bhb_k32_list[] = {
MIDR_ALL_VERSIONS(MIDR_CORTEX_A78),
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A78AE),
MIDR_ALL_VERSIONS(MIDR_CORTEX_A78C),
MIDR_ALL_VERSIONS(MIDR_CORTEX_X1),
MIDR_ALL_VERSIONS(MIDR_CORTEX_A710),
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -86,6 +86,7 @@
#define ARM_CPU_PART_CORTEX_X2 0xD48
#define ARM_CPU_PART_NEOVERSE_N2 0xD49
#define ARM_CPU_PART_CORTEX_A78C 0xD4B
+#define ARM_CPU_PART_CORTEX_X4 0xD82
#define APM_CPU_PART_XGENE 0x000
#define APM_CPU_VAR_POTENZA 0x00
@@ -159,6 +160,7 @@
#define MIDR_CORTEX_X2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X2)
#define MIDR_NEOVERSE_N2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N2)
#define MIDR_CORTEX_A78C MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78C)
+#define MIDR_CORTEX_X4 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X4)
#define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX)
#define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_81XX)
#define MIDR_THUNDERX_83XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_83XX)
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -139,6 +157,5 @@
#define MIDR_NEOVERSE_N2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N2)
#define MIDR_CORTEX_A78C MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78C)
-#define MIDR_NEOVERSE_V2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_V2)
#define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX)
#define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_81XX)
#define MIDR_THUNDERX_83XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_83XX)
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -141,5 +159,4 @@
#define MIDR_CORTEX_A78C MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78C)
-#define MIDR_NEOVERSE_V2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_V2)
#define MIDR_CORTEX_X4 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X4)
#define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX)
#define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_81XX)
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -88,4 +88,4 @@
-#define ARM_CPU_PART_NEOVERSE_V2 0xD4F
+#define ARM_CPU_PART_CORTEX_A720 0xD81
#define ARM_CPU_PART_CORTEX_X4 0xD82
#define ARM_CPU_PART_NEOVERSE_V3 0xD84
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -147,4 +167,4 @@
#define MIDR_CORTEX_A78C MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78C)
#define MIDR_CORTEX_X3 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X3)
#define MIDR_NEOVERSE_V2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_V2)
-#define MIDR_CORTEX_X4 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X4)
+#define MIDR_CORTEX_A720 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A720)
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -150,5 +179,6 @@
#define MIDR_NEOVERSE_V2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_V2)
+#define MIDR_CORTEX_A720 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A720)
#define MIDR_CORTEX_X4 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X4)
#define MIDR_NEOVERSE_V3 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_V3)
#define MIDR_CORTEX_X925 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X925)
-#define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX)
+#define MIDR_CORTEX_A725 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A725)
================================================================================
* DELTA DIFFERENCES - code changes that differ between the patches *
================================================================================
--- b/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -133,6 +133,7 @@
#define HISI_CPU_PART_TSV110 0xD01
#define AMPERE_CPU_PART_AMPERE1 0xAC3
+
#define MICROSOFT_CPU_PART_AZURE_COBALT_100 0xD49 /* Based on r0p0 of ARM Neoverse N2 */
#define MIDR_CORTEX_A53 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A53)
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -178,5 +190,5 @@
-#define MIDR_FUJITSU_A64FX MIDR_CPU_MODEL(ARM_CPU_IMP_FUJITSU, FUJITSU_CPU_PART_A64FX)
-#define MIDR_HISI_TSV110 MIDR_CPU_MODEL(ARM_CPU_IMP_HISI, HISI_CPU_PART_TSV110)
+#define MIDR_APPLE_M2_BLIZZARD_MAX MIDR_CPU_MODEL(ARM_CPU_IMP_APPLE, APPLE_CPU_PART_M2_BLIZZARD_MAX)
+#define MIDR_APPLE_M2_AVALANCHE_MAX MIDR_CPU_MODEL(ARM_CPU_IMP_APPLE, APPLE_CPU_PART_M2_AVALANCHE_MAX)
#define MIDR_AMPERE1 MIDR_CPU_MODEL(ARM_CPU_IMP_AMPERE, AMPERE_CPU_PART_AMPERE1)
/* Fujitsu Erratum 010001 affects A64FX 1.0 and 1.1, (v0r0 and v1r0) */
================================================================================
* ONLY IN PATCH2 - files not modified by patch1 *
================================================================================
--- a/Documentation/arch/arm64/silicon-errata.rst
+++ b/Documentation/arch/arm64/silicon-errata.rst
@@ -243,3 +243,10 @@ stable kernels.
+----------------+-----------------+-----------------+-----------------------------+
| ASR | ASR8601 | #8601001 | N/A |
+----------------+-----------------+-----------------+-----------------------------+
++----------------+-----------------+-----------------+-----------------------------+
+| Microsoft | Azure Cobalt 100| #2139208 | ARM64_ERRATUM_2139208 |
++----------------+-----------------+-----------------+-----------------------------+
+| Microsoft | Azure Cobalt 100| #2067961 | ARM64_ERRATUM_2067961 |
++----------------+-----------------+-----------------+-----------------------------+
+| Microsoft | Azure Cobalt 100| #2253138 | ARM64_ERRATUM_2253138 |
++----------------+-----------------+-----------------+-----------------------------+
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -374,6 +374,7 @@ static const struct midr_range erratum_1463225[] = {
static const struct midr_range trbe_overwrite_fill_mode_cpus[] = {
#ifdef CONFIG_ARM64_ERRATUM_2139208
MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N2),
+ MIDR_ALL_VERSIONS(MIDR_MICROSOFT_AZURE_COBALT_100),
#endif
#ifdef CONFIG_ARM64_ERRATUM_2119858
MIDR_ALL_VERSIONS(MIDR_CORTEX_A710),
@@ -387,6 +388,7 @@ static const struct midr_range trbe_overwrite_fill_mode_cpus[] = {
static const struct midr_range tsb_flush_fail_cpus[] = {
#ifdef CONFIG_ARM64_ERRATUM_2067961
MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N2),
+ MIDR_ALL_VERSIONS(MIDR_MICROSOFT_AZURE_COBALT_100),
#endif
#ifdef CONFIG_ARM64_ERRATUM_2054223
MIDR_ALL_VERSIONS(MIDR_CORTEX_A710),
@@ -399,6 +401,7 @@ static const struct midr_range tsb_flush_fail_cpus[] = {
static struct midr_range trbe_write_out_of_range_cpus[] = {
#ifdef CONFIG_ARM64_ERRATUM_2253138
MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N2),
+ MIDR_ALL_VERSIONS(MIDR_MICROSOFT_AZURE_COBALT_100),
#endif
#ifdef CONFIG_ARM64_ERRATUM_2224489
MIDR_ALL_VERSIONS(MIDR_CORTEX_A710),This is an automated interdiff check for backported commits. |
JIRA PR Check Results7 commit(s) with issues found: Commit
|
|
❌ Validation checks completed with issues View full results: https://github.com/ctrliq/kernel-src-tree/actions/runs/27383376095 |
cve cve-2025-10263 commit-author Mark Rutland <mark.rutland@arm.com> commit - commit-source-sha cfd391e74134db664feb499d43af286380b10ba8 commit-source arm64 upstream-diff silicon-errata.rst at different path (Documentation/arm64/ vs Documentation/arch/arm64/) and required manual conflict resolution due to condensed table formatting and missing entries in our branch. Content is identical. On 4.18, arm64_repeat_tlbi_cpus[] is a flat struct midr_range array, not struct arm64_cpu_capabilities[]. Upstream uses a nested ERRATA_MIDR_RANGE_LIST() with inline array initialization, which sets .type, .matches, and .midr_range_list fields that do not exist on struct midr_range. Flattened to direct MIDR_ALL_VERSIONS() entries in the array. A number of CPUs developed by Arm suffer from errata whereby a broadcast TLBI;DSB sequence may complete before the global observation of writes which are translated by an affected TLB entry. These errata ONLY affect the completion of memory accesses which have been translated by an invalidated TLB entry, and these errata DO NOT affect the actual invalidation of TLB entries. TLB entries are removed correctly. This issue has been assigned CVE ID CVE-2025-10263. To mitigate this issue, Arm recommends that software follows any affected TLBI;DSB sequence with an additional TLBI;DSB, which will ensure that all memory write effects affected by the first TLBI have been globally observed. The additional TLBI can use any operation that is broadcast to affected CPUs, and the additional DSB can use any option that is sufficient to complete the additional TLBI. The ARM64_WORKAROUND_REPEAT_TLBI workaround is sufficient to mitigate the issue. Enable this workaround for affected CPUs, and update the silicon errata documentation accordingly. Note that due to the manner in which Arm develops IP and tracks errata, some CPUs share a common erratum number. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org> (cherry picked from commit cfd391e74134db664feb499d43af286380b10ba8) Signed-off-by: Jonathan Maple <jmaple@ciq.com> (cherry picked from commit a96d5f4) Signed-off-by: Jonathan Maple <jmaple@ciq.com> add to arm64: errata: Mitigate TLBI arm64: errata: Mitigate TLBI errata on various Arm CPUs cve cve-2025-10263 commit-author Mark Rutland <mark.rutland@arm.com> commit - commit-source-sha cfd391e74134db664feb499d43af286380b10ba8 commit-source arm64 upstream-diff silicon-errata.rst at different path (Documentation/arm64/ vs Documentation/arch/arm64/) and required manual conflict resolution due to condensed table formatting and missing entries in our branch. A number of CPUs developed by Arm suffer from errata whereby a broadcast TLBI;DSB sequence may complete before the global observation of writes which are translated by an affected TLB entry. These errata ONLY affect the completion of memory accesses which have been translated by an invalidated TLB entry, and these errata DO NOT affect the actual invalidation of TLB entries. TLB entries are removed correctly. This issue has been assigned CVE ID CVE-2025-10263. To mitigate this issue, Arm recommends that software follows any affected TLBI;DSB sequence with an additional TLBI;DSB, which will ensure that all memory write effects affected by the first TLBI have been globally observed. The additional TLBI can use any operation that is broadcast to affected CPUs, and the additional DSB can use any option that is sufficient to complete the additional TLBI. The ARM64_WORKAROUND_REPEAT_TLBI workaround is sufficient to mitigate the issue. Enable this workaround for affected CPUs, and update the silicon errata documentation accordingly. Note that due to the manner in which Arm develops IP and tracks errata, some CPUs share a common erratum number. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org> (cherry picked from commit cfd391e74134db664feb499d43af286380b10ba8) Signed-off-by: Jonathan Maple <jmaple@ciq.com> (cherry picked from commit a96d5f4) Signed-off-by: Jonathan Maple <jmaple@ciq.com>
cve cve-2025-10263 commit-author Shanker Donthineni <sdonthineni@nvidia.com> commit - commit-source-sha ec7216f92e4ebd485b1c6dc6aa3f6064b71a5768 commit-source arm64 upstream-diff silicon-errata.rst at different path and required manual conflict resolution due to missing entries in our branch. NVIDIA Olympus cores are affected by the TLBI completion issue tracked as CVE-2025-10263. The existing ARM64_ERRATUM_4118414 handling already uses ARM64_WORKAROUND_REPEAT_TLBI to issue an additional broadcast TLBI;DSB sequence and ensure affected memory write effects are globally observed. Add MIDR_NVIDIA_OLYMPUS to the repeat-TLBI match list so the same mitigation is enabled on affected Olympus systems. Also document the NVIDIA Olympus erratum in the arm64 silicon errata table and list it in the Kconfig help text. Signed-off-by: Shanker Donthineni <sdonthineni@nvidia.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Acked-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Will Deacon <will@kernel.org> (cherry picked from commit ec7216f92e4ebd485b1c6dc6aa3f6064b71a5768) Signed-off-by: Brett Mastbergen <bmastbergen@ciq.com> (cherry picked from commit 573f6f2) Signed-off-by: Jonathan Maple <jmaple@ciq.com>
cve cve-2025-10263 commit-author Will Deacon <will@kernel.org> commit - commit-source-sha 1940e70a8144bf75e6df26bf6f600862ea7f7ea1 commit-source arm64 upstream-diff silicon-errata.rst at different path and required manual conflict resolution due to missing entries in our branch. Commit fb091ff ("arm64: Subscribe Microsoft Azure Cobalt 100 to ARM Neoverse N2 errata") states that Microsoft Azure Cobalt 100 CPU "is a Microsoft implemented CPU based on r0p0 of the ARM Neoverse N2 CPU, and therefore suffers from all the same errata.". So enable the workaround for the latest broadcast TLB invalidation bug on these parts. Signed-off-by: Will Deacon <will@kernel.org> (cherry picked from commit 1940e70a8144bf75e6df26bf6f600862ea7f7ea1) Signed-off-by: Brett Mastbergen <bmastbergen@ciq.com> (cherry picked from commit 5257aae) Signed-off-by: Jonathan Maple <jmaple@ciq.com>
8346a0f to
c95facd
Compare
|
🤖 Validation Checks In Progress Workflow run: https://github.com/ctrliq/kernel-src-tree/actions/runs/27385365926 |
🔍 Upstream Linux Kernel Commit Check
This is an automated message from the kernel commit checker workflow. |
🔍 Interdiff Analysis
================================================================================
* DELTA DIFFERENCES - code changes that differ between the patches *
================================================================================
--- b/net/ipv4/ip_output.c
+++ b/net/ipv4/ip_output.c
@@ -1347,9 +1347,6 @@
goto error;
}
- if (!(flags & MSG_NO_SHARED_FRAGS))
- skb_shinfo(skb)->tx_flags |= SKBTX_SHARED_FRAG;
-
if (skb->ip_summed == CHECKSUM_NONE) {
__wsum csum;
csum = csum_page(page, offset, len);
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/net/ipv4/ip_output.c
+++ b/net/ipv4/ip_output.c
@@ -1233,6 +1233,8 @@
if (err < 0)
goto error;
copy = err;
+ if (!(flags & MSG_NO_SHARED_FRAGS))
+ skb_shinfo(skb)->flags |= SKBFL_SHARED_FRAG;
wmem_alloc_delta += copy;
} else if (!zc) {
int i = skb_shinfo(skb)->nr_frags;
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/net/ipv4/ip_output.c
+++ b/net/ipv4/ip_output.c
@@ -1230,6 +1230,6 @@
- goto error;
- }
-
- if (skb->ip_summed == CHECKSUM_NONE) {
- __wsum csum;
- csum = csum_page(page, offset, len);
+ if (err < 0)
+ goto error;
+ copy = err;
+ wmem_alloc_delta += copy;
+ } else if (!zc) {
+ int i = skb_shinfo(skb)->nr_frags;
================================================================================
* ONLY IN PATCH2 - files not modified by patch1 *
================================================================================
--- a/net/ipv6/esp6.c
+++ b/net/ipv6/esp6.c
@@ -915,7 +915,8 @@ static int esp6_input(struct xfrm_state *x, struct sk_buff *skb)
nfrags = 1;
goto skip_cow;
- } else if (!skb_has_frag_list(skb)) {
+ } else if (!skb_has_frag_list(skb) &&
+ !skb_has_shared_frag(skb)) {
nfrags = skb_shinfo(skb)->nr_frags;
nfrags++;
--- a/net/ipv6/ip6_output.c
+++ b/net/ipv6/ip6_output.c
@@ -1794,6 +1794,8 @@ alloc_new_skb:
if (err < 0)
goto error;
copy = err;
+ if (!(flags & MSG_NO_SHARED_FRAGS))
+ skb_shinfo(skb)->flags |= SKBFL_SHARED_FRAG;
wmem_alloc_delta += copy;
} else if (!zc) {
int i = skb_shinfo(skb)->nr_frags;
================================================================================
* DELTA DIFFERENCES - code changes that differ between the patches *
================================================================================
--- b/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -152,18 +152,6 @@
return queue - queue->ctrl->queues;
}
-static inline bool nvme_tcp_recv_pdu_supported(enum nvme_tcp_pdu_type type)
-{
- switch (type) {
- case nvme_tcp_c2h_data:
- case nvme_tcp_r2t:
- case nvme_tcp_rsp:
- return true;
- default:
- return false;
- }
-}
-
static inline struct blk_mq_tags *nvme_tcp_tagset(struct nvme_tcp_queue *queue)
{
u32 queue_idx = nvme_tcp_queue_id(queue);
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -217,6 +217,19 @@
return queue - queue->ctrl->queues;
}
+static inline bool nvme_tcp_recv_pdu_supported(enum nvme_tcp_pdu_type type)
+{
+ switch (type) {
+ case nvme_tcp_c2h_term:
+ case nvme_tcp_c2h_data:
+ case nvme_tcp_r2t:
+ case nvme_tcp_rsp:
+ return true;
+ default:
+ return false;
+ }
+}
+
/*
* Check if the queue is TLS encrypted
*/
@@ -818,6 +831,16 @@
return 0;
hdr = queue->pdu;
+ if (unlikely(hdr->hlen != sizeof(struct nvme_tcp_rsp_pdu))) {
+ if (!nvme_tcp_recv_pdu_supported(hdr->type))
+ goto unsupported_pdu;
+
+ dev_err(queue->ctrl->ctrl.device,
+ "pdu type %d has unexpected header length (%d)\n",
+ hdr->type, hdr->hlen);
+ return -EPROTO;
+ }
+
if (unlikely(hdr->type == nvme_tcp_c2h_term)) {
/*
* C2HTermReq never includes Header or Data digests.
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -149,6 +149,6 @@
return queue - queue->ctrl->queues;
}
-static inline struct blk_mq_tags *nvme_tcp_tagset(struct nvme_tcp_queue *queue)
-{
- u32 queue_idx = nvme_tcp_queue_id(queue);
+/*
+ * Check if the queue is TLS encrypted
+ */
@@ -674,6 +818,6 @@
return 0;
hdr = queue->pdu;
- if (queue->hdr_digest) {
- ret = nvme_tcp_verify_hdgst(queue, queue->pdu, hdr->hlen);
- if (unlikely(ret))
+ if (unlikely(hdr->type == nvme_tcp_c2h_term)) {
+ /*
+ * C2HTermReq never includes Header or Data digests.
================================================================================
* DELTA DIFFERENCES - code changes that differ between the patches *
================================================================================
--- b/net/ipv4/udp_offload.c
+++ b/net/ipv4/udp_offload.c
@@ -11,7 +11,6 @@
*/
#include <linux/skbuff.h>
-#include <net/ip6_checksum.h>
#include <net/udp.h>
#include <net/protocol.h>
#include <net/inet_common.h>
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/net/ipv4/udp_offload.c
+++ b/net/ipv4/udp_offload.c
@@ -273,6 +269,6 @@
if (skb_shinfo(gso_skb)->gso_type & SKB_GSO_FRAGLIST)
return __udp_gso_segment_list(gso_skb, features, is_ipv6);
- mss = skb_shinfo(gso_skb)->gso_size;
- if (gso_skb->len <= sizeof(*uh) + mss)
+ skb_pull(gso_skb, sizeof(*uh));
+
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/net/ipv4/udp_offload.c
+++ b/net/ipv4/udp_offload.c
@@ -273,6 +273,6 @@
bool copy_dtor;
__sum16 check;
__be16 newlen;
- if (skb_shinfo(gso_skb)->gso_type & SKB_GSO_FRAGLIST) {
- /* Detect modified geometry and pass those to skb_segment. */
+ mss = skb_shinfo(gso_skb)->gso_size;
+ if (gso_skb->len <= sizeof(*uh) + mss)
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -1607,7 +1613,6 @@
cpu_detect(c);
get_cpu_vendor(c);
get_cpu_cap(c);
- get_cpu_address_sizes(c);
setup_force_cpu_cap(X86_FEATURE_CPUID);
cpu_parse_early_param();
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -1502,6 +1597,5 @@
get_cpu_vendor(c);
get_cpu_cap(c);
- get_model_name(c); /* RHEL8: get model name for unsupported check */
get_cpu_address_sizes(c);
setup_force_cpu_cap(X86_FEATURE_CPUID);
cpu_parse_early_param();
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -1508,5 +1509,4 @@
get_cpu_cap(c);
- get_model_name(c); /* RHEL8: get model name for unsupported check */
setup_force_cpu_cap(X86_FEATURE_CPUID);
cpu_parse_early_param();
@@ -1520,5 +1599,4 @@
} else {
- identify_cpu_without_cpuid(c);
setup_clear_cpu_cap(X86_FEATURE_CPUID);
}
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/drivers/net/ethernet/microsoft/Kconfig
+++ b/drivers/net/ethernet/microsoft/Kconfig
@@ -20,4 +20,4 @@
depends on PCI_MSI && X86_64
depends on PCI_HYPERV
select AUXILIARY_BUS
- help
+ select PAGE_POOL
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/drivers/net/ethernet/microsoft/Kconfig
+++ b/drivers/net/ethernet/microsoft/Kconfig
@@ -21,4 +21,4 @@
depends on X86_64 || (ARM64 && !CPU_BIG_ENDIAN && ARM64_4K_PAGES)
depends on PCI_HYPERV
select AUXILIARY_BUS
- help
+ select PAGE_POOL
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -217,17 +256,6 @@
struct pci_bus_region region, inverted_region;
const char *res_name = pci_resource_name(dev, res - dev->resource);
- mask = type ? PCI_ROM_ADDRESS_MASK : ~0;
-
- /* No printks while decoding is disabled! */
- if (!dev->mmio_always_on) {
- pci_read_config_word(dev, PCI_COMMAND, &orig_cmd);
- if (orig_cmd & PCI_COMMAND_DECODE_ENABLE) {
- pci_write_config_word(dev, PCI_COMMAND,
- orig_cmd & ~PCI_COMMAND_DECODE_ENABLE);
- }
- }
-
res->name = pci_name(dev);
pci_read_config_dword(dev, pos, &l);
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/drivers/pci/iov.c
+++ b/drivers/pci/iov.c
@@ -740,4 +739,5 @@
struct resource *res;
+ const char *res_name;
struct pci_dev *pdev;
pci_read_config_word(dev, pos + PCI_SRIOV_CTRL, &ctrl);
--- b/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -186,6 +180,7 @@
u64 l64, sz64, mask64;
u16 orig_cmd;
struct pci_bus_region region, inverted_region;
+ const char *res_name = pci_resource_name(dev, res - dev->resource);
mask = type ? PCI_ROM_ADDRESS_MASK : ~0;
================================================================================
* DELTA DIFFERENCES - code changes that differ between the patches *
================================================================================
--- b/drivers/net/ethernet/intel/idpf/idpf_singleq_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_singleq_txrx.c
@@ -1,8 +1,6 @@
// SPDX-License-Identifier: GPL-2.0-only
/* Copyright (C) 2023 Intel Corporation */
-#include <net/libeth/tx.h>
-
#include "idpf.h"
/**
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -1,8 +1,6 @@
// SPDX-License-Identifier: GPL-2.0-only
/* Copyright (C) 2023 Intel Corporation */
-#include <net/libeth/tx.h>
-
#include "idpf.h"
#include "idpf_virtchnl.h"
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/drivers/net/ethernet/intel/idpf/idpf_singleq_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_singleq_txrx.c
@@ -2,6 +2,7 @@
/* Copyright (C) 2023 Intel Corporation */
#include <net/libeth/rx.h>
+#include <net/libeth/tx.h>
#include "idpf.h"
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -2,6 +2,7 @@
/* Copyright (C) 2023 Intel Corporation */
#include <net/libeth/rx.h>
+#include <net/libeth/tx.h>
#include "idpf.h"
#include "idpf_virtchnl.h"
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/drivers/net/ethernet/intel/idpf/idpf_singleq_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_singleq_txrx.c
@@ -2,4 +1,8 @@
/* Copyright (C) 2023 Intel Corporation */
+#include <net/libeth/rx.h>
+
+#include <net/libeth/tx.h>
+
#include "idpf.h"
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -2,5 +1,9 @@
/* Copyright (C) 2023 Intel Corporation */
+#include <net/libeth/rx.h>
+
+#include <net/libeth/tx.h>
+
#include "idpf.h"
#include "idpf_virtchnl.h"
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
@@ -785,7 +785,7 @@
u32 next_to_use;
u32 next_to_clean;
- u32 num_completions;
+ aligned_u64 num_completions;
__cacheline_group_end_aligned(read_write);
__cacheline_group_begin_aligned(cold);
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
@@ -971,4 +920,4 @@
u32 num_completions_pending;
};
-/**
+static inline int idpf_q_vector_to_mem(const struct idpf_q_vector *q_vector)
================================================================================
* DELTA DIFFERENCES - code changes that differ between the patches *
================================================================================
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -247,7 +247,6 @@
struct device *dev = tx_q->dev;
struct idpf_sw_queue *refillq;
int err;
- unsigned int i = 0;
err = idpf_tx_buf_alloc_all(tx_q);
if (err)
@@ -282,7 +281,7 @@
goto err_alloc;
}
- for (i = 0; i < refillq->desc_count; i++)
+ for (unsigned int i = 0; i < refillq->desc_count; i++)
refillq->ring[i] =
FIELD_PREP(IDPF_RFL_BI_BUFID_M, i) |
FIELD_PREP(IDPF_RFL_BI_GEN_M,
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -3551,7 +3630,7 @@
skip_data:
rx_buf->netmem = 0;
- idpf_rx_post_buf_refill(refillq, buf_id);
+ idpf_post_buf_refill(refillq, buf_id);
IDPF_RX_BUMP_NTC(rxq, ntc);
/* skip if it is non EOP desc */
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
@@ -622,6 +622,7 @@
* @cleaned_pkts: Number of packets cleaned for the above said case
* @tx_max_bufs: Max buffers that can be transmitted with scatter-gather
* @stash: Tx buffer stash for Flow-based scheduling mode
+ * @refillq: Pointer to refill queue
* @compl_tag_bufid_m: Completion tag buffer id mask
* @compl_tag_cur_gen: Used to keep track of current completion tag generation
* @compl_tag_gen_max: To determine when compl_tag_cur_gen should be reset
@@ -671,6 +672,7 @@
u16 tx_max_bufs;
struct idpf_txq_stash *stash;
+ struct idpf_sw_queue *refillq;
u16 compl_tag_bufid_m;
u16 compl_tag_cur_gen;
@@ -692,7 +694,7 @@
__cacheline_group_end_aligned(cold);
};
libeth_cacheline_set_assert(struct idpf_tx_queue, 64,
- 112 + sizeof(struct u64_stats_sync),
+ 120 + sizeof(struct u64_stats_sync),
24);
/**
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -244,4 +246,5 @@
struct device *dev = tx_q->dev;
int err;
+ unsigned int i = 0;
err = idpf_tx_buf_alloc_all(tx_q);
@@ -3358,4 +3417,4 @@
idpf_rx_post_buf_refill(refillq, buf_id);
-
IDPF_RX_BUMP_NTC(rxq, ntc);
+
/* skip if it is non EOP desc */
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
@@ -624,6 +619,6 @@
* @cleaned_pkts: Number of packets cleaned for the above said case
* @tx_max_bufs: Max buffers that can be transmitted with scatter-gather
- * @tx_min_pkt_len: Min supported packet length
+ * @stash: Tx buffer stash for Flow-based scheduling mode
* @compl_tag_bufid_m: Completion tag buffer id mask
- * @compl_tag_gen_s: Completion tag generation bit
- * The format of the completion tag will change based on the TXQ
+ * @compl_tag_cur_gen: Used to keep track of current completion tag generation
+ * @compl_tag_gen_max: To determine when compl_tag_cur_gen should be reset
@@ -743,6 +692,2 @@
- u16 tx_max_bufs;
- u16 tx_min_pkt_len;
-
- u16 compl_tag_bufid_m;
- u16 compl_tag_gen_s;
+/**
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
@@ -610,6 +610,8 @@
* @netdev: &net_device corresponding to this queue
* @next_to_use: Next descriptor to use
* @next_to_clean: Next descriptor to clean
+ * @last_re: last descriptor index that RE bit was set
+ * @tx_max_bufs: Max buffers that can be transmitted with scatter-gather
* @cleaned_bytes: Splitq only, TXQ only: When a TX completion is received on
* the TX completion queue, it can be for any TXQ associated
* with that completion queue. This means we can clean up to
@@ -620,7 +622,6 @@
* only once at the end of the cleaning routine.
* @clean_budget: singleq only, queue cleaning budget
* @cleaned_pkts: Number of packets cleaned for the above said case
- * @tx_max_bufs: Max buffers that can be transmitted with scatter-gather
* @stash: Tx buffer stash for Flow-based scheduling mode
* @refillq: Pointer to refill queue
* @compl_tag_bufid_m: Completion tag buffer id mask
@@ -672,7 +675,6 @@
};
u16 cleaned_pkts;
- u16 tx_max_bufs;
struct idpf_txq_stash *stash;
struct idpf_sw_queue *refillq;
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
@@ -607,6 +607,5 @@
- * @desc_count: Number of descriptors
+ * @netdev: &net_device corresponding to this queue
* @next_to_use: Next descriptor to use
* @next_to_clean: Next descriptor to clean
- * @netdev: &net_device corresponding to this queue
* @cleaned_bytes: Splitq only, TXQ only: When a TX completion is received on
* the TX completion queue, it can be for any TXQ associated
@@ -685,5 +622,5 @@
* @cleaned_pkts: Number of packets cleaned for the above said case
* @tx_max_bufs: Max buffers that can be transmitted with scatter-gather
- * @tx_min_pkt_len: Min supported packet length
+ * @stash: Tx buffer stash for Flow-based scheduling mode
* @refillq: Pointer to refill queue
* @compl_tag_bufid_m: Completion tag buffer id mask
@@ -723,4 +660,4 @@
- u16 desc_count;
+ __cacheline_group_begin_aligned(read_write);
u16 next_to_use;
u16 next_to_clean;
@@ -731,5 +668,5 @@
u16 tx_max_bufs;
- u16 tx_min_pkt_len;
+ struct idpf_txq_stash *stash;
struct idpf_sw_queue *refillq;
================================================================================
* DELTA DIFFERENCES - code changes that differ between the patches *
================================================================================
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -2287,4 +2287,55 @@
/**
+ * idpf_tx_dma_map_error - handle TX DMA map errors
+ * @txq: queue to send buffer on
+ * @skb: send buffer
+ * @first: original first buffer info buffer for packet
+ * @idx: starting point on ring to unwind
+ */
+void idpf_tx_dma_map_error(struct idpf_tx_queue *txq, struct sk_buff *skb,
+ struct idpf_tx_buf *first, u16 idx)
+{
+ struct libeth_sq_napi_stats ss = { };
+ struct libeth_cq_pp cp = {
+ .dev = txq->dev,
+ .ss = &ss,
+ };
+
+ u64_stats_update_begin(&txq->stats_sync);
+ u64_stats_inc(&txq->q_stats.dma_map_errs);
+ u64_stats_update_end(&txq->stats_sync);
+
+ /* clear dma mappings for failed tx_buf map */
+ for (;;) {
+ struct idpf_tx_buf *tx_buf;
+
+ tx_buf = &txq->tx_buf[idx];
+ libeth_tx_complete(tx_buf, &cp);
+ if (tx_buf == first)
+ break;
+ if (idx == 0)
+ idx = txq->desc_count;
+ idx--;
+ }
+
+ if (skb_is_gso(skb)) {
+ union idpf_tx_flex_desc *tx_desc;
+
+ /* If we failed a DMA mapping for a TSO packet, we will have
+ * used one additional descriptor for a context
+ * descriptor. Reset that here.
+ */
+ tx_desc = &txq->flex_tx[idx];
+ memset(tx_desc, 0, sizeof(struct idpf_flex_tx_ctx_desc));
+ if (idx == 0)
+ idx = txq->desc_count;
+ idx--;
+ }
+
+ /* Update tail in case netdev_xmit_more was previously true */
+ idpf_tx_buf_hw_update(txq, idx, false);
+}
+
+/**
* idpf_tx_splitq_bump_ntu - adjust NTU and generation
* @txq: the tx ring to wrap
@@ -2335,35 +2386,4 @@
/**
- * idpf_tx_splitq_pkt_err_unmap - Unmap buffers and bump tail in case of error
- * @txq: Tx queue to unwind
- * @params: pointer to splitq params struct
- * @first: starting buffer for packet to unmap
- */
-static void idpf_tx_splitq_pkt_err_unmap(struct idpf_tx_queue *txq,
- struct idpf_tx_splitq_params *params,
- struct idpf_tx_buf *first)
-{
- struct libeth_sq_napi_stats ss = { };
- struct idpf_tx_buf *tx_buf = first;
- struct libeth_cq_pp cp = {
- .dev = txq->dev,
- .ss = &ss,
- };
- u32 idx = 0;
-
- u64_stats_update_begin(&txq->stats_sync);
- u64_stats_inc(&txq->q_stats.dma_map_errs);
- u64_stats_update_end(&txq->stats_sync);
-
- do {
- libeth_tx_complete(tx_buf, &cp);
- idpf_tx_clean_buf_ring_bump_ntc(txq, idx, tx_buf);
- } while (idpf_tx_buf_compl_tag(tx_buf) == params->compl_tag);
-
- /* Update tail in case netdev_xmit_more was previously true. */
- idpf_tx_buf_hw_update(txq, params->prev_ntu, false);
-}
-
-/**
* idpf_tx_splitq_map - Build the Tx flex descriptor
* @tx_q: queue to send buffer on
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -2339,57 +2339,6 @@
return count;
}
-/**
- * idpf_tx_dma_map_error - handle TX DMA map errors
- * @txq: queue to send buffer on
- * @skb: send buffer
- * @first: original first buffer info buffer for packet
- * @idx: starting point on ring to unwind
- */
-void idpf_tx_dma_map_error(struct idpf_tx_queue *txq, struct sk_buff *skb,
- struct idpf_tx_buf *first, u16 idx)
-{
- struct libeth_sq_napi_stats ss = { };
- struct libeth_cq_pp cp = {
- .dev = txq->dev,
- .ss = &ss,
- };
-
- u64_stats_update_begin(&txq->stats_sync);
- u64_stats_inc(&txq->q_stats.dma_map_errs);
- u64_stats_update_end(&txq->stats_sync);
-
- /* clear dma mappings for failed tx_buf map */
- for (;;) {
- struct idpf_tx_buf *tx_buf;
-
- tx_buf = &txq->tx_buf[idx];
- libeth_tx_complete(tx_buf, &cp);
- if (tx_buf == first)
- break;
- if (idx == 0)
- idx = txq->desc_count;
- idx--;
- }
-
- if (skb_is_gso(skb)) {
- union idpf_tx_flex_desc *tx_desc;
-
- /* If we failed a DMA mapping for a TSO packet, we will have
- * used one additional descriptor for a context
- * descriptor. Reset that here.
- */
- tx_desc = &txq->flex_tx[idx];
- memset(tx_desc, 0, sizeof(*tx_desc));
- if (idx == 0)
- idx = txq->desc_count;
- idx--;
- }
-
- /* Update tail in case netdev_xmit_more was previously true */
- idpf_tx_buf_hw_update(txq, idx, false);
-}
-
/**
* idpf_tx_splitq_bump_ntu - adjust NTU and generation
* @txq: the tx ring to wrap
@@ -2438,6 +2387,37 @@
return true;
}
+/**
+ * idpf_tx_splitq_pkt_err_unmap - Unmap buffers and bump tail in case of error
+ * @txq: Tx queue to unwind
+ * @params: pointer to splitq params struct
+ * @first: starting buffer for packet to unmap
+ */
+static void idpf_tx_splitq_pkt_err_unmap(struct idpf_tx_queue *txq,
+ struct idpf_tx_splitq_params *params,
+ struct idpf_tx_buf *first)
+{
+ struct libeth_sq_napi_stats ss = { };
+ struct idpf_tx_buf *tx_buf = first;
+ struct libeth_cq_pp cp = {
+ .dev = txq->dev,
+ .ss = &ss,
+ };
+ u32 idx = 0;
+
+ u64_stats_update_begin(&txq->stats_sync);
+ u64_stats_inc(&txq->q_stats.dma_map_errs);
+ u64_stats_update_end(&txq->stats_sync);
+
+ do {
+ libeth_tx_complete(tx_buf, &cp);
+ idpf_tx_clean_buf_ring_bump_ntc(txq, idx, tx_buf);
+ } while (idpf_tx_buf_compl_tag(tx_buf) == params->compl_tag);
+
+ /* Update tail in case netdev_xmit_more was previously true. */
+ idpf_tx_buf_hw_update(txq, params->prev_ntu, false);
+}
+
/**
* idpf_tx_splitq_map - Build the Tx flex descriptor
* @tx_q: queue to send buffer on
@@ -2482,8 +2462,9 @@
for (frag = &skb_shinfo(skb)->frags[0];; frag++) {
unsigned int max_data = IDPF_TX_MAX_DESC_DATA_ALIGNED;
- if (dma_mapping_error(tx_q->dev, dma))
- return idpf_tx_dma_map_error(tx_q, skb, first, i);
+ if (unlikely(dma_mapping_error(tx_q->dev, dma)))
+ return idpf_tx_splitq_pkt_err_unmap(tx_q, params,
+ first);
first->nr_frags++;
idpf_tx_buf_compl_tag(tx_buf) = params->compl_tag;
@@ -2939,7 +2920,9 @@
static netdev_tx_t idpf_tx_splitq_frame(struct sk_buff *skb,
struct idpf_tx_queue *tx_q)
{
- struct idpf_tx_splitq_params tx_params = { };
+ struct idpf_tx_splitq_params tx_params = {
+ .prev_ntu = tx_q->next_to_use,
+ };
union idpf_flex_tx_ctx_desc *ctx_desc;
struct idpf_tx_buf *first;
unsigned int count;
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -2326,7 +2380,7 @@
* descriptor. Reset that here.
*/
tx_desc = &txq->flex_tx[idx];
- memset(tx_desc, 0, sizeof(struct idpf_flex_tx_ctx_desc));
+ memset(tx_desc, 0, sizeof(*tx_desc));
if (idx == 0)
idx = txq->desc_count;
idx--;
@@ -2817,4 +2871,5 @@
{
struct idpf_tx_splitq_params tx_params = { };
+ union idpf_flex_tx_ctx_desc *ctx_desc;
struct idpf_tx_buf *first;
unsigned int count;
================================================================================
* DELTA DIFFERENCES - code changes that differ between the patches *
================================================================================
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -1915,11 +1915,8 @@
.napi = budget,
};
- tx_buf = &txq->tx_buf[buf_id];
- if (tx_buf->type == LIBETH_SQE_SKB) {
+ if (tx_buf->type == LIBETH_SQE_SKB)
libeth_tx_complete(tx_buf, &cp);
- idpf_post_buf_refill(txq->refillq, buf_id);
- }
while (idpf_tx_buf_next(tx_buf) != IDPF_TXBUF_NULL) {
buf_id = idpf_tx_buf_next(tx_buf);
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -1962,6 +1962,7 @@
idpf_tx_buf_compl_tag(tx_buf) != compl_tag))
return false;
+ tx_buf = &txq->tx_buf[buf_id];
if (tx_buf->type == LIBETH_SQE_SKB) {
if (skb_shinfo(tx_buf->skb)->tx_flags & SKBTX_IN_PROGRESS)
idpf_tx_read_tstamp(txq, tx_buf->skb);
@@ -1965,6 +1966,7 @@
idpf_tx_read_tstamp(txq, tx_buf->skb);
libeth_tx_complete(tx_buf, &cp);
+ idpf_post_buf_refill(txq->refillq, buf_id);
}
idpf_tx_clean_buf_ring_bump_ntc(txq, idx, tx_buf);
@@ -2892,6 +2859,7 @@
struct idpf_tx_buf *first;
unsigned int count;
int tso, idx;
+ u32 buf_id;
count = idpf_tx_desc_count_required(tx_q, skb);
if (unlikely(!count))
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
@@ -703,6 +710,7 @@
dma_addr_t dma;
struct idpf_q_vector *q_vector;
+ u32 buf_pool_size;
__cacheline_group_end_aligned(cold);
};
libeth_cacheline_set_assert(struct idpf_tx_queue, 64,
@@ -706,7 +714,7 @@
};
libeth_cacheline_set_assert(struct idpf_tx_queue, 64,
120 + sizeof(struct u64_stats_sync),
- 24);
+ 32);
/**
* struct idpf_buf_queue - software structure representing a buffer queue
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -1915,8 +1965,12 @@
idpf_tx_buf_compl_tag(tx_buf) != compl_tag))
return false;
- if (tx_buf->type == LIBETH_SQE_SKB)
+ if (tx_buf->type == LIBETH_SQE_SKB) {
+ if (skb_shinfo(tx_buf->skb)->tx_flags & SKBTX_IN_PROGRESS)
+ idpf_tx_read_tstamp(txq, tx_buf->skb);
+
libeth_tx_complete(tx_buf, &cp);
+ }
idpf_tx_clean_buf_ring_bump_ntc(txq, idx, tx_buf);
@@ -2744,4 +2798,4 @@
- struct idpf_flex_tx_ctx_desc *desc;
+ union idpf_flex_tx_ctx_desc *desc;
int i = txq->next_to_use;
txq->tx_buf[i].type = LIBETH_SQE_CTX;
@@ -2802,6 +2856,6 @@
struct idpf_tx_buf *first;
unsigned int count;
- int tso;
+ int tso, idx;
count = idpf_tx_desc_count_required(tx_q, skb);
if (unlikely(!count))
@@ -2840,4 +2962,4 @@
- u64_stats_update_end(&tx_q->stats_sync);
+ idpf_tx_set_tstamp_desc(ctx_desc, idx);
}
/* record the location of the first descriptor for this packet */
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
@@ -692,5 +694,9 @@
struct idpf_q_vector *q_vector;
-} ____cacheline_aligned;
+ __cacheline_group_end_aligned(cold);
+};
+libeth_cacheline_set_assert(struct idpf_tx_queue, 64,
+ 120 + sizeof(struct u64_stats_sync),
+ 24);
/**
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -2909,7 +2926,7 @@
};
union idpf_flex_tx_ctx_desc *ctx_desc;
struct idpf_tx_buf *first;
- unsigned int count;
+ u32 count, buf_count = 1;
int tso, idx;
u32 buf_id;
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -2768,7 +2821,8 @@
};
+ union idpf_flex_tx_ctx_desc *ctx_desc;
struct idpf_tx_buf *first;
unsigned int count;
- int tso;
+ int tso, idx;
u32 buf_id;
count = idpf_tx_desc_count_required(tx_q, skb);
================================================================================
* DELTA DIFFERENCES - code changes that differ between the patches *
================================================================================
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -1557,6 +1557,82 @@
wake_up(&vport->sw_marker_wq);
}
+/**
+ * idpf_tx_clean_stashed_bufs - clean bufs that were stored for
+ * out of order completions
+ * @txq: queue to clean
+ * @compl_tag: completion tag of packet to clean (from completion descriptor)
+ * @cleaned: pointer to stats struct to track cleaned packets/bytes
+ * @budget: Used to determine if we are in netpoll
+ */
+static void idpf_tx_clean_stashed_bufs(struct idpf_tx_queue *txq,
+ u16 compl_tag,
+ struct libeth_sq_napi_stats *cleaned,
+ int budget)
+{
+ struct idpf_tx_stash *stash;
+ struct hlist_node *tmp_buf;
+ struct libeth_cq_pp cp = {
+ .dev = txq->dev,
+ .ss = cleaned,
+ .napi = budget,
+ };
+
+ /* Buffer completion */
+ hash_for_each_possible_safe(txq->stash->sched_buf_hash, stash, tmp_buf,
+ hlist, compl_tag) {
+ if (unlikely(idpf_tx_buf_compl_tag(&stash->buf) != compl_tag))
+ continue;
+
+ hash_del(&stash->hlist);
+ libeth_tx_complete(&stash->buf, &cp);
+
+ /* Push shadow buf back onto stack */
+ idpf_buf_lifo_push(&txq->stash->buf_stack, stash);
+ }
+}
+
+/**
+ * idpf_stash_flow_sch_buffers - store buffer parameters info to be freed at a
+ * later time (only relevant for flow scheduling mode)
+ * @txq: Tx queue to clean
+ * @tx_buf: buffer to store
+ */
+static int idpf_stash_flow_sch_buffers(struct idpf_tx_queue *txq,
+ struct idpf_tx_buf *tx_buf)
+{
+ struct idpf_tx_stash *stash;
+
+ if (unlikely(tx_buf->type <= LIBETH_SQE_CTX))
+ return 0;
+
+ stash = idpf_buf_lifo_pop(&txq->stash->buf_stack);
+ if (unlikely(!stash)) {
+ net_err_ratelimited("%s: No out-of-order TX buffers left!\n",
+ netdev_name(txq->netdev));
+
+ return -ENOMEM;
+ }
+
+ /* Store buffer params in shadow buffer */
+ stash->buf.skb = tx_buf->skb;
+ stash->buf.bytes = tx_buf->bytes;
+ stash->buf.packets = tx_buf->packets;
+ stash->buf.type = tx_buf->type;
+ stash->buf.nr_frags = tx_buf->nr_frags;
+ dma_unmap_addr_set(&stash->buf, dma, dma_unmap_addr(tx_buf, dma));
+ dma_unmap_len_set(&stash->buf, len, dma_unmap_len(tx_buf, len));
+ idpf_tx_buf_compl_tag(&stash->buf) = idpf_tx_buf_compl_tag(tx_buf);
+
+ /* Add buffer to buf_hash table to be freed later */
+ hash_add(txq->stash->sched_buf_hash, &stash->hlist,
+ idpf_tx_buf_compl_tag(&stash->buf));
+
+ tx_buf->type = LIBETH_SQE_EMPTY;
+
+ return 0;
+}
+
#define idpf_tx_splitq_clean_bump_ntc(txq, ntc, desc, buf) \
do { \
if (unlikely(++(ntc) == (txq)->desc_count)) { \
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
@@ -663,6 +663,7 @@
* @cleaned_pkts: Number of packets cleaned for the above said case
* @tx_min_pkt_len: Min supported packet length
* @refillq: Pointer to refill queue
+ * @compl_tag_bufid_m: Completion tag buffer id mask
* @compl_tag_gen_s: Completion tag generation bit
* The format of the completion tag will change based on the TXQ
* descriptor ring size so that we can maintain roughly the same level
@@ -683,6 +684,9 @@
* --------------------------------
*
* This gives us 8*8160 = 65280 possible unique values.
+ * @compl_tag_cur_gen: Used to keep track of current completion tag generation
+ * @compl_tag_gen_max: To determine when compl_tag_cur_gen should be reset
+ * @stash: Tx buffer stash for Flow-based scheduling mode
* @stats_sync: See struct u64_stats_sync
* @q_stats: See union idpf_tx_queue_stats
* @q_id: Queue id
@@ -724,6 +728,14 @@
u16 tx_min_pkt_len;
struct idpf_sw_queue *refillq;
+ u16 compl_tag_bufid_m;
+ u16 compl_tag_gen_s;
+
+ u16 compl_tag_cur_gen;
+ u16 compl_tag_gen_max;
+
+ struct idpf_txq_stash *stash;
+
struct u64_stats_sync stats_sync;
struct idpf_tx_queue_stats q_stats;
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -1602,87 +1462,6 @@
spin_unlock_bh(&tx_tstamp_caps->status_lock);
}
-/**
- * idpf_tx_clean_stashed_bufs - clean bufs that were stored for
- * out of order completions
- * @txq: queue to clean
- * @compl_tag: completion tag of packet to clean (from completion descriptor)
- * @cleaned: pointer to stats struct to track cleaned packets/bytes
- * @budget: Used to determine if we are in netpoll
- */
-static void idpf_tx_clean_stashed_bufs(struct idpf_tx_queue *txq,
- u16 compl_tag,
- struct libeth_sq_napi_stats *cleaned,
- int budget)
-{
- struct idpf_tx_stash *stash;
- struct hlist_node *tmp_buf;
- struct libeth_cq_pp cp = {
- .dev = txq->dev,
- .ss = cleaned,
- .napi = budget,
- };
-
- /* Buffer completion */
- hash_for_each_possible_safe(txq->stash->sched_buf_hash, stash, tmp_buf,
- hlist, compl_tag) {
- if (unlikely(idpf_tx_buf_compl_tag(&stash->buf) != compl_tag))
- continue;
-
- hash_del(&stash->hlist);
-
- if (stash->buf.type == LIBETH_SQE_SKB &&
- (skb_shinfo(stash->buf.skb)->tx_flags & SKBTX_IN_PROGRESS))
- idpf_tx_read_tstamp(txq, stash->buf.skb);
-
- libeth_tx_complete(&stash->buf, &cp);
-
- /* Push shadow buf back onto stack */
- idpf_buf_lifo_push(&txq->stash->buf_stack, stash);
- }
-}
-
-/**
- * idpf_stash_flow_sch_buffers - store buffer parameters info to be freed at a
- * later time (only relevant for flow scheduling mode)
- * @txq: Tx queue to clean
- * @tx_buf: buffer to store
- */
-static int idpf_stash_flow_sch_buffers(struct idpf_tx_queue *txq,
- struct idpf_tx_buf *tx_buf)
-{
- struct idpf_tx_stash *stash;
-
- if (unlikely(tx_buf->type <= LIBETH_SQE_CTX))
- return 0;
-
- stash = idpf_buf_lifo_pop(&txq->stash->buf_stack);
- if (unlikely(!stash)) {
- net_err_ratelimited("%s: No out-of-order TX buffers left!\n",
- netdev_name(txq->netdev));
-
- return -ENOMEM;
- }
-
- /* Store buffer params in shadow buffer */
- stash->buf.skb = tx_buf->skb;
- stash->buf.bytes = tx_buf->bytes;
- stash->buf.packets = tx_buf->packets;
- stash->buf.type = tx_buf->type;
- stash->buf.nr_frags = tx_buf->nr_frags;
- dma_unmap_addr_set(&stash->buf, dma, dma_unmap_addr(tx_buf, dma));
- dma_unmap_len_set(&stash->buf, len, dma_unmap_len(tx_buf, len));
- idpf_tx_buf_compl_tag(&stash->buf) = idpf_tx_buf_compl_tag(tx_buf);
-
- /* Add buffer to buf_hash table to be freed later */
- hash_add(txq->stash->sched_buf_hash, &stash->hlist,
- idpf_tx_buf_compl_tag(&stash->buf));
-
- tx_buf->type = LIBETH_SQE_EMPTY;
-
- return 0;
-}
-
#define idpf_tx_splitq_clean_bump_ntc(txq, ntc, desc, buf) \
do { \
if (unlikely(++(ntc) == (txq)->desc_count)) { \
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
@@ -598,7 +565,6 @@
* only once at the end of the cleaning routine.
* @clean_budget: singleq only, queue cleaning budget
* @cleaned_pkts: Number of packets cleaned for the above said case
- * @stash: Tx buffer stash for Flow-based scheduling mode
* @refillq: Pointer to refill queue
* @compl_tag_bufid_m: Completion tag buffer id mask
* @compl_tag_cur_gen: Used to keep track of current completion tag generation
@@ -600,9 +566,6 @@
* @cleaned_pkts: Number of packets cleaned for the above said case
* @stash: Tx buffer stash for Flow-based scheduling mode
* @refillq: Pointer to refill queue
- * @compl_tag_bufid_m: Completion tag buffer id mask
- * @compl_tag_cur_gen: Used to keep track of current completion tag generation
- * @compl_tag_gen_max: To determine when compl_tag_cur_gen should be reset
* @cached_tstamp_caps: Tx timestamp capabilities negotiated with the CP
* @tstamp_task: Work that handles Tx timestamp read
* @stats_sync: See struct u64_stats_sync
@@ -633,7 +596,6 @@
u16 desc_count;
u16 tx_min_pkt_len;
- u16 compl_tag_gen_s;
struct net_device *netdev;
__cacheline_group_end_aligned(read_mostly);
@@ -650,7 +612,6 @@
};
u16 cleaned_pkts;
- struct idpf_txq_stash *stash;
struct idpf_sw_queue *refillq;
u16 compl_tag_bufid_m;
@@ -653,10 +614,6 @@
struct idpf_txq_stash *stash;
struct idpf_sw_queue *refillq;
- u16 compl_tag_bufid_m;
- u16 compl_tag_cur_gen;
- u16 compl_tag_gen_max;
-
struct idpf_ptp_vport_tx_tstamp_caps *cached_tstamp_caps;
struct work_struct *tstamp_task;
@@ -674,7 +631,7 @@
__cacheline_group_end_aligned(cold);
};
libeth_cacheline_set_assert(struct idpf_tx_queue, 64,
- 120 + sizeof(struct u64_stats_sync),
+ 104 + sizeof(struct u64_stats_sync),
32);
/**
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -3,4 +3,4 @@
-#include "idpf.h"
+#include "idpf_ptp.h"
#include "idpf_virtchnl.h"
struct idpf_tx_stash {
@@ -1725,6 +1770,11 @@
continue;
hash_del(&stash->hlist);
+
+ if (stash->buf.type == LIBETH_SQE_SKB &&
+ (skb_shinfo(stash->buf.skb)->tx_flags & SKBTX_IN_PROGRESS))
+ idpf_tx_read_tstamp(txq, stash->buf.skb);
+
libeth_tx_complete(&stash->buf, &cp);
/* Push shadow buf back onto stack */
--- b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.h
@@ -713,9 +663,7 @@
+ u16 desc_count;
+
+ u16 tx_min_pkt_len;
+ u16 compl_tag_gen_s;
+
+ struct net_device *netdev;
+ __cacheline_group_end_aligned(read_mostly);
- * --------------------------------
- *
- * This gives us 8*8160 = 65280 possible unique values.
- * @compl_tag_cur_gen: Used to keep track of current completion tag generation
- * @compl_tag_gen_max: To determine when compl_tag_cur_gen should be reset
- * @stash: Tx buffer stash for Flow-based scheduling mode
- * @stats_sync: See struct u64_stats_sync
- * @q_stats: See union idpf_tx_queue_stats
- * @q_id: Queue id
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/kernel/exit.c
+++ b/kernel/exit.c
@@ -475,6 +475,18 @@
BUG_ON(mm != current->active_mm);
/* more a memory barrier than a real lock */
task_lock(current);
+ /*
+ * When a thread stops operating on an address space, the loop
+ * in membarrier_private_expedited() may not observe that
+ * tsk->mm, and the loop in membarrier_global_expedited() may
+ * not observe a MEMBARRIER_STATE_GLOBAL_EXPEDITED
+ * rq->membarrier_state, so those would not issue an IPI.
+ * Membarrier requires a memory barrier after accessing
+ * user-space memory, before clearing tsk->mm or the
+ * rq->membarrier_state.
+ */
+ smp_mb__after_spinlock();
+ local_irq_disable();
current->mm = NULL;
mmap_read_unlock(mm);
enter_lazy_tlb(mm, current);
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/include/linux/sched/mm.h
+++ b/include/linux/sched/mm.h
@@ -395,4 +363,4 @@
}
#endif
-#ifdef CONFIG_IOMMU_SVA
+#endif /* _LINUX_SCHED_MM_H */
--- b/kernel/exit.c
+++ b/kernel/exit.c
@@ -472,7 +472,6 @@
BUG_ON(mm != current->active_mm);
/* more a memory barrier than a real lock */
task_lock(current);
- current->user_dumpable = (get_dumpable(mm) == SUID_DUMP_USER);
current->mm = NULL;
mmap_read_unlock(mm);
enter_lazy_tlb(mm, current);
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -136,2 +133,3 @@
#define MIDR_CORTEX_X1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X1)
+#define MIDR_CORTEX_A510 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A510)
#define MIDR_CORTEX_A710 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A710)
================================================================================
* ONLY IN PATCH1 - files not modified by patch2 *
================================================================================
--- b/arch/arm64/kernel/proton-pack.c
+++ /dev/null
@@ -1,1157 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * Handle detection, reporting and mitigation of Spectre v1, v2, v3a and v4, as
- * detailed at:
- *
- * https://developer.arm.com/support/arm-security-updates/speculative-processor-vulnerability
- *
- * This code was originally written hastily under an awful lot of stress and so
- * aspects of it are somewhat hacky. Unfortunately, changing anything in here
- * instantly makes me feel ill. Thanks, Jann. Thann.
- *
- * Copyright (C) 2018 ARM Ltd, All Rights Reserved.
- * Copyright (C) 2020 Google LLC
- *
- * "If there's something strange in your neighbourhood, who you gonna call?"
- *
- * Authors: Will Deacon <will@kernel.org> and Marc Zyngier <maz@kernel.org>
- */
-
-#include <linux/arm-smccc.h>
-#include <linux/bpf.h>
-#include <linux/cpu.h>
-#include <linux/device.h>
-#include <linux/nospec.h>
-#include <linux/prctl.h>
-#include <linux/sched/task_stack.h>
-
-#include <asm/debug-monitors.h>
-#include <asm/insn.h>
-#include <asm/spectre.h>
-#include <asm/traps.h>
-#include <asm/vectors.h>
-#include <asm/virt.h>
-
-/*
- * We try to ensure that the mitigation state can never change as the result of
- * onlining a late CPU.
- */
-static void update_mitigation_state(enum mitigation_state *oldp,
- enum mitigation_state new)
-{
- enum mitigation_state state;
-
- do {
- state = READ_ONCE(*oldp);
- if (new <= state)
- break;
-
- /* Userspace almost certainly can't deal with this. */
- if (WARN_ON(system_capabilities_finalized()))
- break;
- } while (cmpxchg_relaxed(oldp, state, new) != state);
-}
-
-/*
- * Spectre v1.
- *
- * The kernel can't protect userspace for this one: it's each person for
- * themselves. Advertise what we're doing and be done with it.
- */
-ssize_t cpu_show_spectre_v1(struct device *dev, struct device_attribute *attr,
- char *buf)
-{
- return sprintf(buf, "Mitigation: __user pointer sanitization\n");
-}
-
-/*
- * Spectre v2.
- *
- * This one sucks. A CPU is either:
- *
- * - Mitigated in hardware and advertised by ID_AA64PFR0_EL1.CSV2.
- * - Mitigated in hardware and listed in our "safe list".
- * - Mitigated in software by firmware.
- * - Mitigated in software by a CPU-specific dance in the kernel and a
- * firmware call at EL2.
- * - Vulnerable.
- *
- * It's not unlikely for different CPUs in a big.LITTLE system to fall into
- * different camps.
- */
-static enum mitigation_state spectre_v2_state;
-
-static bool __read_mostly __nospectre_v2;
-static int __init parse_spectre_v2_param(char *str)
-{
- __nospectre_v2 = true;
- return 0;
-}
-early_param("nospectre_v2", parse_spectre_v2_param);
-
-static bool spectre_v2_mitigations_off(void)
-{
- bool ret = __nospectre_v2 || cpu_mitigations_off();
-
- if (ret)
- pr_info_once("spectre-v2 mitigation disabled by command line option\n");
-
- return ret;
-}
-
-static const char *get_bhb_affected_string(enum mitigation_state bhb_state)
-{
- switch (bhb_state) {
- case SPECTRE_UNAFFECTED:
- return "";
- default:
- case SPECTRE_VULNERABLE:
- return ", but not BHB";
- case SPECTRE_MITIGATED:
- return ", BHB";
- }
-}
-
-static bool _unprivileged_ebpf_enabled(void)
-{
-#ifdef CONFIG_BPF_SYSCALL
- return !sysctl_unprivileged_bpf_disabled;
-#else
- return false;
-#endif
-}
-
-ssize_t cpu_show_spectre_v2(struct device *dev, struct device_attribute *attr,
- char *buf)
-{
- enum mitigation_state bhb_state = arm64_get_spectre_bhb_state();
- const char *bhb_str = get_bhb_affected_string(bhb_state);
- const char *v2_str = "Branch predictor hardening";
-
- switch (spectre_v2_state) {
- case SPECTRE_UNAFFECTED:
- if (bhb_state == SPECTRE_UNAFFECTED)
- return sprintf(buf, "Not affected\n");
-
- /*
- * Platforms affected by Spectre-BHB can't report
- * "Not affected" for Spectre-v2.
- */
- v2_str = "CSV2";
- fallthrough;
- case SPECTRE_MITIGATED:
- if (bhb_state == SPECTRE_MITIGATED && _unprivileged_ebpf_enabled())
- return sprintf(buf, "Vulnerable: Unprivileged eBPF enabled\n");
-
- return sprintf(buf, "Mitigation: %s%s\n", v2_str, bhb_str);
- case SPECTRE_VULNERABLE:
- fallthrough;
- default:
- return sprintf(buf, "Vulnerable\n");
- }
-}
-
-static enum mitigation_state spectre_v2_get_cpu_hw_mitigation_state(void)
-{
- u64 pfr0;
- static const struct midr_range spectre_v2_safe_list[] = {
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A35),
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A53),
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A55),
- MIDR_ALL_VERSIONS(MIDR_BRAHMA_B53),
- MIDR_ALL_VERSIONS(MIDR_HISI_TSV110),
- MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_2XX_SILVER),
- MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_3XX_SILVER),
- MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_4XX_SILVER),
- { /* sentinel */ }
- };
-
- /* If the CPU has CSV2 set, we're safe */
- pfr0 = read_cpuid(ID_AA64PFR0_EL1);
- if (cpuid_feature_extract_unsigned_field(pfr0, ID_AA64PFR0_CSV2_SHIFT))
- return SPECTRE_UNAFFECTED;
-
- /* Alternatively, we have a list of unaffected CPUs */
- if (is_midr_in_range_list(read_cpuid_id(), spectre_v2_safe_list))
- return SPECTRE_UNAFFECTED;
-
- return SPECTRE_VULNERABLE;
-}
-
-static enum mitigation_state spectre_v2_get_cpu_fw_mitigation_state(void)
-{
- int ret;
- struct arm_smccc_res res;
-
- arm_smccc_1_1_invoke(ARM_SMCCC_ARCH_FEATURES_FUNC_ID,
- ARM_SMCCC_ARCH_WORKAROUND_1, &res);
-
- ret = res.a0;
- switch (ret) {
- case SMCCC_RET_SUCCESS:
- return SPECTRE_MITIGATED;
- case SMCCC_ARCH_WORKAROUND_RET_UNAFFECTED:
- return SPECTRE_UNAFFECTED;
- default:
- fallthrough;
- case SMCCC_RET_NOT_SUPPORTED:
- return SPECTRE_VULNERABLE;
- }
-}
-
-bool has_spectre_v2(const struct arm64_cpu_capabilities *entry, int scope)
-{
- WARN_ON(scope != SCOPE_LOCAL_CPU || preemptible());
-
- if (spectre_v2_get_cpu_hw_mitigation_state() == SPECTRE_UNAFFECTED)
- return false;
-
- if (spectre_v2_get_cpu_fw_mitigation_state() == SPECTRE_UNAFFECTED)
- return false;
-
- return true;
-}
-
-enum mitigation_state arm64_get_spectre_v2_state(void)
-{
- return spectre_v2_state;
-}
-
-DEFINE_PER_CPU_READ_MOSTLY(struct bp_hardening_data, bp_hardening_data);
-
-static void install_bp_hardening_cb(bp_hardening_cb_t fn)
-{
- __this_cpu_write(bp_hardening_data.fn, fn);
-
- /*
- * Vinz Clortho takes the hyp_vecs start/end "keys" at
- * the door when we're a guest. Skip the hyp-vectors work.
- */
- if (!is_hyp_mode_available())
- return;
-
- __this_cpu_write(bp_hardening_data.slot, HYP_VECTOR_SPECTRE_DIRECT);
-}
-
-/* Called during entry so must be noinstr */
-static noinstr void call_smc_arch_workaround_1(void)
-{
- arm_smccc_1_1_smc(ARM_SMCCC_ARCH_WORKAROUND_1, NULL);
-}
-
-/* Called during entry so must be noinstr */
-static noinstr void call_hvc_arch_workaround_1(void)
-{
- arm_smccc_1_1_hvc(ARM_SMCCC_ARCH_WORKAROUND_1, NULL);
-}
-
-/* Called during entry so must be noinstr */
-static noinstr void qcom_link_stack_sanitisation(void)
-{
- u64 tmp;
-
- asm volatile("mov %0, x30 \n"
- ".rept 16 \n"
- "bl . + 4 \n"
- ".endr \n"
- "mov x30, %0 \n"
- : "=&r" (tmp));
-}
-
-static bp_hardening_cb_t spectre_v2_get_sw_mitigation_cb(void)
-{
- u32 midr = read_cpuid_id();
- if (((midr & MIDR_CPU_MODEL_MASK) != MIDR_QCOM_FALKOR) &&
- ((midr & MIDR_CPU_MODEL_MASK) != MIDR_QCOM_FALKOR_V1))
- return NULL;
-
- return qcom_link_stack_sanitisation;
-}
-
-static enum mitigation_state spectre_v2_enable_fw_mitigation(void)
-{
- bp_hardening_cb_t cb;
- enum mitigation_state state;
-
- state = spectre_v2_get_cpu_fw_mitigation_state();
- if (state != SPECTRE_MITIGATED)
- return state;
-
- if (spectre_v2_mitigations_off())
- return SPECTRE_VULNERABLE;
-
- switch (arm_smccc_1_1_get_conduit()) {
- case SMCCC_CONDUIT_HVC:
- cb = call_hvc_arch_workaround_1;
- break;
-
- case SMCCC_CONDUIT_SMC:
- cb = call_smc_arch_workaround_1;
- break;
-
- default:
- return SPECTRE_VULNERABLE;
- }
-
- /*
- * Prefer a CPU-specific workaround if it exists. Note that we
- * still rely on firmware for the mitigation at EL2.
- */
- cb = spectre_v2_get_sw_mitigation_cb() ?: cb;
- install_bp_hardening_cb(cb);
- return SPECTRE_MITIGATED;
-}
-
-void spectre_v2_enable_mitigation(const struct arm64_cpu_capabilities *__unused)
-{
- enum mitigation_state state;
-
- WARN_ON(preemptible());
-
- state = spectre_v2_get_cpu_hw_mitigation_state();
- if (state == SPECTRE_VULNERABLE)
- state = spectre_v2_enable_fw_mitigation();
-
- update_mitigation_state(&spectre_v2_state, state);
-}
-
-/*
- * Spectre-v3a.
- *
- * Phew, there's not an awful lot to do here! We just instruct EL2 to use
- * an indirect trampoline for the hyp vectors so that guests can't read
- * VBAR_EL2 to defeat randomisation of the hypervisor VA layout.
- */
-bool has_spectre_v3a(const struct arm64_cpu_capabilities *entry, int scope)
-{
- static const struct midr_range spectre_v3a_unsafe_list[] = {
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A57),
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
- {},
- };
-
- WARN_ON(scope != SCOPE_LOCAL_CPU || preemptible());
- return is_midr_in_range_list(read_cpuid_id(), spectre_v3a_unsafe_list);
-}
-
-void spectre_v3a_enable_mitigation(const struct arm64_cpu_capabilities *__unused)
-{
- struct bp_hardening_data *data = this_cpu_ptr(&bp_hardening_data);
-
- if (this_cpu_has_cap(ARM64_SPECTRE_V3A))
- data->slot += HYP_VECTOR_INDIRECT;
-}
-
-/*
- * Spectre v4.
- *
- * If you thought Spectre v2 was nasty, wait until you see this mess. A CPU is
- * either:
- *
- * - Mitigated in hardware and listed in our "safe list".
- * - Mitigated in hardware via PSTATE.SSBS.
- * - Mitigated in software by firmware (sometimes referred to as SSBD).
- *
- * Wait, that doesn't sound so bad, does it? Keep reading...
- *
- * A major source of headaches is that the software mitigation is enabled both
- * on a per-task basis, but can also be forced on for the kernel, necessitating
- * both context-switch *and* entry/exit hooks. To make it even worse, some CPUs
- * allow EL0 to toggle SSBS directly, which can end up with the prctl() state
- * being stale when re-entering the kernel. The usual big.LITTLE caveats apply,
- * so you can have systems that have both firmware and SSBS mitigations. This
- * means we actually have to reject late onlining of CPUs with mitigations if
- * all of the currently onlined CPUs are safelisted, as the mitigation tends to
- * be opt-in for userspace. Yes, really, the cure is worse than the disease.
- *
- * The only good part is that if the firmware mitigation is present, then it is
- * present for all CPUs, meaning we don't have to worry about late onlining of a
- * vulnerable CPU if one of the boot CPUs is using the firmware mitigation.
- *
- * Give me a VAX-11/780 any day of the week...
- */
-static enum mitigation_state spectre_v4_state;
-
-/* This is the per-cpu state tracking whether we need to talk to firmware */
-DEFINE_PER_CPU_READ_MOSTLY(u64, arm64_ssbd_callback_required);
-
-enum spectre_v4_policy {
- SPECTRE_V4_POLICY_MITIGATION_DYNAMIC,
- SPECTRE_V4_POLICY_MITIGATION_ENABLED,
- SPECTRE_V4_POLICY_MITIGATION_DISABLED,
-};
-
-static enum spectre_v4_policy __read_mostly __spectre_v4_policy;
-
-static const struct spectre_v4_param {
- const char *str;
- enum spectre_v4_policy policy;
-} spectre_v4_params[] = {
- { "force-on", SPECTRE_V4_POLICY_MITIGATION_ENABLED, },
- { "force-off", SPECTRE_V4_POLICY_MITIGATION_DISABLED, },
- { "kernel", SPECTRE_V4_POLICY_MITIGATION_DYNAMIC, },
-};
-static int __init parse_spectre_v4_param(char *str)
-{
- int i;
-
- if (!str || !str[0])
- return -EINVAL;
-
- for (i = 0; i < ARRAY_SIZE(spectre_v4_params); i++) {
- const struct spectre_v4_param *param = &spectre_v4_params[i];
-
- if (strncmp(str, param->str, strlen(param->str)))
- continue;
-
- __spectre_v4_policy = param->policy;
- return 0;
- }
-
- return -EINVAL;
-}
-early_param("ssbd", parse_spectre_v4_param);
-
-/*
- * Because this was all written in a rush by people working in different silos,
- * we've ended up with multiple command line options to control the same thing.
- * Wrap these up in some helpers, which prefer disabling the mitigation if faced
- * with contradictory parameters. The mitigation is always either "off",
- * "dynamic" or "on".
- */
-static bool spectre_v4_mitigations_off(void)
-{
- bool ret = cpu_mitigations_off() ||
- __spectre_v4_policy == SPECTRE_V4_POLICY_MITIGATION_DISABLED;
-
- if (ret)
- pr_info_once("spectre-v4 mitigation disabled by command-line option\n");
-
- return ret;
-}
-
-/* Do we need to toggle the mitigation state on entry to/exit from the kernel? */
-static bool spectre_v4_mitigations_dynamic(void)
-{
- return !spectre_v4_mitigations_off() &&
- __spectre_v4_policy == SPECTRE_V4_POLICY_MITIGATION_DYNAMIC;
-}
-
-static bool spectre_v4_mitigations_on(void)
-{
- return !spectre_v4_mitigations_off() &&
- __spectre_v4_policy == SPECTRE_V4_POLICY_MITIGATION_ENABLED;
-}
-
-ssize_t cpu_show_spec_store_bypass(struct device *dev,
- struct device_attribute *attr, char *buf)
-{
- switch (spectre_v4_state) {
- case SPECTRE_UNAFFECTED:
- return sprintf(buf, "Not affected\n");
- case SPECTRE_MITIGATED:
- return sprintf(buf, "Mitigation: Speculative Store Bypass disabled via prctl\n");
- case SPECTRE_VULNERABLE:
- fallthrough;
- default:
- return sprintf(buf, "Vulnerable\n");
- }
-}
-
-enum mitigation_state arm64_get_spectre_v4_state(void)
-{
- return spectre_v4_state;
-}
-
-static enum mitigation_state spectre_v4_get_cpu_hw_mitigation_state(void)
-{
- static const struct midr_range spectre_v4_safe_list[] = {
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A35),
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A53),
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A55),
- MIDR_ALL_VERSIONS(MIDR_BRAHMA_B53),
- MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_3XX_SILVER),
- MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_4XX_SILVER),
- { /* sentinel */ },
- };
-
- if (is_midr_in_range_list(read_cpuid_id(), spectre_v4_safe_list))
- return SPECTRE_UNAFFECTED;
-
- /* CPU features are detected first */
- if (this_cpu_has_cap(ARM64_SSBS))
- return SPECTRE_MITIGATED;
-
- return SPECTRE_VULNERABLE;
-}
-
-static enum mitigation_state spectre_v4_get_cpu_fw_mitigation_state(void)
-{
- int ret;
- struct arm_smccc_res res;
-
- arm_smccc_1_1_invoke(ARM_SMCCC_ARCH_FEATURES_FUNC_ID,
- ARM_SMCCC_ARCH_WORKAROUND_2, &res);
-
- ret = res.a0;
- switch (ret) {
- case SMCCC_RET_SUCCESS:
- return SPECTRE_MITIGATED;
- case SMCCC_ARCH_WORKAROUND_RET_UNAFFECTED:
- fallthrough;
- case SMCCC_RET_NOT_REQUIRED:
- return SPECTRE_UNAFFECTED;
- default:
- fallthrough;
- case SMCCC_RET_NOT_SUPPORTED:
- return SPECTRE_VULNERABLE;
- }
-}
-
-bool has_spectre_v4(const struct arm64_cpu_capabilities *cap, int scope)
-{
- enum mitigation_state state;
-
- WARN_ON(scope != SCOPE_LOCAL_CPU || preemptible());
-
- state = spectre_v4_get_cpu_hw_mitigation_state();
- if (state == SPECTRE_VULNERABLE)
- state = spectre_v4_get_cpu_fw_mitigation_state();
-
- return state != SPECTRE_UNAFFECTED;
-}
-
-static int ssbs_emulation_handler(struct pt_regs *regs, u32 instr)
-{
- if (user_mode(regs))
- return 1;
-
- if (instr & BIT(PSTATE_Imm_shift))
- regs->pstate |= PSR_SSBS_BIT;
- else
- regs->pstate &= ~PSR_SSBS_BIT;
-
- arm64_skip_faulting_instruction(regs, 4);
- return 0;
-}
-
-static struct undef_hook ssbs_emulation_hook = {
- .instr_mask = ~(1U << PSTATE_Imm_shift),
- .instr_val = 0xd500401f | PSTATE_SSBS,
- .fn = ssbs_emulation_handler,
-};
-
-static enum mitigation_state spectre_v4_enable_hw_mitigation(void)
-{
- static bool undef_hook_registered = false;
- static DEFINE_RAW_SPINLOCK(hook_lock);
- enum mitigation_state state;
-
- /*
- * If the system is mitigated but this CPU doesn't have SSBS, then
- * we must be on the safelist and there's nothing more to do.
- */
- state = spectre_v4_get_cpu_hw_mitigation_state();
- if (state != SPECTRE_MITIGATED || !this_cpu_has_cap(ARM64_SSBS))
- return state;
-
- raw_spin_lock(&hook_lock);
- if (!undef_hook_registered) {
- register_undef_hook(&ssbs_emulation_hook);
- undef_hook_registered = true;
- }
- raw_spin_unlock(&hook_lock);
-
- if (spectre_v4_mitigations_off()) {
- sysreg_clear_set(sctlr_el1, 0, SCTLR_ELx_DSSBS);
- set_pstate_ssbs(1);
- return SPECTRE_VULNERABLE;
- }
-
- /* SCTLR_EL1.DSSBS was initialised to 0 during boot */
- set_pstate_ssbs(0);
- return SPECTRE_MITIGATED;
-}
-
-/*
- * Patch a branch over the Spectre-v4 mitigation code with a NOP so that
- * we fallthrough and check whether firmware needs to be called on this CPU.
- */
-void __init spectre_v4_patch_fw_mitigation_enable(struct alt_instr *alt,
- __le32 *origptr,
- __le32 *updptr, int nr_inst)
-{
- BUG_ON(nr_inst != 1); /* Branch -> NOP */
-
- if (spectre_v4_mitigations_off())
- return;
-
- if (cpus_have_final_cap(ARM64_SSBS))
- return;
-
- if (spectre_v4_mitigations_dynamic())
- *updptr = cpu_to_le32(aarch64_insn_gen_nop());
-}
-
-/*
- * Patch a NOP in the Spectre-v4 mitigation code with an SMC/HVC instruction
- * to call into firmware to adjust the mitigation state.
- */
-void __init smccc_patch_fw_mitigation_conduit(struct alt_instr *alt,
- __le32 *origptr,
- __le32 *updptr, int nr_inst)
-{
- u32 insn;
-
- BUG_ON(nr_inst != 1); /* NOP -> HVC/SMC */
-
- switch (arm_smccc_1_1_get_conduit()) {
- case SMCCC_CONDUIT_HVC:
- insn = aarch64_insn_get_hvc_value();
- break;
- case SMCCC_CONDUIT_SMC:
- insn = aarch64_insn_get_smc_value();
- break;
- default:
- return;
- }
-
- *updptr = cpu_to_le32(insn);
-}
-
-static enum mitigation_state spectre_v4_enable_fw_mitigation(void)
-{
- enum mitigation_state state;
-
- state = spectre_v4_get_cpu_fw_mitigation_state();
- if (state != SPECTRE_MITIGATED)
- return state;
-
- if (spectre_v4_mitigations_off()) {
- arm_smccc_1_1_invoke(ARM_SMCCC_ARCH_WORKAROUND_2, false, NULL);
- return SPECTRE_VULNERABLE;
- }
-
- arm_smccc_1_1_invoke(ARM_SMCCC_ARCH_WORKAROUND_2, true, NULL);
-
- if (spectre_v4_mitigations_dynamic())
- __this_cpu_write(arm64_ssbd_callback_required, 1);
-
- return SPECTRE_MITIGATED;
-}
-
-void spectre_v4_enable_mitigation(const struct arm64_cpu_capabilities *__unused)
-{
- enum mitigation_state state;
-
- WARN_ON(preemptible());
-
- state = spectre_v4_enable_hw_mitigation();
- if (state == SPECTRE_VULNERABLE)
- state = spectre_v4_enable_fw_mitigation();
-
- update_mitigation_state(&spectre_v4_state, state);
-}
-
-static void __update_pstate_ssbs(struct pt_regs *regs, bool state)
-{
- u64 bit = compat_user_mode(regs) ? PSR_AA32_SSBS_BIT : PSR_SSBS_BIT;
-
- if (state)
- regs->pstate |= bit;
- else
- regs->pstate &= ~bit;
-}
-
-void spectre_v4_enable_task_mitigation(struct task_struct *tsk)
-{
- struct pt_regs *regs = task_pt_regs(tsk);
- bool ssbs = false, kthread = tsk->flags & PF_KTHREAD;
-
- if (spectre_v4_mitigations_off())
- ssbs = true;
- else if (spectre_v4_mitigations_dynamic() && !kthread)
- ssbs = !test_tsk_thread_flag(tsk, TIF_SSBD);
-
- __update_pstate_ssbs(regs, ssbs);
-}
-
-/*
- * The Spectre-v4 mitigation can be controlled via a prctl() from userspace.
- * This is interesting because the "speculation disabled" behaviour can be
- * configured so that it is preserved across exec(), which means that the
- * prctl() may be necessary even when PSTATE.SSBS can be toggled directly
- * from userspace.
- */
-static void ssbd_prctl_enable_mitigation(struct task_struct *task)
-{
- task_clear_spec_ssb_noexec(task);
- task_set_spec_ssb_disable(task);
- set_tsk_thread_flag(task, TIF_SSBD);
-}
-
-static void ssbd_prctl_disable_mitigation(struct task_struct *task)
-{
- task_clear_spec_ssb_noexec(task);
- task_clear_spec_ssb_disable(task);
- clear_tsk_thread_flag(task, TIF_SSBD);
-}
-
-static int ssbd_prctl_set(struct task_struct *task, unsigned long ctrl)
-{
- switch (ctrl) {
- case PR_SPEC_ENABLE:
- /* Enable speculation: disable mitigation */
- /*
- * Force disabled speculation prevents it from being
- * re-enabled.
- */
- if (task_spec_ssb_force_disable(task))
- return -EPERM;
-
- /*
- * If the mitigation is forced on, then speculation is forced
- * off and we again prevent it from being re-enabled.
- */
- if (spectre_v4_mitigations_on())
- return -EPERM;
-
- ssbd_prctl_disable_mitigation(task);
- break;
- case PR_SPEC_FORCE_DISABLE:
- /* Force disable speculation: force enable mitigation */
- /*
- * If the mitigation is forced off, then speculation is forced
- * on and we prevent it from being disabled.
- */
- if (spectre_v4_mitigations_off())
- return -EPERM;
-
- task_set_spec_ssb_force_disable(task);
- fallthrough;
- case PR_SPEC_DISABLE:
- /* Disable speculation: enable mitigation */
- /* Same as PR_SPEC_FORCE_DISABLE */
- if (spectre_v4_mitigations_off())
- return -EPERM;
-
- ssbd_prctl_enable_mitigation(task);
- break;
- case PR_SPEC_DISABLE_NOEXEC:
- /* Disable speculation until execve(): enable mitigation */
- /*
- * If the mitigation state is forced one way or the other, then
- * we must fail now before we try to toggle it on execve().
- */
- if (task_spec_ssb_force_disable(task) ||
- spectre_v4_mitigations_off() ||
- spectre_v4_mitigations_on()) {
- return -EPERM;
- }
-
- ssbd_prctl_enable_mitigation(task);
- task_set_spec_ssb_noexec(task);
- break;
- default:
- return -ERANGE;
- }
-
- spectre_v4_enable_task_mitigation(task);
- return 0;
-}
-
-int arch_prctl_spec_ctrl_set(struct task_struct *task, unsigned long which,
- unsigned long ctrl)
-{
- switch (which) {
- case PR_SPEC_STORE_BYPASS:
- return ssbd_prctl_set(task, ctrl);
- default:
- return -ENODEV;
- }
-}
-
-static int ssbd_prctl_get(struct task_struct *task)
-{
- switch (spectre_v4_state) {
- case SPECTRE_UNAFFECTED:
- return PR_SPEC_NOT_AFFECTED;
- case SPECTRE_MITIGATED:
- if (spectre_v4_mitigations_on())
- return PR_SPEC_NOT_AFFECTED;
-
- if (spectre_v4_mitigations_dynamic())
- break;
-
- /* Mitigations are disabled, so we're vulnerable. */
- fallthrough;
- case SPECTRE_VULNERABLE:
- fallthrough;
- default:
- return PR_SPEC_ENABLE;
- }
-
- /* Check the mitigation state for this task */
- if (task_spec_ssb_force_disable(task))
- return PR_SPEC_PRCTL | PR_SPEC_FORCE_DISABLE;
-
- if (task_spec_ssb_noexec(task))
- return PR_SPEC_PRCTL | PR_SPEC_DISABLE_NOEXEC;
-
- if (task_spec_ssb_disable(task))
- return PR_SPEC_PRCTL | PR_SPEC_DISABLE;
-
- return PR_SPEC_PRCTL | PR_SPEC_ENABLE;
-}
-
-int arch_prctl_spec_ctrl_get(struct task_struct *task, unsigned long which)
-{
- switch (which) {
- case PR_SPEC_STORE_BYPASS:
- return ssbd_prctl_get(task);
- default:
- return -ENODEV;
- }
-}
-
-/*
- * Spectre BHB.
- *
- * A CPU is either:
- * - Mitigated by a branchy loop a CPU specific number of times, and listed
- * in our "loop mitigated list".
- * - Mitigated in software by the firmware Spectre v2 call.
- * - Has the ClearBHB instruction to perform the mitigation.
- * - Has the 'Exception Clears Branch History Buffer' (ECBHB) feature, so no
- * software mitigation in the vectors is needed.
- * - Has CSV2.3, so is unaffected.
- */
-static enum mitigation_state spectre_bhb_state;
-
-enum mitigation_state arm64_get_spectre_bhb_state(void)
-{
- return spectre_bhb_state;
-}
-
-enum bhb_mitigation_bits {
- BHB_LOOP,
- BHB_FW,
- BHB_HW,
- BHB_INSN,
-};
-static unsigned long system_bhb_mitigations;
-
-/*
- * This must be called with SCOPE_LOCAL_CPU for each type of CPU, before any
- * SCOPE_SYSTEM call will give the right answer.
- */
-u8 spectre_bhb_loop_affected(int scope)
-{
- u8 k = 0;
- static u8 max_bhb_k;
-
- if (scope == SCOPE_LOCAL_CPU) {
- static const struct midr_range spectre_bhb_k32_list[] = {
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A78),
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A78AE),
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A78C),
- MIDR_ALL_VERSIONS(MIDR_CORTEX_X1),
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A710),
- MIDR_ALL_VERSIONS(MIDR_CORTEX_X2),
- MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N2),
- MIDR_ALL_VERSIONS(MIDR_NEOVERSE_V1),
- {},
- };
- static const struct midr_range spectre_bhb_k24_list[] = {
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A76),
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A77),
- MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N1),
- {},
- };
- static const struct midr_range spectre_bhb_k8_list[] = {
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A57),
- {},
- };
-
- if (is_midr_in_range_list(read_cpuid_id(), spectre_bhb_k32_list))
- k = 32;
- else if (is_midr_in_range_list(read_cpuid_id(), spectre_bhb_k24_list))
- k = 24;
- else if (is_midr_in_range_list(read_cpuid_id(), spectre_bhb_k8_list))
- k = 8;
-
- max_bhb_k = max(max_bhb_k, k);
- } else {
- k = max_bhb_k;
- }
-
- return k;
-}
-
-static enum mitigation_state spectre_bhb_get_cpu_fw_mitigation_state(void)
-{
- int ret;
- struct arm_smccc_res res;
-
- arm_smccc_1_1_invoke(ARM_SMCCC_ARCH_FEATURES_FUNC_ID,
- ARM_SMCCC_ARCH_WORKAROUND_3, &res);
-
- ret = res.a0;
- switch (ret) {
- case SMCCC_RET_SUCCESS:
- return SPECTRE_MITIGATED;
- case SMCCC_ARCH_WORKAROUND_RET_UNAFFECTED:
- return SPECTRE_UNAFFECTED;
- default:
- fallthrough;
- case SMCCC_RET_NOT_SUPPORTED:
- return SPECTRE_VULNERABLE;
- }
-}
-
-static bool is_spectre_bhb_fw_affected(int scope)
-{
- static bool system_affected;
- enum mitigation_state fw_state;
- bool has_smccc = arm_smccc_1_1_get_conduit() != SMCCC_CONDUIT_NONE;
- static const struct midr_range spectre_bhb_firmware_mitigated_list[] = {
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
- {},
- };
- bool cpu_in_list = is_midr_in_range_list(read_cpuid_id(),
- spectre_bhb_firmware_mitigated_list);
-
- if (scope != SCOPE_LOCAL_CPU)
- return system_affected;
-
- fw_state = spectre_bhb_get_cpu_fw_mitigation_state();
- if (cpu_in_list || (has_smccc && fw_state == SPECTRE_MITIGATED)) {
- system_affected = true;
- return true;
- }
-
- return false;
-}
-
-static bool supports_ecbhb(int scope)
-{
- u64 mmfr1;
-
- if (scope == SCOPE_LOCAL_CPU)
- mmfr1 = read_sysreg_s(SYS_ID_AA64MMFR1_EL1);
- else
- mmfr1 = read_sanitised_ftr_reg(SYS_ID_AA64MMFR1_EL1);
-
- return cpuid_feature_extract_unsigned_field(mmfr1,
- ID_AA64MMFR1_ECBHB_SHIFT);
-}
-
-bool is_spectre_bhb_affected(const struct arm64_cpu_capabilities *entry,
- int scope)
-{
- WARN_ON(scope != SCOPE_LOCAL_CPU || preemptible());
-
- if (supports_csv2p3(scope))
- return false;
-
- if (supports_clearbhb(scope))
- return true;
-
- if (spectre_bhb_loop_affected(scope))
- return true;
-
- if (is_spectre_bhb_fw_affected(scope))
- return true;
-
- return false;
-}
-
-static void this_cpu_set_vectors(enum arm64_bp_harden_el1_vectors slot)
-{
- const char *v = arm64_get_bp_hardening_vector(slot);
-
- if (slot < 0)
- return;
-
- __this_cpu_write(this_cpu_vector, v);
-
- /*
- * When KPTI is in use, the vectors are switched when exiting to
- * user-space.
- */
- if (arm64_kernel_unmapped_at_el0())
- return;
-
- write_sysreg(v, vbar_el1);
- isb();
-}
-
-void spectre_bhb_enable_mitigation(const struct arm64_cpu_capabilities *entry)
-{
- bp_hardening_cb_t cpu_cb;
- enum mitigation_state fw_state, state = SPECTRE_VULNERABLE;
- struct bp_hardening_data *data = this_cpu_ptr(&bp_hardening_data);
-
- if (!is_spectre_bhb_affected(entry, SCOPE_LOCAL_CPU))
- return;
-
- if (arm64_get_spectre_v2_state() == SPECTRE_VULNERABLE) {
- /* No point mitigating Spectre-BHB alone. */
- } else if (!IS_ENABLED(CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY)) {
- pr_info_once("spectre-bhb mitigation disabled by compile time option\n");
- } else if (cpu_mitigations_off()) {
- pr_info_once("spectre-bhb mitigation disabled by command line option\n");
- } else if (supports_ecbhb(SCOPE_LOCAL_CPU)) {
- state = SPECTRE_MITIGATED;
- set_bit(BHB_HW, &system_bhb_mitigations);
- } else if (supports_clearbhb(SCOPE_LOCAL_CPU)) {
- /*
- * Ensure KVM uses the indirect vector which will have ClearBHB
- * added.
- */
- if (!data->slot)
- data->slot = HYP_VECTOR_INDIRECT;
-
- this_cpu_set_vectors(EL1_VECTOR_BHB_CLEAR_INSN);
- state = SPECTRE_MITIGATED;
- set_bit(BHB_INSN, &system_bhb_mitigations);
- } else if (spectre_bhb_loop_affected(SCOPE_LOCAL_CPU)) {
- /*
- * Ensure KVM uses the indirect vector which will have the
- * branchy-loop added. A57/A72-r0 will already have selected
- * the spectre-indirect vector, which is sufficient for BHB
- * too.
- */
- if (!data->slot)
- data->slot = HYP_VECTOR_INDIRECT;
-
- this_cpu_set_vectors(EL1_VECTOR_BHB_LOOP);
- state = SPECTRE_MITIGATED;
- set_bit(BHB_LOOP, &system_bhb_mitigations);
- } else if (is_spectre_bhb_fw_affected(SCOPE_LOCAL_CPU)) {
- fw_state = spectre_bhb_get_cpu_fw_mitigation_state();
- if (fw_state == SPECTRE_MITIGATED) {
- /*
- * Ensure KVM uses one of the spectre bp_hardening
- * vectors. The indirect vector doesn't include the EL3
- * call, so needs upgrading to
- * HYP_VECTOR_SPECTRE_INDIRECT.
- */
- if (!data->slot || data->slot == HYP_VECTOR_INDIRECT)
- data->slot += 1;
-
- this_cpu_set_vectors(EL1_VECTOR_BHB_FW);
-
- /*
- * The WA3 call in the vectors supersedes the WA1 call
- * made during context-switch. Uninstall any firmware
- * bp_hardening callback.
- */
- cpu_cb = spectre_v2_get_sw_mitigation_cb();
- if (__this_cpu_read(bp_hardening_data.fn) != cpu_cb)
- __this_cpu_write(bp_hardening_data.fn, NULL);
-
- state = SPECTRE_MITIGATED;
- set_bit(BHB_FW, &system_bhb_mitigations);
- }
- }
-
- update_mitigation_state(&spectre_bhb_state, state);
-}
-
-/* Patched to NOP when enabled */
-void noinstr spectre_bhb_patch_loop_mitigation_enable(struct alt_instr *alt,
- __le32 *origptr,
- __le32 *updptr, int nr_inst)
-{
- BUG_ON(nr_inst != 1);
-
- if (test_bit(BHB_LOOP, &system_bhb_mitigations))
- *updptr++ = cpu_to_le32(aarch64_insn_gen_nop());
-}
-
-/* Patched to NOP when enabled */
-void noinstr spectre_bhb_patch_fw_mitigation_enabled(struct alt_instr *alt,
- __le32 *origptr,
- __le32 *updptr, int nr_inst)
-{
- BUG_ON(nr_inst != 1);
-
- if (test_bit(BHB_FW, &system_bhb_mitigations))
- *updptr++ = cpu_to_le32(aarch64_insn_gen_nop());
-}
-
-/* Patched to correct the immediate */
-void noinstr spectre_bhb_patch_loop_iter(struct alt_instr *alt,
- __le32 *origptr, __le32 *updptr, int nr_inst)
-{
- u8 rd;
- u32 insn;
- u16 loop_count = spectre_bhb_loop_affected(SCOPE_SYSTEM);
-
- BUG_ON(nr_inst != 1); /* MOV -> MOV */
-
- if (!IS_ENABLED(CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY))
- return;
-
- insn = le32_to_cpu(*origptr);
- rd = aarch64_insn_decode_register(AARCH64_INSN_REGTYPE_RD, insn);
- insn = aarch64_insn_gen_movewide(rd, loop_count, 0,
- AARCH64_INSN_VARIANT_64BIT,
- AARCH64_INSN_MOVEWIDE_ZERO);
- *updptr++ = cpu_to_le32(insn);
-}
-
-/* Patched to mov WA3 when supported */
-void noinstr spectre_bhb_patch_wa3(struct alt_instr *alt,
- __le32 *origptr, __le32 *updptr, int nr_inst)
-{
- u8 rd;
- u32 insn;
-
- BUG_ON(nr_inst != 1); /* MOV -> MOV */
-
- if (!IS_ENABLED(CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY) ||
- !test_bit(BHB_FW, &system_bhb_mitigations))
- return;
-
- insn = le32_to_cpu(*origptr);
- rd = aarch64_insn_decode_register(AARCH64_INSN_REGTYPE_RD, insn);
-
- insn = aarch64_insn_gen_logical_immediate(AARCH64_INSN_LOGIC_ORR,
- AARCH64_INSN_VARIANT_32BIT,
- AARCH64_INSN_REG_ZR, rd,
- ARM_SMCCC_ARCH_WORKAROUND_3);
- if (WARN_ON_ONCE(insn == AARCH64_BREAK_FAULT))
- return;
-
- *updptr++ = cpu_to_le32(insn);
-}
-
-/* Patched to NOP when not supported */
-void __init spectre_bhb_patch_clearbhb(struct alt_instr *alt,
- __le32 *origptr, __le32 *updptr, int nr_inst)
-{
- BUG_ON(nr_inst != 2);
-
- if (test_bit(BHB_INSN, &system_bhb_mitigations))
- return;
-
- *updptr++ = cpu_to_le32(aarch64_insn_gen_nop());
- *updptr++ = cpu_to_le32(aarch64_insn_gen_nop());
-}
-
-#ifdef CONFIG_BPF_SYSCALL
-#define EBPF_WARN "Unprivileged eBPF is enabled, data leaks possible via Spectre v2 BHB attacks!\n"
-void unpriv_ebpf_notify(int new_state)
-{
- if (spectre_v2_state == SPECTRE_VULNERABLE ||
- spectre_bhb_state != SPECTRE_MITIGATED)
- return;
-
- if (!new_state)
- pr_err("WARNING: %s", EBPF_WARN);
-}
-#endif
================================================================================
* ONLY IN PATCH2 - files not modified by patch1 *
================================================================================
--- a/arch/arm64/kernel/proton-pack.c
+++ b/arch/arm64/kernel/proton-pack.c
@@ -853,6 +853,7 @@ u8 spectre_bhb_loop_affected(int scope)
if (scope == SCOPE_LOCAL_CPU) {
static const struct midr_range spectre_bhb_k32_list[] = {
MIDR_ALL_VERSIONS(MIDR_CORTEX_A78),
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A78AE),
MIDR_ALL_VERSIONS(MIDR_CORTEX_A78C),
MIDR_ALL_VERSIONS(MIDR_CORTEX_X1),
MIDR_ALL_VERSIONS(MIDR_CORTEX_A710),
################################################################################
! REJECTED PATCH2 HUNKS - could not be compared; manual review needed !
################################################################################
--- b/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -86,6 +86,7 @@
#define ARM_CPU_PART_CORTEX_X2 0xD48
#define ARM_CPU_PART_NEOVERSE_N2 0xD49
#define ARM_CPU_PART_CORTEX_A78C 0xD4B
+#define ARM_CPU_PART_CORTEX_X4 0xD82
#define APM_CPU_PART_XGENE 0x000
#define APM_CPU_VAR_POTENZA 0x00
@@ -159,6 +160,7 @@
#define MIDR_CORTEX_X2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X2)
#define MIDR_NEOVERSE_N2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N2)
#define MIDR_CORTEX_A78C MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78C)
+#define MIDR_CORTEX_X4 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X4)
#define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX)
#define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_81XX)
#define MIDR_THUNDERX_83XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_83XX)
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -139,6 +157,5 @@
#define MIDR_NEOVERSE_N2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N2)
#define MIDR_CORTEX_A78C MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78C)
-#define MIDR_NEOVERSE_V2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_V2)
#define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX)
#define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_81XX)
#define MIDR_THUNDERX_83XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_83XX)
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -141,5 +159,4 @@
#define MIDR_CORTEX_A78C MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78C)
-#define MIDR_NEOVERSE_V2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_V2)
#define MIDR_CORTEX_X4 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X4)
#define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX)
#define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_81XX)
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -88,4 +88,4 @@
-#define ARM_CPU_PART_NEOVERSE_V2 0xD4F
+#define ARM_CPU_PART_CORTEX_A720 0xD81
#define ARM_CPU_PART_CORTEX_X4 0xD82
#define ARM_CPU_PART_NEOVERSE_V3 0xD84
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -147,4 +167,4 @@
#define MIDR_CORTEX_A78C MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78C)
#define MIDR_CORTEX_X3 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X3)
#define MIDR_NEOVERSE_V2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_V2)
-#define MIDR_CORTEX_X4 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X4)
+#define MIDR_CORTEX_A720 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A720)
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -150,5 +179,6 @@
#define MIDR_NEOVERSE_V2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_V2)
+#define MIDR_CORTEX_A720 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A720)
#define MIDR_CORTEX_X4 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X4)
#define MIDR_NEOVERSE_V3 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_V3)
#define MIDR_CORTEX_X925 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X925)
-#define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX)
+#define MIDR_CORTEX_A725 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A725)
================================================================================
* DELTA DIFFERENCES - code changes that differ between the patches *
================================================================================
--- b/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -133,6 +133,7 @@
#define HISI_CPU_PART_TSV110 0xD01
#define AMPERE_CPU_PART_AMPERE1 0xAC3
+
#define MICROSOFT_CPU_PART_AZURE_COBALT_100 0xD49 /* Based on r0p0 of ARM Neoverse N2 */
#define MIDR_CORTEX_A53 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A53)
================================================================================
* CONTEXT DIFFERENCES - surrounding code differences between the patches *
================================================================================
--- b/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -178,5 +190,5 @@
-#define MIDR_FUJITSU_A64FX MIDR_CPU_MODEL(ARM_CPU_IMP_FUJITSU, FUJITSU_CPU_PART_A64FX)
-#define MIDR_HISI_TSV110 MIDR_CPU_MODEL(ARM_CPU_IMP_HISI, HISI_CPU_PART_TSV110)
+#define MIDR_APPLE_M2_BLIZZARD_MAX MIDR_CPU_MODEL(ARM_CPU_IMP_APPLE, APPLE_CPU_PART_M2_BLIZZARD_MAX)
+#define MIDR_APPLE_M2_AVALANCHE_MAX MIDR_CPU_MODEL(ARM_CPU_IMP_APPLE, APPLE_CPU_PART_M2_AVALANCHE_MAX)
#define MIDR_AMPERE1 MIDR_CPU_MODEL(ARM_CPU_IMP_AMPERE, AMPERE_CPU_PART_AMPERE1)
/* Fujitsu Erratum 010001 affects A64FX 1.0 and 1.1, (v0r0 and v1r0) */
================================================================================
* ONLY IN PATCH2 - files not modified by patch1 *
================================================================================
--- a/Documentation/arch/arm64/silicon-errata.rst
+++ b/Documentation/arch/arm64/silicon-errata.rst
@@ -243,3 +243,10 @@ stable kernels.
+----------------+-----------------+-----------------+-----------------------------+
| ASR | ASR8601 | #8601001 | N/A |
+----------------+-----------------+-----------------+-----------------------------+
++----------------+-----------------+-----------------+-----------------------------+
+| Microsoft | Azure Cobalt 100| #2139208 | ARM64_ERRATUM_2139208 |
++----------------+-----------------+-----------------+-----------------------------+
+| Microsoft | Azure Cobalt 100| #2067961 | ARM64_ERRATUM_2067961 |
++----------------+-----------------+-----------------+-----------------------------+
+| Microsoft | Azure Cobalt 100| #2253138 | ARM64_ERRATUM_2253138 |
++----------------+-----------------+-----------------+-----------------------------+
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -374,6 +374,7 @@ static const struct midr_range erratum_1463225[] = {
static const struct midr_range trbe_overwrite_fill_mode_cpus[] = {
#ifdef CONFIG_ARM64_ERRATUM_2139208
MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N2),
+ MIDR_ALL_VERSIONS(MIDR_MICROSOFT_AZURE_COBALT_100),
#endif
#ifdef CONFIG_ARM64_ERRATUM_2119858
MIDR_ALL_VERSIONS(MIDR_CORTEX_A710),
@@ -387,6 +388,7 @@ static const struct midr_range trbe_overwrite_fill_mode_cpus[] = {
static const struct midr_range tsb_flush_fail_cpus[] = {
#ifdef CONFIG_ARM64_ERRATUM_2067961
MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N2),
+ MIDR_ALL_VERSIONS(MIDR_MICROSOFT_AZURE_COBALT_100),
#endif
#ifdef CONFIG_ARM64_ERRATUM_2054223
MIDR_ALL_VERSIONS(MIDR_CORTEX_A710),
@@ -399,6 +401,7 @@ static const struct midr_range tsb_flush_fail_cpus[] = {
static struct midr_range trbe_write_out_of_range_cpus[] = {
#ifdef CONFIG_ARM64_ERRATUM_2253138
MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N2),
+ MIDR_ALL_VERSIONS(MIDR_MICROSOFT_AZURE_COBALT_100),
#endif
#ifdef CONFIG_ARM64_ERRATUM_2224489
MIDR_ALL_VERSIONS(MIDR_CORTEX_A710),This is an automated interdiff check for backported commits. |
JIRA PR Check Results7 commit(s) with issues found: Commit
|
|
❌ Validation checks completed with issues View full results: https://github.com/ctrliq/kernel-src-tree/actions/runs/27385365926 |
https://ciqinc.atlassian.net/browse/KERNEL-1145
Update process (This kernel CentOS base for 4.18.0-553.132.1.el8_10)
rlc-8/4.18.0-553.132.1.el8_10branch fromrocky8_10rlc-8/4.18.0-553.126.1.el8_10into new branch (skipping unneeded code)Rebase Log
BUILD
KSelfTest