diff options
| author | Serge Semin <fancer.lancer@gmail.com> | 2025-06-10 17:11:13 +0300 |
|---|---|---|
| committer | Serge Semin <fancer.lancer@gmail.com> | 2026-08-13 20:31:42 +0300 |
| commit | 36658042c830defcf2cc5b3b535827a7742b082b (patch) | |
| tree | c76f3abaad2a488d7eab5f8027949c67e5d961db | |
| parent | 754290560513a88af6ca6e6b148a155df8e63851 (diff) | |
| download | linux-36658042c830defcf2cc5b3b535827a7742b082b.tar.gz linux-36658042c830defcf2cc5b3b535827a7742b082b.zip | |
net: stmmac: dwmac1000: Fix long-broken Tx Jumbo-frames support
As soon as the Jumbo frames with MTU over 8K enabled it's possible to have
a garbage or truncated frame transmitted to a recipient. The problem is
multi-dimensional and mainly connected with over-complicated and weakly
tested DW GMAC Jumbos implementation.
First if all let's clarify the DW GMAC descriptors capabilities crucial
for the denoted problems. DW MAC and GMAC support two types of the
DMA-descriptors Normal and Enhanced - the former descriptor can have up to
two 2K buffers attached, meanwhile the later one can be initialized with
up to two 8K buffers. The DMA-descriptors can be allocated either as a
continuous array of descriptors (ring mode) or as a set of individually
linked descriptors (chain mode). All of that affects the size of the
buffers support by each descriptor since in the chain mode the descriptor
field responsible for the buffer2 pointer is utilized for linking the
adjacent descriptors. Thus here is the maximum buffers size
Normal Tx descriptor: 2K (chain), 2x2K (ring)
Enhanced Tx descriptor: 8K (chain), 2x8K (ring)
The main problem happens when an SKB with a Jumboed fragment is being
transmitted. The stmmac_xmit() method allocates only one descriptor in for
each segment that case. So if it's a Chained Enhanced descriptor then over
8K fragment just won't fit in there and will be simply truncated. If it's
a Ringed Enhanced Descriptor then the buffers length will be properly
initialized by the enh_desc_prepare_tx_desc() method, but the descriptor
won't have the buffer2 pointer field initialized and a garbage from the
DMA address 0x0 will be transmitted.
The same problem happens for the Normal descriptors except that it happens
for over 2K fragments. Moreover SKBs with Jumboed head won't be properly
handled for the Ringed Normal Descriptors mode either. See the buffer2
offset utilized in the stmmac_jumbo_frm() method for the buffer2 pointer.
It's always 4K meanwhile the buffer size in this case is 2K. Thus some
data will be missed in the sent out frame.
So to speak the Jumbos support is almost completely broken for DW
MAC/GMAC. The only working case is when an SKB with Jumboed head is
submitted and the Enhanced Tx descriptor is utilized.
Let's fix the denoted problems. But instead of fixing each part of the
affected code and making it even more complicated let's do that in a more
elegant way. The jumbos support can be significantly simplified by
implementing a allocation-based pattern already developed for the TSO
feature. The only difference is that a simple
stmmac_desc_ops::prepare_tx_desc() method will be utilized for the
Tx DMA-descriptors initialization. Thus a lot of redundant code in the
chain_mode.c/ring_mode.c modules will be replaced with a single
stmmac_tx_allocator() method utilized for the DMA-descriptors allocation
for all SKB head and fragments.
As a nice side-effect of this change the stmmac_xmit() and
stmmac_tso_xmit() methods now look very much similar which makes the code
much more readable and maintainable.
Fixes: 286a83721720 ("stmmac: add CHAINED descriptor mode support (V4)")
Fixes: 47dd7a540b8a ("net: add support for STMicroelectronics Ethernet controllers.")
Signed-off-by: Serge Semin <fancer.lancer@gmail.com>
---
Note after this change there is no longer need in the GMAC max MTU 9K
constraint for Tx. It can be up to 16K from now. The only thing left to
fix is the Rx path. Alas it will be harder to implement.
| -rw-r--r-- | drivers/net/ethernet/stmicro/stmmac/chain_mode.c | 91 | ||||
| -rw-r--r-- | drivers/net/ethernet/stmicro/stmmac/descs_com.h | 52 | ||||
| -rw-r--r-- | drivers/net/ethernet/stmicro/stmmac/dwmac4_descs.c | 10 | ||||
| -rw-r--r-- | drivers/net/ethernet/stmicro/stmmac/dwxgmac2_descs.c | 10 | ||||
| -rw-r--r-- | drivers/net/ethernet/stmicro/stmmac/enh_desc.c | 13 | ||||
| -rw-r--r-- | drivers/net/ethernet/stmicro/stmmac/hwif.h | 11 | ||||
| -rw-r--r-- | drivers/net/ethernet/stmicro/stmmac/norm_desc.c | 13 | ||||
| -rw-r--r-- | drivers/net/ethernet/stmicro/stmmac/ring_mode.c | 84 | ||||
| -rw-r--r-- | drivers/net/ethernet/stmicro/stmmac/stmmac.h | 1 | ||||
| -rw-r--r-- | drivers/net/ethernet/stmicro/stmmac/stmmac_main.c | 200 |
10 files changed, 169 insertions, 316 deletions
diff --git a/drivers/net/ethernet/stmicro/stmmac/chain_mode.c b/drivers/net/ethernet/stmicro/stmmac/chain_mode.c index d6118745d681d..fb5749bfb3480 100644 --- a/drivers/net/ethernet/stmicro/stmmac/chain_mode.c +++ b/drivers/net/ethernet/stmicro/stmmac/chain_mode.c @@ -14,95 +14,6 @@ #include "stmmac.h" -static int jumbo_frm(struct stmmac_tx_queue *tx_q, struct sk_buff *skb, - int csum) -{ - unsigned int nopaged_len = skb_headlen(skb); - struct stmmac_priv *priv = tx_q->priv_data; - unsigned int entry = tx_q->cur_tx; - unsigned int bmax, buf_len, des2; - unsigned int i = 1, len; - struct dma_desc *desc; - - if (priv->extend_desc) - desc = (struct dma_desc *)(tx_q->dma_etx + entry); - else - desc = tx_q->dma_tx + entry; - - if (priv->plat->enh_desc) - bmax = BUF_SIZE_8KiB; - else - bmax = BUF_SIZE_2KiB; - - buf_len = min_t(unsigned int, nopaged_len, bmax); - len = nopaged_len - buf_len; - - des2 = dma_map_single(priv->device, skb->data, - buf_len, DMA_TO_DEVICE); - desc->des2 = cpu_to_le32(des2); - if (dma_mapping_error(priv->device, des2)) - return -1; - tx_q->tx_skbuff_dma[entry].buf = des2; - tx_q->tx_skbuff_dma[entry].len = buf_len; - /* do not close the descriptor and do not set own bit */ - stmmac_prepare_tx_desc(priv, desc, 1, buf_len, csum, STMMAC_CHAIN_MODE, - 0, false, skb->len); - - while (len != 0) { - tx_q->tx_skbuff[entry] = NULL; - entry = STMMAC_GET_ENTRY(entry, priv->dma_conf.dma_tx_size); - - if (priv->extend_desc) - desc = (struct dma_desc *)(tx_q->dma_etx + entry); - else - desc = tx_q->dma_tx + entry; - - if (len > bmax) { - des2 = dma_map_single(priv->device, - (skb->data + bmax * i), - bmax, DMA_TO_DEVICE); - desc->des2 = cpu_to_le32(des2); - if (dma_mapping_error(priv->device, des2)) - return -1; - tx_q->tx_skbuff_dma[entry].buf = des2; - tx_q->tx_skbuff_dma[entry].len = bmax; - stmmac_prepare_tx_desc(priv, desc, 0, bmax, csum, - STMMAC_CHAIN_MODE, 1, false, skb->len); - len -= bmax; - i++; - } else { - des2 = dma_map_single(priv->device, - (skb->data + bmax * i), len, - DMA_TO_DEVICE); - desc->des2 = cpu_to_le32(des2); - if (dma_mapping_error(priv->device, des2)) - return -1; - tx_q->tx_skbuff_dma[entry].buf = des2; - tx_q->tx_skbuff_dma[entry].len = len; - /* last descriptor can be set now */ - stmmac_prepare_tx_desc(priv, desc, 0, len, csum, - STMMAC_CHAIN_MODE, 1, - !skb_is_nonlinear(skb), skb->len); - len = 0; - } - } - - tx_q->cur_tx = entry; - - return entry; -} - -static bool is_jumbo_frm(unsigned int len, bool enh_desc) -{ - bool ret = false; - - if ((enh_desc && (len > BUF_SIZE_8KiB)) || - (!enh_desc && (len > BUF_SIZE_2KiB))) - ret = true; - - return ret; -} - static void init_dma_chain(void *des, dma_addr_t phy_addr, unsigned int size, unsigned int extend_desc) { @@ -150,7 +61,5 @@ static void refill_desc3(struct stmmac_rx_queue *rx_q, struct dma_desc *p) const struct stmmac_mode_ops chain_mode_ops = { .init = init_dma_chain, - .is_jumbo_frm = is_jumbo_frm, - .jumbo_frm = jumbo_frm, .refill_desc3 = refill_desc3, }; diff --git a/drivers/net/ethernet/stmicro/stmmac/descs_com.h b/drivers/net/ethernet/stmicro/stmmac/descs_com.h index 0019f389181f7..bedb18538d519 100644 --- a/drivers/net/ethernet/stmicro/stmmac/descs_com.h +++ b/drivers/net/ethernet/stmicro/stmmac/descs_com.h @@ -38,22 +38,25 @@ static inline void enh_desc_end_tx_desc_on_ring(struct dma_desc *p, int end) p->des0 &= cpu_to_le32(~ETDES0_END_RING); } -/* The maximum buffer 1 size is 8KiB - 1. However, we limit to 4KiB. */ static inline void enh_set_tx_desc_len_on_ring(struct dma_desc *p, int len) { - unsigned int buffer1_max_length = BUF_SIZE_4KiB; + if (unlikely(len > ETDES1_BUFFER1_SIZE_MASK)) { + u32 buf2_len = FIELD_PREP(ETDES1_BUFFER2_SIZE_MASK, + len - ETDES1_BUFFER1_SIZE_MASK); - if (unlikely(len > buffer1_max_length)) { - p->des1 |= cpu_to_le32(FIELD_PREP(ETDES1_BUFFER2_SIZE_MASK, - len - buffer1_max_length) | - FIELD_PREP(ETDES1_BUFFER1_SIZE_MASK, - buffer1_max_length)); + p->des1 |= cpu_to_le32(ETDES1_BUFFER1_SIZE_MASK | buf2_len); + p->des3 = cpu_to_le32(le32_to_cpu(p->des2) + ETDES1_BUFFER1_SIZE_MASK); } else { - p->des1 |= cpu_to_le32(FIELD_PREP(ETDES1_BUFFER1_SIZE_MASK, - len)); + p->des1 |= cpu_to_le32(FIELD_PREP(ETDES1_BUFFER1_SIZE_MASK, len)); } } +static inline unsigned int enh_desc_tx_desc_len_on_ring(void) +{ + return FIELD_MAX(ETDES1_BUFFER1_SIZE_MASK) + + FIELD_MAX(ETDES1_BUFFER2_SIZE_MASK); +} + /* Normal descriptors */ static inline void ndesc_rx_set_on_ring(struct dma_desc *p, int end, int bfsize) { @@ -77,22 +80,25 @@ static inline void ndesc_end_tx_desc_on_ring(struct dma_desc *p, int end) p->des1 &= cpu_to_le32(~TDES1_END_RING); } -/* The maximum buffer 1 size is 2KiB - 1, limited by the mask width */ static inline void norm_set_tx_desc_len_on_ring(struct dma_desc *p, int len) { - unsigned int buffer1_max_length = BUF_SIZE_2KiB - 1; + if (unlikely(len > TDES1_BUFFER1_SIZE_MASK)) { + u32 buf2_len = FIELD_PREP(TDES1_BUFFER2_SIZE_MASK, + len - TDES1_BUFFER1_SIZE_MASK); - if (unlikely(len > buffer1_max_length)) { - p->des1 |= cpu_to_le32(FIELD_PREP(TDES1_BUFFER2_SIZE_MASK, - len - buffer1_max_length) | - FIELD_PREP(TDES1_BUFFER1_SIZE_MASK, - buffer1_max_length)); + p->des1 |= cpu_to_le32(TDES1_BUFFER1_SIZE_MASK | buf2_len); + p->des3 = cpu_to_le32(le32_to_cpu(p->des2) + TDES1_BUFFER1_SIZE_MASK); } else { - p->des1 |= cpu_to_le32(FIELD_PREP(TDES1_BUFFER1_SIZE_MASK, - len)); + p->des1 |= cpu_to_le32(FIELD_PREP(TDES1_BUFFER1_SIZE_MASK, len)); } } +static inline unsigned int ndesc_tx_desc_len_on_ring(void) +{ + return FIELD_MAX(TDES1_BUFFER1_SIZE_MASK) + + FIELD_MAX(TDES1_BUFFER2_SIZE_MASK); +} + /* Specific functions used for Chain mode */ /* Enhanced descriptors */ @@ -115,6 +121,11 @@ static inline void enh_set_tx_desc_len_on_chain(struct dma_desc *p, int len) p->des1 |= cpu_to_le32(len & ETDES1_BUFFER1_SIZE_MASK); } +static inline unsigned int enh_desc_tx_desc_len_on_chain(void) +{ + return FIELD_MAX(ETDES1_BUFFER1_SIZE_MASK); +} + /* Normal descriptors */ static inline void ndesc_rx_set_on_chain(struct dma_desc *p, int end) { @@ -135,6 +146,11 @@ static inline void norm_set_tx_desc_len_on_chain(struct dma_desc *p, int len) p->des1 |= cpu_to_le32(len & TDES1_BUFFER1_SIZE_MASK); } +static inline unsigned int ndesc_tx_desc_len_on_chain(void) +{ + return FIELD_MAX(TDES1_BUFFER1_SIZE_MASK); +} + /* Functions used for all ring/chain modes, enhanced/normal descriptors */ static inline int com_desc_rx_coe_rdes0(int ipc_err, int type, int payload_err) diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac4_descs.c b/drivers/net/ethernet/stmicro/stmmac/dwmac4_descs.c index 7918759369bb9..90c1567b19b64 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac4_descs.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac4_descs.c @@ -191,11 +191,6 @@ static int dwmac4_wrback_get_rx_status(struct stmmac_extra_stats *x, return ret; } -static int dwmac4_rd_get_tx_len(struct dma_desc *p) -{ - return (le32_to_cpu(p->des2) & TDES2_BUFFER1_SIZE_MASK); -} - static int dwmac4_get_tx_owner(struct dma_desc *p) { return (le32_to_cpu(p->des3) & TDES3_OWN) >> TDES3_OWN_SHIFT; @@ -338,6 +333,11 @@ static void dwmac4_rd_init_tx_desc(struct dma_desc *p, int mode, int end) p->des3 = 0; } +static unsigned int dwmac4_rd_get_tx_len(int mode) +{ + return FIELD_MAX(TDES2_BUFFER1_SIZE_MASK); +} + static void dwmac4_rd_prepare_tx_desc(struct dma_desc *p, int is_fs, int len, bool csum_flag, int mode, bool tx_own, bool ls, unsigned int tot_pkt_len) diff --git a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_descs.c b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_descs.c index 98135173a321d..a4b40adeda18b 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_descs.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_descs.c @@ -129,11 +129,6 @@ static int dwxgmac2_get_rx_status(struct stmmac_extra_stats *x, return ret; } -static int dwxgmac2_get_tx_len(struct dma_desc *p) -{ - return (le32_to_cpu(p->des2) & XGMAC_TDES2_B1L); -} - static int dwxgmac2_get_tx_owner(struct dma_desc *p) { return (le32_to_cpu(p->des3) & XGMAC_TDES3_OWN) > 0; @@ -247,6 +242,11 @@ static void dwxgmac2_init_tx_desc(struct dma_desc *p, int mode, int end) p->des3 = 0; } +static unsigned int dwxgmac2_get_tx_len(int mode) +{ + return FIELD_MAX(XGMAC_TDES2_B1L); +} + static void dwxgmac2_prepare_tx_desc(struct dma_desc *p, int is_fs, int len, bool csum_flag, int mode, bool tx_own, bool ls, unsigned int tot_pkt_len) diff --git a/drivers/net/ethernet/stmicro/stmmac/enh_desc.c b/drivers/net/ethernet/stmicro/stmmac/enh_desc.c index ed91ffb022722..6e2a9320bfb2e 100644 --- a/drivers/net/ethernet/stmicro/stmmac/enh_desc.c +++ b/drivers/net/ethernet/stmicro/stmmac/enh_desc.c @@ -70,11 +70,6 @@ static int enh_desc_get_tx_status(struct stmmac_extra_stats *x, return ret; } -static int enh_desc_get_tx_len(struct dma_desc *p) -{ - return (le32_to_cpu(p->des1) & ETDES1_BUFFER1_SIZE_MASK); -} - static int enh_desc_rx_ext_status(unsigned int rdes4, struct stmmac_extra_stats *x) { @@ -334,6 +329,14 @@ static int enh_desc_get_tx_ls(struct dma_desc *p) return (le32_to_cpu(p->des0) & ETDES0_LAST_SEGMENT) >> 29; } +static unsigned int enh_desc_get_tx_len(int mode) +{ + if (mode == STMMAC_CHAIN_MODE) + return enh_desc_tx_desc_len_on_chain(); + else + return enh_desc_tx_desc_len_on_ring(); +} + static void enh_desc_release_tx_desc(struct dma_desc *p, int mode, dma_addr_t np, bool hwts_tx) { diff --git a/drivers/net/ethernet/stmicro/stmmac/hwif.h b/drivers/net/ethernet/stmicro/stmmac/hwif.h index ed961ea1f939a..085e2b0624785 100644 --- a/drivers/net/ethernet/stmicro/stmmac/hwif.h +++ b/drivers/net/ethernet/stmicro/stmmac/hwif.h @@ -66,8 +66,8 @@ struct stmmac_desc_ops { bool (*get_rx_vlan_valid)(struct dma_desc *p); /* Return the transmit status looking at the TDES1 */ int (*tx_status)(struct stmmac_extra_stats *x, struct dma_desc *p); - /* Get the buffer size from the descriptor */ - int (*get_tx_len)(struct dma_desc *p); + /* Get the buffer(s) size for a single descriptor */ + unsigned int (*get_tx_len)(int mode); /* Handle extra events on specific interrupts hw dependent */ void (*set_rx_owner)(struct dma_desc *p, int disable_rx_ic); /* Get the receive frame size */ @@ -548,9 +548,6 @@ struct stmmac_rx_queue; struct stmmac_mode_ops { void (*init) (void *des, dma_addr_t phy_addr, unsigned int size, unsigned int extend_desc); - bool (*is_jumbo_frm)(unsigned int len, bool enh_desc); - int (*jumbo_frm)(struct stmmac_tx_queue *tx_q, struct sk_buff *skb, - int csum); int (*set_16kib_bfsize)(int mtu); void (*init_desc3)(struct dma_desc *p); void (*refill_desc3)(struct stmmac_rx_queue *rx_q, struct dma_desc *p); @@ -558,10 +555,6 @@ struct stmmac_mode_ops { #define stmmac_mode_init(__priv, __args...) \ stmmac_do_void_callback(__priv, mode, init, __args) -#define stmmac_is_jumbo_frm(__priv, __args...) \ - stmmac_do_callback(__priv, mode, is_jumbo_frm, __args) -#define stmmac_jumbo_frm(__priv, __args...) \ - stmmac_do_callback(__priv, mode, jumbo_frm, __args) #define stmmac_set_16kib_bfsize(__priv, __args...) \ stmmac_do_callback(__priv, mode, set_16kib_bfsize, __args) #define stmmac_init_desc3(__priv, __args...) \ diff --git a/drivers/net/ethernet/stmicro/stmmac/norm_desc.c b/drivers/net/ethernet/stmicro/stmmac/norm_desc.c index 6f90b0b6f54c6..3ca188c934bc5 100644 --- a/drivers/net/ethernet/stmicro/stmmac/norm_desc.c +++ b/drivers/net/ethernet/stmicro/stmmac/norm_desc.c @@ -55,11 +55,6 @@ static int ndesc_get_tx_status(struct stmmac_extra_stats *x, return ret; } -static int ndesc_get_tx_len(struct dma_desc *p) -{ - return (le32_to_cpu(p->des1) & RDES1_BUFFER1_SIZE_MASK); -} - /* This function verifies if each incoming frame has some errors * and, if required, updates the multicast statistics. * In case of success, it returns good_frame because the GMAC device @@ -208,6 +203,14 @@ static int ndesc_get_tx_ls(struct dma_desc *p) return (le32_to_cpu(p->des1) & TDES1_LAST_SEGMENT) >> 30; } +static unsigned int ndesc_get_tx_len(int mode) +{ + if (mode == STMMAC_CHAIN_MODE) + return ndesc_tx_desc_len_on_chain(); + else + return ndesc_tx_desc_len_on_ring(); +} + static void ndesc_release_tx_desc(struct dma_desc *p, int mode, dma_addr_t np, bool hwts_tx) { diff --git a/drivers/net/ethernet/stmicro/stmmac/ring_mode.c b/drivers/net/ethernet/stmicro/stmmac/ring_mode.c index 3110b10675cd5..ff3ddb800d049 100644 --- a/drivers/net/ethernet/stmicro/stmmac/ring_mode.c +++ b/drivers/net/ethernet/stmicro/stmmac/ring_mode.c @@ -14,88 +14,6 @@ #include "stmmac.h" -static int jumbo_frm(struct stmmac_tx_queue *tx_q, struct sk_buff *skb, - int csum) -{ - unsigned int nopaged_len = skb_headlen(skb); - struct stmmac_priv *priv = tx_q->priv_data; - unsigned int entry = tx_q->cur_tx; - unsigned int bmax, len, des2; - struct dma_desc *desc; - - if (priv->extend_desc) - desc = (struct dma_desc *)(tx_q->dma_etx + entry); - else - desc = tx_q->dma_tx + entry; - - if (priv->plat->enh_desc) - bmax = BUF_SIZE_8KiB; - else - bmax = BUF_SIZE_2KiB; - - len = nopaged_len - bmax; - - if (nopaged_len > BUF_SIZE_8KiB) { - - des2 = dma_map_single(priv->device, skb->data, bmax, - DMA_TO_DEVICE); - desc->des2 = cpu_to_le32(des2); - if (dma_mapping_error(priv->device, des2)) - return -1; - - tx_q->tx_skbuff_dma[entry].buf = des2; - tx_q->tx_skbuff_dma[entry].len = bmax; - tx_q->tx_skbuff_dma[entry].is_jumbo = true; - - desc->des3 = cpu_to_le32(des2 + BUF_SIZE_4KiB); - stmmac_prepare_tx_desc(priv, desc, 1, bmax, csum, - STMMAC_RING_MODE, 0, false, skb->len); - tx_q->tx_skbuff[entry] = NULL; - entry = STMMAC_GET_ENTRY(entry, priv->dma_conf.dma_tx_size); - - if (priv->extend_desc) - desc = (struct dma_desc *)(tx_q->dma_etx + entry); - else - desc = tx_q->dma_tx + entry; - - des2 = dma_map_single(priv->device, skb->data + bmax, len, - DMA_TO_DEVICE); - desc->des2 = cpu_to_le32(des2); - if (dma_mapping_error(priv->device, des2)) - return -1; - tx_q->tx_skbuff_dma[entry].buf = des2; - tx_q->tx_skbuff_dma[entry].len = len; - tx_q->tx_skbuff_dma[entry].is_jumbo = true; - - desc->des3 = cpu_to_le32(des2 + BUF_SIZE_4KiB); - stmmac_prepare_tx_desc(priv, desc, 0, len, csum, - STMMAC_RING_MODE, 1, !skb_is_nonlinear(skb), - skb->len); - } else { - des2 = dma_map_single(priv->device, skb->data, - nopaged_len, DMA_TO_DEVICE); - desc->des2 = cpu_to_le32(des2); - if (dma_mapping_error(priv->device, des2)) - return -1; - tx_q->tx_skbuff_dma[entry].buf = des2; - tx_q->tx_skbuff_dma[entry].len = nopaged_len; - tx_q->tx_skbuff_dma[entry].is_jumbo = true; - desc->des3 = cpu_to_le32(des2 + BUF_SIZE_4KiB); - stmmac_prepare_tx_desc(priv, desc, 1, nopaged_len, csum, - STMMAC_RING_MODE, 0, !skb_is_nonlinear(skb), - skb->len); - } - - tx_q->cur_tx = entry; - - return entry; -} - -static bool is_jumbo_frm(unsigned int len, bool enh_desc) -{ - return len >= BUF_SIZE_4KiB; -} - static void refill_desc3(struct stmmac_rx_queue *rx_q, struct dma_desc *p) { struct stmmac_priv *priv = rx_q->priv_data; @@ -120,8 +38,6 @@ static int set_16kib_bfsize(int mtu) } const struct stmmac_mode_ops ring_mode_ops = { - .is_jumbo_frm = is_jumbo_frm, - .jumbo_frm = jumbo_frm, .refill_desc3 = refill_desc3, .init_desc3 = init_desc3, .set_16kib_bfsize = set_16kib_bfsize, diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac.h b/drivers/net/ethernet/stmicro/stmmac/stmmac.h index 74f7817b6af4e..1e8ae9b17ca3d 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac.h +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac.h @@ -90,7 +90,6 @@ struct stmmac_tx_info { bool map_as_page; unsigned len; bool last_segment; - bool is_jumbo; enum stmmac_txbuf_type buf_type; struct xsk_tx_metadata_compl xsk_meta; }; diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c index 529a9bf406995..7d47b0ea1a2b0 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c @@ -2844,7 +2844,6 @@ static bool stmmac_xdp_xmit_zc(struct stmmac_priv *priv, u32 queue, u32 budget) tx_q->tx_skbuff_dma[entry].map_as_page = false; tx_q->tx_skbuff_dma[entry].len = xdp_desc.len; tx_q->tx_skbuff_dma[entry].last_segment = true; - tx_q->tx_skbuff_dma[entry].is_jumbo = false; stmmac_set_desc_addr(priv, tx_desc, dma_addr); @@ -3012,7 +3011,6 @@ static int stmmac_tx_clean(struct stmmac_priv *priv, int budget, u32 queue, } tx_q->tx_skbuff_dma[entry].last_segment = false; - tx_q->tx_skbuff_dma[entry].is_jumbo = false; if (xdpf && tx_q->tx_skbuff_dma[entry].buf_type == STMMAC_TXBUF_T_XDP_TX) { @@ -4752,6 +4750,57 @@ flush_ring: } /** + * stmmac_tx_allocator - Tx DMA descriptors allocator for cross-descriptor frames + * @priv: driver private structure + * @des: buffer start address + * @total_len: total length to fill in descriptors + * @last_segment: condition for the last descriptor + * @queue: TX queue index + * Description: + * This function fills descriptor and request new descriptors according to + * buffer length to fill. The method specifically dedicated for Jumbo-frames + * which don't fit to a single descriptor. + */ +static void stmmac_tx_allocator(struct stmmac_priv *priv, dma_addr_t des, + int total_len, bool last_segment, u32 queue) +{ + struct stmmac_tx_queue *tx_q = &priv->dma_conf.tx_queue[queue]; + u32 buff_size, buff_max; + struct dma_desc *desc; + int tmp_len; + + buff_max = stmmac_get_tx_len(priv, priv->mode); + tmp_len = total_len; + + while (tmp_len > 0) { + dma_addr_t curr_addr; + + tx_q->cur_tx = STMMAC_GET_ENTRY(tx_q->cur_tx, + priv->dma_conf.dma_tx_size); + WARN_ON(tx_q->tx_skbuff[tx_q->cur_tx]); + + if (likely(priv->extend_desc)) + desc = &tx_q->dma_etx[tx_q->cur_tx].basic; + else if (tx_q->tbs & STMMAC_TBS_AVAIL) + desc = &tx_q->dma_entx[tx_q->cur_tx].basic; + else + desc = &tx_q->dma_tx[tx_q->cur_tx]; + + curr_addr = des + (total_len - tmp_len); + stmmac_set_desc_addr(priv, desc, curr_addr); + + buff_size = tmp_len >= buff_max ? buff_max : tmp_len; + + /* Prepare _not_ first descriptor and set it' own bit */ + stmmac_prepare_tx_desc(priv, desc, false, buff_size, false, + priv->mode, true, + last_segment && (tmp_len <= buff_max), 0); + + tmp_len -= buff_max; + } +} + +/** * stmmac_xmit - Tx entry point of the driver * @skb : the socket buffer * @dev : device pointer @@ -4761,21 +4810,19 @@ flush_ring: */ static netdev_tx_t stmmac_xmit(struct sk_buff *skb, struct net_device *dev) { - bool enh_desc, has_vlan, set_ic, drop = false, is_jumbo = false; + unsigned int first_entry, tx_packets, nopaged_len, descbuf_len; struct stmmac_priv *priv = netdev_priv(dev); - unsigned int nopaged_len = skb_headlen(skb); u32 queue = skb_get_queue_mapping(skb); int nfrags = skb_shinfo(skb)->nr_frags; - unsigned int first_entry, tx_packets; int gso = skb_shinfo(skb)->gso_type; + bool has_vlan, set_ic, drop = false; struct stmmac_txq_stats *txq_stats; struct dma_edesc *tbs_desc = NULL; + int i, first_tx, csum_insertion; netdev_tx_t ret = NETDEV_TX_OK; struct dma_desc *desc, *first; struct stmmac_tx_queue *tx_q; - int i, csum_insertion = 0; struct netdev_queue *nq; - int entry, first_tx; dma_addr_t des; u32 sdu_len; @@ -4818,8 +4865,7 @@ static netdev_tx_t stmmac_xmit(struct sk_buff *skb, struct net_device *dev) /* Check if VLAN can be inserted by HW */ has_vlan = stmmac_vlan_insert(priv, skb, tx_q); - entry = tx_q->cur_tx; - first_entry = entry; + first_entry = tx_q->cur_tx; WARN_ON(tx_q->tx_skbuff[first_entry]); /* Check if CSUM should be inserted by HW */ @@ -4830,46 +4876,45 @@ static netdev_tx_t stmmac_xmit(struct sk_buff *skb, struct net_device *dev) } if (likely(priv->extend_desc)) - desc = (struct dma_desc *)(tx_q->dma_etx + entry); + desc = &tx_q->dma_etx[first_entry].basic; else if (tx_q->tbs & STMMAC_TBS_AVAIL) - desc = &tx_q->dma_entx[entry].basic; + desc = &tx_q->dma_entx[first_entry].basic; else - desc = tx_q->dma_tx + entry; + desc = &tx_q->dma_tx[first_entry]; first = desc; if (has_vlan) stmmac_set_desc_vlan(priv, first, STMMAC_VLAN_INSERT); - enh_desc = priv->plat->enh_desc; - /* To program the descriptors according to the size of the frame */ - if (enh_desc) - is_jumbo = stmmac_is_jumbo_frm(priv, skb->len, enh_desc); + des = dma_map_single(priv->device, skb->data, skb_headlen(skb), + DMA_TO_DEVICE); + if (dma_mapping_error(priv->device, des)) { + drop = true; + goto flush_ring; + } - if (unlikely(is_jumbo)) { - entry = stmmac_jumbo_frm(priv, tx_q, skb, csum_insertion); - if (unlikely(entry < 0) && (entry != -EINVAL)) { - drop = true; - goto flush_ring; - } + tx_q->tx_skbuff_dma[first_entry].buf = des; + tx_q->tx_skbuff_dma[first_entry].len = skb_headlen(skb); + tx_q->tx_skbuff_dma[first_entry].map_as_page = false; + tx_q->tx_skbuff_dma[first_entry].buf_type = STMMAC_TXBUF_T_SKB; + + stmmac_set_desc_addr(priv, first, des); + + nopaged_len = skb_headlen(skb); + descbuf_len = stmmac_get_tx_len(priv, priv->mode); + if (nopaged_len > descbuf_len) { + stmmac_tx_allocator(priv, des + descbuf_len, + nopaged_len - descbuf_len, + (nfrags == 0), queue); + nopaged_len = descbuf_len; } for (i = 0; i < nfrags; i++) { const skb_frag_t *frag = &skb_shinfo(skb)->frags[i]; - int len = skb_frag_size(frag); - bool last_segment = (i == (nfrags - 1)); - - entry = STMMAC_GET_ENTRY(entry, priv->dma_conf.dma_tx_size); - WARN_ON(tx_q->tx_skbuff[entry]); - - if (likely(priv->extend_desc)) - desc = (struct dma_desc *)(tx_q->dma_etx + entry); - else if (tx_q->tbs & STMMAC_TBS_AVAIL) - desc = &tx_q->dma_entx[entry].basic; - else - desc = tx_q->dma_tx + entry; - des = skb_frag_dma_map(priv->device, frag, 0, len, + des = skb_frag_dma_map(priv->device, frag, 0, + skb_frag_size(frag), DMA_TO_DEVICE); if (dma_mapping_error(priv->device, des)) { netdev_err(priv->dev, "Frag Tx DMA map failed\n"); @@ -4877,30 +4922,27 @@ static netdev_tx_t stmmac_xmit(struct sk_buff *skb, struct net_device *dev) goto flush_ring; /* should reuse desc w/o issues */ } - tx_q->tx_skbuff_dma[entry].buf = des; - - stmmac_set_desc_addr(priv, desc, des); - - tx_q->tx_skbuff_dma[entry].map_as_page = true; - tx_q->tx_skbuff_dma[entry].len = len; - tx_q->tx_skbuff_dma[entry].last_segment = last_segment; - tx_q->tx_skbuff_dma[entry].buf_type = STMMAC_TXBUF_T_SKB; + stmmac_tx_allocator(priv, des, skb_frag_size(frag), + (i == nfrags - 1), queue); - /* Prepare the descriptor and set the own bit too */ - stmmac_prepare_tx_desc(priv, desc, 0, len, csum_insertion, - priv->mode, 1, last_segment, skb->len); + tx_q->tx_skbuff_dma[tx_q->cur_tx].buf = des; + tx_q->tx_skbuff_dma[tx_q->cur_tx].len = skb_frag_size(frag); + tx_q->tx_skbuff_dma[tx_q->cur_tx].map_as_page = true; + tx_q->tx_skbuff_dma[tx_q->cur_tx].buf_type = STMMAC_TXBUF_T_SKB; } + tx_q->tx_skbuff_dma[tx_q->cur_tx].last_segment = true; + /* Only the last descriptor gets to point to the skb. */ - tx_q->tx_skbuff[entry] = skb; - tx_q->tx_skbuff_dma[entry].buf_type = STMMAC_TXBUF_T_SKB; + tx_q->tx_skbuff[tx_q->cur_tx] = skb; + tx_q->tx_skbuff_dma[tx_q->cur_tx].buf_type = STMMAC_TXBUF_T_SKB; /* According to the coalesce parameter the IC bit for the latest * segment is reset and the timer re-started to clean the tx status. * This approach takes care about the fragments: desc is the first * element in case of no SG. */ - tx_packets = (entry + 1) - first_tx; + tx_packets = (tx_q->cur_tx + 1) - first_tx; tx_q->tx_count_frames += tx_packets; if ((skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP) && priv->hwts_tx_en) @@ -4917,11 +4959,11 @@ static netdev_tx_t stmmac_xmit(struct sk_buff *skb, struct net_device *dev) if (set_ic) { if (likely(priv->extend_desc)) - desc = &tx_q->dma_etx[entry].basic; + desc = &tx_q->dma_etx[tx_q->cur_tx].basic; else if (tx_q->tbs & STMMAC_TBS_AVAIL) - desc = &tx_q->dma_entx[entry].basic; + desc = &tx_q->dma_entx[tx_q->cur_tx].basic; else - desc = &tx_q->dma_tx[entry]; + desc = &tx_q->dma_tx[tx_q->cur_tx]; tx_q->tx_count_frames = 0; stmmac_set_tx_ic(priv, desc); @@ -4932,14 +4974,13 @@ static netdev_tx_t stmmac_xmit(struct sk_buff *skb, struct net_device *dev) * ndo_start_xmit will fill this descriptor the next time it's * called and stmmac_tx_clean may clean up to this descriptor. */ - entry = STMMAC_GET_ENTRY(entry, priv->dma_conf.dma_tx_size); - tx_q->cur_tx = entry; + tx_q->cur_tx = STMMAC_GET_ENTRY(tx_q->cur_tx, priv->dma_conf.dma_tx_size); if (netif_msg_pktdata(priv)) { netdev_dbg(priv->dev, - "%s: curr=%d dirty=%d f=%d, e=%d, first=%p, nfrags=%d", + "%s: curr=%d dirty=%d entry=%d, first=%p, nfrags=%d", __func__, tx_q->cur_tx, tx_q->dirty_tx, first_entry, - entry, first, nfrags); + first, nfrags); netdev_dbg(priv->dev, ">>> frame to be transmitted: "); print_pkt(skb->data, skb->len); @@ -4960,41 +5001,11 @@ static netdev_tx_t stmmac_xmit(struct sk_buff *skb, struct net_device *dev) if (priv->sarc_type) stmmac_set_desc_sarc(priv, first, priv->sarc_type); - /* Ready to fill the first descriptor and set the OWN bit w/o any - * problems because all the descriptors are actually ready to be - * passed to the DMA engine. - */ - if (likely(!is_jumbo)) { - bool last_segment = (nfrags == 0); - - des = dma_map_single(priv->device, skb->data, - nopaged_len, DMA_TO_DEVICE); - if (dma_mapping_error(priv->device, des)) { - netdev_err(priv->dev, "First Tx DMA map failed\n"); - drop = true; - goto flush_ring; - } - - tx_q->tx_skbuff_dma[first_entry].buf = des; - tx_q->tx_skbuff_dma[first_entry].buf_type = STMMAC_TXBUF_T_SKB; - tx_q->tx_skbuff_dma[first_entry].map_as_page = false; - - stmmac_set_desc_addr(priv, first, des); - - tx_q->tx_skbuff_dma[first_entry].len = nopaged_len; - tx_q->tx_skbuff_dma[first_entry].last_segment = last_segment; - - if (unlikely((skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP) && - priv->hwts_tx_en)) { - /* declare that device is doing timestamping */ - skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS; - stmmac_enable_tx_timestamp(priv, first); - } - - /* Prepare the first descriptor setting the OWN bit too */ - stmmac_prepare_tx_desc(priv, first, 1, nopaged_len, - csum_insertion, priv->mode, 0, last_segment, - skb->len); + if (unlikely((skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP) && + priv->hwts_tx_en)) { + /* declare that device is doing timestamping */ + skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS; + stmmac_enable_tx_timestamp(priv, first); } if (tx_q->tbs & STMMAC_TBS_EN) { @@ -5004,7 +5015,11 @@ static netdev_tx_t stmmac_xmit(struct sk_buff *skb, struct net_device *dev) stmmac_set_desc_tbs(priv, tbs_desc, ts.tv_sec, ts.tv_nsec); } - stmmac_set_tx_owner(priv, first); + /* Complete the first descriptor before granting the DMA */ + stmmac_prepare_tx_desc(priv, first, 1, nopaged_len, + csum_insertion, priv->mode, true, + tx_q->tx_skbuff_dma[first_entry].last_segment, + skb->len); skb_tx_timestamp(skb); @@ -5330,7 +5345,6 @@ static int stmmac_xdp_xmit_xdpf(struct stmmac_priv *priv, int queue, tx_q->tx_skbuff_dma[entry].map_as_page = false; tx_q->tx_skbuff_dma[entry].len = xdpf->len; tx_q->tx_skbuff_dma[entry].last_segment = true; - tx_q->tx_skbuff_dma[entry].is_jumbo = false; tx_q->xdpf[entry] = xdpf; |
