[PATCH v4 00/14] Introduce the lwIP network stack
Jerome Forissier
jerome.forissier at linaro.org
Mon Jul 1 11:58:22 CEST 2024
On 6/28/24 17:48, Tim Harvey wrote:
> On Fri, Jun 28, 2024 at 2:50 AM Jerome Forissier
> <jerome.forissier at linaro.org> wrote:
>>
>> Hi Tim,
>>
>> On 6/26/24 18:00, Tim Harvey wrote:
>>> On Tue, Jun 25, 2024 at 1:02 AM Jerome Forissier
>>> <jerome.forissier at linaro.org> wrote:
>>>>
>>>> On 6/25/24 00:28, Tim Harvey wrote:
>>>>> On Sat, Jun 22, 2024 at 1:09 AM Maxim Uvarov <muvarov at gmail.com> wrote:
>>>>>>
>>>>>> пт, 21 июн. 2024 г. в 21:42, Fabio Estevam <festevam at gmail.com>:
>>>>>>>
>>>>>>> Hi Tim and Jerome,
>>>>>>>
>>>>>>> On Fri, Jun 21, 2024 at 1:08 PM Tim Harvey <tharvey at gateworks.com> wrote:
>>>>>>>
>>>>>>>> I tried your to-upstream/v5-wip branch
>>>>>>>> (042bea36eb9731079a3d7afffe3774d79e06ac5d) and it behaves the same. Do
>>>>>>>> you have something else to try/test?
>>>>>>>
>>>>>>> Yes, when I tested older versions from Maxim I could never get lwIP to
>>>>>>> work with i.MX.
>>>>>>>
>>>>>>> Jerome,
>>>>>>>
>>>>>>> Please try to run the lwIP series on any i.MX board, if possible.
>>>>>>>
>>>>>>> Thanks
>>>>>>
>>>>>> Packet not for us means that incoming packet DST MAC does not match to
>>>>>> the MAC address inside lwip. I.e. to MAC address set into lwip when
>>>>>> lwip_init was done. Original U-Boot network stack does not compare
>>>>>> MACs but lwip does. There is something specific on this board, in
>>>>>> general lwip with debug should print out
>>>>>> MAC used during initialization. This MAC should match to the MAC from
>>>>>> the incoming packet.
>>>>>>
>>>>>
>>>>> It seems 'packet not for us' can mean a lot of things.
>>>>
>>>> Yeah :-/ in this case I believe the traces are caused by unrelated traffic
>>>> (UDP port 138 is used by NetBIOS).
>>>>
>>>>
>>>>> I added a bit of debugging around 'DHCP packet accepted' and found I'm
>>>>> not receiving any packets from my DHCP server. So I connected directly
>>>>> to another board (isolated network) where I ran my own server and
>>>>> tcpdump and I don't see packets coming from lwip:
>>>>
>>>> Ha! That's interesting.
>>>>
>>>>> without lwip my server shows:
>>>>> # tcpdump -i eth1
>>>>> tcpdump: listening on eth1, link-type EN10MB (Ethernet), snapshot
>>>>> length 262144 bytes
>>>>> tcpdump: listening on eth1, link-type EN10MB (Ethernet), snapshot
>>>>> length 262144 bytes
>>>>> 22:18:01.043992 IP (tos 0x0, ttl 255, id 23, offset 0, flags [DF],
>>>>> proto UDP (17), length 328)
>>>>> 22:18:01.044391 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none],
>>>>> proto UDP (17), length 328)
>>>>> 22:18:01.044454 IP (tos 0x0, ttl 255, id 24, offset 0, flags [DF],
>>>>> proto UDP (17), length 328)
>>>>> 22:18:01.044752 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none],
>>>>> proto UDP (17), length 328)
>>>>>
>>>>> # tcpdump -i eth1
>>>>> tcpdump: listening on eth1, link-type EN10MB (Ethernet), snapshot
>>>>> length 262144 bytes
>>>>> 22:22:41.602178 [|llc]
>>>>> 22:22:41.709978 [|llc]
>>>>> 22:22:41.867947 [|llc]
>>>>> 22:22:42.105202 [|llc]
>>>>> 22:22:42.502091 [|llc]
>>>>> 22:22:43.219079 [|llc]
>>>>> 22:22:44.497979 [|llc]
>>>>> 22:22:45.776884 [|llc]
>>>>> 22:22:47.054773 [|llc]
>>>>> 22:22:48.333667 [|llc]
>>>>> 22:22:49.611559 [|llc]
>>>>> 22:22:50.890469 [|llc]
>>>>>
>>>>> What actual hardware has this been tested with? I think Tom mentioned
>>>>> he tested with an rpi of some sort.
>>>>
>>>> Yes, I believe he tested on RPi 3B and me too. Ilias has tested on NVIDIA
>>>> Jetson Nano.
>>>>
>>>>> I don't know what the meaning of the llc msg is above but you can see
>>>>> there is no DHCP request.
>>>>
>>>> Some more tracing is needed. Can you try enabling all traces in
>>>> lib/lwip/u-boot/lwipopts.h? i.e., replace all LWIP_DBG_OFF with
>>>> LWIP_DBG_ON in the #if defined(CONFIG_LWIP_DEBUG) section. And of course
>>>> enabled CONFIG_LWIP_DEBUG.
>>>
>>> Here's what I see with all those enabled on an imx8mm-venice-gw73xx-0x:
>>> Net: eth0: ethernet at 30be0000 [PRIME]
>>> GSC : boot watchdog disabled
>>> Thermal protection:enabled at 96C
>>> Hit any key to stop autoboot: 0
>>> u-boot=> dhcp || echo fail
>>> netif: added interface et IP addr 0.0.0.0 netmask 0.0.0.0 gw 0.0.0.0
>>> netif: setting default interface et
>>> dhcp_start(netif=00000000fdf23bb0) et0
>>> dhcp_start(): mallocing new DHCP client
>>> dhcp_start(): allocated dhcp
>>> dhcp_start(): starting DHCP configuration
>>> udp_bind(ipaddr = 0.0.0.0, port = 68)
>>> udp_bind: bound to 0.0.0.0, port 68)
>>> udp_connect: connected to 0.0.0.0, port 67)
>>> dhcp_discover()
>>> pbuf_alloc(length=308)
>>> pbuf_alloc(length=308) == 00000000fdf23d10
>>> transaction id xid(42021)
>>> dhcp_discover: making request
>>> dhcp_discover: sendto(DISCOVER, IP_ADDR_BROADCAST, LWIP_IANA_PORT_DHCP_SERVER)
>>> pbuf_add_header: old 00000000fdf23d5e new 00000000fdf23d56 (8)
>>> udp_send: added header in given pbuf 00000000fdf23d10
>>> udp_send: sending datagram of length 316
>>> udp_send: UDP packet length 316
>>> inet_chksum_pseudo(): checksumming pbuf 00000000fdf23d10 (has next
>>> 0000000000000000)
>>> inet_chksum_pseudo(): pbuf chain lwip_chksum()=807
>>> udp_send: UDP checksum 0xf7f8
>>> udp_send: ip_output_if (,,,,0x11,)
>>> pbuf_add_header: old 00000000fdf23d56 new 00000000fdf23d42 (20)
>>> ip4_output_if: et0
>>> IP header:
>>> +-------------------------------+
>>> | 4 | 5 | 0x00 | 336 | (v, hl, tos, len)
>>> +-------------------------------+
>>> | 0 |000| 0 | (id, flags, offset)
>>> +-------------------------------+
>>> | 255 | 17 | 0xba9d | (ttl, proto, chksum)
>>> +-------------------------------+
>>> | 0 | 0 | 0 | 0 | (src)
>>> +-------------------------------+
>>> | 255 | 255 | 255 | 255 | (dest)
>>> +-------------------------------+
>>> ip4_output_if: call netif->output()
>>> pbuf_add_header: old 00000000fdf23d42 new 00000000fdf23d34 (14)
>>> ethernet_output: sending packet 00000000fdf23d10
>>> ^^^ at this point a 350 byte packet gets sent to fecmxc_send for the
>>> NIC which looks like this via print_hex_dump_bytes():
>>> 00000000: ff ff ff ff ff ff 00 d0 12 ba f8 cc 08 00 45 00 ..............E.
>>> 00000010: 01 50 00 00 00 00 ff 11 ba 9d 00 00 00 00 ff ff .P..............
>>> 00000020: ff ff 00 44 00 43 01 3c f8 f7 01 01 06 00 00 04 ...D.C.<........
>>> 00000030: 20 21 00 00 00 00 00 00 00 00 00 00 00 00 00 00 !..............
>>> 00000040: 00 00 00 00 00 00 00 d0 12 ba f8 cc 00 00 00 00 ................
>>> 00000050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 00000060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 00000070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 00000080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 00000090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 000000a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 000000b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 000000c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 000000d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 000000e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 000000f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 00000100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 00000110: 00 00 00 00 00 00 63 82 53 63 35 01 01 39 02 05 ......c.Sc5..9..
>>> 00000120: dc 37 03 01 03 1c ff 00 00 00 00 00 00 00 00 00 .7..............
>>> 00000130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 00000140: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 00000150: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ..............
>>
>> Up to this point everything *seems* OK
>>
>>> on my dhcp server 'tcpdump -i eth1 -X -vvv' shows:
>>> tcpdump: listening on eth1, link-type EN10MB (Ethernet), snapshot
>>> length 262144 bytes
>>> 15:34:25.341167 [|llc]
>>> 0x0000: 0000 0000 0000 0000 0000 343d f2fd 0000 ..........4=....
>>> 0x0010: 0000 5e01 5e01 8000 0100 0000 0000 0000 ..^.^...........
>>> 0x0020: 0000 0000 0000 ffff ffff ffff 00d0 12ba ................
>>> 0x0030: f8cc 0800 4500 0150 001e 0000 ff11 ba7f ....E..P........
>>> 0x0040: 0000 0000 ffff ffff 0044 0043 013c 6d78 .........D.C.<mx
>>> 0x0050: 0101 0600 1255 994f 0000 0000 0000 0000 .....U.O........
>>> 0x0060: 0000 0000 0000 0000 0000 0000 00d0 12ba ................
>>> 0x0070: f8cc 0000 0000 0000 0000 0000 0000 0000 ................
>>> 0x0080: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>> 0x0090: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>> 0x00a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>> 0x00b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>> 0x00c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>> 0x00d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>> 0x00e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>> 0x00f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>> 0x0100: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>> 0x0110: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>> 0x0120: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>> 0x0130: 0000 0000 0000 0000 0000 0000 6382 5363 ............c.Sc
>>> 0x0140: 3501 0139 0205 dc37 0301 031c ff00 0000 5..9...7........
>>
>> But here it shows the DHCP packet is not received.
>>
>>> Comparing to w/o LWIP:
>>> u-boot=> dhcp || echo fail
>>> BOOTP broadcast 1
>>> fecmxc_send ethernet at 30be0000 342
>>> 00000000: ff ff ff ff ff ff 00 d0 12 ba f8 cc 08 00 45 00 ..............E.
>>> 00000010: 01 48 00 00 40 00 ff 11 7a a5 00 00 00 00 ff ff .H.. at ...z.......
>>> 00000020: ff ff 00 44 00 43 01 34 00 00 01 01 06 00 12 bb ...D.C.4........
>>> 00000030: 82 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .d..............
>>> 00000040: 00 00 00 00 00 00 00 d0 12 ba f8 cc 00 00 00 00 ................
>>> 00000050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 00000060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 00000070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 00000080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 00000090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 000000a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 000000b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 000000c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 000000d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 000000e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 000000f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 00000100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 00000110: 00 00 00 00 00 00 63 82 53 63 35 01 01 39 02 02 ......c.Sc5..9..
>>> 00000120: 40 5d 02 00 16 5e 03 01 00 00 3c 0c 55 2d 42 6f @]...^....<.U-Bo
>>> 00000130: 6f 74 2e 61 72 6d 76 38 37 05 01 03 06 0c 11 ff ot.armv87.......
>>> 00000140: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> 00000150: 00 00 00 00 00 00 ......
>>> ^^^ Note the packet is 8 bytes shorter
>>
>> Yes but that's just padding IIUC.
>>
>>> and on the dhcp server:
>>> tcpdump: listening on eth1, link-type EN10MB (Ethernet), snapshot
>>> length 262144 bytes
>>> 15:45:12.548077 IP (tos 0x0, ttl 255, id 0, offset 0, flags [DF],
>>> proto UDP (17), length 328)
>>> 0.0.0.0.bootpc > 255.255.255.255.bootps: [no cksum] BOOTP/DHCP,
>>> Request from 00:d0:12:ba:f8:cc (oui Unknown), length 300, xid
>>> 0x12bb8264, Flags [none] (0x0000)
>>> Client-Ethernet-Address 00:d0:12:ba:f8:cc (oui Unknown)
>>> Vendor-rfc1048 Extensions
>>> Magic Cookie 0x63825363
>>> DHCP-Message (53), length 1: Discover
>>> MSZ (57), length 2: 576
>>> ARCH (93), length 2: 22
>>> NDI (94), length 3: 1.0.0
>>> Vendor-Class (60), length 12: "U-Boot.armv8"
>>> Parameter-Request (55), length 5:
>>> Subnet-Mask (1), Default-Gateway (3), Domain-Name-Server (6),
>>> Hostname (12)
>>> RP (17)
>>> END (255), length 0
>>> PAD (0), length 0, occurs 22
>>> 0x0000: 4500 0148 0000 4000 ff11 7aa5 0000 0000 E..H.. at ...z.....
>>> 0x0010: ffff ffff 0044 0043 0134 0000 0101 0600 .....D.C.4......
>>> 0x0020: 12bb 8264 0000 0000 0000 0000 0000 0000 ...d............
>>> 0x0030: 0000 0000 0000 0000 00d0 12ba f8cc 0000 ................
>>> 0x0040: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>> 0x0050: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>> 0x0060: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>> 0x0070: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>> 0x0080: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>> 0x0090: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>> 0x00a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>> 0x00b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>> 0x00c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>> 0x00d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>> 0x00e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>> 0x00f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>> 0x0100: 0000 0000 0000 0000 6382 5363 3501 0139 ........c.Sc5..9
>>> 0x0110: 0202 405d 0200 165e 0301 0000 3c0c 552d ..@]...^....<.U-
>>> 0x0120: 426f 6f74 2e61 726d 7638 3705 0103 060c Boot.armv87.....
>>> 0x0130: 11ff 0000 0000 0000 0000 0000 0000 0000 ................
>>> 0x0140: 0000 0000 0000 0000 ........
>>
>> All good there, as expected.
>>
>>>> I do have a i.MX8M Plus EVK board but will not be able to use it before
>>>> tomorrow.
>>>
>>> Please let me know your findings. I can test imx8mp as well but would
>>> have to reconfigure everything.
>>
>> So I did some tests on my imx8mp and I got DHCP to work on the second
>> ethernet port (ENET2) but *not* on the first one (ENET1). They appear to
>> be using different drivers (drivers/net/fec_mxc.c for ENET1 and
>> drivers/net/dwc_eth_qos.c for ENET2). No matter what I do, there is no
>> packet going out of ENET1.
>>
>> With a network cable plugged into ENET2 (ethernet at 30bf0000):
>>
>> ...
>> Net: eth0: ethernet at 30be0000, eth1: ethernet at 30bf0000 [PRIME]
>> Hit any key to stop autoboot: 0
>> u-boot=>
>> u-boot=> dhcp
>> ethernet at 30bf0000 Waiting for PHY auto negotiation to complete....... done
>> DHCP client bound to address 192.168.0.39 (2066 ms)
>> u-boot=>
>>
>> Now moving the cable to ENET1 (ethernet at 30be0000) and resetting the board:
>>
>> ...
>> Net: eth0: ethernet at 30be0000, eth1: ethernet at 30bf0000 [PRIME]
>> Hit any key to stop autoboot: 0
>> u-boot=>
>> u-boot=> setenv ethact ethernet at 30be0000
>> u-boot=> dhcp
>> u-boot=> echo $?
>> 1
>>
>> I enabled the debug traces in fecmxc_send() but I see nothing wrong,
>> the function is called with a packet and it returns success yet nothing
>> is sent out on the wire. Any idea what could be wrong?
>> My plan is to add more traces to the driver and compare NET vs. NET_LWIP.
>>
>
> Jerome,
>
> That is consistent with what I see then. The imx8mm (and all imx all
> the way back to imx28) use the FEC ethernet MAC so I would assume you
> see the issue on all of those SOC's. The imx8mp introduced a 2nd
> different MAC, the eqos which you are showing working.
>
> If you look at my results above a packet 'is' received by the DHCP
> server but its missing something where tcpdump shows an '[|llc]' and I
> don't know what that means - something wrong with the link layer?
>
> I feel like something is wrong in the header or packet alignment.
Good catch :) Here is a quick workaroud that works for me:
diff --git a/drivers/net/fec_mxc.c b/drivers/net/fec_mxc.c
index 2dc1364beec..85bf1a2b52e 100644
--- a/drivers/net/fec_mxc.c
+++ b/drivers/net/fec_mxc.c
@@ -658,7 +658,7 @@ static void fecmxc_halt(struct udevice *dev)
* @param[in] length Data count in bytes
* Return: 0 on success
*/
-static int fecmxc_send(struct udevice *dev, void *packet, int length)
+static int fecmxc_send_(struct udevice *dev, void *packet, int length)
{
unsigned int status;
u32 size;
@@ -796,6 +796,21 @@ out:
return ret;
}
+static int fecmxc_send(struct udevice *dev, void *packet, int length)
+{
+ void *pp;
+ int ret;
+
+ pp = memalign(64, length);
+ if (!pp)
+ return -ENOMEM;
+ memcpy(pp, packet, length);
+ ret = fecmxc_send_(dev, pp, length);
+ free(pp);
+
+ return ret;
+}
+
/**
* Pull one frame from the card
* @param[in] dev Our ethernet device to handle
I believe the root of the problem is that lwIP uses malloc() to allocate
the outgoing packets and doesn't enforce any alignement. I need to
revisit that wrt. PKTALIGN.
Thanks,
--
Jerome
> Maybe the key is to understand what the tcpdump output of '[llc]'
> means:
> - lwip fec mac destination tcpdump shows '15:34:25.341167 [|llc]'
> - no-lwip fec mac destination tcptdump shows '15:45:12.548077 IP (tos
> 0x0, ttl 255, id 0, offset 0, flags [DF], proto UDP (17), length 328)
>
> (Adding Joe and Ramon, the network maintainers to cc)
>
> Regards,
>
> Tim
More information about the U-Boot
mailing list