[lwip-devel] [RFC PATCH 0/5] LWIP stack integration
Ilias Apalodimas
ilias.apalodimas at linaro.org
Wed Jun 7 11:46:50 CEST 2023
Hi Maxim,
On Tue, 6 Jun 2023 at 17:33, Maxim Uvarov <maxim.uvarov at linaro.org> wrote:
>
> Greetings,
>
> I implemented the tftp client (that was quick due to lwip has example app for tftp), and did some more measurements.
> I uploaded patches here if somebody want to do his own measurements:
> https://github.com/muvarov/uboot-lwip
>
> measure 1:
> 976K - total (total means lwip with all 3 commands ping, tftp, wget)
> 971K - total - tftp (total, but disable/minus tftp)
> 965K - total - tftp - wget (disable tftp and wget)
> 963K - total - tftp - wget - ping (disable tftp, wget, ping)
> 931K - no lwip
>
> result 1: lwip tftp (+ udp protocol) protocol 976-971k = 5kb
> result 2: lwip ping command 965- 963 = 2kb
> result 3: lwip wget command 971- 965 = 6kb
> result 4: lwip core stack with apps 976 - 931 = 45kb
So tftp = 5kb, wget = 6kb ping =2kb and lwip = 32kb
>
> measure 2:
> 890K - no CONFIG_NET_CMD
> 930K - + lwip tftp only
> 937K - + full lwip (ping wget tftp)
>
> result 1: 937-890=47kb ( lwip + all 3 commands)
> result 2: 937-930=7kb (ping and lwip command)
I am not sure I understand this measurement. How is this different
from measurement 1 where the entire binary was 976K?
>
> measure 3:
> 904K - no lwip, CMD_NET_TFTP=y
> 900K - no lwip, CMD_NET_TFTP=n
> result 1: original u-boot tftp command 904-900=4kb
> 890K - no lwip, CMD_NET=n
> result 2: 900-890=10k original u-boot net/IP stack.
>
> My findings for all that measurements and lwip configuration:
> 1. The original u-boot net stack (packet process and up layers) is 10k vs lwip 40k (the very minimal settings were 30k).
> 2. Network applications size is about the same 4kb for tftp original command 5kb for lwip.
> 3. It's quite easy to reuse LWIP examples to implement the same functionality for the U-boot.
> 4. I still think that there are other criterias which might have more priority than size (bug free code, code reuse, development speed, compatible API to posix and etc).
Yes, there are other criteria and certainly having a complete network
stack might be worth it in many cases, but we need to keep in mind
30kb might be a lot for some systems.
I personally think this is decent and we can optimize lwip more in the
future. Tom, Simon, how about adding lwip as 'experimental' and
making it depend on !CMD_NET or something similar?
Thanks
/Ilias
>
> BR,
> Maxim.
>
> On Thu, 25 May 2023 at 02:18, Simon Goldschmidt <goldsimon at gmx.de> wrote:
>>
>> Hi Maxim, Tom,
>>
>> On 24.05.2023 16:05, Maxim Uvarov wrote:
>> > On Tue, 23 May 2023 at 03:23, Tom Rini <trini at konsulko.com> wrote:
>> >
>> >> On Mon, May 22, 2023 at 12:40:49PM -0400, Maxim Uvarov wrote:
>> >>> On Mon, 22 May 2023 at 10:20, Tom Rini <trini at konsulko.com> wrote:
>> >>>
>> >>>> On Mon, May 22, 2023 at 04:33:57PM +0300, Ilias Apalodimas wrote:
>> >>>>> Hi Maxim
>> >>>>>
>> >>>>> On Mon, 22 May 2023 at 12:01, Maxim Uvarov <maxim.uvarov at linaro.org>
>> >>>> wrote:
>> >>>>>>
>> >>>>>> My measurements for binary after LTO looks like:
>> >>>>>>
>> >>>>>> U-boot WGET | LWIP WGET + ping | LWIP WGET| diff bytes| diff %
>> >>>>>> 870728 | 915000 | 912560 |
>> >>>> 41832 | 4.8
>> >>>>>
>> >>>>>
>> >>>>> I think you'll need to analyze that a bit more. First of all I don't
>> >>>>> think the '+ping' tab is useful. What is is trying to achieve?
>> >>>>
>> >>>
>> >>> To show the difference of extra bytes if we add a ping app on top.
>> >>>
>> >>>
>> >>>>> - How was LWIP compiled?
>> >>>>
>> >>>
>> >>> It has a really huge configuration. I tried to turn off everything off
>> >>> everything what can impact on size but still make http app work:
>> >>> #define LWIP_HAVE_LOOPIF 0
>> >>> #define LWIP_NETCONN 0
>> >>> #define LWIP_SOCKET 0
>> >>> #define SO_REUSE 0
>> >>> #define LWIP_STATS 0
>> >>> #define PPP_SUPPORT 0
>> >>>
>> >>> Disabling loopback:
>> >>> #define LWIP_NETIF_LOOPBACK 0
>> >>> can lower to 912288 bytes.
>> >>>
>> >>> And it's the same compilation option (optimization for size) as the main
>> >>> u-boot. I will do more experiments, but I think the goal is not to turn
>> >> off
>> >>> everything.
>> >>>
>> >>>
>> >>>>> - Was ipv6 supported?
>> >>>>
>> >>>
>> >>> No. I.e. when I sent results it was enabled on the compilation stage but
>> >>> not used. I just checked that size remains the same if IPv6 is not even
>> >>> compiled.
>> >>>
>> >>>
>> >>>>> - Can we strip it down even further?
>> >>>>>
>> >>>>
>> >>>
>> >>> There is always room for optimization. I think I tried to turn off
>> >>> everything that is configurable with defines. I can play with disable IP
>> >>> reassembly and things like that or figure out which functions have more
>> >>> size and if it's possible to exclude them.
>> >>>
>> >>>
>> >>>>> In general please give as much information as you can with what we
>> >>>>> gain in functionality from LWIP with those extra bytes of code.
>> >>>>
>> >>>>
>> >>> The main idea is to reuse a maintainable IP stack outside of U-boot.
>> >> LWIP
>> >>> can give a nice separation between IP stack code and network application
>> >>> code. I.e. application should not take care about any TCP details (SYN,
>> >>> ACK, retransmission, reassembly etc) and should open connection and use
>> >>> functions similar to recv() and send() to transfer data. Data means
>> >>> application data, no network packets. And LWIP allows
>> >>> us to do that.
>> >>> Because LWIP has an API similar to sockets, it has to be very easy to
>> >> port
>> >>> a linux application to LWIP. Then you can test it with a tap device. Then
>> >>> copy sources to U-boot, add a small integration layer (cmd command to
>> >>> call), compile and use.
>> >>>
>> >>> So my suggestion was:
>> >>> - do not maintain new network stack code in the current U-boot. Use lwip
>> >>> sources as an external project. All bugs related to network stack go to
>> >>> lwip project first, then sync with U-boot.
>> >>> - maintain network apps code* or
>> >>> -- inside U-boot. Write our own code for application and maintain it
>> >>> inside U-boot.
>> >>> -- inside LWIP. Add examples to LWIP which are suitable for both
>> >> U-boot
>> >>> and LWIP.
>> >>>
>> >>> * Let's define a U-boot network application as a cmd command. It might be
>> >>> ping, wget (http or https download), telnet, arp dns etc..
>> >>>
>> >>> Let's consider the real use case, like HTTPS download client. We need to
>> >>> enable TLS connection, validate certificates, then do http download.
>> >>> Looking at the current code of wget command it's quite difficult to
>> >>> implement this due to the application having some protol level things. On
>> >>> the other side we can find embedTLS examples to do https download on
>> >>> sockets. If LWIP socket API is ported then the only thing you need to do
>> >> is
>> >>> change socket() -> lwip_socket(), recv()->lwip_recv(),
>> >> send()->lwip_send()
>> >>> and etc, even function names are similar. If LWIP socket API is not
>> >>> supported, then use callback API for recv() and send(), which are also
>> >>> easy.
>> >>>
>> >>> So yes we add extra bytes, but that will allow us to write more complex
>> >>> apps, use standard debug tools, use applications with very minimal
>> >>> integration changes, use help from the LWIP community to fix protocol
>> >> bugs,
>> >>> etc..
>> >>> Bunch of things already implemented there:
>> >>> - ipv6
>> >>> - dhcp
>> >>> - snmp
>> >>> - igmp
>> >>> - dns
>> >>> - tcp and udp and raw.
>> >>> - loopback
>> >>> - netconn
>> >>> - socket
>> >>> - stats
>> >>> - ppp
>> >>> (I just followed configurable defines).
>> >>>
>> >>>
>> >>> And please make sure to disable the previous support, my guess fro that
>> >>>> much growth is that you didn't.
>> >>>>
>> >>>
>> >>> # CONFIG_PROT_TCP is not set
>> >>> # CONFIG_PROT_UDP is not set
>> >>> # CONFIG_UDP_CHECKSUM is not set
>> >>> # CONFIG_UDP_FUNCTION_FASTBOOT is not set
>> >>> # CONFIG_CMD_WGET is not set
>> >>
>> >> I think you need to step back and figure out a better way to measure the
>> >> size change and growth.
>> >>
>> >> I am not interested in a path that long term means two networking stacks
>> >> in U-Boot.
>> >>
>> >> I am not interested in massively growing the overall binary size for
>> >> every platform. Given how much larger just TCP support is, that's
>> >> strongly implying a huge growth for the older use cases too.
>> >>
>> >> But I also suspect given the overall reputation that LWIP enjoys,
>> >> there's something amiss here.
>> >>
>> >> --
>> >> Tom
>> >>
>> >
>> > +cc lwip-devel@ mailing list, maybe they have something to add.
>>
>> I do think using lwIP instead of "inventing yet another IP stack" is a
>> good idea! However, in terms of code size, lwIP will lose against what's
>> in U-Boot at present. And this is only natural, as lwIP is a "full-size"
>> stack supporting multiple concurrently running applications while the
>> current IP stack in U-Boot is rather "crippled" down to just what the
>> implementor needed at the time of writing.
>>
>> One example of this is that (if I remember correctly), U-Boot only has
>> one single network packet buffer, while lwIP has support for multiple
>> buffers. When speaking of TCP (forgive me if I'm wrong, I've lost track
>> of that development in U-Boot about 3 years ago), we're comparing "we
>> have implemented everything we need so that it just kind of works" to
>> "we can easily add a HTTPS client to download something over the
>> internet just by enabling more compile options".
>>
>> Also, when comparing lwIP to U-Boot TCP code size, keep in mind that
>> U-Boot TCP (at least that of some years ago) is far from complete when
>> compared to lwIP!
>>
>> lwIP is meant to be highly configurable and we're always open to add yet
>> more options to leave out more code when it's not needed. However, I
>> think there are some design decisions that will make lwIP larger than
>> the current IP stack in U-Boot. To me, that's a natural result of having
>> a "generic code" approach vs "developed to our needs". However, while
>> DHCP + BOOTP and even a simple network console was rather easy to
>> implement, I would not recommend implementing your own HTTPS download
>> but rather using the existing lwIP + apps for that.
>>
>> In the end, I cannot take the decision from you. In my opinion, lwIP
>> would be the better decision in terms of future work load and
>> compatibility, but in the short run, it *will* lead to bigger binaries
>> at least in some setups. And I do know from my past that it sometimes
>> has been a pain to try and stuff a new U-Boot release into the existing
>> space of flash or RAM, so that's not an easy decision.
>>
>> If you do take the lwIP approach however, let us know if we can help!
>>
>> Regards,
>> Simon
>>
>> >
>> > My measurements say that the current U-boot IP stack + wget command adds an
>> > additional 9 Kbytes.
>> > The minimal configuration of LWIP with wget command is 30 Kbytes.
>> > (compiled out all asserts, debugs, not used protocols etc.).
>> >
>> > And the most bigger functions are tcp in/out itself:
>> > * These functions are generally called in the order (ip_input() ->)
>> > * tcp_input() -> * tcp_process() -> tcp_receive() (-> application).
>> >
>> > +tcp_input - 4364 +4364
>> > https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_in.c#n118
>> > +tcp_receive - 3444 +3444
>> > https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_in.c#n1154
>> > +tcp_write - 2192 +2192
>> > https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_out.c#n393
>> > +ip4_reass - 2096 +2096
>> > https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/ipv4/ip4_frag.c#n503
>> > +tcp_output - 1616 +1616
>> > https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_out.c#n1241
>> >
>> > If we transfer current net commands to lwip then we can decrease the size,
>> > because of functions reuse.
>> > And if we turn on all features in lwip it will be about 50 Kbytes.
>> >
>> > BR,
>> > Maxim.
>> >
>> >
>> > _______________________________________________
>> > lwip-devel mailing list
>> > lwip-devel at nongnu.org
>> > https://lists.nongnu.org/mailman/listinfo/lwip-devel
More information about the U-Boot
mailing list