[PATCHv8 00/15] net/lwip: add lwip library for the network stack

[Simon Goldschmidt]

On 13.09.2023 09:53, Ilias Apalodimas wrote:
> Hi Maxim,
>
> On Wed, 13 Sept 2023 at 10:32, Maxim Uvarov <maxim.uvarov at linaro.org> wrote:
>>
>>
>>
>> On Wed, 13 Sept 2023 at 01:27, Simon Glass <sjg at google.com> wrote:
>>>
>>> Hi Maxim,
>>>
>>> On Tue, 12 Sept 2023 at 05:42, Maxim Uvarov <maxim.uvarov at linaro.org> wrote:
>>>>
>>>> On Fri, 8 Sept 2023 at 19:59, Tom Rini <trini at konsulko.com> wrote:
>>>>
>>>>> On Fri, Sep 08, 2023 at 07:53:05PM +0600, Maxim Uvarov wrote:
>>>>>
>>>>>> Before apply these patches it  is needed to create lwIP merge into
>>>>> U-Boot:
>>>>>> git subtree add --prefix net/lwip/lwip-external
>>>>> https://git.savannah.nongnu.org/git/lwip.git master --squash
>>>>>> or
>>>>>> create git submodule, depends how it's more easy to maintain external
>>>>>> library.
>>>>>
>>>>> So, I think we're going to go with subtree.  Please work out how to
>>>>> integrate the above in to the build process automatically (and such that
>>>>> we can maintain it via upgrades moving forward).
>>>>>
>>>>> --
>>>>> Tom
>>>>>
>>>>
>>>> I did not find a good way to friend git format-patch, git am and subtree.
>>>> And now with using subtree I can provide my thoughts, in general I do not
>>>> see any big advantages
>>>> with maintaining subtree code.
>>>>
>>>> Problem is that:
>>>>
>>>> 1. subtree does some root reset. So rebase looks like:
>>>> label onto
>>>>
>>>>
>>>>
>>>> # Branch acbc0469a49de7055141cc730aa9c728e61b6de2-2
>>>>
>>>> reset [new root]
>>>>
>>>> pick acbc0469a4 Squashed 'net/lwip/lwip-external/' content from commit
>>>> 84fde1ebbf
>>>> label acbc0469a49de7055141cc730aa9c728e61b6de2-2
>>>>
>>>>
>>>>
>>>> reset onto
>>>>
>>>> merge -C ec4a128c8d acbc0469a49de7055141cc730aa9c728e61b6de2-2 # Merge
>>>> commit 'acbc0469a49de7055141cc730aa9c728e61b6de2' as
>>>> 'net/lwip/lwip-external'
>>>> pick 739681a6f5 net/lwip: add doc/develop/net_lwip.rst
>>>>
>>>> pick f0ecab85e0 net/lwip: integrate lwIP library
>>>>
>>>> 2. if  --rebase-merges option was not provided to rebase, then rebase will
>>>> omit subtree directory and try to apply rebase patches to root directory.
>>>> I.e. in current case squashed commit for lwip, will be applied to uboot
>>>> root directory instead of ./net/lwip/lwip-external.
>>>>
>>>> 3. due to broken rebases without --rebase-merges more likely things like
>>>> git bisect also will not work.
>>>>
>>>> 4.  change in subtree code ./net/lwip/lwip-external/../something.c will
>>>> create a git commit which looks like a standard U-Boot commit. I.e. path
>>>> starts with ./net/lwip/lwip-external/
>>>
>>> I don't really understand most of the above, but I take it that
>>> subtree has some problems...I did find an article about subtree [1]
>>>
>>>>
>>>> 5. lwip maintains code with a mailing list.  So we don't need to push
>>>> subtree git somewhere to create a pull request.
>>>>
>>>> 6. I rechecked the latest edk2 and they use submodules now. (openssl,
>>>> libfdt, berkeley-softfloat-3 and others).
>>>>
>>>> 7. dynamic download also looks horrible for me. I.e. create subtree in
>>>> Makefile on compilation process. I think maintaining that will give you a
>>>> bunch of problems. I think we should not touch git structure after cloning.
>>>>
>>>> So what I can here suggest:
>>>> 1.  lwip source code is 9.4M.  If we compare all code it will be 564M in
>>>> total. So just having 1 commit witn copy of lwip library will work here.
>>>
>>> So we add a 9.4MB patch for the code we need? I suppose that is OK,
>>> although it is much larger than net/ today (0.5MB).
>>>
>>> What is the churn on lwip? E.g. would it be easy to add a commit every
>>> few months to bring in upstream changes?
>>>
>>>>
>>>> or
>>>>
>>>> 2. use git submodules. Size of the project will be lower.  Submodule will
>>>> not allow you to use local changes. I.e. change needs to be merged into the
>>>> upstream project and then you can update git HEAD for the submodule.
>>>
>>> I really don't want to work with a submodule project. I've just had
>>> too many problems.
>>>
>>>>
>>>> or
>>>>
>>>> 3. inside u-boot.git create branch lib-lwip and clone lwip repo there. Then
>>>> use git submoule to connect this branch as a folder to the main U-Boot code.
>>>
>>> It really needs to be properly part of U-Boot.
>>>
>>
>> Ok. Then more likely we don't need all the git history of lwip inside uboot.git. Then the option with a single commit is more preferable.
>> Then we can use part 2 of this article, how to  go with standard git commands:
>>
>> 1.
>> <cmd>
>> git remote add -f lwip https://git.savannah.nongnu.org/git/lwip.git
>> git read-tree --prefix=net/lwip/lwip-external -u lwip/master
>> git commit -m "lwip merge sha: xxxx"
>> </cmd>
>> this will create a git format-patch friendly commit. Then we send it to the mailing list  and apply.
>> I hope the mailing list will allow us to send a 7.8 MB patch.
>>
>>
>> Then if for development you need  to pull he history of lwip, you can do it with:
>> git pull -s subtree lwip  master --allow-unrelated-histories
>> (but I think nobody will need this.)
>>
>> New update of the lwip net/lwip/lwip-external dir will be done with:
>> git pull -s subtree lwip  master --allow-unrelated-histories --squash
>> Squash commit also has to be git format-patch friendly.
>>
>> If you are ok with that proposal I will send v9 with the first patch created with steps above.
>
> We've gone through this before.  The whole purpose of this is not
> having to maintain patches.
> Simon, instead of "I had problems in the past", can you elaborate a bit more?
>
> Tom said he is fine with subtrees instead of submodules and I know for
> a fact EDK2 doesn't have too many issues with submodules.
> Their documentation is pretty clear on building and requires
>
> git clone https://github.com/tianocore/edk2.git
> cd edk2
> git submodule update --init
>
> Perhaps the situation has improved since you had issues?

While I don't really care how you solve this technically, I'd *strongly*
be interested for U-Boot to use *unmodified* lwIP sources where an
explicit reference to an lwIP commit is used. I'd rather integrate
bugfixes for U-Boot into lwIP than having the sources drift apart.

Regards,
Simon

>
> Thanks
> /Ilias
>>
>> Thanks,
>> Maxim.
>>
>>
>>
>>
>>
>>
>>>
>>>>
>>>> BR,
>>>> Maxim.
>>>
>>> Regards,
>>> Simon
>>>
>>> [1] https://www.atlassian.com/git/tutorials/git-subtree