[U-Boot] [PATCH v4 0/3] mtd, ubi, ubifs: resync with Linux-3.14

[Scott Wood]

On Thu, 2014-06-19 at 12:34 +0200, Heiko Schocher wrote:
> Hello Tom, Scott,
> 
> Am 18.06.2014 23:42, schrieb Tom Rini:
> > On Wed, Jun 18, 2014 at 04:24:39PM -0500, Scott Wood wrote:
> >> On Wed, 2014-06-18 at 09:55 +0200, Heiko Schocher wrote:
> >>> Hello Scott,
> > [snip]
> >>>> history to see what Linux version corresponded to the last mtd sync, and
> >>>> generate a diff relative to that.
> >>>
> >>> I think, we should have the linux code as the reference, and on top
> >>> of this, we should add our U-Boot changes ... Thats the reason
> >>> for the git tree [1].
> >>
> >> One of the strengths of version control is the ability to do three-way
> >> merges, rather than throw away the downstream code and reassemble it
> >> from scratch.
> >
> > Indeed.
> >
> > [snip]
> >>
> >> What does #ifndef __UBOOT__ tell you that a diff between the U-Boot and
> >> Linux files, as of the SHA1 that was last merged into U-Boot, doesn't
> >> tell you?
> >>
> >>> There we have the "simple copy from linux patch" [2]. It overwrites
> >>> the U-Boot specific changes, yes, but only in an intermediate step
> >>> in this git tree ... (Maybe I should add in the commit message, that
> >>> this is a simple copy from linux sources)
> >>>
> >>> Then a U-Boot specifc patch [3], which introduces again hopefully
> >>> all U-Boot specific changes... so *all* U-Boot specific changs are
> >>> documented. At last a seperate licencse file patch [4] added which
> >>> changes the license text.
> >>>
> >>> My hope was, that future linux mtd syncs could look like:
> >>>
> >>> a) copy new linux source files to u-boot tree [1] (make a new branch
> >>>      go back to commit [2] ... and copy new files from linux)
> >>>      (commit this, to have all linux changes since last sync
> >>>       with linux documented)
> >>> b) apply [3] and fix problems
> >>>      All U-Boot changes documented
> >>> c) apply [4] and fix problems
> >>> d) add newly added U-Boot specific patches
> >>>      (mark them with _UBOOT_ define if they are not yet marked with it)
> >>>      (maybe squash them to step b ?)
> >>>
> >>>      ->  new git tree ready
> >>>
> >>> e) squash all patches into one patch and send it to mainline ...
> >>>      so the mainline patch has only new changes with all U-Boot
> >>>      specific changes.
> >>
> >> If you take over as NAND custodian, you can of course do the updates as
> >> you please (unless Tom or Wolfgang disagree), but until then I insist on
> >> doing a proper merge, and not littering the code with #ifndef __UBOOT__.
> >>
> >>> Thats sounds easy ... and you have all linux and all U-Boot
> >>> changes visible and documented ... I must admit, I did not
> >>> tried your proposal to look into the linux changes since
> >>> the last mtd sync ... but that sounds the more tricky way
> >>> to get this sync, as the history can be long ... and shouldn;t
> >>> be the result the same?
> >>
> >> Why does the length of the history matter?  I'm not asking you to go
> >> patch by patch.  Just take a diff between the SHA1 of last merge and the
> >> SHA1 you want to merge now, and apply that patch.
> >
> > But it's not even that hard to do so!  git format-patch -o mtd-resync
> > v3.7..v3.14 drivers/mtd/nand will give you only the relevant patches,
> 
> "git format-patch -o mtd-resync v3.7..v3.14 drivers/mtd/nand"
> results in 440 patches ... :-(

This is why I suggested taking one big diff rather than going
patch-to-patch.  Then you don't have to resolve the same conflicts over
and over.

You should also exclude files that U-Boot doesn't have, and include
header files that are relevant.

> "git am" first patch immediately fails ...
> 
> Ok, one step back ... I try only "drivers/mtd/mtdcore.c":
> 
> $ git format-patch -o mtdcore v3.7..v3.14 drivers/mtd/mtdcore.c
> mtdcore/0001-mtd-convert-to-idr_alloc.patch
> mtdcore/0002-mtd-add-const-qualifier-to-a-couple-of-register-func.patch
> mtdcore/0003-mtd-mtdcore-remove-few-useless-ifdef-s.patch
> mtdcore/0004-mtd-merge-mtdchar-module-with-mtdcore.patch
> mtdcore/0005-Include-missing-linux-slab.h-inclusions.patch
> mtdcore/0006-drivers-avoid-format-string-in-dev_set_name.patch
> mtdcore/0007-mtd-add-a-new-sys-node-to-show-the-ecc-step-size.patch
> mtdcore/0008-mtd-add-MTD_MLCNANDFLASH-case-for-mtd_type_show.patch
> mtdcore/0009-mtd-Move-major-number-definitions-to-major.h.patch
> mtdcore/0010-mtd-remove-duplicated-include-from-mtdcore.c.patch
> mtdcore/0011-mtd-convert-to-use-ATTRIBUTE_GROUPS.patch
> $
> 
> Only 11 patches, great ... but:
> 
> $ git am -3 mtdcore/0001-mtd-convert-to-idr_alloc.patch
> Wende an: mtd: convert to idr_alloc()
> fatal: sha1 information is lacking or useless (drivers/mtd/mtdcore.c).

If you fetch the Linux tree into your U-boot repository, then -3 will
work.

> > and they should mostly apply, and then you can squash them after the
> > fact.
> 
> I try to try this ASAP and report (first steps see above ...). But as a
> lot of linux code is missing in U-Boot code, I feel, that a lot of chunks
> will not apply ... but maybe (hopefully) I am wrong ...

If the conflict is due to a change that applies to code that is missing
because it doesn't make sense in U-Boot, then just delete that conflict
and move on.  Unlike having the change silently apply to an ifndef
region, you can first evaluate the change to see if it should apply to
some alternative U-Boot version of the code.

> Thats the question we must solve, do we want the complete Linux MTD code
> in U-Boot and mark not used code?

No.

> If no, I delete this pieces of code...
> 
> Hmm... looking in the history of drivers/mtd/ubi it seems the code
> is basically from 2008!
> http://git.denx.de/?p=u-boot.git;a=history;f=drivers/mtd/ubi;hb=7d2357c1999ff1f93f795282526230a8bd176106
> 
> Hmm.. what was the Linux base of this ?

See commit dfe64e2c89731a3f9950d7acd8681b68df2bae03 "mtd: resync with
Linux-3.7.1"

> and ubifs from 2009?
> http://git.denx.de/?p=u-boot.git;a=history;f=fs/ubifs;hb=7d2357c1999ff1f93f795282526230a8bd176106
> Base: "2.6.29-rc6" ... uh, very, very old!
> 
> I see there no real syncs with newer linux code ... is this true?
> Just some fixups ...

CCing Kyungmin who is the UBI maintainer.

> I really fear a patch "git format-patch -o ubifs-resync 2.6.29-rc6..v3.14 fs/ubifs"
> (The result from this command are 355 patches ...)
> 
> So, in sum, this are 795 patches + missing ubi patches which must be
> applied/reworked/reviewed... Sounds painful...

I don't know about ubifs, but the core NAND code (and apparently the
core UBI code) is currently synced with Linux 3.7.1.

> So another question is:
> How to generate an update-sync patch:
> 
> version a):
> - copy the new linux files from Linux v3.14 to U-Boot
> - work in the U-Boot specific changes
> - make a patch from this
> 
> Is this really different to:
> 
> version b):
> - git format-patch -o mtd-resync v3.7..v3.14 drivers/mtd/nand
> - apply them
> - make a patch from this

It is different because "work in the U-Boot specific changes" is a
manual, error-prone step compared to letting git do most of the work.

> Currently it is differrent, as I introduced the complete linux code
> and marked unused/different code in U-Boot with __UBOOT__. If I delete
> this pieces ... version a) and b) should be the same.
> 
> I try it ... if I not give up due to count of patches ...

Again, there's no need to go patch-by-patch.  We haven't done so on
prior updates.

-Scott