[U-Boot] dfu: dfu and UBI Volumes

[Benoît Thébaudeau]

Dear Pantelis Antoniou,

On Tuesday, May 28, 2013 5:05:12 PM, Pantelis Antoniou wrote:
> Hi Tom,
> 
> On May 28, 2013, at 6:01 PM, Tom Rini wrote:
> 
> > On Tue, May 28, 2013 at 07:50:46AM +0200, Wolfgang Denk wrote:
> >> Dear Tom,
> >> 
> >> In message <20130527233735.GZ17119 at bill-the-cat> you wrote:
> >>> 
> >>>> Where exactly is this 8 MB limit coming into play?
> >>> 
> >>> In buffering the data.  We cannot write a chunk of a file to a
> >>> filesystem and then append to it, we don't have the API today.
> >> 
> >> Sorry, I still don't get it. Assuming I have a GiB of RAM, why can I
> >> not load a 256 MiB file to RAM, and then write it to a file system?
> >> 
> >> I have definitely dealt with images and files bigger than 8 MiB in
> >> thepast, so I really don't see where any buffer problem could be.
> > 
> > I thought I might not have been clear about where this limit comes from,
> > after I sent the email.  The problem we have, and this is only for
> > writing to a filesystem (_not_ writing of a filesystem) is that we do
> > not have the API for appending to files, only create/overwrite.  So we
> > must read the whole file into memory, and then write it out.  The DFU
> > protocol doesn't have (I would swear anyhow) a part where it says "I'm
> > about to send you a blob of X bytes", so we cannot know at the start how
> > much data is coming our way.
> > 
> > Today we "solve" this with a statically defined
> > CONFIG_SYS_DFU_MAX_FILE_SIZE.  Looking at things again, I think this is
> > buggy right now in that we need to also whack DFU_DATA_BUF_SIZE to also
> > be that same value.  Going forward, we may be able to switch this to
> > (and both of these are off the top of my head) a getenv to see how much
> > space to malloc, or just making it a malloc and adding some compile-time
> > check to ensure that the malloc area is at least as big as
> > CONFIG_SYS_DFU_MAX_FILE_SIZE.
> > 
> 
> Correct, the DFU protocol doesn't have a method to inform you before hand
> about the size of the transfer about to happen.
> 
> The only possible solution I see at this point is to have an environment
> variable, i.e. dfubuf that controls the size of the buffer.
> 
> Upon start of a dfu transfer we can allocate the buffer, and do our
> thing.

I don't know the details of the DFU implementation in U-Boot, but the
specification leaves the choice between programming the firmware on-the-fly
during the download, and later during the manifestation phase (or a mix of
both). Hence, there is not need for a global firmware buffer if U-Boot goes for
the on-the-fly programming strategy. The only buffer constraint would be
wTransferSize (chosen by U-Boot for the control endpoint) in that case. See
"7. Manifestation Phase" on page 26 here:
http://www.usb.org/developers/devclass_docs/DFU_1.1.pdf

Of course this can't yet apply to writing files on file systems since the
current API in U-Boot misses the append feature, but this could be applied to
program raw memory partitions, including UBI images.

Best regards,
Benoît