[U-Boot] dfu: dfu and UBI Volumes

[Pantelis Antoniou]

Hi Benoît

On May 28, 2013, at 7:31 PM, Benoît Thébaudeau wrote:

> Dear Pantelis Antoniou,
> 
> On Tuesday, May 28, 2013 5:05:12 PM, Pantelis Antoniou wrote:
>> Hi Tom,
>> 
>> On May 28, 2013, at 6:01 PM, Tom Rini wrote:
>> 
>>> On Tue, May 28, 2013 at 07:50:46AM +0200, Wolfgang Denk wrote:
>>>> Dear Tom,
>>>> 
>>>> In message <20130527233735.GZ17119 at bill-the-cat> you wrote:
>>>>> 
>>>>>> Where exactly is this 8 MB limit coming into play?
>>>>> 
>>>>> In buffering the data.  We cannot write a chunk of a file to a
>>>>> filesystem and then append to it, we don't have the API today.
>>>> 
>>>> Sorry, I still don't get it. Assuming I have a GiB of RAM, why can I
>>>> not load a 256 MiB file to RAM, and then write it to a file system?
>>>> 
>>>> I have definitely dealt with images and files bigger than 8 MiB in
>>>> thepast, so I really don't see where any buffer problem could be.
>>> 
>>> I thought I might not have been clear about where this limit comes from,
>>> after I sent the email.  The problem we have, and this is only for
>>> writing to a filesystem (_not_ writing of a filesystem) is that we do
>>> not have the API for appending to files, only create/overwrite.  So we
>>> must read the whole file into memory, and then write it out.  The DFU
>>> protocol doesn't have (I would swear anyhow) a part where it says "I'm
>>> about to send you a blob of X bytes", so we cannot know at the start how
>>> much data is coming our way.
>>> 
>>> Today we "solve" this with a statically defined
>>> CONFIG_SYS_DFU_MAX_FILE_SIZE.  Looking at things again, I think this is
>>> buggy right now in that we need to also whack DFU_DATA_BUF_SIZE to also
>>> be that same value.  Going forward, we may be able to switch this to
>>> (and both of these are off the top of my head) a getenv to see how much
>>> space to malloc, or just making it a malloc and adding some compile-time
>>> check to ensure that the malloc area is at least as big as
>>> CONFIG_SYS_DFU_MAX_FILE_SIZE.
>>> 
>> 
>> Correct, the DFU protocol doesn't have a method to inform you before hand
>> about the size of the transfer about to happen.
>> 
>> The only possible solution I see at this point is to have an environment
>> variable, i.e. dfubuf that controls the size of the buffer.
>> 
>> Upon start of a dfu transfer we can allocate the buffer, and do our
>> thing.
> 
> I don't know the details of the DFU implementation in U-Boot, but the
> specification leaves the choice between programming the firmware on-the-fly
> during the download, and later during the manifestation phase (or a mix of
> both). Hence, there is not need for a global firmware buffer if U-Boot goes for
> the on-the-fly programming strategy. The only buffer constraint would be
> wTransferSize (chosen by U-Boot for the control endpoint) in that case. See
> "7. Manifestation Phase" on page 26 here:
> http://www.usb.org/developers/devclass_docs/DFU_1.1.pdf
> 

The problem is not DFU TBH, it's that since we don't have an option to append
to a file, we have to have the whole file transferred in RAM and written in
one go. The raw medium dfu methods in u-boot don't have a problem.

> Of course this can't yet apply to writing files on file systems since the
> current API in U-Boot misses the append feature, but this could be applied to
> program raw memory partitions, including UBI images.
> 

It already happens for raw memory partitions, it's the UBI images being discussed.
> Best regards,
> Benoît

Regards

-- Pantelis