[U-Boot] [PATCH 1/7] JFFS2: Bug fix for summary support

[Baidu Liu]

Hi,Detlev :

2011/4/30 Detlev Zundel <dzu at denx.de>:
> Hi Baidu,
>
>> Hi,  Detlev :
>>
>> 2011/4/29 Detlev Zundel <dzu at denx.de>:
>>> Hi,
>>>
>>>>  1/ Get the latest DIRENT
>>>>  For example, if you create a file in linux jffs2 which config summary
>>>>  support, then you delete the file , you will not see the file  in
>>>>  linux jffs2. But you can also see this file in uboot after you reset
>>>>  the system. That is because all the nodes in jffs2 which config summary
>>>>  will not be marked as obsolete. The deleted file's DIRENT node will be
>>>>  seen in uboot. So what we need to do is to get the latest DIRENT whose
>>>>  ino in DIRENT is 0.
>>>
>>> Sorry, but I do not understand that last sentence.  Can you clarify this
>>> please?
>>
>> This is just give an example. If you create a file, then you DELETE it
>> in jffs2 .
>> In previous code where you do NOT config SUMMARY. There will be two
>> DIRENT nodes in flash. One is obsoleted where the ino=1, another one
>> is valid where the ino=0. When we type "ls " command, the file will
>> NOT shown because uboot will not show the DIRENT where the ino =0
>> which means the file is deleted.
>> But when we config SUMMARY in  kernel jffs2. The two DIRENTs are BOTH
>> valid. When we type "ls " command, the file will be  SHOWN  because we
>> find the first DIRENT where the ino is not 0. The result is  obviously
>> wrong.
>
> The more I read the jffs2 code, the more I do not understand.  In my
> eyes, the summary feature should just be a shortcut for not scanning the
> whole eraseblock, but I fail to see substantial differences.
> Specifically, I do not understand why your example should be a problem.
> So maybe you should correct me where I'm wrong:
>
> - we create a file in Linux.  So we get a dirent with versino = 1, ino
>  <> 0.  This will be in the summary of its jeb.
NO, the summary inode is only written by jffs2 when one jeb is almost full.
In some jeb which are not full, there is no summary node.

> - we delete the file in Linux.  So we get a dirent with version = 2 and
>  ino = 0.  If this is not in the same jeb than the previous dirent,
>  then it will be in the summary of its own jeb, otherwise it will
>  obsolete the old dirent in the summary directly.
Firstly, if the jeb is not almost full, there is NO summary inode.
secondly, The most important different between the jffs2 not configed
SUMMARY and the jffs2 configed jffs2 is :
There is no obsolete inode in jffs2 configed SUMMARY.

> Is this correct?
>
> Now when U-Boot reads the summaries, it will enter both dirents into its
> list and 'jffs2_1pass_find_inode' will find the dirent with version 2
> having ino = 0.  So the file should not be listed.
Acturally, the both dirent is belong to the same file. The latest
version represent the last status of the file

> Where is this going wrong?
>
> I'm really trying hard to understand the changes, but to comment on them
> sensibly, I have to understand what is going on...
>
>> So what we need to do is to find the DIRENT with the latest version.
>> In this example ,the second DIRENT has highest version which means
>> latest. And the ino is 0. Then we will not show it.
>
> Yes, I think I understand what you try to do, I explicitely said that I
> did not understand this sentence:
>
> "So what we need to do is to get the latest DIRENT whose ino in DIRENT
> is 0"
>
> What does this mean? Do you mean "Before we list any DIRENT, we first
> check if we find a later DIRENT which has ino = 0.  If this is the case,
> then the file has been erased and should not be displayed"?
Yes, this is happened when the summary is configed. Because all the
inode will be scanned when summary is configed.

> Alas as I wrote above, I do not understand why this is not handled
> already by current code.
>
>>>>  Than we will not see this file in uboot which is
>>>>  what we want.
>>>>
>>>>  2/ Add CONFIG_SYS_JFFS2_SORT_FRAGMENTS definition,if we config jffs2
>>>>  summary in uboot.
>>>>  All the inodes of a file will not marked as obsolete, if they do not
>>>>  sort in the list struct b_node *, the latest data in inode may be
>>>>  overwritten by the older one.
>>>
>>> Also I have trouble making sense of this text.
>>>
>> There may have many data inode in jfffs2 which represent the same
>> range in the same file, when we use SUMMARY. They are overlapped. But
>> only the data inode which has the latest version represent the actual
>> data. So we need to sort the data inode, where we make the latest data
>> inode after the obsoleted data inode. Then the latest data inode will
>> not be overwritten when use use "fsload" command in uboot.
>
> Sorry, I am still not understanding this.  Can you please state
> explicitely where you think there is an error and how your changes fix
> the error?
>
> Using summary or not, we can have multiple fragments for a file in a
> JFFS2 filesystem and the code seems to handle this fine.  The summary
> should only be a optimization for the scanning phase, it should not
> behave differently.  This is why I want to understand this.  SUMMARY and
> SORT_FRAGMENTS should be orthogonal.  If they are not, then we most
> certainly have another problem.
>
> [...]
>
>>>> diff --git a/include/jffs2/jffs2.h b/include/jffs2/jffs2.h
>>>> index 651f94c..5b006c0 100644
>>>> --- a/include/jffs2/jffs2.h
>>>> +++ b/include/jffs2/jffs2.h
>>>> @@ -41,6 +41,16 @@
>>>>  #include <asm/types.h>
>>>>  #include <jffs2/load_kernel.h>
>>>>
>>>> +#ifdef CONFIG_JFFS2_SUMMARY
>>>> +#ifndef CONFIG_SYS_JFFS2_SORT_FRAGMENTS
>>>> +/*
>>>> +we should define CONFIG_SYS_JFFS2_SORT_FRAGMENTS,if
>>>> +CONFIG_JFFS2_SUMMARY is enabled.
>>>> +*/
>>>> +#define CONFIG_SYS_JFFS2_SORT_FRAGMENTS
>>>> +#endif
>>>> +#endif
>>>> +
>>>>  #define JFFS2_SUPER_MAGIC 0x72b6
>>>>
>>>>  /* Values we may expect to find in the 'magic' field */
>>>
>>> If JFFS2_SUMMARY _needs_ SORT_FRAGMENTS, then we should say so, i.e.
>>>
>>> /*
>>> CONFIG_JFFS2_SUMMARY will not work correctly without
>>> CONFIG_SYS_JFFS2_SORT_FRAGMENTS
>>> */
>>>
>>> Alas, technically I do not understand why that is the case.  So I invite
>>> people more knowledgeable with JFFS2 to comment on this bit.
>>>
>>> [time passes]
>>>
>>> Wait a minute - I tried to understand the code here - is it possible
>>> that SORT_FRAGMENTS really is needed _whenever_ we have a read-write
>>> JFFS2 filesystem?  I.e. even without summary support we will have
>>> problems without SORT_FRAGMENTS?
>>>
>> No, SORT_FRAGMENTS only need when SUMMARY is configed.
>> Because all the nodes including the DIRENT inode and DATA inode are
>> all valid. So we need to sort them.
>
> What?  On JFFS2 we can always have multiple DIRENT and DATA nodes and we
> always have to find the highest version.
For DIRENT, we shall find the latest DIRENT with latest version.
For INODE, because we do not create fully structure to represent the
information in the jffs2 while the kernel does. So we should sort the
inode to avoid the latest data be overwritten by the old one

You can dump the flash image in the flash where the summary is
configed, the cleanmark is still 0x1985 2003  after the jffs2 write
inode in the jeb.
And all the DIRENTs are 0xe001, all the data inode is 0xe002.

While the summary is not configed, the obsoleted DIRENT  is 0xc001.
And the obsoleted data inode is 0xc002.

We should do our best to  make the UBOOT work well. And for jffs2, the
best way to know the principle is to exercise, after this, then you
will know why I do these changes.

We can see the summary MACRO has been added in UBOOT for a long time.
But it does not work. So we can get the conclusion that people do not
use the uboot to read file from jffs2 when summary is configed.

UBOOT should be more frequently used in board. Not just put some
unused functions in it.