[U-Boot] ext4: massive corruption with ext4write
Aaron Williams
Aaron.Williams at cavium.com
Tue Jul 24 10:53:32 UTC 2018
On Monday, July 23, 2018 11:25:41 PM PDT Aaron Williams wrote:
> External Email
>
> On Monday, July 23, 2018 11:15:59 PM PDT Aaron Williams wrote:
>
> > External Email
> >
> >
> >
> > Hi all,
> >
> >
> >
> > It looks like after a certain amount of data has been written that all
> > hell
breaks loose with the ext4 filesystem.
> >
> >
> >
> > In my case, I have the following files on a 64G USB thumb drive with two
> > partitions, a small FAT partition with the rest of the space dedicated to
> > ext4
>
> created using mkfs.ext4 /dev/sdg2
>
> >
> >
> > total 477632
> > -rwxr-xr-x 1 root root 152777216 Jul 23 18:11 Image
> > drwx------ 2 root root 16384 Jul 23 18:10 lost+found
> > -rwxr-xr-x 1 root root 90706976 Dec 31 1969 test.64
> > -rwxr-xr-x 1 root root 152777216 Dec 31 1969 test.img
> > -rwxr-xr-x 1 root root 50 Dec 31 1969 test.txt
> > -rwxr-xr-x 1 root root 1841408 Jul 23 18:12 u-boot-octeon_ebb7304.bin
> > -rwxr-xr-x 1 root root 90706976 Jul 23 18:11 vmlinux.64
> >
> >
> >
> > Everything is fine until I wrote the file test.64, which is basically a
> > copy
>
> of vmlinux.64.
>
> >
> >
> > The first few files were written using my host Linux system, namely
> > Image,
> > u-
>
> boot-octeon_ebb7304.bin and vmlinux.64.
>
> >
> >
> > From within U-Boot I performed the following commands:
> >
> >
> >
> > # ext4load usb 0:2 $loadaddr Image
> > # ext4write usb 0:2 $fileaddr /test.img $filesize
> > # tftpboot $loadaddr test.txt
> > # ext4write usb 0:2 $fileaddr /test.txt $filesize
> > # ext4load usb 0:2 $loadaddr vmlinux.64
> > # ext4write usb 0:2 $fileaddr /test.64 $filesize
> >
> >
> >
> > Everything is fine on the drive until I write test.64. At this point, I
> > get
a
>
> huge list of errors:
>
> >
> >
> > # fsck.ext4 -v -n /dev/sdg2
> > e2fsck 1.42.11 (09-Jul-2014)
> > Group descriptor 0 has invalid unused inodes count 57374. Fix? no
> >
> >
> >
> > /dev/sdg2 contains a file system with errors, check forced.
> > Pass 1: Checking inodes, blocks, and sizes
> > Deleted inode 130913 has zero dtime. Fix? no
> >
> >
> >
> > Inode 130915 is in use, but has dtime set. Fix? no
> >
> >
> >
> > Inode 130915 has imagic flag set. Clear? no
> >
> >
> >
> > Inode 130915 has a extra size (8223) which is invalid
> > Fix? no
> >
> >
> >
> > Inode 130916 is in use, but has dtime set. Fix? no
> >
> >
> >
> > Inode 130916 has imagic flag set. Clear? no
> >
> >
> >
> > Inode 130916 has a extra size (8223) which is invalid
> > Fix? no
> >
> >
> >
> > ...
> > Illegal block #11 (2435760161) in inode 131070. IGNORED.
> > Illegal indirect block (4026556192) in inode 131070. IGNORED.
> > Illegal double indirect block (2433138721) in inode 131070. IGNORED.
> > Illegal triple indirect block (2434195456) in inode 131070. IGNORED.
> > Error while iterating over blocks in inode 131070: Illegal triply
> > indirect
> > block found
> >
> >
> >
> >
> > and many many more errors.
> >
> >
> >
> > Note that I am using the very latest ext4 code from the master branch.
> > This
>
> is not a USB problem because I can reproduce this problem with an SD
>
> > card. This problem also occurs on two different platforms, one being
> > aarch64 little endian and the other being MIPS64 big endian. The
> > filesystem code is identical since the latest code has been backported to
> > our older MIPS bootloader.
> >
> >
> >
> > My guess is that all hell is breaking loose when a file spans multiple
> > block
>
> groups.
>
> >
> >
> > -Aaron Williams
> >
> >
> >
> > On Thursday, July 19, 2018 7:35:46 PM PDT Aaron Williams wrote:
> >
> >
> >
> > >
> > >
> > >
> > > Hi all,
> > >
> > >
> > >
> > >
> > >
> > > I am sometimes seeing issues when using ext4write where fsck later
> > > complains
> >
> >
> >
> > that the group descriptor has an invalid number of unused
> >
> >
> >
> > > inodes. Is this a known problem?
> > >
> > >
> > >
> > >
> > >
> > > Also, I think the assert(offset == sizeof(*desc)); in
> > > ext4fs_checksum_update()
> >
> >
> >
> > is invalid since with ext4 the descriptor is
> >
> >
> >
> > > larger than the offset. When debugging was enabled I'd always hit this
> > > assert.
> > >
> > >
> > >
> > >
> > >
> > > Also, in ext4fs_write, the debug statement should say blocks and not
> > > bytes.
> >
> >
> >
> > >
> > >
> > >
> > > -Aaron
> > >
> > >
> > >
> > >
> > >
> > > --
> > > Aaron Williams
> > > Senior Software Engineer
> > > Cavium, Inc.
> > > (408) 943-7198 (510) 789-8988 (cell)
> > > _______________________________________________
> > > U-Boot mailing list
> > > U-Boot at lists.denx.de
> > > https://lists.denx.de/listinfo/u-boot
> >
> >
> >
> >
> > --
> > Aaron Williams
> > Senior Software Engineer
> > Cavium, Inc.
> > (408) 943-7198 (510) 789-8988 (cell)
> > _______________________________________________
> > U-Boot mailing list
> > U-Boot at lists.denx.de
> > https://lists.denx.de/listinfo/u-boot
>
>
> Here is the output after the write operation from fsck.ext4 -v /dev/sdg2
>
> fsck.ext4 -v /dev/sdg2
> e2fsck 1.42.11 (09-Jul-2014)
> Group descriptor 0 has invalid unused inodes count 57374. Fix<y>? yes
> Pass 1: Checking inodes, blocks, and sizes
>
> Running additional passes to resolve blocks claimed by more than one
> inode...
Pass 1B: Rescanning for multiply-claimed blocks
> Multiply-claimed block(s) in inode 7: 98307 98308 98309 98310 98311 98312
> 98313 98314 98315 98316 98317 98318 98319 98320 98321 98322 98323 98324
> 98325
98326 98327 98328 98329 98330 98331 98332 98333 98334 98335 98336
> 98337 98338 98339 98340 98341 98342 98343 98344 98345 98346 98347 98348
> 98349 98350 98351 98352 98353 98354 98355 98356 98357 98358 98359 98360
> 98361 98362 98363 98364 98365 98366 98367 98368 98369 98370 98371 98372
> 98373 98374 98375 98376 98377 98378 98379 98380 98381 98382 98383 98384
> 98385 98386 98387 98388 98389 98390 98391 98392 98393 98394 98395 98396
> 98397 98398 98399 98400 98401 98402 98403 98404 98405 98406 98407 98408
> 98409 98410 98411 98412 98413 98414 98415 98416 98417 98418 98419 98420
> 98421 98422 98423 98424 98425 98426 98427 98428 98429 98430 98431 98432
> 98433 98434 98435 98436 98437 98438 98439 98440 98441 98442 98443 98444
> 98445 98446 98447 98448 98449 98450 98451 98452 98453 98454 98455 98456
> 98457 98458 98459 98460 98461 98462 98463 98464 98465 98466 98467 98468
> 98469 98470 98471 98472 98473 98474 98475 98476 98477 98478 98479 98480
> 98481 98482 98483 98484 98485 98486 98487 98488 98489 98490 98491 98492
> 98493 98494 98495 98496 98497 98498 98499 98500 98501 98502 98503 98504
> 98505 98506 98507 98508 98509 98510 98511 98512 98513 98514 98515 98516
> 98517 98518 98519 98520 98521 98522 98523 98524 98525 98526 98527 98528
> 98529 98530 98531 98532 98533 98534 98535 98536 98537 98538 98539 98540
> 98541 98542 98543 98544 98545 98546 98547 98548 98549 98550 98551 98552
> 98553 98554 98555 98556 98557 98558 98559 98560 98561 98562 98563 98564
> 98565 98566 98567 98568 98569 98570 98571 98572 98573 98574 98575 98576
> 98577 98578 98579 98580 98581 98582 98583 98584 98585 98586 98587 98588
> 98589 98590 98591 98592 98593 98594 98595 98596 98597 98598 98599 98600
> 98601 98602 98603 98604 98605 98606 98607 98608 98609 98610 98611 98612
> 98613 98614 98615 98616 98617 98618 98619 98620 98621 98622 98623 98624
> 98625 98626 98627 98628 98629 98630 98631 98632 98633 98634 98635 98636
> 98637 98638 98639 98640 98641 98642 98643 98644 98645 98646 98647 98648
> 98649 98650 98651 98652 98653 98654 98655 98656 98657 98658 98659 98660
> 98661 98662 98663 98664 98665 98666 98667 98668 98669 98670 98671 98672
> 98673 98674 98675 98676 98677 98678 98679 98680 98681 98682 98683 98684
> 98685 98686 98687 98688 98689 98690 98691 98692 98693 98694 98695 98696
> 98697 98698 98699 98700 98701 98702 98703 98704 98705 98706 98707 98708
> 98709 98710 98711 98712 98713 98714 98715 98716 98717 98718 98719 98720
> 98721 98722 98723 98724 98725 98726 98727 98728 98729 98730 98731 98732
> 98733 98734 98735 98736 98737 98738 98739 98740 98741 98742 98743 98744
> 98745 98746 98747 98748 98749 98750 98751 98752 98753 98754 98755 98756
> 98757 98758 98759 98760 98761 98762 98763 98764 98765 98766 98767 98768
> 98769 98770 98771 98772 98773 98774 98775 98776 98777 98778 98779 98780
> 98781 98782 98783 98784 98785 98786 98787 98788 98789 98790 98791 98792
> 98793 98794 98795 98796 98797 98798 98799 98800 98801 98802 98803 98804
> 98805 98806 98807 98808 98809 98810 98811 98812 98813 98814 98815 98816
> 98817 98818 98819 98820 98821 98822 98823 98824 98825 98826 98827 98828
> 98829 98830 98831 98832 98833 98834 98835 98836 98837 98838 98839 98840
> 98841 98842 98843 98844 98845 98846 98847 98848 98849 98850 98851 98852
> 98853 98854 98855 98856 98857 98858 98859 98860 98861 98862 98863 98864
> 98865 98866 98867 98868 98869 98870 98871 98872 98873 98874 98875 98876
> 98877 98878 98879 98880 98881 98882 98883 98884 98885 98886 98887 98888
> 98889 98890 98891 98892 98893 98894 98895 98896 98897 98898 98899 98900
> 98901 98902 98903 98904 98905 98906 98907 98908 98909 98910 98911 98912
> 98913 98914 98915 98916 98917 98918 98919 98920 98921 98922 98923 98924
> 98925 98926 98927 98928 98929 98930 98931 98932 98933 98934 98935 98936
> 98937 98938 98939 98940 98941 98942 98943 98944 98945 98946 98947 98948
> 98949 98950 98951 98952 98953 98954 98955 98956 98957 98958 98959 98960
> 98961 98962 98963 98964 98965 98966 98967 98968 98969 98970 98971 98972
> 98973 98974 98975 98976 98977 98978 98979 98980 98981 98982 98983 98984
> 98985 98986 98987 98988 98989 98990 98991 98992 98993 98994 98995 98996
> 98997 98998 98999 99000 99001 99002 99003 99004 99005 99006 99007 99008
> 99009 99010 99011 99012 99013 99014 99015 99016 99017 99018 99019 99020
> 99021 99022 99023 99024 99025 99026 99027 99028 99029 99030 99031 99032
> 99033 99034 99035 99036 99037 99038 99039 99040 99041 99042 99043 99044
> 99045 99046 99047 99048 99049 99050 99051 99052 99053 99054 99055 99056
> 99057 99058 99059 99060 99061 99062 99063 99064 99065 99066 99067 99068
> 99069 99070 99071 99072 99073 99074 99075 99076 99077 99078 99079 99080
> 99081 99082 99083 99084 99085 99086 99087 99088 99089 99090 99091 99092
> 99093 99094 99095 99096 99097 99098 99099 99100 99101 99102 99103 99104
> 99105 99106 99107 99108 99109 99110 99111 99112 99113 99114 99115 99116
> 99117 99118 99119 99120 99121 99122 99123 99124 99125 99126 99127 99128
> 99129 99130 99131 99132 99133 99134 99135 99136 99137 99138 99139 99140
> 99141 99142 99143 99144 99145 99146 99147 99148 99149 99150 99151 99152
> 99153 99154 99155 99156 99157 99158 99159 99160 99161 99162 99163 99164
> 99165 99166 99167 99168 99169 99170 99171 99172 99173 99174 99175 99176
> 99177 99178 99179 99180 99181 99182 99183 99184 99185 99186 99187 99188
> 99189 99190 99191 99192 99193 99194 99195 99196 99197 99198 99199 99200
> 99201 99202 99203 99204 99205 99206 99207 99208 99209 99210 99211 99212
> 99213 99214 99215 99216 99217 99218 99219 99220 99221 99222 99223 99224
> 99225 99226 99227 99228 99229 99230 99231 99232 99233 99234 99235 99236
> 99237 99238 99239 99240 99241 99242 99243 99244 99245 99246 99247 99248
> 99249 99250 99251 99252 99253 99254 99255 99256 99257 99258 99259 99260
> 99261 99262 99263 99264 99265 99266 99267 99268 99269 99270 99271 99272
> 99273 99274 99275 99276 99277 99278 99279 99280 99281 99282 99283 99284
> 99285 99286 99287 99288 99289 99290 99291 99292 99293 99294 99295 99296
> 99297 99298 99299 99300 99301 99302 99303 99304 99305 99306 99307 99308
> 99309 99310 99311 99312 99313 99314 99315 99316 99317 99318 99319 99320
> 99321 99322 99323 99324 99325 99326 99327 99328
> Multiply-claimed block(s) in inode 15: 98304 98305 98306 98307 98308 98309
> 98310 98311 98312 98313 98314 98315 98316 98317 98318 98319 98320 98321
> 98322
98323 98324 98325 98326 98327 98328 98329 98330 98331 98332 98333
> 98334 98335 98336 98337 98338 98339 98340 98341 98342 98343 98344 98345
> 98346 98347 98348 98349 98350 98351 98352 98353 98354 98355 98356 98357
> 98358 98359 98360 98361 98362 98363 98364 98365 98366 98367 98368 98369
> 98370 98371 98372 98373 98374 98375 98376 98377 98378 98379 98380 98381
> 98382 98383 98384 98385 98386 98387 98388 98389 98390 98391 98392 98393
> 98394 98395 98396 98397 98398 98399 98400 98401 98402 98403 98404 98405
> 98406 98407 98408 98409 98410 98411 98412 98413 98414 98415 98416 98417
> 98418 98419 98420 98421 98422 98423 98424 98425 98426 98427 98428 98429
> 98430 98431 98432 98433 98434 98435 98436 98437 98438 98439 98440 98441
> 98442 98443 98444 98445 98446 98447 98448 98449 98450 98451 98452 98453
> 98454 98455 98456 98457 98458 98459 98460 98461 98462 98463 98464 98465
> 98466 98467 98468 98469 98470 98471 98472 98473 98474 98475 98476 98477
> 98478 98479 98480 98481 98482 98483 98484 98485 98486 98487 98488 98489
> 98490 98491 98492 98493 98494 98495 98496 98497 98498 98499 98500 98501
> 98502 98503 98504 98505 98506 98507 98508 98509 98510 98511 98512 98513
> 98514 98515 98516 98517 98518 98519 98520 98521 98522 98523 98524 98525
> 98526 98527 98528 98529 98530 98531 98532 98533 98534 98535 98536 98537
> 98538 98539 98540 98541 98542 98543 98544 98545 98546 98547 98548 98549
> 98550 98551 98552 98553 98554 98555 98556 98557 98558 98559 98560 98561
> 98562 98563 98564 98565 98566 98567 98568 98569 98570 98571 98572 98573
> 98574 98575 98576 98577 98578 98579 98580 98581 98582 98583 98584 98585
> 98586 98587 98588 98589 98590 98591 98592 98593 98594 98595 98596 98597
> 98598 98599 98600 98601 98602 98603 98604 98605 98606 98607 98608 98609
> 98610 98611 98612 98613 98614 98615 98616 98617 98618 98619 98620 98621
> 98622 98623 98624 98625 98626 98627 98628 98629 98630 98631 98632 98633
> 98634 98635 98636 98637 98638 98639 98640 98641 98642 98643 98644 98645
> 98646 98647 98648 98649 98650 98651 98652 98653 98654 98655 98656 98657
> 98658 98659 98660 98661 98662 98663 98664 98665 98666 98667 98668 98669
> 98670 98671 98672 98673 98674 98675 98676 98677 98678 98679 98680 98681
> 98682 98683 98684 98685 98686 98687 98688 98689 98690 98691 98692 98693
> 98694 98695 98696 98697 98698 98699 98700 98701 98702 98703 98704 98705
> 98706 98707 98708 98709 98710 98711 98712 98713 98714 98715 98716 98717
> 98718 98719 98720 98721 98722 98723 98724 98725 98726 98727 98728 98729
> 98730 98731 98732 98733 98734 98735 98736 98737 98738 98739 98740 98741
> 98742 98743 98744 98745 98746 98747 98748 98749 98750 98751 98752 98753
> 98754 98755 98756 98757 98758 98759 98760 98761 98762 98763 98764 98765
> 98766 98767 98768 98769 98770 98771 98772 98773 98774 98775 98776 98777
> 98778 98779 98780 98781 98782 98783 98784 98785 98786 98787 98788 98789
> 98790 98791 98792 98793 98794 98795 98796 98797 98798 98799 98800 98801
> 98802 98803 98804 98805 98806 98807 98808 98809 98810 98811 98812 98813
> 98814 98815 98816 98817 98818 98819 98820 98821 98822 98823 98824 98825
> 98826 98827 98828 98829 98830 98831 98832 98833 98834 98835 98836 98837
> 98838 98839 98840 98841 98842 98843 98844 98845 98846 98847 98848 98849
> 98850 98851 98852 98853 98854 98855 98856 98857 98858 98859 98860 98861
> 98862 98863 98864 98865 98866 98867 98868 98869 98870 98871 98872 98873
> 98874 98875 98876 98877 98878 98879 98880 98881 98882 98883 98884 98885
> 98886 98887 98888 98889 98890 98891 98892 98893 98894 98895 98896 98897
> 98898 98899 98900 98901 98902 98903 98904 98905 98906 98907 98908 98909
> 98910 98911 98912 98913 98914 98915 98916 98917 98918 98919 98920 98921
> 98922 98923 98924 98925 98926 98927 98928 98929 98930 98931 98932 98933
> 98934 98935 98936 98937 98938 98939 98940 98941 98942 98943 98944 98945
> 98946 98947 98948 98949 98950 98951 98952 98953 98954 98955 98956 98957
> 98958 98959 98960 98961 98962 98963 98964 98965 98966 98967 98968 98969
> 98970 98971 98972 98973 98974 98975 98976 98977 98978 98979 98980 98981
> 98982 98983 98984 98985 98986 98987 98988 98989 98990 98991 98992 98993
> 98994 98995 98996 98997 98998 98999 99000 99001 99002 99003 99004 99005
> 99006 99007 99008 99009 99010 99011 99012 99013 99014 99015 99016 99017
> 99018 99019 99020 99021 99022 99023 99024 99025 99026 99027 99028 99029
> 99030 99031 99032 99033 99034 99035 99036 99037 99038 99039 99040 99041
> 99042 99043 99044 99045 99046 99047 99048 99049 99050 99051 99052 99053
> 99054 99055 99056 99057 99058 99059 99060 99061 99062 99063 99064 99065
> 99066 99067 99068 99069 99070 99071 99072 99073 99074 99075 99076 99077
> 99078 99079 99080 99081 99082 99083 99084 99085 99086 99087 99088 99089
> 99090 99091 99092 99093 99094 99095 99096 99097 99098 99099 99100 99101
> 99102 99103 99104 99105 99106 99107 99108 99109 99110 99111 99112 99113
> 99114 99115 99116 99117 99118 99119 99120 99121 99122 99123 99124 99125
> 99126 99127 99128 99129 99130 99131 99132 99133 99134 99135 99136 99137
> 99138 99139 99140 99141 99142 99143 99144 99145 99146 99147 99148 99149
> 99150 99151 99152 99153 99154 99155 99156 99157 99158 99159 99160 99161
> 99162 99163 99164 99165 99166 99167 99168 99169 99170 99171 99172 99173
> 99174 99175 99176 99177 99178 99179 99180 99181 99182 99183 99184 99185
> 99186 99187 99188 99189 99190 99191 99192 99193 99194 99195 99196 99197
> 99198 99199 99200 99201 99202 99203 99204 99205 99206 99207 99208 99209
> 99210 99211 99212 99213 99214 99215 99216 99217 99218 99219 99220 99221
> 99222 99223 99224 99225 99226 99227 99228 99229 99230 99231 99232 99233
> 99234 99235 99236 99237 99238 99239 99240 99241 99242 99243 99244 99245
> 99246 99247 99248 99249 99250 99251 99252 99253 99254 99255 99256 99257
> 99258 99259 99260 99261 99262 99263 99264 99265 99266 99267 99268 99269
> 99270 99271 99272 99273 99274 99275 99276 99277 99278 99279 99280 99281
> 99282 99283 99284 99285 99286 99287 99288 99289 99290 99291 99292 99293
> 99294 99295 99296 99297 99298 99299 99300 99301 99302 99303 99304 99305
> 99306 99307 99308 99309 99310 99311 99312 99313 99314 99315 99316 99317
> 99318 99319 99320 99321 99322 99323 99324 99325 99326 99327 99328
> Pass 1C: Scanning directories for inodes with multiply-claimed blocks
> Pass 1D: Reconciling multiply-claimed blocks
> (There are 1 inodes containing multiply-claimed blocks.)
>
> File /test.img (inode #15, mod time Wed Dec 31 16:00:00 1969)
> has 1025 multiply-claimed block(s), shared with 2 file(s):
> <filesystem metadata>
> <The group descriptor inode> (inode #7, mod time Mon Jul 23
> 18:10:08
2018)
> Clone multiply-claimed blocks<y>?
> Pass 2: Checking directory structure
> Pass 3: Checking directory connectivity
> Pass 4: Checking reference counts
> Pass 5: Checking group summary information
>
> /dev/sdg2: ***** FILE SYSTEM WAS MODIFIED *****
>
> 17 inodes used (0.00%, out of 1913184)
> 0 non-contiguous files (0.0%)
> 0 non-contiguous directories (0.0%)
> # of inodes with ind/dind/tind blocks: 2/2/0
> Extent depth histogram: 6
> 283495 blocks used (3.71%, out of 7640576)
> 0 bad blocks
> 1 large file
>
> 6 regular files
> 2 directories
> 0 character device files
> 0 block device files
> 0 fifos
> 0 links
> 0 symbolic links (0 fast symbolic links)
> 0 sockets
> ------------
> 8 files
>
> -Aaron
>
I found another endian issue. In ext4fs_get_new_inode_no() it is setting bgd-
>bg_itable_unused without performing the proper endian conversion.
Additionally, if it's a 64-bit filesystem, it does not set
bg_itable_unused_high.
-Aaron
More information about the U-Boot
mailing list