Loading from squashfs with zstd compression slower than ext4

[Tom Rini]

On Fri, Jun 02, 2023 at 11:34:33AM +1000, Jonathan Liu wrote:
> Hi Tom,
> 
> On Fri, 2 Jun 2023 at 03:09, Tom Rini <trini at konsulko.com> wrote:
> >
> > On Thu, Jun 01, 2023 at 02:10:21PM +1000, Jonathan Liu wrote:
> > > Hi Simon,
> > >
> > > On Thu, 1 Jun 2023 at 11:27, Jonathan Liu <net147 at gmail.com> wrote:
> > > >
> > > > Hi Simon,
> > > >
> > > > On Thu, 1 Jun 2023 at 03:13, Simon Glass <sjg at chromium.org> wrote:
> > > > >
> > > > > Hi Jonathan,
> > > > >
> > > > > On Wed, 31 May 2023 at 06:52, Jonathan Liu <net147 at gmail.com> wrote:
> > > > > >
> > > > > > Hi All,
> > > > > >
> > > > > > I noticed that loading from squashfs with zstd compression is
> > > > > > significantly slower than ext4 on RK3399 using U-Boot 2022.01. The
> > > > > > filesystem is located on eMMC.
> > > > > >
> > > > > > I have CONFIG_BLOCK_CACHE=y in .config and am using the generic load command.
> > > > > >
> > > > > > ext4:
> > > > > > 15682578 bytes read in 366 ms (40.9 MiB/s)
> > > > > >
> > > > > > SquashFS with zstd compression:
> > > > > > 15685001 bytes read in 2339 ms (6.4 MiB/s)
> > > > > >
> > > > > > Any suggestions on how to speed up loading from SquashFS with zstd compression?
> > > >
> > > > >
> > > > > Could it just be that zstd is slow? Have you tried lz4?
> > > > >
> > >
> > > >
> > > > I tried LZ4 but get "Error: unknown compression type."
> > > > It seems the SquashFS decompressor only supports lzo, gzip and zstd -
> > > > https://source.denx.de/u-boot/u-boot/-/blob/master/fs/squashfs/sqfs_decompressor.c
> > > >
> > > > ext4:
> > > > 15682578 bytes read in 366 ms (40.9 MiB/s)
> > > >
> > > > SquashFS with lzo compression (~129 MiB squashfs):
> > > > 15686657 bytes read in 1032 ms (14.5 MiB/s)
> > > >
> > > > SquashFS with gzip compression (~116 MiB squashfs):
> > > > 15686657 bytes read in 1494 ms (10 MiB/s)
> > > >
> > > > SquashFS with zstd compression (~107 MiB squashfs):
> > > > 15685001 bytes read in 2339 ms (6.4 MiB/s)
> > >
> > > SquashFS with zstd compression seems to be even slower after zstd was
> > > updated to 1.5.2 in U-Boot.
> > >
> > > SquashFS with zstd compression (~107 MiB squashfs), with zstd 1.5.2
> > > update cherry-picked from
> > > https://source.denx.de/u-boot/u-boot/-/commit/4b9b25d943ff95c7421cab261333fc29852fe3b1:
> > > 15686657 bytes read in 2726 ms (5.5 MiB/s)
> 
> >
> > OK, so is zstd supposed to be fast decompression? Are there some
> > optimizations in the library we could be enabling (or at least asking to
> > enable via Kconfig) but aren't?
> >
> 
> CONFIG_ZSTD_LIB_MINIFY enabled (default)
> 15686657 bytes read in 2707 ms (5.5 MiB/s)
> 
> CONFIG_ZSTD_LIB_MINIFY disabled
> 15686657 bytes read in 2342 ms (6.4 MiB/s)
> 
> It seems the zstd decompression performance regression after update to
> zstd 1.5.2 is caused by CONFIG_ZSTD_LIB_MINIFY being enabled by
> default.

It might be worth updating the help messages then to note the tradeoff
in performance vs size there.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20230601/58affa34/attachment.sig>