[PATCH v2 0/7] Add video damage tracking

[Matthias Brugger]

On 10/06/2022 00:59, Alexander Graf wrote:
> This patch set speeds up graphics output on ARM by a factor of 60x.
> 
> On most ARM SBCs, we keep the frame buffer in DRAM and map it as cached,
> but need it accessible by the display controller which reads directly
> from a later point of consistency. Hence, we flush the frame buffer to
> DRAM on every change. The full frame buffer.
> 
> Unfortunately, with the advent of 4k displays, we are seeing frame buffers
> that can take a while to flush out. This was reported by Da Xue with grub,
> which happily print 1000s of spaces on the screen to draw a menu. Every
> printed space triggers a cache flush.
> 
> This patch set implements the easiest mitigation against this problem:
> Damage tracking. We remember the lowest common denominator region that was
> touched since the last video_sync() call and only flush that. The most
> typical writer to the frame buffer is the video console, which always
> writes rectangles of characters on the screen and syncs afterwards.
> 
> With this patch set applied, we reduce drawing a large grub menu (with
> serial console attached for size information) on an RK3399-ROC system
> at 1440p from 55 seconds to less than 1 second.
> 
> Version 2 also implements VIDEO_COPY using this mechanism, reducing its
> overhead compared to before as well. So even x86 systems should be faster
> with this now :).
> 

Series tested on RPi4, thanks a lot!

Tested-by: Matthias Brugger <mbrugger at suse.com>

> 
> Alternatives considered:
> 
>    1) Lazy sync - Sandbox does this. It only calls video_sync(true) ever
>       so often. We are missing timers to do this generically.
> 
>    2) Double buffering - We could try to identify whether anything changed
>       at all and only draw to the FB if it did. That would require
>       maintaining a second buffer that we need to scan.
> 
>    3) Text buffer - Maintain a buffer of all text printed on the screen with
>       respective location. Don't write if the old and new character are
>       identical. This would limit applicability to text only and is an
>       optimization on top of this patch set.
> 
>    4) Hash screen lines - Create a hash (sha256?) over every line when it
>       changes. Only flush when it does. I'm not sure if this would waste
>       more time, memory and cache than the current approach. It would make
>       full screen updates much more expensive.
> 
> v1 -> v2:
> 
>    - new patch: video: Use VIDEO_DAMAGE for VIDEO_COPY
>    - Remove ifdefs
>    - Fix dcache range; we were flushing too much before
>    - Fix ranges in truetype target
>    - Limit rotate to necessary damange
> 
> 
> Alexander Graf (7):
>    dm: video: Add damage tracking API
>    dm: video: Add damage notification on display clear
>    vidconsole: Add damage notifications to all vidconsole drivers
>    video: Add damage notification on bmp display
>    efi_loader: GOP: Add damage notification on BLT
>    video: Only dcache flush damaged lines
>    video: Use VIDEO_DAMAGE for VIDEO_COPY
> 
>   configs/chromebook_coral_defconfig   |   1 +
>   configs/chromebook_link_defconfig    |   1 +
>   configs/chromebook_samus_defconfig   |   1 +
>   configs/minnowmax_defconfig          |   1 +
>   configs/sandbox_defconfig            |   1 +
>   configs/xilinx_zynqmp_virt_defconfig |   1 +
>   drivers/video/Kconfig                |  21 ++-
>   drivers/video/console_normal.c       |  22 ++--
>   drivers/video/console_rotate.c       |  87 ++++++++-----
>   drivers/video/console_truetype.c     |  30 +++--
>   drivers/video/vidconsole-uclass.c    |  16 ---
>   drivers/video/video-uclass.c         | 185 ++++++++++++++++-----------
>   drivers/video/video_bmp.c            |   7 +-
>   include/video.h                      |  54 +++-----
>   include/video_console.h              |  49 -------
>   lib/efi_loader/efi_gop.c             |   7 +-
>   16 files changed, 247 insertions(+), 237 deletions(-)
>