Passing boot logs to Linux?

[Csókás Bence]

On 2023. 12. 20. 20:58, Dragan Simic wrote:
> On 2023-12-20 20:24, Csókás Bence wrote:
>> On 2023. 12. 20. 9:29, Dragan Simic wrote:
>>> On 2023-12-20 08:52, Csókás Bence wrote:
>>>> That's what I read as well. Is there support for U-boot to write and
>>>> Linux to read PStores?
>>>
>>> No and yes, but U-Boot can already read pstore.  Please see
>>> doc/usage/cmd/pstore.rst for the U-Boot part, and
>>> Documentation/admin-guide/pstore-blk.rst for the Linux kernel part.
>>
>> Irrelevant, as we only want to write out the console log to U-Boot, and
>> not the other way around (that's for collecting panic logs, which is
>> already implemented).
> 
> Quite frankly, this is a bit confusing to me.  Why should it be 
> irrelevant, if we want to store the U-Boot console logs into the pstore, 
> and read them later in Linux?  That's exactly what you asked about, and 
> it was a good question.

Because we want to *write* PStore from U-Boot. The read being already 
implemented is not very useful, we could only re-read an already-written 
console log (which is not helpful for us).

>>> Another benefit of using pstore would be no permanently wasted RAM for
>>> the recorded console contents.  Also, having the data recorded to a
>>> storage device also goes along with providing permanent records.
>>
>> I'm positively *not gonna save boot logs to disk*, as most embedded
>> systems have Flash-based storage media; writing to them on every boot
>> would be devastating.
> 
> Writing 30-40 KB of data to an even remotely modern flash-based storage 
> media on each boot is pretty much nothing.  OTOH, writing 30-40 KB of 
> data to an SPI chip on each boot would be rather bad, but we aren't 
> going to do that for sure.
> 
> To put things into perspective, I'm writing this on a Pinebook Pro with 
> an eMMC chip, and it has stored many GBs of data over a few years.  Even 
> high-endurance microSD cards are rather reliable in that regard.

Not every system has eMMC/uSD, and as you said, these arguments don't 
hold for a 4 MB SPI NAND, for example, one you might find in an OpenWrt 
router for example. Whereas RAM is quite cheap nowadays.

>> Plus, I don't want the console subsystem to depend on any file/disk
>> operations/drivers.
> 
> Well, the console would still work as usual even if logging to disk 
> would fail for any reason, which is similar to the serial console still 
> working if the graphical console fails.  Moreover, if the disk fails, 
> the system isn't be able to boot, so any RAM-based console logs would be 
> lost in that case.  All this makes the RAM-based logging no more 
> resilient to disk failures.

Correction: if disk *reads* fail, as well as writes, then the system 
will not boot. However, typical failure of Flash media is that it 
becomes read-only.

> I still think that using disk-based pstore is a better option.  Just as 
> you don't want to wear out your flash disks with 30-40 KB of data, I 
> also don't want to waste 30-40 KB of RAM.

As I said, you could just unload the log after you're done processing 
it. 40 KB RAM is less, than what `sshd` uses, for instance (860k on my 
laptop, but it can probably be less, maybe even 10x less, so 80-90k?), 
so you could, in your init, process the in-RAM log, then unload it, then 
start your other services, thereby reclaiming that RAM.

> Also, embedded devices such 
> as the PinePhone would benefit additionally from using disk-based 
> logging, because people often can't do anything in the field if booting 
> fails for some reason, so RAM-based logs would end up being lost, 
> preventing any post-mortem debugging later.

Post-mortem analysis could be interesting, true.

> Perhaps we can actually offer both options, i.e. both logging to RAM and 
> to disk, so everyone can choose what they prefer.  I'd also be fine with 
> working on such a dual approach, but it would obviously be more complex.

Perhaps that is the best option. But, as Sean has pointed out, U-Boot 
doesn't support disk-backed PStore anyways... So then, should we 
implement RAM-based PStore writing for the time being, and then maybe 
expand it later?

Bence