You are not logged in.

#1 2021-12-01 22:40:34

Colonel Panic
Member
Registered: 2018-11-13
Posts: 402

A fix for grub when os-prober isn't working

Hi. It sometimes happens that when you want to update grub from within a distro, you end up with only that distro listed in the grub menu because os-prober didn't work.

The fix is to open the grub file you find in /etc/default in a text editor, and then either add or uncomment this line;

GRUB_DISABLE_OS_PROBER=false

and then, after saving the grub file, run sudo update-grub as usual. The grub updater should then probe for any other distros on your hard drive and update your grub menu in full as normal.

Last edited by Colonel Panic (2021-12-02 11:55:41)

Offline

#2 2021-12-02 08:09:46

unklar
Back to the roots 1.9
From: #! BL
Registered: 2015-10-31
Posts: 1,656

Re: A fix for grub when os-prober isn't working

^The corresponding log @CP is referring to is this one (from sid(uction)):

zcat /usr/share/doc/grub-common/NEWS.Debian.gz
grub2 (2.06-1) UNRELEASED; urgency=medium

  * Boot menu entries for other operating systems are no longer generated by
    default.  To re-enable this, set GRUB_DISABLE_OS_PROBER=false in
    /etc/default/grub.

 -- Colin Watson <cjwatson@debian.org>  Wed, 18 Aug 2021 13:03:23 +0100

It means that in the future package grub2 the switch for os-prober is basically OFF. This means that there is basically NO automatic inclusion of other OS in the menu of grub.
In siduction this has now been implemented with package 2.06-2

In the current test version of [beryllium] is the package 2.04-20
I assume that with the release of Beryllium then also there the above package 2.06-2 of grub2 is included, which forces the dual/multiboot user to act. Unless..., the distributions fade in a suitable reference with the installation.  big_smile

Last edited by unklar (2021-12-02 08:14:15)

Offline

#3 2021-12-02 11:27:38

rbh
Moderator
From: Sweden/Vasterbotten/Rusfors
Registered: 2016-08-11
Posts: 1,345

Re: A fix for grub when os-prober isn't working

unklar wrote:

In siduction this has now been implemented with package 2.06-2

In the current test version of [beryllium] is the package 2.04-20
I assume that with the release of Beryllium then also there the above package 2.06-2 of grub2 is included, which forces the dual/multiboot user to act. Unless..., the distributions fade in a suitable reference with the installation.  big_smile

No, grub2 in both Debian Stable and Testing is version 2.04.
2.06 is still only in sid (debian unstable).
https://packages.debian.org/search?suit … ords=grub2


// Regards rbh

Please read before requesting help: Guide to getting help,
Introduction to the Bunsenlabs Lithium Desktop and other help topics under "Help Resources" on the BunsenLabs menu

Offline

#4 2021-12-02 11:52:59

Colonel Panic
Member
Registered: 2018-11-13
Posts: 402

Re: A fix for grub when os-prober isn't working

Thanks to you both for replying. I'd just updated a newish installation of *Siduction and was getting a message that os-prober wasn't picking up any other distros, and managed to find a fix for it (the one I describe above, which worked; I'm now posting from a different distro).

What I'm puzzled by is why the grub developers would make that change in the first place, which seems to reduce rather than extend grub's capabilities?

* P.S. Unklar, well guessed! One reason why I didn't mention the distro I was using was that when I googled for a solution I saw that other distros such as Manjaro had the same issue, so I wanted my thread to be as general as possible.

Last edited by Colonel Panic (2021-12-02 15:14:55)

Offline

#5 2021-12-02 15:22:36

rbh
Moderator
From: Sweden/Vasterbotten/Rusfors
Registered: 2016-08-11
Posts: 1,345

Re: A fix for grub when os-prober isn't working

Colonel Panic wrote:

What I'm puzzled by is why the grub developers would make that change in the first place, which seems to reduce rather than extend grub's capabilities?

From the grub manual v 2.06:
"‘GRUB_DISABLE_OS_PROBER’
The grub-mkconfig has a feature to use the external os-prober program to discover other operating systems installed on the same machine and generate appropriate menu entries for them. It is disabled by default since automatic and silent execution of os-prober, and creating boot entries based on that data, is a potential attack vector. Set this option to ‘false’ to enable this feature in the grub-mkconfig command."

In Grub 2.04, it is absent but can be added. When running BL/Debian from usb HD, I usually prefer to disable os-prober.


// Regards rbh

Please read before requesting help: Guide to getting help,
Introduction to the Bunsenlabs Lithium Desktop and other help topics under "Help Resources" on the BunsenLabs menu

Offline

#6 2021-12-02 15:34:15

sleekmason
zoom
Registered: 2018-05-22
Posts: 691
Website

Re: A fix for grub when os-prober isn't working

Thank you for posting this. 
@rbh, What are your concerns about attacks or whatever with osprober?

I was going to fix this in Lilidog as well, but it sounds like it is no longer a good idea?
That's a bummer. 

What are the other distributions doing about it? Just letting it go I suppose.

Offline

#7 2021-12-02 16:10:38

unklar
Back to the roots 1.9
From: #! BL
Registered: 2015-10-31
Posts: 1,656

Re: A fix for grub when os-prober isn't working

For me as a multi-user, this is quite a big step forward.

This automatic os-prober caused the grub.cfg of the main bootloader to become more and more bloated, errors occurred and the cleanup was always tedious.
That's why years ago I switched to removing the execution rights of all os-prober involved:

chmod -x 30_os-prober

Only the os-prober of my main bootloader remained untouched.
So the innovation does not really affect me.   tongue

Offline

#8 2021-12-02 20:07:08

rbh
Moderator
From: Sweden/Vasterbotten/Rusfors
Registered: 2016-08-11
Posts: 1,345

Re: A fix for grub when os-prober isn't working

sleekmason wrote:

 
@rbh, What are your concerns about attacks or whatever with osprober?

None for the multibooting desktop well safeguarded behind firewalls.
If I would setup an public server, I sure would disable os_prober.

I was going to fix this in Lilidog as well, but it sounds like it is no longer a good idea?

I think it is a good default to disable os_prober.
If you need to use it, put up your shields and regularly check your systems integrity.


// Regards rbh

Please read before requesting help: Guide to getting help,
Introduction to the Bunsenlabs Lithium Desktop and other help topics under "Help Resources" on the BunsenLabs menu

Offline

#9 2021-12-02 21:19:47

Colonel Panic
Member
Registered: 2018-11-13
Posts: 402

Re: A fix for grub when os-prober isn't working

Thanks for the replies rbh, unklar and sleekmason (and unklar, I quite agree that os-prober leaves a messy file to clean up).

It doesn't look like it's particularly safe to use os-prober; are there any good alternatives to it (and for that matter, any viable alternative boot managers to grub in 2021)?

Last edited by Colonel Panic (2021-12-02 21:22:03)

Offline

#10 2021-12-03 00:38:56

sleekmason
zoom
Registered: 2018-05-22
Posts: 691
Website

Re: A fix for grub when os-prober isn't working

Iv'e never had an issue with it, but I am pretty much just testing my own stuff, or close enough to it.

Kinda sounds like leaving it out for others is the way to go long term. I guess it is easy enough to write a quick snippet to follow as a fix if anyone wants to use it.

The below is part of the research/testing I did while writing the kernel guide. https://forums.bunsenlabs.org/viewtopic.php?id=4870

Maybe one of these below can be helpful? On my personal stuff, I will probably still add osprober to the build, but not for others.  When building, adding GRUB_DISABLE_OS_PROBER=false to /etc/default/grub, causes grub during the install process to ask which version of the /etc/default/grub file you want to use.
Handy I suppose, but maybe a bit much for anyone who doesn't know what is going on during the install process. I do think grub is the way to go overall though. very versatile. Good topic:)

--------------------------

Another way to add a distro to the Grub boot menu, is by placing the below into /etc/grub.d/40_custom. In this example, I am dual booting with antiX as my second distribution on sda5.

menuentry 'Antix' {
    set root=(hd0,5)
    linux /vmlinuz root=/dev/sda5 ro quiet
    initrd /initrd.img
}

Then:

update-grub

If you want, you can replace; root=/dev/sda5, with root=UUID=$12345uuid... where $12345uuid... IS the actual UUID of /dev/sda5. This method may be more reliable, as fewer errors can develop from improper partition naming.

Yet one more way, and my preferred method, is with chainloading.  This snippet is also placed in /etc/grub.d/40_custom.  What it does is place a final menu entry that opens the grub screen of the other distro.  In this example, I am booting "Patience" on sda5:

menuentry 'Chainload patience (auf /dev/sda5)' {
	set root='hd0,msdos5'
	configfile /boot/grub/grub.cfg
}

Last edited by sleekmason (2021-12-03 00:39:30)

Offline

#11 2021-12-03 02:23:35

DeepDayze
Like sands through an hourglass...
From: In Linux Land
Registered: 2017-05-28
Posts: 1,436

Re: A fix for grub when os-prober isn't working

Colonel Panic wrote:

Thanks for the replies rbh, unklar and sleekmason (and unklar, I quite agree that os-prober leaves a messy file to clean up).

It doesn't look like it's particularly safe to use os-prober; are there any good alternatives to it (and for that matter, any viable alternative boot managers to grub in 2021)?

I believe I've seen somewhere that there might be an internal os-prober to do the work so that no external program needs to be called to enumerate the OS's on the partition(s) and create the entries for those OS's detected.

Last edited by DeepDayze (2021-12-03 02:24:13)


Real Men Use Linux

Offline

#12 2021-12-03 09:16:41

AndrewSmart
Member
Registered: 2019-06-10
Posts: 41

Re: A fix for grub when os-prober isn't working

rbh wrote:

When running BL/Debian from usb HD, I usually prefer to disable os-prober.

I run BL/Debian from usb flashdrive. `apt-get purge grub2` :o)

When needed I install and use GRUB2 within a chroot. No need for GRUB2 on main system.

On the usb flashdrive, os-prober would error, it couldn't figure out the device under the / overlayfs used in live-persistence. IIRC that error would occur when update-grub was triggered by apt-get and would prevent apt-get from installing things! Awful.

`apt-get purge grub2` and the chroot is salvation from that predicament. IIRC os-prober doesn't error within the chroot like it does on main system. Also I never use update-grub/grub-mkconfig either, just grub-install.

Not sure if I'd tried the GRUB_DISABLE_OS_PROBER=TRUE option, or if that option existed back then. I know I commented out the os-prober line in the relevant script to get apt to work again, before I ended up with `apt-get purge grub2` and the chroot solution.

I edit grub.cfg manually. I couldn't get the grub-mkconfig.d wizard stuff to output what I wanted- less effort to just edit grub.cfg manually.

That is my learned no-headaches workflow with GRUB2.

Last edited by AndrewSmart (2021-12-03 13:31:01)

Offline

#13 2021-12-03 13:00:42

Colonel Panic
Member
Registered: 2018-11-13
Posts: 402

Re: A fix for grub when os-prober isn't working

Thanks Andrew (and DeepDayze and sleekmason too). The thing about grub is that despite its flaws it works just well enough to deter you from going elsewhere, and I'd guess that the same is true of os-prober. Manual editing of the grub.cfg file sounds like a good idea except that you need to know what the UUIDs are.

Last edited by Colonel Panic (2021-12-03 13:53:39)

Offline

#14 2021-12-03 13:21:56

unklar
Back to the roots 1.9
From: #! BL
Registered: 2015-10-31
Posts: 1,656

Re: A fix for grub when os-prober isn't working

DeepDayze wrote:
Colonel Panic wrote:

Thanks for the replies rbh, unklar and sleekmason (and unklar, I quite agree that os-prober leaves a messy file to clean up).

It doesn't look like it's particularly safe to use os-prober; are there any good alternatives to it (and for that matter, any viable alternative boot managers to grub in 2021)?

I believe I've seen somewhere that there might be an internal os-prober to do the work so that no external program needs to be called to enumerate the OS's on the partition(s) and create the entries for those OS's detected.

I am not aware of anything significantly new regarding bootloaders.

My favorites are still

systemd-boot (bootctl)
coreboot (libreboot)

The former would be the easiest for me. But that requires (U)efi, which none of my machines have. Tratitionally I was always an opponent of "efi".

Second, coreboot can be done by myself. Mentally, however, I feel too old for it and the hardware available for it would also already be 10 years and older.

Now I have always squinted after something younger. Here, however, my government has imposed a "budget freeze" devil on me that I have now said to myself before Christmas, well, you might as well wait for the right machinebig_smile


@AndrewSmart, you should check it out.

Offline

#15 2021-12-03 13:32:34

unklar
Back to the roots 1.9
From: #! BL
Registered: 2015-10-31
Posts: 1,656

Re: A fix for grub when os-prober isn't working

Colonel Panic wrote:

Thanks Andrew (and DeepDayze and sleekmason too). The thing about grub is that despite its flaws it works just well enough to deter you from going elsewhere, and the same is true of os-prober. Manual editing of the grub.cfg file sounds like a good idea except that you need to know what the UUIDs are.

Manual editing of grub.cfg is a "no go", for everyone (experts included) and should only be done in really exceptional cases.
I have not had such a case yet.

Why don't you use the ones provided for that:
40_custom
41_custom

or your own,
42_my_script etc.  ?

Last edited by unklar (2021-12-03 13:33:11)

Offline

#16 2021-12-03 13:34:59

AndrewSmart
Member
Registered: 2019-06-10
Posts: 41

Re: A fix for grub when os-prober isn't working

I've warmed up to EFI since I first grumbled about having to learn something new. Not too bad once I figured out the --removable and --no-nvram flags to grub-install. EDIT: Also that a 4MB FAT12 EFI System Partition (ESP) works fine, contrary to most everyone saying you need at least a 100MB FAT32 partition.

sleekmason wrote:

If you want, you can replace; root=/dev/sda5, with root=UUID=$12345uuid... where $12345uuid... IS the actual UUID of /dev/sda5. This method may be more reliable, as fewer errors can develop from improper partition naming.

Yet one more way, and my preferred method, is with chainloading.  This snippet is also placed in /etc/grub.d/40_custom.  What it does is place a final menu entry that opens the grub screen of the other distro.  In this example, I am booting "Patience" on sda5:

menuentry 'Chainload patience (auf /dev/sda5)' {
	set root='hd0,msdos5'
	configfile /boot/grub/grub.cfg
}

In the Debian buster->bullseye upgrade notes, they wrote a significant change is that the /dev/sd* device letter assignments by the kernel are no longer deterministic from boot to boot! So using the UUID is the only reliable way to go.
EDIT: Reference: https://www.debian.org/releases/bullsey … erministic

Changing that `root=/dev/sda5` kernel parameter to `root=/dev/disk/by-uuid/$my_uuid`, where:

set my_uuid=ea4649a0-39c1-408b-9942-f7efbf7d7e37

I presume the GRUB hd0,msdos5 remains deterministic (no idea really) but I think it'd be best anyway to do that by UUID to handle changes:

#Instead of:
set root='hd0,msdos5'

#Do search by uuid:
set my_uuid=ea4649a0-39c1-408b-9942-f7efbf7d7e37
search --no-floppy --fs-uuid $my_uuid --set=root  # $root is now e.g. (hd0,5), or (hd1,5), etc

I think that's how it should work.

Last edited by AndrewSmart (2021-12-03 14:10:21)

Offline

#17 2021-12-03 13:51:25

AndrewSmart
Member
Registered: 2019-06-10
Posts: 41

Re: A fix for grub when os-prober isn't working

unklar wrote:
Colonel Panic wrote:

Thanks Andrew (and DeepDayze and sleekmason too). The thing about grub is that despite its flaws it works just well enough to deter you from going elsewhere, and the same is true of os-prober. Manual editing of the grub.cfg file sounds like a good idea except that you need to know what the UUIDs are.

Manual editing of grub.cfg is a "no go", for everyone (experts included) and should only be done in really exceptional cases.
I have not had such a case yet.

Why don't you use the ones provided for that:
40_custom
41_custom

or your own,
42_my_script etc.  ?

Off the top of my head I think the 10_linux one was non-functional. And I couldn't figure out how to get the 40_custom one to be the default selection. Every time I'd boot without manually selecting the 40_custom I'd have to reboot and try again. Might be more issues I don't remember. I do remember being upset and going back to manually editing grub.cfg smile

Offline

#18 2021-12-03 14:50:56

DeepDayze
Like sands through an hourglass...
From: In Linux Land
Registered: 2017-05-28
Posts: 1,436

Re: A fix for grub when os-prober isn't working

Colonel Panic wrote:

Thanks Andrew (and DeepDayze and sleekmason too). The thing about grub is that despite its flaws it works just well enough to deter you from going elsewhere, and I'd guess that the same is true of os-prober. Manual editing of the grub.cfg file sounds like a good idea except that you need to know what the UUIDs are.

Yes you can add your own menu entries manually, even for your Windows partition(s). However you have to ensure the syntax is correct or it will fail. To get the UUID's blkid is your friend!

There are some tips and tricks for managing your Grub2 setups all over the place.

Note: I liked the old legacy grub's single config file arrangement and too bad Grub2 was made more complex.

Last edited by DeepDayze (2021-12-03 14:52:33)


Real Men Use Linux

Offline

#19 2021-12-03 15:15:30

unklar
Back to the roots 1.9
From: #! BL
Registered: 2015-10-31
Posts: 1,656

Re: A fix for grub when os-prober isn't working

AndrewSmart wrote:

I presume the GRUB hd0,msdos5 remains deterministic (no idea really) but I think it'd be best anyway to do that by UUID to handle changes:

I think you are confusing grub with the kernel   wink

set root='hd0,msdos5'

is the naming convention of grub2. You can't just change this to the UUID of the partition at this point.

AndrewSmart wrote:

Off the top of my head I think the 10_linux one was non-functional. And I couldn't figure out how to get the 40_custom one to be the default selection. Every time I'd boot without manually selecting the 40_custom I'd have to reboot and try again. Might be more issues I don't remember. I do remember being upset and going back to manually editing grub.cfg

Usually the scripts of grub2 look like this:

00_header
05_debian_theme
10_linux
20_linux_xen
20_memtest86+
30_os-prober
30_uefi-firmware
40_custom
41_custom

and their order has this purpose:

wiki_ubuntu_de wrote:

As you can see, the script names are headed by a number (XX_). The scripts are processed in ascending order according to this number. If two scripts have the same number, their further alphanumeric sequence determines the processing.

A script with the designation 10_own (script) is called before 10_linux for processing. In this way the order of the menu entries can be influenced.

A classic way to do this is to rename the file 30_os-prober to 09_os-prober in the /etc/grub.d/ directory in order to arrange the Windows entries before the Linux entries.

Offline

#20 2021-12-03 16:25:16

jeffreyC
Member
Registered: 2019-09-07
Posts: 89

Re: A fix for grub when os-prober isn't working

On any computer built with EFI/UEFI you could look at using rEFInd boot manager, it probes for installed OSes so osprober is unneeded and can work with the boot loader built into the Linux kernel, so GRUB is unneeded.

It is in the Debian repos.

http://www.rodsbooks.com/refind/

ETA:
I do not use it, yet, but think with the need to install/upgrade I very well may.

Last edited by jeffreyC (2021-12-03 16:27:04)

Offline

#21 2021-12-03 18:58:03

rbh
Moderator
From: Sweden/Vasterbotten/Rusfors
Registered: 2016-08-11
Posts: 1,345

Re: A fix for grub when os-prober isn't working

Colonel Panic wrote:

Thanks for the replies rbh, unklar and sleekmason (and unklar, I quite agree that os-prober leaves a messy file to clean up).

It doesn't look like it's particularly safe to use os-prober; are there any good alternatives to it (and for that matter, any viable alternative boot managers to grub in 2021)?

Note that on a multiboot pc, you must disable os-prober for all os but one, the one you use most. Then there will never be any mess. No need to abandon grub...


// Regards rbh

Please read before requesting help: Guide to getting help,
Introduction to the Bunsenlabs Lithium Desktop and other help topics under "Help Resources" on the BunsenLabs menu

Offline

#22 2021-12-03 19:11:56

sleekmason
zoom
Registered: 2018-05-22
Posts: 691
Website

Re: A fix for grub when os-prober isn't working

Just saw this and thought it to be relevant. Based on grub.
https://launchpad.net/burg

*Edit - or not.  Doesn't look active there anyway.

Last edited by sleekmason (2021-12-03 19:13:29)

Offline

#23 2021-12-03 19:26:25

rbh
Moderator
From: Sweden/Vasterbotten/Rusfors
Registered: 2016-08-11
Posts: 1,345

Re: A fix for grub when os-prober isn't working

AndrewSmart wrote:
rbh wrote:

When running BL/Debian from usb HD, I usually prefer to disable os-prober.

I run BL/Debian from usb flashdrive. `apt-get purge grub2` yikes)

When needed I install and use GRUB2 within a chroot. No need for GRUB2 on main system.

On the usb flashdrive, os-prober would error, it couldn't figure out the device under the / overlayfs used in live-persistence.

I want to have a full installation on usb-hd. Nice to use when travelling and using public pc. Also good good for rescue-missions.

When installing live system to flashdrive, I think the drive boots with syslinux, hence no need there for grub.

Not sure if I'd tried the GRUB_DISABLE_OS_PROBER=TRUE option, or if that option existed back then.

Think it came with GRU2 in 2014.


// Regards rbh

Please read before requesting help: Guide to getting help,
Introduction to the Bunsenlabs Lithium Desktop and other help topics under "Help Resources" on the BunsenLabs menu

Offline

#24 2021-12-04 04:01:35

Colonel Panic
Member
Registered: 2018-11-13
Posts: 402

Re: A fix for grub when os-prober isn't working

rbh wrote:
Colonel Panic wrote:

Thanks for the replies rbh, unklar and sleekmason (and unklar, I quite agree that os-prober leaves a messy file to clean up).

It doesn't look like it's particularly safe to use os-prober; are there any good alternatives to it (and for that matter, any viable alternative boot managers to grub in 2021)?

Note that on a multiboot pc, you must disable os-prober for all os but one, the one you use most. Then there will never be any mess. No need to abandon grub...

Right, so if I disable os-prober for any other distro on my hard drive but Bunsen, I'll be safe (and not leave a mess in my grub.cfg file) if I then run update-grub in Bunsen?

Offline

#25 2021-12-04 09:13:21

unklar
Back to the roots 1.9
From: #! BL
Registered: 2015-10-31
Posts: 1,656

Re: A fix for grub when os-prober isn't working

Colonel Panic wrote:

Right, so if I disable os-prober for any other distro on my hard drive but Bunsen, I'll be safe (and not leave a mess in my grub.cfg file) if I then run update-grub in Bunsen?

Since the os-prober of bunsen reads the grub.cfg of all other distributions, an update-grub must also be executed there afterwards when their os-prober is deactivated.
So that these grub.cfg's are "clean".

Finally the update-grub in bunsen.
Now understood?  big_smile

Offline

Board footer

Powered by FluxBB