Repair Your System Using BunsenLabs Live and Chroot

tknomanzr · 2016-01-29 18:34:28

While I regularly regularly follow our topic Tracking Debian Testing/Unstable with Bunsenlabs, sometimes systems tracking Sid can break horribly and we need to be able to repair a system that we are unable to boot.

For this topic, I deliberately broke my system, in order to repair it via chroot. First, let's see what packages were involved in the breakage. This output will be useful later when we go to repair the system:

The following packages will be REMOVED:
xserver-xorg-input-all xserver-xorg-input-evdev xserver-xorg-video-all xserver-xorg-video-intel xserver-xorg-video-openchrome xserver-xorg-video-siliconmotion

as per Imbecil. While it is generally accepted that messing about with xorg's stuff can lead to unbootable systems and/or giant headaches, I went ahead with the dist-upgrade for the specific purpose of later repairing it. Sure enough, a full dist-upgrade led to a system, that while it was bootable, was unable to accept any input, which was pretty much what I expected would happen.

The two packages in question (for me at least, your system might differ) were xserver-xorg-input-all and xserver-xorg-input-evdev. A little digging around in aptitude was able to show me that these were the affected packages. I don't use Aptitude all the time but it is a very useful tool for tracking through dependency chains to determine a point of breakage.

But first, we need to set up our chroot environment. For this, I used a BunsenLabs live usb key. I then had to get a working internet connection (a separate issue, but be aware you do need internet).

1. Get the Required Binaries

sudo apt install binutils debootstrap

2. Create a Mountpoint

sudo mkdir -p /mnt/chroot

3. Determine root partition using lsblk

lsblk -f

You are looking for the root partition of the Debian install that you are attempting to repair
Sample code:

tknomanzr@wtfbox-bl:~$ lsblk -f
NAME   FSTYPE LABEL           UUID                                 MOUNTPOINT
sda                                                                
├─sda1 ntfs   System Reserved 2494DC2D94DC02EA                     
└─sda2 ntfs                   0E4CE9A64CE98933                     
sdb                                                                
├─sdb1 ext4                   48e37151-9bbd-4ed9-bc0f-a0e28ac7fbef /
├─sdb2 swap                   9b545b59-ee1a-48df-af01-4df11537d6e4 [SWAP]
└─sdb3 ext4                   d5cd252f-7808-4edd-9ff7-731cd06d9622 /home

so we can see that in my instance, the root partition was mounted to /dev/sdb1

4. Mount the Necessary Filesystems

sudo mount -t ext4 /dev/sdb1 /mnt/chroot

If you have a separate /boot partition, you will need to mount it as well. Here is a sample for a separate ext2 /boot partition. If you are using GPT partition scheme for UEFI support, you will need to mount the /boot partition as the type listed in your output from lsblk. <-- I am not able to confirm this, as I boot in legacy mode.
The next step is to mount /proc, /dev and /sys partitions.

sudo mount -t proc none /mnt/chroot/proc
sudo mount -o bind /dev /mnt/chroot/dev
sudo mount -o bind /sys /mnt/chroot/sys

5. Copy /etc/resolv.conf over to the chroot

sudo cp /etc/resolv.conf /mnt/chroot/etc/resolv.conf

This is important if you need internet from the chroot environment.

6. Enter the chroot Environment

Once all necessary filesystems are in place, chroot into your system:

sudo chroot /mnt/chroot

7. Perform Necessary Repairs

Once you have successfully chrooted into your system, any command you issue will be issued against that system. So, for instance, we can use apt to install our missing xserver-xorg packages.

sudo apt update
sudo apt install xserver-xorg-input-evdev xserver-xorg-input-all
sudo apt dist-upgrade

Now, keep in mind that you may still need to use apt with the -f flag to fix missing dependencies, use aptitude to track back which packages is causing apt to fail and if it is failing and unavailable, be ready to wait it out til the offending package gets fixed.
Once I rebooted, I could determine that installing the necessary packages via chroot did indeed fix my install.

EDITS:

Added stanza about copying /etc/resolv.conf over if internet is needed.

Last edited by tknomanzr (2016-01-30 13:19:57)

Head_on_a_Stick · 2016-01-29 19:49:20

Nice guide tk, thanks!

I think it's OK to leave out the "-t" bit when using mount(8), it will do that automagically

I'm lazy so I use:

for i in /sys /proc /dev; do mount --bind $i /mnt/chroot$i; done

to bind mount /proc /sys & /dev

unklar · 2016-01-29 21:15:51

It should not be a chroot necessary.
Or do you use here another Debian?

http://forum.siduction.org/index.php?to … 8#msg49728

tknomanzr · 2016-01-30 00:46:24

Indeed, I was not aware of that little trick to get to a tty. I still feel like being able to repair from a chroot if necessary is a good thing to know. One reason I posted the walkthrough was so I had an easily searchable reference if it ever becomes a necessity.

dolly · 2016-01-30 06:46:39

Thanks for the guide teknomanzer. Just in case I will need it.

johnraff · 2016-01-30 07:45:44

Many thanks for this!

(You've already seen this discussion on #!? It is a bit old though...)

Snap · 2016-01-30 07:54:35

Great howto, tknomanzr. Thanks.

tknomanzr · 2016-01-30 13:15:13

If I had, it didnt make sense to me at the time. I did notice that I forgot to mention the need to copy etc/resolv.conf over to the chroot. I will fix the OP with that info.
Learning how to properly chroot has been something I have been wanting to understand for sometime. The Debian wiki is somewhat obtuse about it, as all of it's chroot stuff directly referenced debootstrap, which is fine for building systems, but makes no mention of how to go about repairing one.
Sadly, Crunchbang's search relevancy seems to be gradually fading as I get less and less hits from there in search engine results. Thanks do @xaos for the original post on the #! forums, though.

Last edited by tknomanzr (2016-01-30 13:29:22)

nobody0 · 2016-01-30 15:41:04

^ Without getting to the internet, you wouldn't have done #7 of your OP, would you?

Edit: Anyway, have a look here about using chroot.

Last edited by nobody0 (2016-01-30 15:48:43)

johnraff · 2016-02-07 07:52:23

A recent blog post on using chroot to repair Sid: http://shallowsky.com/blog/linux/instal … hroot.html

Eon · 2017-11-04 07:24:02

So, apparently I'm in a need to repair my BunsenLabs setup. As I am running BL in a dual boot setup with Windows 10,and I recently updated Windows 10 to Fall Creators Update, when I boot, it shows this:

 
error: unknown filesystem
Entering rescue mode... 
grub rescue>

Seeing that the above guide was tested on sid, I would like to ask if it is reliable for me to follow? Since I am using stable BL setup.

Or should I create a new topic in Basic Help & Support section?

tknomanzr · 2017-11-04 08:24:35

The same methods will work for Stable. I just listed Sid because that is where I need to do these types of repairs.

Head_on_a_Stick · 2017-11-04 11:55:38

Eon wrote:

should I create a new topic in Basic Help & Support section?

Please do, I am interesting in finding out what Windows has done to your system.

If you have a UEFI system then this guide should work:

https://wiki.debian.org/GrubEFIReinstall

The only difference will be that GRUB's loader will be placed at /boot/efi/EFI/bunsenlabs/grubx64.efi

If you have a non-UEFI system then a simple chroot & `grub-install` should fix things.

Eon · 2017-11-04 12:18:17

Hi HoaS,
Sorry, but seems the issue has been fixed. I simply followed this guide: https://forums.bunsenlabs.org/viewtopic … 598#p52598

It seems Windows update had removed grub from EFI system partition, or modified it. So all I did is chroot and reinstalled GRUB. This time again, you saved my day. So thank you very much.

Head_on_a_Stick · 2017-11-04 14:25:17

^ Nice work

Take that Redmond! ]:D

BLizgreat! · 2017-11-04 23:06:13

Just another thanks and job well done Tknomanzr. Learned to actually do this originally via that Xaos52 how-to and it's good to see people keeping information like this current. Cause absolutely have to agree, it's the cats meow and has saved me tech-gawds only know how much time over the years fixing system breakage.

Went stable branch, haven't messed with Sid in quite awhile. Simply due to updatitus ( a flood of updates) and potential borkage. Though seldom had any problems while tracking unstable, even without taking practical precautions. Often neglecting to, it was surprisingly stable in my experience.

PS, Oops ... thanks Hoas too, being able to combine mounting xyz into one terminal command is really convenient and cool to know.

Last edited by BLizgreat! (2017-11-04 23:08:32)

Head_on_a_Stick · 2017-11-04 23:18:00

BLizgreat! wrote:

being able to combine mounting xyz into one terminal command is really convenient and cool to know

The Bourne shell syntax is rather clumsy, I prefer Plan9's native rc(1) shell for that sort of thing, the compound commands are very elegant:

for(i in proc sys dev dev/pts)mount --bind /$i /mnt/$

https://packages.debian.org/stretch/9base

unklar · 2017-11-05 13:49:18

^^ I'm not sure since when systemd that supports, but...

In the age of systemd chroots you now like this, which simplifies things a lot

mount /dev/sdXY /mnt
systemd-nspawn -D /mnt

then

grub-install ... etc pp
exit

With bunsenlabs I have already tested this in a VM and it actually provides me with a container. In this one I can work like in the previous chroot, so it is only a command (!) to get there.

https://www.freedesktop.org/software/sy … spawn.html
https://wiki.archlinux.org/index.php/Systemd-nspawn

Last edited by unklar (2017-11-05 13:57:18)

Head_on_a_Stick · 2017-11-05 13:54:41

^ Yes, systemd-nspawn can be very useful but the safety features enabled for the container may prevent GRUB from installing correctly in certain configurations, this is why I recommend the old-fashioned `chroot` method, it's only an extra line anyway

unklar · 2017-11-05 13:58:54

Yes, HoaS,

of course, I don't have your knowledge.

#1 2016-01-29 18:34:28