Linux Server

Server Won't Boot? Use Rescue Mode to Mount Disks, Fix fstab and GRUB, and Copy Your Data Off

10 steps 14 min read 7 views 0
On this page

A server that will not boot usually still has all its data. The cause is often small: a wrong line in /etc/fstab, a kernel update that broke the bootloader, a full disk, a file system that needs checking, or a firewall rule that locked you out. A rescue system boots from the network instead of your disks, so you can mount the disks, fix the problem and boot normally again, or copy your data to safety first. This guide covers the common repairs for Linux servers.

On IMIDC dedicated servers, start the rescue system with the Rescue Mode button in the client area and work in it through the VNC Console. See the dedicated server control panel guide. On other providers the steps inside the rescue system are the same.

Step 1: Find Out Why It Does Not Boot

Before changing anything, open the console and look at the last messages on the screen:

  • "You are in emergency mode" or "Dependency failed for /data": usually a bad fstab entry or a missing disk. Fix fstab (Step 4).
  • "grub rescue>", "no such partition" or "Reboot and select proper boot device": the bootloader is damaged. Reinstall GRUB (Step 6).
  • Kernel panic after an update: boot the previous kernel from the GRUB menu if it appears, otherwise rebuild the initramfs in the rescue system (Step 6).
  • The system boots but you cannot connect: check the network and firewall from the console first; you may not need rescue mode at all (Step 7).

Step 2: Identify the Disks

In the rescue system, list disks and partitions:

lsblk -f
fdisk -l

Look for the root partition (often the largest ext4 or xfs partition) and, on UEFI servers, a small vfat partition (the EFI system partition). If you see linux_raid_member or LVM2_member, assemble those first:

# Software RAID (mdadm)
mdadm --assemble --scan
cat /proc/mdstat

# LVM
vgscan
vgchange -ay
lvs

Step 3: Check the File System

Run a file system check while the partition is not mounted. Replace the device names with yours:

# ext4
fsck -f /dev/sda2

# XFS: try a normal mount first, use xfs_repair only if mounting fails
xfs_repair /dev/mapper/vg0-root
Never run fsck or xfs_repair on a mounted file system. Avoid xfs_repair -L unless a normal repair is impossible: it discards the log and can lose recent writes. If the data is important, copy it off first (Step 8).

Step 4: Mount the System and Fix fstab

mkdir -p /mnt/sys
mount /dev/sda2 /mnt/sys          # your root partition or LV
ls /mnt/sys                       # you should see etc, home, var ...
nano /mnt/sys/etc/fstab

Comment out the line for a disk that no longer exists by adding # in front of it, and add nofail to optional data disks so a missing disk cannot stop the boot again:

UUID=1234-abcd  /data  ext4  defaults,nofail  0  2

Use UUIDs from blkid rather than names like /dev/sdb1, which can change between boots. Mounting disks correctly is explained in partitioning and mounting a data disk.

Disk full

If the root file system is 100% full, services fail and some systems stop booting. With the disk mounted, find and remove large files, typically old logs or backups:

du -xh /mnt/sys --max-depth=2 2>/dev/null | sort -h | tail -20

More cleanup ideas are in freeing up disk space on Linux.

Step 5: Enter the Installed System with chroot

For repairs that need the system's own tools (GRUB, package manager, passwd), chroot into it:

mount /dev/sda1 /mnt/sys/boot            # only if /boot is a separate partition
mount /dev/sda15 /mnt/sys/boot/efi       # UEFI only: the vfat EFI partition
for d in dev proc sys run; do mount --bind /$d /mnt/sys/$d; done
chroot /mnt/sys /bin/bash

You are now working inside your installed system. To reset a forgotten root password here, just run passwd root; other methods are in resetting a Linux root password.

Step 6: Reinstall GRUB and Rebuild the Initramfs

Inside the chroot:

# Debian / Ubuntu, legacy BIOS boot
grub-install /dev/sda
update-grub
update-initramfs -u -k all

# Debian / Ubuntu, UEFI boot
grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=debian
update-grub

# Rocky / AlmaLinux / CentOS
grub2-mkconfig -o /boot/grub2/grub.cfg
dracut -f --regenerate-all

With software RAID, run grub-install on every member disk (for example /dev/sda and /dev/sdb) so the server can boot from either.

Step 7: Undo a Firewall or SSH Lockout

If the server boots but you cannot connect after changing the firewall or SSH settings, fix it from the chroot (or directly in the VNC console if the system still boots):

# Check the SSH configuration for errors
sshd -t

# Temporarily disable the firewall at next boot
systemctl disable ufw 2>/dev/null; systemctl disable firewalld 2>/dev/null

After you are back in, re-enable the firewall with the right port open, as in opening ports with firewalld and ufw.

Step 8: Copy Your Data Off

If the repair fails or you plan to reinstall, copy important data to another server first. From the rescue system:

# Copy a directory to a backup server over SSH
rsync -aHAX --info=progress2 /mnt/sys/var/www/ user@backup-server:/backup/www/

# Stream a compressed archive without needing local space
tar -C /mnt/sys -czf - home etc var/lib/mysql | ssh user@backup-server "cat > /backup/server.tar.gz"

For databases, copying the raw files from a stopped server works when you restore them to the same database version. Otherwise, boot the system once it is repaired and use mysqldump.

Step 9: Leave Rescue Mode

exit                                     # leave the chroot
for d in run sys proc dev; do umount /mnt/sys/$d; done
umount -R /mnt/sys
sync

Then exit rescue mode (on IMIDC: Current Task → Exit Rescue) so the server boots from its own disks, and watch the boot in the VNC console.

Windows Servers

Choose the Windows rescue type to start a Windows environment where you can copy files from the system drive, or run chkdsk C: /f and bcdboot C:\Windows to repair the boot files.

FAQ

lsblk shows no disks

The RAID controller may need a driver the rescue system lacks, or a disk may have failed. Open a ticket so our engineers can check the hardware.

Should I just reinstall?

Only after your data is safe. A reinstall wipes the disks; the rescue system lets you take a copy first.

Still stuck after following these steps? Open a support ticket and the IMIDC 24/7 technical team will help. Include the server IP, a screenshot of the boot error from the console and the output of lsblk -f so we can pinpoint the issue faster.

Was this answer helpful?

Related Tutorials