How To Set Up Software RAID1 On A Running System (Incl. GRUB Configuration) (Debian Etch)

Version 1.0
Author: Falko Timme

This guide explains how to set up software RAID1 on an already running Debian Etch system. The GRUB bootloader will be configured in such a way that the system will still be able to boot if one of the hard drives fails (no matter which one).

I do not issue any guarantee that this will work for you!

 

1 Preliminary Note

In this tutorial I'm using a Debian Etch system with two hard drives, /dev/sda and /dev/sdb which are identical in size. /dev/sdb is currently unused, and /dev/sda has the following partitions:

  • /dev/sda1: /boot partition, ext3;
  • /dev/sda2: swap;
  • /dev/sda3: / partition, ext3

In the end I want to have the following situation:

  • /dev/md0 (made up of /dev/sda1 and /dev/sdb1): /boot partition, ext3;
  • /dev/md1 (made up of /dev/sda2 and /dev/sdb2): swap;
  • /dev/md2 (made up of /dev/sda3 and /dev/sdb3): / partition, ext3

This is the current situation:

df -h

server1:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda3             4.4G  729M  3.4G  18% /
tmpfs                 126M     0  126M   0% /lib/init/rw
udev                   10M   56K   10M   1% /dev
tmpfs                 126M     0  126M   0% /dev/shm
/dev/sda1             137M   12M  118M  10% /boot
server1:~#

fdisk -l

server1:~# fdisk -l

Disk /dev/sda: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          18      144553+  83  Linux
/dev/sda2              19          80      498015   82  Linux swap / Solaris
/dev/sda3              81         652     4594590   83  Linux

Disk /dev/sdb: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sdb doesn't contain a valid partition table
server1:~#

 

2 Installing mdadm

The most important tool for setting up RAID is mdadm. Let's install it like this:

apt-get install initramfs-tools mdadm

You will be asked the following question:

MD arrays needed for the root filesystem: <-- all

Afterwards, we load a few kernel modules (to avoid a reboot):

modprobe md
modprobe linear
modprobe multipath
modprobe raid0
modprobe raid1
modprobe raid5
modprobe raid6
modprobe raid10

Now run

cat /proc/mdstat

The output should look as follows:

server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
unused devices: <none>
server1:~#

 

3 Preparing /dev/sdb

To create a RAID1 array on our already running system, we must prepare the /dev/sdb hard drive for RAID1, then copy the contents of our /dev/sda hard drive to it, and finally add /dev/sda to the RAID1 array.

First, we copy the partition table from /dev/sda to /dev/sdb so that both disks have exactly the same layout:

sfdisk -d /dev/sda | sfdisk /dev/sdb

The output should be as follows:

server1:~# sfdisk -d /dev/sda | sfdisk /dev/sdb
Checking that no-one is using this disk right now ...
OK

Disk /dev/sdb: 652 cylinders, 255 heads, 63 sectors/track

sfdisk: ERROR: sector 0 does not have an msdos signature
 /dev/sdb: unrecognized partition table type
Old situation:
No partitions found
New situation:
Units = sectors of 512 bytes, counting from 0

   Device Boot    Start       End   #sectors  Id  System
/dev/sdb1   *        63    289169     289107  83  Linux
/dev/sdb2        289170   1285199     996030  82  Linux swap / Solaris
/dev/sdb3       1285200  10474379    9189180  83  Linux
/dev/sdb4             0         -          0   0  Empty
Successfully wrote the new partition table

Re-reading the partition table ...

If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
to zero the first 512 bytes:  dd if=/dev/zero of=/dev/foo7 bs=512 count=1
(See fdisk(8).)
server1:~#

The command

fdisk -l

should now show that both HDDs have the same layout:

server1:~# fdisk -l

Disk /dev/sda: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          18      144553+  83  Linux
/dev/sda2              19          80      498015   82  Linux swap / Solaris
/dev/sda3              81         652     4594590   83  Linux

Disk /dev/sdb: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           1          18      144553+  83  Linux
/dev/sdb2              19          80      498015   82  Linux swap / Solaris
/dev/sdb3              81         652     4594590   83  Linux
server1:~#

Next we must change the partition type of our three partitions on /dev/sdb to Linux raid autodetect:

fdisk /dev/sdb

server1:~# fdisk /dev/sdb

Command (m for help):
 <-- m
Command action
   a   toggle a bootable flag
   b   edit bsd disklabel
   c   toggle the dos compatibility flag
   d   delete a partition
   l   list known partition types
   m   print this menu
   n   add a new partition
   o   create a new empty DOS partition table
   p   print the partition table
   q   quit without saving changes
   s   create a new empty Sun disklabel
   t   change a partition's system id
   u   change display/entry units
   v   verify the partition table
   w   write table to disk and exit
   x   extra functionality (experts only)

Command (m for help):
 <-- t
Partition number (1-4): <-- 1
Hex code (type L to list codes): <-- L

 0  Empty           1e  Hidden W95 FAT1 80  Old Minix       be  Solaris boot
 1  FAT12           24  NEC DOS         81  Minix / old Lin bf  Solaris
 2  XENIX root      39  Plan 9          82  Linux swap / So c1  DRDOS/sec (FAT-
 3  XENIX usr       3c  PartitionMagic  83  Linux           c4  DRDOS/sec (FAT-
 4  FAT16 <32M      40  Venix 80286     84  OS/2 hidden C:  c6  DRDOS/sec (FAT-
 5  Extended        41  PPC PReP Boot   85  Linux extended  c7  Syrinx
 6  FAT16           42  SFS             86  NTFS volume set da  Non-FS data
 7  HPFS/NTFS       4d  QNX4.x          87  NTFS volume set db  CP/M / CTOS / .
 8  AIX             4e  QNX4.x 2nd part 88  Linux plaintext de  Dell Utility
 9  AIX bootable    4f  QNX4.x 3rd part 8e  Linux LVM       df  BootIt
 a  OS/2 Boot Manag 50  OnTrack DM      93  Amoeba          e1  DOS access
 b  W95 FAT32       51  OnTrack DM6 Aux 94  Amoeba BBT      e3  DOS R/O
 c  W95 FAT32 (LBA) 52  CP/M            9f  BSD/OS          e4  SpeedStor
 e  W95 FAT16 (LBA) 53  OnTrack DM6 Aux a0  IBM Thinkpad hi eb  BeOS fs
 f  W95 Ext'd (LBA) 54  OnTrackDM6      a5  FreeBSD         ee  EFI GPT
10  OPUS            55  EZ-Drive        a6  OpenBSD         ef  EFI (FAT-12/16/
11  Hidden FAT12    56  Golden Bow      a7  NeXTSTEP        f0  Linux/PA-RISC b
12  Compaq diagnost 5c  Priam Edisk     a8  Darwin UFS      f1  SpeedStor
14  Hidden FAT16 <3 61  SpeedStor       a9  NetBSD          f4  SpeedStor
16  Hidden FAT16    63  GNU HURD or Sys ab  Darwin boot     f2  DOS secondary
17  Hidden HPFS/NTF 64  Novell Netware  b7  BSDI fs         fd  Linux raid auto
18  AST SmartSleep  65  Novell Netware  b8  BSDI swap       fe  LANstep
1b  Hidden W95 FAT3 70  DiskSecure Mult bb  Boot Wizard hid ff  BBT
1c  Hidden W95 FAT3 75  PC/IX
Hex code (type L to list codes):
 <-- fd
Changed system type of partition 1 to fd (Linux raid autodetect)

Command (m for help):
 <-- t
Partition number (1-4): <-- 2
Hex code (type L to list codes): <-- fd
Changed system type of partition 2 to fd (Linux raid autodetect)

Command (m for help):
 <-- t
Partition number (1-4): <-- 3
Hex code (type L to list codes): <-- fd
Changed system type of partition 3 to fd (Linux raid autodetect)

Command (m for help):
 <-- w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
server1:~#

To make sure that there are no remains from previous RAID installations on /dev/sdb, we run the following commands:

mdadm --zero-superblock /dev/sdb1
mdadm --zero-superblock /dev/sdb2
mdadm --zero-superblock /dev/sdb3

If there are no remains from previous RAID installations, each of the above commands will throw an error like this one (which is nothing to worry about):

server1:~# mdadm --zero-superblock /dev/sdb1
mdadm: Unrecognised md component device - /dev/sdb1
server1:~#

Otherwise the commands will not display anything at all.

Share this page:

14 Comment(s)

Add comment

Comments

From:

Thanks for a great howto! It was a life saver, as I have never set up RAID before, let alone on a running system. The install did not go perfectly however and so I thought I might share my notes and a couple of suggestions. Luckily I did a backup of the entire system before beginning, so I was able to restore the system and begin again after I could not get the system to boot off the RAID array. N.B. I did the install on a Debian testing system (lenny/amd64), but I've checked that everything applies to etch as well.

1. If the disks are not brand new, mdadm will detect the previous filesystem when creating the array and ask if you want to continue. Answer, 'yes'. I also got a segfault error from mdadm and a warning that the disk was 'dirty'. The warning could probably be avoided by zeroing the entire disk with dd. Despite the error and warning everything worked as it should.

2. Instead of: cp /etc/mdadm/mdadm.conf /etc/mdadm/mdadm.conf_orig mdadm --examine --scan >> /etc/mdadm/mdadm.conf do: mv /etc/mdadm/mdadm.conf /etc/mdadm/mdadm.conf_orig /usr/share/mdadm/mkconf >> /etc/mdadm/mdadm.conf This will create a proper mdadm.conf and remove the control file in /var/lib/mdadm (if arrays are found). If you do not remove the control file, you should get a warning message when updating the initrd images.

3. When editing GRUBS's menu.lst, I followed the advice in the comments and put the stanza for the RAID array, before the '### BEGIN AUTOMAGIC KERNELS LIST' line. If you put your custom stanzas inside the AUTOMAGIC area, they will be overwritten during the next kernel upgrade. Instead of: update-initramfs -u I had to do: dpkg-reconfigure mdadm When asked to specify the arrays needed for the root filesystem, I answered with the appropriate devices (in my case only /dev/md0) instead of selecting the default, 'all'. Otherwise I kept to the default answers. After the initrd images had been created, I updated GRUB: update-grub

4. Instead of using the GRUB shell, I used grub-install to install the boot loader on the hard drives: grub-install /dev/sda grub-install /dev/sdb

5. After having added both disks to the arrays, it was time to update the initrd again. First I executed: dpkg-reconfigure mdadm and was informed that the initrd would not be updated, because it was a custom image. The configure script informed me that I could force the update by running 'update-initramfs' with the '-t' option, so that is what I did: update-initramfs -u -t

6. Every time you update the initrd image, you also have to re-install GRUB in the MBR's: grub-install /dev/sda grub-install /dev/sdb Otherwise the system will not boot and you will be thrown into the GRUB shell. Other notes: It is normal for 'fdisk -l' to report stuff like, 'Disk /dev/md0 doesn't contain a valid partition table'. This is because fdisk cannot read md arrays correctly. If you forget to re-install GRUB in the MBR's, after updating your initrd, and get the GRUB shell on reboot. Do the following: Boot from a Debian Installer CD (full or netinst) of the same architecture as your install (so if you're running amd64, it has to be a amd64 CD). Boot the CD in 'rescue' mode. After networking has been set up and the disks have been detected press CTRL+ALT+F2, followed by Enter, to get a prompt. Execute the following commands (md0=/boot and md2=/): mkdir /mnt/mydisk mount /dev/md2 /mnt/mydisk mount /dev/md0 /mnt/mydisk/boot mount -t proc none /mnt/mydisk/proc mount -o bind /dev /mnt/mydisk/dev chroot /mnt/mydisk grub-install /dev/sda grub-install /dev/sdb exit umount /mnt/mydisk/proc umount /mnt/mydisk/dev umount /mnt/mydisk/boot umount /mnt/mydisk reboot

From:

Thank you so much for this detailed howto.  It saved me a lot of pain and worked perfectly on Ubuntu 8.04.1 LTS.

From: nochids

I am relatively new to linux and am completely dependent on these tutorials.  I bought a server and installed Suse 10.3.  After running Ubuntu on my desktop and laptop, I decided to change the server to run Ubuntu as well (I didn't uninstall Suse - just installed over it??).  After installing the server based on "The Perfect Ubuntu Server 8.04" (http://www.howtoforge.com/perfect-server-ubuntu8.04-lts) I installed the ISPConfig as detailed at the end.  Then to install RAID, I followed the tutorial perfectly I think,  but at the end of step 6 after rebooting, I still show sda1 rather than md0.

 [email protected]:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/costarica-root
                      228G  1.4G  216G   1% /
varrun                502M  108K  502M   1% /var/run
varlock               502M     0  502M   0% /var/lock
udev                  502M   76K  502M   1% /dev
devshm                502M     0  502M   0% /dev/shm
/dev/sda1             236M   26M  198M  12% /boot
[email protected]:~#

 vi /etc/fstab shows the following:

# /etc/fstab: static file system information.
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
proc            /proc           proc    defaults        0       0
# /dev/mapper/costarica-root
UUID=2e3442d3-c650-480a-a923-4775de238b7f /               ext3    relatime,errors=remount-ro,usrquota,grpquota 0       1
# /dev/md0
UUID=251a68c2-1497-433b-b415-d49ca8f2125e /boot           ext3    relatime        0       2
# /dev/mapper/costarica-swap_1
UUID=03c6c32e-38bd-4707-9df5-dcdd3049825a none            swap    sw              0       0
/dev/scd0       /media/cdrom0   udf,iso9660 user,noauto,exec,utf8 0       0

 

It looks different than the one in the tutorial but I attribute that to being ubuntu rather than debian.

 

Can anyone shed some light on this for me?  Did I miss a step or are there other steps involved because of Ubuntu?

 

Thanks for any help.

 

Jason.

 

From: Anonymous

Dear all,

 this howto just worked for me flawlessly for my brand-new Debian Lenny (testing) today (03-Jan-2009) !!!

No issues, no problems at all. I had several different partitions, even extended ones, I only had to follow on paper, which partition goes into which numbered array - that's it ;-)

(And my boot partition wasn't /boot but simply /, I did everything accordingly - flawless!!!)

 

THANK YOU VERY MUCH for this HowTo, I've NEVER EVER raid-ed before and it's a success :)

md0 = /

md1 = swap

md2 = /home

This all on an Abit NF7-S2, BIOS-Raid OFF, 2 x SATA2 Samsung 320G, Sempron 2800+, 2x512 DDR400 ;-)

 

 lol:~# df -m
Filesystem           1M-blocks      Used Available Use% Mounted on
/dev/md0                 18778      2312     15512  13% /
tmpfs                      506         0       506   0% /lib/init/rw
udev                        10         1        10   2% /dev
tmpfs                      506         0       506   0% /dev/shm
/dev/md2                280732       192    266281   1% /home
lol:~#


Cheers from Europe !

 

From: Anonymous

Great tutorial,

my compliments

From: Johan Boulé

I wonder, what's the point in having the swap on a raid1? shouldn't it be better to add /dev/sda2 and /dev/sdb2 directly as two separate swap devices?

From: Froi

Can I apply this How-to to my PPC Debian Etch? Thanks!

From: nord

Nice howto!  
I would like to correct some minor errors though:

Ext2 on boot instead of ext3... Ext3 on /boot is just a waste of space and resources.  You dont need journaling for your boot partition :)

and why make a raid array for swap?  swap will stripe data as a raid0 anyway.. just tell linux to swap to two different physical disks and voila. Striping made easy :p

Happy raiding :p

(If you suddenly need a lot of swapspace, you can use "swapon" command to swap to memorysticks or whatever you need, unlike fstab fixing, swapon wil get resetted on reboot)  ;)

From: Andy Beverley

I spent hours trying to work out not only how to set up a software RAID, but also how to do it on a boot partition. I didn't even come close to looking at a live system. I got nowhere until I found this HOWTO which does it all very well. Thank you!

Andy

From: Anonymous

It works just perfectly with ubuntu 8.04

Thanks for the brilliant how-to

From: Alex Dekker

You might like to put a link somewhere in this howto to your newer howto detailing the install with Grub2. I spent some time following this howto and tripping up on Grub2 and doing lots of googling, before finally realising that what I thought were google hits on your existing howto were actually pointing to a separate but very similarly named howto, that covers Grub2!

From: bob143

I did manage to lose all my existing data following this. I was not doing this with a root partition so I had no issues with partitions being in use and I specified both disks in the create command  rather than the "missing" placeholder - maybe that was my problem.

From: Losteron

Hi thanks for the tutorial.

I got a question about /etc/fstab file.

My File contains must UUIDs not sda or sdb.

Can I just replace them with sda and sdb?

 

Regards

From: linux fan

Of course, you mean not "cp /etc/mdadm/mdadm.conf_orig /etc/mdadm/mdadm.conf" but "cp /etc/mdadm/mdadm.conf /etc/mdadm/mdadm.conf_orig", right?