OS or root disk mirroring in Solaris 8
Yes, a blast from the past, but as I have to do this, and patch upgrades, for a client soon, I felt like refreshing my memory. In the following example, I will be truncating output to refer only to the disks that I am interested in.
First, what are we working with:
So we are working with a Solaris 8 system, a Sun-Fire V440, end of life was 2007!
# uname -a SunOS syd71 5.8 Generic_108528-27 sun4u sparc SUNW,Sun-Fire-V440
Any mirroring in place?
Use the unix df command to list mounted filesystems; if the filesystem path contains either “md” or “vx” then some form of volume management is in use. In this case, there doesn’t appear to be any.
syd71# df -k
Filesystem kbytes used avail capacity Mounted on
/dev/dsk/c1t0d0s0 4034088 1695730 2298018 43% /
/proc 0 0 0 0% /proc
fd 0 0 0 0% /dev/fd
mnttab 0 0 0 0% /etc/mnttabc
/dev/dsk/c1t0d0s3 3008649 1498560 1449917 51% /var
swap 9056952 24 9056928 1% /var/run
swap 9057000 72 9056928 1% /tmp
/dev/dsk/c6t600C0FF0000000000048CB0F02547800d0s0
70310728 53873312 15031204 79% /data2
/dev/dsk/c6t600C0FF0000000000048CB4F7554FE00d0s0
70310728 2584548 66319968 4% /data1
Best also check the swap file configuration:
syd71# swap -l swapfile dev swaplo blocks free /dev/dsk/c1t0d0s1 32,9 16 12289776 12289776
As you can see from the highlighted line above, the swap partition is only on the root disk, slice 1.
Is Solstice Disksuite installed?
We are going to use Solaris Volume Manager (SVM; formerly known as Online: DiskSuite, and later Solstice DiskSuite). This is a software package for creating, modifying and controlling RAID-0 (concatenation and stripe) volumes, RAID-1 (mirror) volumes, RAID 0+1 volumes, RAID 1+0 volumes, RAID-5 volumes, and soft partitions. It has been superseded by ZFS in Solaris 10 onwards, and many users employ Veritas Volume Manager (VxVM) as an alternative. It’s free, and was always a good option for root disk mirroring. From this point onwards I will refer to it as SVM as everything else is just too long to type!
syd71# pkginfo -i | grep SUNWmd system SUNWmdb Modular Debugger system SUNWmdbdm Modular Debugger Demo Source system SUNWmdbx Modular Debugger (64-bit) system SUNWmdg Solstice DiskSuite Tool system SUNWmdi Sun Multipath I/O Drivers system SUNWmdiu Sun Multipath I/O Drivers (usr) system SUNWmdix Sun Multipath I/O Drivers (64-bit) system SUNWmdnr Solstice DiskSuite Log Daemon Configuration Files system SUNWmdnu Solstice DiskSuite Log Daemon system SUNWmdr Solstice DiskSuite Drivers system SUNWmdu Solstice DiskSuite Commands system SUNWmdx Solstice DiskSuite Drivers(64-bit)
What version of SVM is it?
syd71# pkginfo SUNWmdr system SUNWmdr Solstice DiskSuite Drivers syd71# pkginfo -l SUNWmdr PKGINST: SUNWmdr NAME: Solstice DiskSuite Drivers CATEGORY: system ARCH: sparc VERSION: 4.2.1,REV=1999.12.03.10.00 BASEDIR: / VENDOR: Sun Microsystems, Inc. DESC: Solstice DiskSuite Drivers PSTAMP: 12/03/99-10:06:11 INSTDATE: Jun 27 2005 16:50 VSTOCK: 258-6252-11 HOTLINE: Please contact your local service provider STATUS: completely installed FILES: 28 installed pathnames 8 shared pathnames 8 directories 11 executables 1027 blocks used (approx)
Version 4.2.1, which is probably the latest available for Solaris 8.
Do we have some free disks in order to set up mirroring?
Use the format command to list disks configured on the system
# format
AVAILABLE DISK SELECTIONS:
0. c1t0d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107>
/pci@1f,700000/scsi@2/sd@0,0
1. c1t2d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107>
/pci@1f,700000/scsi@2/sd@2,0
2. c1t3d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107>
/pci@1f,700000/scsi@2/sd@3,0
3. c6t600C0FF0000000000048CB0F02547800d0 <SUN-StorEdge3510-327P cyl 34873 alt 2 hd 64 sec 64>
/scsi_vhci/ssd@g600c0ff0000000000048cb0f02547800
4. c6t600C0FF0000000000048CB4F7554FE00d0 <SUN-StorEdge3510-327P cyl 34873 alt 2 hd 64 sec 64>
/scsi_vhci/ssd@g600c0ff0000000000048cb4f7554fe00
5. c6t600C0FF0000000000048CB594EB54200d0 <SUN-StorEdge3510-327P cyl 35424 alt 2 hd 127 sec 127>
/scsi_vhci/ssd@g600c0ff0000000000048cb594eb54200
Specify disk (enter its number):
The first disk, highlighted on line 3, is the existing root disk. The disks highlighted on lines 5 and 7 are possible mirror candidates as their size and geometry are identical to the existing root disk.
Verify that nothing is using the selected mirror disks by initially examining the partition tables. On the root disk we see a selection of specifically sized and labeled partitions (highlighted):
format> verify Primary label contents: Volume name = < > ascii name = <SUN36G cyl 24620 alt 2 hd 27 sec 107> pcyl = 24622 ncyl = 24620 acyl = 2 nhead = 27 nsect = 107 Part Tag Flag Cylinders Size Blocks 0 root wm 0 - 2835 3.91GB (2836/0/0) 8193204 1 swap wu 2836 - 7089 5.86GB (4254/0/0) 12289806 2 backup wm 0 - 24619 33.92GB (24620/0/0) 71127180 3 var wm 7090 - 9216 2.93GB (2127/0/0) 6144903 4 unassigned wm 9217 - 12052 3.91GB (2836/0/0) 8193204 5 home wm 12053 - 24104 16.60GB (12052/0/0) 34818228 6 unassigned wm 0 0 (0/0/0) 0 7 unassigned wm 0 0 (0/0/0) 0
Note the following about the root disk:
- Partition 2 (line 15) represents the entire disk and is rarely used, except for Oracle ASM
- Partition 6 and 7 haven’t been used, and are available for SVM meta databases
- The disk has 24619 usable cylinders, of which 24104 are used. The remainder will be used for partition 7 meta databases
And on the selected mirror disk, we see what appears to be a standard “empty” partition:
Specify disk (enter its number)[0]: 1 selecting c1t2d0 [disk formatted] format> verify Primary label contents: Volume name = <> ascii name = pcyl = 24622 ncyl = 24620 acyl = 2 nhead = 27 nsect = 107 Part Tag Flag Cylinders Size Blocks 0 root wm 0 - 90 128.37MB (91/0/0) 262899 1 swap wu 91 - 181 128.37MB (91/0/0) 262899 2 backup wu 0 - 24619 33.92GB (24620/0/0) 71127180 3 unassigned wm 0 0 (0/0/0) 0 4 unassigned wm 0 0 (0/0/0) 0 5 unassigned wm 0 0 (0/0/0) 0 6 usr wm 182 - 24619 33.67GB (24438/0/0) 70601382 7 unassigned wm 0 0 (0/0/0) 0
And there is nothing unusual in the vfstab (i.e. no mentions of the selected mirror disk being used in any way):
syd71# cat vfstab #device device mount FS fsck mount mount #to mount to fsck point type pass at boot options # #/dev/dsk/c1d0s2 /dev/rdsk/c1d0s2 /usr ufs 1 yes - fd - /dev/fd fd - no - /proc - /proc proc - no - /dev/dsk/c1t0d0s1 - - swap - no - /dev/dsk/c1t0d0s0 /dev/rdsk/c1t0d0s0 / ufs 1 no logging /dev/dsk/c1t0d0s3 /dev/rdsk/c1t0d0s3 /var ufs 1 no logging /dev/dsk/c1t0d0s5 /dev/rdsk/c1t0d0s5 /apps ufs 2 yes logging /dev/dsk/c1t0d0s4 /dev/rdsk/c1t0d0s4 /opt ufs 2 yes logging /dev/dsk/c6t600C0FF0000000000048CB0F02547800d0s0 /dev/rdsk/c6t600C0FF0000000000048CB0F02547800d0s0 /data2 ufs 2 yes logging,forcedir ectio /dev/dsk/c6t600C0FF0000000000048CB4F7554FE00d0s0 /dev/rdsk/c6t600C0FF0000000000048CB4F7554FE00d0s0 /data1 ufs 2 yes logging,forcedir ectio /dev/dsk/c6t600C0FF0000000000048CB594EB54200d0s0 /dev/rdsk/c6t600C0FF0000000000048CB594EB54200d0s0 /data ufs 2 yes logging,forcedir ectio swap - /tmp tmpfs - yes -
Copy the partition setup to the chosen mirror disk:
SVM is not a “whole disk” mirroring solution, rather it works on a partition by partition basis. The target partition needs to exist before you set it up as a mirror. When attempting to mirror a complete disk, the easy way to do this is to use the prtvtoc command to print out the “volume table of contents” and then write it to the target disk:
prtvtoc /dev/rdsk/c1t0d0s2 | fmthard -s - /dev/rdsk/c1t2d0s2
Note 1: Be sure you get the drive names absolutely correct, or you WILL destroy data! This will NOT ask you if you are sure, and there is NO WAY to undo this if you get it wrong!
Note 2: It’s a good idea to add the partition for your SVM state databases before opting the partition table to the second disk. You will need this partition in the next step.
Establish the state databases:
These small databases contain all of the information SVM needs to operate, and the configuration information of the RAID volumes we create.
metadb -af -c 3 /dev/dsk/c1t0d0s7 metadb -af -c 3 /dev/dsk/c1t2d0s7
The parameters above are:
- –a: add databases
- –f: force, basically create the initial databases
- –c: the number of copies in case of database corruption. Additionally SVM requires a majority of databases to be available for correct operation; in an ideal world you would place additional copies on a third drive. In reality most people configure a parameter to override this requirement.
Create the RAID devices:
Next we initialise the partitions which we want to mirror. From the root disk partition table listed above, they are:
- 0 — root
- 1 — swap
- 3 — var
- 4 — opt
- 5 — apps
metainit -f d01 1 1 c1t0d0s0 metainit -f d11 1 1 c1t0d0s1 metainit -f d31 1 1 c1t0d0s3 metainit -f d41 1 1 c1t0d0s4 metainit -f d51 1 1 c1t0d0s5
metainit -f d02 1 1 c1t2d0s0 metainit -f d12 1 1 c1t2d0s1 metainit -f d32 1 1 c1t2d0s3 metainit -f d42 1 1 c1t2d0s4 metainit -f d52 1 1 c1t2d0s5
The “1 1″ parameter tells metainit to create 1 stripe with 1 partition.
Create the Mirrors:
Next we create the parent mirror devices, and attach the first side, the existing root disk:
metainit d00 -m d01 metainit d10 -m d11 metainit d30 -m d31 metainit d40 -m d41 metainit d50 -m d51
We won’t add the second side of the mirrors until we configure the system to boot from mirror device, and perform a reboot.
Configure the root device:
Now that we have SVM mirror devices created, we need to configure Solaris to access them correctly. SVM creates new device files under /dev/md/dsk and /dev/md/rdsk and unless the vfstab file is updated to point to these new device files, it will not be taking advantage of the RAID configuration. For the root partition, the metaroot command will perform this task for you. If it only did that, it would be a pretty useless command, as you need to edit the vfstab file to update the other partition entries as well. Luckily, it also updates the /etc/system kernel configuration file with the necessary entries for SVM.
metaroot d00
Modify vfstab for any other root disk mounts:
Now we need to update the vfstab entries for swap, var, opt, apps:
#device device mount FS fsck mount mount #to mount to fsck point type pass at boot options # #/dev/dsk/c1d0s2 /dev/rdsk/c1d0s2 /usr ufs 1 yes - fd - /dev/fd fd - no - /proc - /proc proc - no - /dev/md/dsk/d10 - - swap - no - /dev/md/dsk/d00 /dev/md/rdsk/d10 / ufs 1 no logging /dev/md/dsk/d30 /dev/md/rdsk/d30 /var ufs 1 no logging /dev/md/dsk/d50 /dev/md/rdsk/d50 /apps ufs 2 yes logging /dev/md/dsk/d40 /dev/md/rdsk/d40 /opt ufs 2 yes logging /dev/dsk/c6t600C0FF0000000000048CB0F02547800d0s0 /dev/rdsk/c6t600C0FF0000000000048CB0F02547800d0s0 /data2 ufs 2 yes logging,forcedir ectio /dev/dsk/c6t600C0FF0000000000048CB4F7554FE00d0s0 /dev/rdsk/c6t600C0FF0000000000048CB4F7554FE00d0s0 /data1 ufs 2 yes logging,forcedir ectio /dev/dsk/c6t600C0FF0000000000048CB594EB54200d0s0 /dev/rdsk/c6t600C0FF0000000000048CB594EB54200d0s0 /data ufs 2 yes logging,forcedir ectio swap - /tmp tmpfs - yes -
Lock the filesystem and shutdown the server:
To ensure that no further changes are made to the filesystems prior to booting onto the new volumes, run the locks command and reboot immediately.
# lockfs -fa # /usr/sbin/shutdown -y -g0 -i0
While at the OBP prompt, configure device aliases and the boot-device parameter to automatically boot from either drive in the event of a failure:
% show-disks
Pick your mirrored disk from the list and then setup an alias:
% nvalias mirror ^y (that is a <control>-y to paste the device path)
Change your boot-device to first try the normal disk alias, then use your mirror disk:
% setenv boot-device disk mirror % reset-all
When the system is booting, you will see errors relating to SVM drivers as follows:
forceload of misc/md_trans failed forceload of misc/md_raid failed forceload of misc/md_hotspares failed
You can safely ignore these messages, as they relate to SVM elements which we haven’t created. If you wish, you can create a volume for transaction logs, a RAID 5 volume, and define a disk as a hotspare in order to remove these errors 
Alternatively, you could edit out those driver references in the /etc/system file.
Attach the second mirrors:
metattach d00 d02 metattach d10 d12 metattach d30 d32 metattach d40 d42 metattach d50 d52
After doing the met attach, there will be lots of disk I/O while the mirrors sync. You can use the metastat command to view status of operation:
metastat | grep State | egrep -v Okay
Other things to fix, the Solaris boot block and the State Database Qurom:
In the event of a primary disk failure, the system should keep running from the second drive. If you reboot though, the system may not start up as there is no “bootblock” on the second disk. Let’s fix that:
# installboot /usr/platform/`uname -i`/lib/fs/ufs/bootblk /dev/rdsk/c1t2d0s0
If you only have two mirrored root disks (DiskSuite 4.2.1, Solaris 8), put this setting in your /etc/system:
set md:mirrored_root_flag=1
And you’re done.