Administrators Jon Posted May 8, 2011 Administrators Posted May 8, 2011 It looks like one of the disks in the server has died. Server will go offline at some point for a replacement. Hopefully the ISP support guys replace the correct one and I can resync the RAID array. Prepare for outages.
Administrators Jon Posted May 9, 2011 Author Administrators Posted May 9, 2011 Replacement drive has been hotswapped into the server but I need to reboot it in the next few hours and put it in rescue mode so I can properly sync up the drives/add boot records to the new drive to prepare for future disk fails. One of two things will happen: 1. I'll reboot and not be able to fix/sync the drives or get the server to boot properly and I'll have to reimage the server from scratch and restore from backup 2. It'll all be fine...
Administrators Jon Posted May 9, 2011 Author Administrators Posted May 9, 2011 I think the new disk might actually have worked. I'm so glad my ISP provides a direct serial terminal connection to the server and a "rescue" environment to help with this. Disk recovery might take a while though from this progress report: Personalities : [raid1] md3 : active raid1 sdb3[0] sda3[2] 970888192 blocks [2/1] [U_] [>....................] recovery = 0.8% (8610240/970888192) finish=783.2min speed=20474K/sec md1 : active raid1 sdb1[1] sda1[0] 3911680 blocks [2/2] [UU] Once that's finished I'll probably reboot with a file system check for good measure.
Administrators Jon Posted May 10, 2011 Author Administrators Posted May 10, 2011 Server rebooted and disk check succeeded - everything should be ok now.
Recommended Posts