Project

General

Profile

Feature #14090

Move to SDDs after failure of one HDD in the RAID array of the domUs

Added by Philippe May almost 2 years ago. Updated almost 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Start date:
23/05/2022
Due date:
% Done:

0%

Close

Related issues

Related to GIS - Support #14072: Gisaf very slow Resolved 18/05/2022

History

#1 Updated by Philippe May almost 2 years ago

#2 Updated by Philippe May almost 2 years ago

Discussed with Giulio last Saturday about the options and there are few options.

A simple option is to just replace the 2TB failing HDD and add it to the existing RAID array.

Another option is to switch to SSDs. We have 400 GB free space on the SDD (which hosts only the data of the dom0, a 30GB partition). The space required for the domUs is currently around 50 GB.

The cost difference is really minor (and actually in favor of small size SSD).

#3 Updated by Philippe May almost 2 years ago

So i decided to move all domUs to SSD in RAID.

This requires to buy a new SSD (for ex. same model CT480BX500SSD1). Will ask CSR for this purchase.

mdadm can create a degraded array with only 1 drive so the process can be started immediately.

#4 Updated by Philippe May almost 2 years ago

  • Created a 400GB partition (type: Linux RAID) on /dev/sdc4
  • Create the array
mdadm --create /dev/md1 --level=1 --raid-devices=2 /dev/sdc4 missing
  • Add the array to the VG
root@dream:~# vgextend dream.csr /dev/md1
  Physical volume "/dev/md1" successfully created.
  Volume group "dream.csr" successfully extended

#5 Updated by Philippe May almost 2 years ago

Received the new SSD and installed it physically yesterday in the server (/dev/sdb).

Formatted following almost the same structure than the other SSD (using cfdisk):

root@dream:~# fdisk -l /dev/sdb
Disk /dev/sdb: 465.76 GiB, 500107862016 bytes, 976773168 sectors
Disk model: CT500MX500SSD1  
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 81A9A107-F811-D442-9B8D-F5EA7EFFFBCF

Device        Start       End   Sectors   Size Type
/dev/sdb1      2048   1050623   1048576   512M EFI System
/dev/sdb2   1050624  63965183  62914560    30G Linux filesystem
/dev/sdb3  63965184 904148991 840183808 400.6G Linux RAID

Added the RAID partition /dev/sdb3 to the RAID1 created previously, which so far had only one partition of the other SSD /dev/sdc4:

mdadm  /dev/md127 --add /dev/sdb3

Use pvmove command (see https://www.tecmint.com/lvm-storage-migration/) to migrate the "extends" (lvs) of the vg. For example:

pvmove -n /dev/dream.csr/infra.csr.av-disk -A y /dev/md0 /dev/md127

... done without even stopping the domUs!

This command is handy to see the location of the lvs on the pvs (below, after moving all lvs):

root@dream:~# lvs -o+devices
  LV                  VG        Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert Devices          
  gisaf2.csr.av-disk  dream.csr -wi-ao----  10.00g                                                     /dev/md127(37120)
  gisaf2.csr.av-swap  dream.csr -wi-ao----   1.00g                                                     /dev/md127(36864)
  gisdb.csr.av-disk   dream.csr -wi-ao----  10.00g                                                     /dev/md127(34304)
  gisdb.csr.av-swap   dream.csr -wi-ao----   1.00g                                                     /dev/md127(34048)
  infra.csr.av-disk   dream.csr -wi-ao----  10.00g                                                     /dev/md127(26112)
  infra.csr.av-swap   dream.csr -wi-ao----   1.00g                                                     /dev/md127(25856)
  jupyter.csr.av-disk dream.csr -wi-ao----  20.00g                                                     /dev/md127(28928)
  jupyter.csr.av-swap dream.csr -wi-ao----   1.00g                                                     /dev/md127(28672)
  samba.csr.av-disk   dream.csr -wi-a----- 100.00g                                                     /dev/md127(0)    
  samba.csr.av-swap   dream.csr -wi-a-----   1.00g                                                     /dev/md127(25600)

Finally remove the old RAID1:

root@dream:~# vgreduce dream.csr /dev/md0 
  Removed "/dev/md0" from volume group "dream.csr" 

Done!

#6 Updated by Philippe May almost 2 years ago

  • Status changed from New to Resolved
  • Subject changed from Reorganize storage disks on server after failure of one HDD in the RAID array of the domUs to Move to SDDs after failure of one HDD in the RAID array of the domUs

Also available in: Atom PDF