Upload
others
View
13
Download
0
Embed Size (px)
Citation preview
Scott Gilliland
LVM and Raid
Why aren't flat partitions enough?
Redundancy
Speed
Live changes
Multipath I/O
Shared partitions
Raid Levels
►Raid-0: Block-level Striping
►Raid-1: Mirroring
►Raid-2: Byte striping
►Raid-3: Byte striping w/ parity disk
►Raid-4: Block-level striping with parity disk
►Raid-5: Block-level striping with distributed parity to cover the loss of any disk
►Raid-6: Block-level striping with distributed parity to cover the loss of any two disks
Raid Levels
►Raid-0: Block-level Striping
►Raid-1: Mirroring
►Raid-2: Byte striping
►Raid-3: Byte striping w/ parity disk
►Raid-4: Block-level striping with parity disk
►Raid-5: Block-level striping with distributed parity to cover the loss of any disk
►Raid-6: Block-level striping with distributed parity to cover the loss of any two disks
Raid 0
►Reads and writes are split between disks
♦ For n disks, rates go up by n
►Loss of any disk looses all data
♦ In theory, you still have n-1/n of your data, but really, you lose your data.
►Great if you have lots of disks, but don't care about your data
Raid 1
►N copies of all data
►Each write has to update all disks
♦ Ow, that's slow!
►Reads can be spread across all disks
♦ Like Raid 0, but without the data loss!
►Great if you only have two disks, and want redundancy♦ Like if you're building a machine with old drives
♦ For more n≥3, use Raid 5
Raid 5
►Distributes the parity information across all disks
►Can loose any one disk
►Reads use n-1/n disks
►Writes write to all n disks♦ Or worse – a read of the whole stripe may be needed
►Stripe sizes matter
♦ Especially for large numbers of small,disparate reads
►Usually a safe bet for 3 or more disks
Raid 6
►Usually defined as being Raid 5, but with twice the parity
►Can loose up to two disks
►Preferred over 'hot-spares'
♦ If you have an extra disk, you may as well use it
♦ How else will you know if your hot spare is bad?
Linux Software Raid
►Combines multiple block devices into one
♦ Disks or partitions
►Often called the MD system
♦ Stands for Multi Disk
►Modern arrays use a 'Superblock'
♦ Contains disk UID and array config● Allows for auto-detection
♦ Actually at end of disk
►Configured via mdadm
Linux Software Raid (cont)
►Status from /proc/mdstat
♦ And more modernly /sys/md
►You can even stack Raid sets
♦ Raid 0+1, Raid 0+5
►Can be 'reshaped' live
►Can be set to come up on boot
♦ But your bootloader will need a flat disk● Or it will need to 'understand' RAID
Typical Raid 1
►Two disks, one partition each
►Partitions mirrored as /dev/md0
►Each has a bootloader, set to boot with root=/dev/md0
►If boot disk fails, other disk can boot to the degraded array
scott@galileo:~$ cat /proc/mdstatPersonalities : [raid1]md0 : active raid1 hda1[0] hdc1[1] 78123968 blocks [2/2] [UU]
unused devices: <none>
Typical Raid 5
►A single or mirrored set of disks to boot
►A Raid 5 set as root, or as storage♦ The mirrored set often covers all disks, as same-
sized disks will all have the same extra space
♦ Here, the spare space is striped together as /tmp
-bash-3.2$ cat /proc/mdstatPersonalities : [raid6] [raid5] [raid4] [raid0]md1 : active raid0 sdc1[0] sdd1[1] sde1[2] sdf1[3] 10248192 blocks 256k chunks
md0 : active raid5 sdf2[5] sde2[4] sdd2[3] sdc2[2] sdb3[1] sda3[0] 4870987520 blocks level 5, 256k chunk, algorithm 2 [6/6][UUUUUU]
mdadm
Seems to be the best way to change an array
Create:# mdadm --create /dev/md0 --level=5 --raid-devices=3 \ /dev/sdc1 /dev/sdd1 /dev/sde1
Set as faulty, remove, add:# mdadm --manage /dev/md0 --fail /dev/sdc1# mdadm --manage /dev/md0 --remove /dev/sdc1# mdadm --manage /dev/md0 --add /dev/sdc1
LVM (and LVM2)
►Can be considered an alternative to traditional partitioning♦ Can use any block device: partitions and full disks
►Allows for rearranging of storage, mirroring, striping, and snapshots
►With a daemon, can handle shared storage and multipath♦ Ex: two servers with a shared SAN disk between
them
►Does not do Raid 5 or 6.
♦ Or Raid 2, Raid 3, or Raid 4.
LVM Basics
►3 main types of entities:
♦ PV – Physical Volume● This is a disk you want to store data on.
♦ VG – Volume Group● A pool of PV's on a machine. You usually just want
one.
♦ LV – Logical Volume● An exposed block device. May be spanned,
striped, mirrored, or a 'snapshot'.
LVM Snapshots
►Allows you to take a full-disk image at a single point in time
►Uses Copy-on-write
♦ Only needs additional space when the device is written
♦ Needs physical space allocated for changes
►Useful for taking backups that are self-consistent
♦ Make snapshot, backup contents off-disk, erase snapshot
LVM configuration
►Command-line tools are simplest♦ All the details in LVM(8)
►System-config-lvm is the Redhat GUI tool♦ Only handles LVM, making MD+LVM configurations
difficult
►EVMS tool has an LVM2 module♦ Not actively maintained?
♦ Also has an MD module for software RAID.
►KVPM is a kde4-based manager♦ Seems kinda beta, but works
♦ Can parse MD devices
♦ Handles filesystems
Command-line LVM
Most of:[pv|vg|lv][change|convert|create|display|extend|reduce|remove|rename|resize|s|scan]
lvm> pvs PV VG Fmt Attr PSize PFree /dev/md0 test lvm2 a- 1.91G 1.62G /dev/sdc2 test lvm2 a- 1.44G 974.00M /dev/sde2 test lvm2 a- 1.39G 1.20Glvm> vgs VG #PV #LV #SN Attr VSize VFree test 3 2 0 wz--n- 4.75G 3.77Glvm> lvs LV VG Attr LSize Origin Snap% Move Log Copy% Convert mirrored test mwn-a- 300.00M mirrored_mlog 100.00 striped test -wn-a- 400.00M
Command-line LVM
PVS, VGS, and LVS can take additional fields
scott@bravil:/usr/src/kvpm-0.4.5b$ sudo pvs -o pv_fmt,pv_size,dev_size,pv_free,pv_used,pv_name,pv_attr,pv_pe_count,pv_pe_alloc_count,pv_tags,pvseg_start,pvseg_size,pe_start,lv_name Fmt PSize DevSize PFree Used PV Attr PE Alloc PV Tags Start SSize 1st PE LV lvm2 1.44G 1.44G 724.00M 750.00M /dev/sdc2 a- 737 375 0 100 192.00K striped lvm2 1.44G 1.44G 724.00M 750.00M /dev/sdc2 a- 737 375 100 150 192.00K mirrored lvm2 1.44G 1.44G 724.00M 750.00M /dev/sdc2 a- 737 375 250 125 192.00K striped lvm2 1.44G 1.44G 724.00M 750.00M /dev/sdc2 a- 737 375 375 362 192.00K lvm2 486.00M 486.31M 486.00M 0 /dev/sdd1 a- 243 0 0 243 192.00K lvm2 1.39G 1.40G 978.00M 450.00M /dev/sde2 a- 714 225 0 100 192.00K striped lvm2 1.39G 1.40G 978.00M 450.00M /dev/sde2 a- 714 225 100 1 192.00K lvm2 1.39G 1.40G 978.00M 450.00M /dev/sde2 a- 714 225 101 125 192.00K striped lvm2 1.39G 1.40G 978.00M 450.00M /dev/sde2 a- 714 225 226 488 192.00K
sudo lvs -o +segtype,stripes,stripesize,devices,seg_count,copy_percent LV VG Attr LSize Origin Snap% Move Log Copy% Convert Type #Str Stripe Devices #Seg Copy% mirrored test mwn-a- 300.00M mirrored_mlog 31.33 mirror 2 0 mirrored_mimage_0(0),mirrored_mimage_1(0) 1 31.33 striped test -wn-a- 900.00M striped 2 4.00K /dev/sdc2(0),/dev/sde2(0) 2 striped test -wn-a- 900.00M striped 2 4.00K /dev/sdc2(250),/dev/sde2(101) 2
KVPM
System-config-lvm