$Date: 2002/01/22 14:49:34 $ Logical Volume Manager Abstract Goals: ------ Implement a very flexible virtual disk subsystem to handle disk storage. Online allocation and relocation of storage. Online extension and reduction of storage. Function: --------- The Logical Volume Manager (LVM) adds an additional layer between the physical peripherals and the i/o interface in the kernel to get a logical view of disks. This allows the concatenation of several disks (so-called physical volumes or PVs) to form a storage pool (so-called Volume Group or VG) with allocation units called physical extents (called PE). Parts out of this VG then can be allocated in form of so-called Logical Volumes or LVs in units called logical extents or LEs. Each logical extent is mapped to a corresponding physical extent of equal size. These physical extents are mapped to offsets and blocks on the disk(s). The LVs can then be used through device special files similar to /dev/sd[a-z]* or /dev/hd[a-z]* named /dev/VolumeGroupName/LogicalVolumeName. But going beyond this, you are able to extend or reduce VGs AND LVs at runtime. Concept: -------- The configuration information for the physical volume, volume group and logical volume(s) is stored on each physical volume and in automatically created backup files, which are stored in the /etc/lvmtab.d directory. The config area on the disk(s) is called Volume Group Descriptor Area or VGDA. A LVM driver holds mapping tables between the LEs of LVs and the PEs of PVs. These tables are created/updated/deleted by superuser LVM commands. The main mapping function of the driver is called with a logical block in a LV from functions in /usr/src/linux/drivers/block/ll_rw_blk.c (in functions ll_rw_block() and ll_rw_swap_file() ) and looks up the corresponding physical block/disk pair in a table. Then it returns this pair to the calling ll_rw_*() function causing a physical i/o request to the disk block(s) to be queued. Example: -------- If the capacity of a LV gets too small and your VG containing this LV is full, you could add another PV to that VG and simply extend the LV afterwards. If you reduce or delete a LV you can use the freed capacity for different LVs of the same VG. The above scenario looks like this: /------------------------------------------\ | /--------\ VG 1 /--------\ | | | | | | | | | PV 1 | ...... | PV n | | | | /-----------------------\ | | | | \-------LV 1------------/ | | | \--------/ \--------/ | \------------------------------------------/ PV 1 could be /dev/sdc1 PV n could be /dev/sde1 VG 1 could be vg00 LV 1 could be /dev/test_vg/test_lv The configuration steps for getting the above scenario are: 1. after installing LVM do an "insmod lvm" or setup kerneld/kmod to load it automatic (see INSTALL) 2. set up partitions (#1) on both disks with partition type 0x8e (I used this type to avoid using Linux primary partitions etc.) 3. do a "pvcreate /dev/sd[ce]1" For testing purposes you can use more than one primary and/or extended partition on a disk. Don't do that for normal LVM operation for performance reasons. Unless you have to, don't stripe logical volumes over physical volumes associated to partitions on the same disk. 4. do a "vgcreate test_vg /dev/sd[ce]1" (vgcreate activates the volume group too) 5. do a "lvcreate -L100 -ntest_lv test_vg" to get a 100MB linear LV or a "lvcreate -i2 -I4 -l100 -nanother_test_lv test_vg" to get a 100 LE large logical volume with 2 stripes and stripe size 4 KB. 6. use created LVs as you want to. For example generate a filesystem in one with "mke2fs /dev/test_vg/test_lv" and mount it. Overview and concept of commands: --------------------------------- I grouped and named LVM commands analogous to HP's. So the commands for physical volume handling all start with pv, those for volume group handling start with vg and the ones for logical volumes start with lv. e2fsadm - administration wrapper for logical volume including filesystem resizing for lvextend, lvreduce, e2fsck and resize2fs lvchange - change attributes of a logical volume lvcreate - create a logical volume lvdisplay - display logical volume config data lvextend - extend a logical volume in size lvreduce - reduce a logical volume in size lvremove - remove a logical volume lvrename - rename an inactive logical volume lvscan - find all existing logical volumes lvmchange - emergency program to change attributes of the LVM lvmdiskscan - scan all disks / partitions and multiple devices and list them lvmsadc - statistic data collector lvmsar - statistic data reporter pvchange - change attributes of physical volumes pvcreate - create a physical volume pvdata - debug list physical volume group descriptor area pvdisplay - display physical volume config information pvmove - move logical extents to a different physical volume pvscan - find all existing physical volumes vgcfgbackup - backup all volume group descriptor areas vgcfgrestore - restore volume group descriptor area(s) to disk(s) vgchange - activate/deactivate volume group(s) vgck - check volume group descriptor area for consistency vgcreate - create a volume group from physical volume(s) vgdisplay - display volume group config information vgexport - export volume group (make it unknown to the system) vgextend - extend a volume group by one or more physical volumes vgimport - import a volume group (make it known to the/another system) vgmerge - merge two volume groups into one vgmknodes - create volume group directory with all logical volume specials vgreduce - reduce a volume group by one or more empty physical volume(s) vgremove - remove an empty volume group vgrename - rename an inactive volume group vgscan - scan for volume groups vgsplit - split one volume group into two Example LVM session output: --------------------------- # create physical volumes on 9 SCSI disk primary partition 1 pvcreate /dev/sd[b-eg-k]1 pvcreate -- /dev/sdb1 has an invalid physical volume identifier pvcreate -- physical volume on /dev/sdb1 successfully created pvcreate -- reinitializing physical volume pvcreate -- physical volume on /dev/sdc1 successfully created pvcreate -- reinitializing physical volume pvcreate -- physical volume on /dev/sdd1 successfully created pvcreate -- reinitializing physical volume pvcreate -- physical volume on /dev/sde1 successfully created pvcreate -- reinitializing physical volume pvcreate -- physical volume on /dev/sdg1 successfully created pvcreate -- reinitializing physical volume pvcreate -- physical volume on /dev/sdh1 successfully created pvcreate -- reinitializing physical volume pvcreate -- physical volume on /dev/sdi1 successfully created pvcreate -- reinitializing physical volume pvcreate -- physical volume on /dev/sdj1 successfully created pvcreate -- reinitializing physical volume pvcreate -- physical volume on /dev/sdk1 successfully created # create a volume group with default physical extent size # from these physical volumes vgcreate my_first_vg /dev/sd[b-eg-k]1 vgcreate -- INFO: using default physical extent size 4096 KB vgcreate -- INFO: maximum logical volume size is 256 GB vgcreate -- doing automatic backup of vg05 vgcreate -- volume group my_first_vg successfully created # Oops ;-) # Don't like the limitations caused by default physical extent size # --> deactivate and delete volume vgchange -an my_first_vg vgchange -- my_first_vg successfully deactivated vgremove my_first_vg vgremove -- volume group my_first_vg successfully removed # create a volume group with physical extent size of 8192 KB # from these physical volumes vgcreate -s 8192 my_first_vg /dev/sd[b-eg-k]1 vgcreate -- INFO: maximum logical volume size is 512 GB vgcreate -- doing automatic backup of my_first_vg vgcreate -- volume group my_first_vg successfully created # display volume group config vgdisplay my_first_vg --- Volume group --- VG Name my_first_vg VG Write Access read/write VG Status available VG # 1 MAX LV 31 Cur LV 0 Open LV 0 MAX LV Size 512 GB Max PV 256 Cur PV 9 Act PV 9 VG Size 12636 MB PE Size 8192 KB Total PE 1579 Alloc PE / Size 0 / 0 KB Free PE / Size 1579 / 12632 MB # do it again Sam but verbose vgdisplay -v my_first_vg --- Volume group --- VG Name my_first_vg VG Write Access read/write VG Status available VG # 1 MAX LV 31 Cur LV 0 Open LV 0 MAX LV Size 512 GB Max PV 256 Cur PV 9 Act PV 9 VG Size 12636 MB PE Size 8192 KB Total PE 1579 Alloc PE / Size 0 / 0 KB Free PE / Size 1579 / 12636 MB --- No logical volumes defined in my_first_vg --- --- Physical volumes --- PV Name (#) /dev/sdb1 (1) PV Status available / allocatable Total PE / Free PE 131 / 131 PV Name (#) /dev/sdc1 (2) PV Status available / allocatable Total PE / Free PE 131 / 131 PV Name (#) /dev/sdd1 (3) PV Status available / allocatable Total PE / Free PE 131 / 131 PV Name (#) /dev/sde1 (4) PV Status available / allocatable Total PE / Free PE 131 / 131 PV Name (#) /dev/sdg1 (5) PV Status available / allocatable Total PE / Free PE 131 / 131 PV Name (#) /dev/sdh1 (6) PV Status available / allocatable Total PE / Free PE 131 / 131 PV Name (#) /dev/sdi1 (7) PV Status available / allocatable Total PE / Free PE 131 / 131 PV Name (#) /dev/sdj1 (8) PV Status available / allocatable Total PE / Free PE 125 / 125 PV Name (#) /dev/sdk1 (9) PV Status available / allocatable Total PE / Free PE 537 / 537 # create a linear physical volume with all space of the volume group lvcreate -l1579 my_first_vg lvcreate -- doing automatic backup of my_first_vg lvcreate -- logical volume /dev/my_first_vg/lvol1 successfully created # create an ext2fs on newly created logical volume mke2fs /dev/my_first_vg/lvol1 mke2fs 1.10, 24-Apr-97 for EXT2 FS 0.5b, 95/08/09 Linux ext2 filesystem format Filesystem label= 3235840 inodes, 12939264 blocks 646963 blocks (5.00%) reserved for the super user First data block=1 Block size=1024 (log=0) Fragment size=1024 (log=0) 1580 block groups 8192 blocks per group, 8192 fragments per group 2048 inodes per group Superblock backups stored on blocks: 8193, 16385, 24577, 32769, 40961, 49153, 57345, 65537, 73729, 81921, 90113, 98305, 106497, 114689, 122881, 131073, 139265, 147457, Writing superblocks and filesystem accounting information: done 0.66user 26.55system 3:06.62elapsed 14%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (97major+522minor)pagefaults 0swaps # and mount it (not very exiting :-) ) mount /dev/my_first_vg/lvol1 /mnt