OpenStack Storage: Horses for Courses
1) OpenStack Cinder (Block Storage):
* Architected to provide traditional block level storage resources to OpenStack services
* Presents persistent block level storage volumes for use with OpenStack Nova compute instance
* Manages the creation, attaching and detaching of these volumes between a storage system different host servers
2) Cinder Features
1\ Grizzly
* Create/delete volumes
* Boot VM instance from volume
* Attach/detach volume to instance
* Point in time copy (snapshots of volumes)
* per tenant usage quotas
2\ Havana
* Attach volume to multiple hosts
* Read-only volumes
* ACLs
* Disk encryption
* FC SAN Zone / Access Control management
* Transfer volume ownership
* Volume Migration
* Work towards locality between instances and volumes
* Scheduler hints
* Volume rate limiting
* Customized scheduling filters
All in one: More intelligent scheduler, in order to support multiple host/heterogeneous back-end storage. Support for different volume type, the user can choose according to their own needs the volume of different performance. etc. -_-!
3) Architectural Overview
* scheduler: decides which host gets each volume
* volume: manages dynamically attachable block devices.
4) Example: High-Level Data and Control Flow
Create/Attach that Volume process:
1\ User: POST http://9.186.105.44:8776/v1/{tenant_id}/volumes
2\ Cinder-api: CALL cinder.volume.API.create()
3\ Cinder-api: RPC CAST create_volume to cinder-scheduler
4\ Cinder-scheduler : SELECT cinder-volume
5\ Cinder-scheduler: RPC CAST create_volume to cinder-volume
6\ Cinder-volume: CALL driver.create_volume()
Current drivers:
* Coraid (AoE)
* EMC VMAX/VNX (iSCSI)
* GlusterFS (GlusterFS)
* HP 3PAR (iSCSI/FC)
* HP LeftHand (iSCSI)
* Huawei T-series/Dorado (iSCSI)
* IBM Storwize family/SVC (iSCSI/FC)
* IBM XIV (iSCSI), LVM (iSCSI)
* NetApp (iSCSI/NFS)
* Nexenta (iSCSI)
* NFS (NFS)
* RBD (Ceph)
* Scality SOFS (scality)
* Sheepdog (sheepdog)
* Solaris (iSCSI)
* SolidFire (iSCSI)
* Windows Server 2012 (iSCSI) Zadara (iSCSI)
5) Guide
1. (External storage device)
pvcreate /dev/diskname
vgcreate cinder-volumes /dev/diskname
2.
fdisk -l
fdisk /dev/diskname ->n ->p ->w
fdisk -l
mkfs.ext4 /dev/diskname
mkdir mount_dir
mount -t ext4 /dev/diskname ./mount_dir
6) trend ?!
"OpenStack H版與 Ceph 整合的現狀"
http://www.ustack.com/blog/openstack_and_ceph/
"Ceph In UnitedStak"
http://www.ustack.com/blog/ceph-in-unitedstak-2/
7) Cinder Support Matrix：
https://wiki.openstack.org/wiki/CinderSupportMatrix
8) Troubleshooting
->
Attach volume to instance hold on "Attaching" state:
1. stop_service tgt
2. mv /etc/init/tgt.conf /etc/init/tgt.conf.disabled
3. restart_service iscsitarget
->
Create volume error and log like this:
It's because --tid=X for ietadm allready exists. delete the new created volume with status error and type
via console the following command:
ietadm
--op delete --tid=X
9) Refrences
http://www.stonefly.com/resources/File-Level-Storage-vs-Block-Level-Storage.asp
http://www.techrepublic.com/blog/the-enterprise-cloud/block-level-storage-vs-file-level-storage-a-comparison/
http://www.ibm.com/developerworks/linux/library/l-ceph/index.html?ca=dgr-lnxw01CEPHdth-LX
http://zh.community.dell.com/techcenter/b/weblog/archive/2011/01/11/comparing-performance-between-iscsi-fcoe-and-fc.aspx
--2014年4月1日
cinder產生的lvm死活都刪不掉，not found device
dmsetup ls 。。。。。
dmsetup remove lv_demo1dat 把它幹掉
lvremove cinder-volumes/volume-b9869d42-418f-4d7c-b4bf-951b035d1817
Do you really want to remove active logical volume volume-b9869d42-418f-4d7c-b4bf-951b035d1817? [y/n]: y
device-mapper: remove ioctl on failed: Device or resource busy
Unable to deactivate cinder–volumes-volume–b9869d42–418f–4d7c–b4bf–951b035d1817 (253:59)
Unable to deactivate logical volume “volume-b9869d42-418f-4d7c-b4bf-951b035d1817″
lsof /dev/cinder-volumes/volume-b9869d42-418f-4d7c-b4bf-951b035d1817
tgt-admin -s | grep b98
Target 61: iqn.2010-10.org.openstack:volume-b9869d42-418f-4d7c-b4bf-951b035d1817
Backing store path: /dev/cinder-volumes/volume-b9869d42-418f-4d7c-b4bf-951b035d1817
tgt-admin –delete iqn.2010-10.org.openstack:volume-b9869d42-418f-4d7c-b4bf-951b035d1817
http://www.jaddog.org/page/2/
http://xiezhenye.com/tag/lvm