Openstack CPU/Disk/network QoS 功能

在openstack中Instance Recource Quota這個feature很早就加進來了，不過一直沒有全部測試過這個feature。今天在havana版本中測試了一下這個功能。

首先，這個feature在社區的wiki如下：

https://wiki.openstack.org/wiki/InstanceResourceQuota

測試過程中發現，這個wiki有些信息已經和代碼匹配不上了。

從總體上來說這個feature的功能是通過cgroup和tc來實現的。cgroup限制CPU/Disk的IO，tc來限制network IO.

首先看一下CPU限制，目前支持一下參數：

quota:cpu_shares
quota:cpu_period
quota:cpu_quota

我僅測試了quota:cpu_period，測試步驟如下：

nova flavor-key  test_resource_quato set quota:cpu_period=110000

通過CLI設置了flavor的Extra Specs. 當然也可以通過dashboard設置，不過目前存在以下bug，雖然可以實際設進去，但是在dashboard上會重定向錯誤：

https://bugs.launchpad.net/horizon/+bug/1268860

然後使用這個flavor啓動虛機，在計算節點上查看cgroup如下：

[root@ci91szcmp005vcpu0]# pwd
/cgroup/cpu/libvirt/qemu/instance-00000010/vcpu0
[root@ci91szcmp005vcpu0]# cat cpu.cfs_period_us
110000
[root@ci91szcmp005vcpu0]# cd ../vcpu1/
[root@ci91szcmp005vcpu1]# cat cpu.cfs_period_us
110000

默認的cpu period是100000，這裏已經改成了110000，說明起作用了。

對於Network IO的測試我以前寫過一個blog，在havana版本中沒有太大的變化，只是tc的作用對象從vnetXX變爲了tap設備。在此不再累述，可以參考以下blog：

http://blog.csdn.net/matt_mao/article/details/15339729

另外目前支持的network IO的參數如下：

quota:vif_inbound_average
quota:vif_inbound_peak
quota:vif_inbound_burst
quota:vif_outbound_average
quota:vif_outbound_peak
quota:vif_outbound_burst

測試Disk IO時發現了以下錯誤日誌：

2014-01-1603:25:18.9923158 ERROR nova.compute.manager [req-8e364bca-9313-4c38-a900-34d2b7f3f700 ad3955246b8d45c8a808b83e4d51c246 88f530c7b2484336824c87801d6d4950] [instance: ab6f99d6-06f4-4ea2-b4d3-fa3dce322587] Error: unsupported configuration: block I/O throttling not supported withthisQEMU binary
2014-01-1603:25:18.9923158 TRACE nova.compute.manager [instance: ab6f99d6-06f4-4ea2-b4d3-fa3dce322587] Traceback (most recent call last):
2014-01-1603:25:18.9923158 TRACE nova.compute.manager [instance: ab6f99d6-06f4-4ea2-b4d3-fa3dce322587]   File"/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1037, in _build_instance
2014-01-1603:25:18.9923158 TRACE nova.compute.manager [instance: ab6f99d6-06f4-4ea2-b4d3-fa3dce322587]     set_access_ip=set_access_ip)
2014-01-1603:25:18.9923158 TRACE nova.compute.manager [instance: ab6f99d6-06f4-4ea2-b4d3-fa3dce322587]   File"/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1410, in _spawn
2014-01-1603:25:18.9923158 TRACE nova.compute.manager [instance: ab6f99d6-06f4-4ea2-b4d3-fa3dce322587]     LOG.exception(_('Instance failed to spawn'), instance=instance)
2014-01-1603:25:18.9923158 TRACE nova.compute.manager [instance: ab6f99d6-06f4-4ea2-b4d3-fa3dce322587]   File"/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1407, in _spawn
2014-01-1603:25:18.9923158 TRACE nova.compute.manager [instance: ab6f99d6-06f4-4ea2-b4d3-fa3dce322587]     block_device_info)
2014-01-1603:25:18.9923158 TRACE nova.compute.manager [instance: ab6f99d6-06f4-4ea2-b4d3-fa3dce322587]   File"/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 2070, in spawn
2014-01-1603:25:18.9923158 TRACE nova.compute.manager [instance: ab6f99d6-06f4-4ea2-b4d3-fa3dce322587]     block_device_info, context=context)
2014-01-1603:25:18.9923158 TRACE nova.compute.manager [instance: ab6f99d6-06f4-4ea2-b4d3-fa3dce322587]   File"/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 3205, in _create_domain_and_network
2014-01-1603:25:18.9923158 TRACE nova.compute.manager [instance: ab6f99d6-06f4-4ea2-b4d3-fa3dce322587]     domain = self._create_domain(xml, instance=instance, power_on=power_on)
2014-01-1603:25:18.9923158 TRACE nova.compute.manager [instance: ab6f99d6-06f4-4ea2-b4d3-fa3dce322587]   File"/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 3148, in _create_domain
2014-01-1603:25:18.9923158 TRACE nova.compute.manager [instance: ab6f99d6-06f4-4ea2-b4d3-fa3dce322587]     domain.XMLDesc(0))
2014-01-1603:25:18.9923158 TRACE nova.compute.manager [instance: ab6f99d6-06f4-4ea2-b4d3-fa3dce322587]   File"/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 3143, in _create_domain
2014-01-1603:25:18.9923158 TRACE nova.compute.manager [instance: ab6f99d6-06f4-4ea2-b4d3-fa3dce322587]     domain.createWithFlags(launch_flags)
2014-01-1603:25:18.9923158 TRACE nova.compute.manager [instance: ab6f99d6-06f4-4ea2-b4d3-fa3dce322587]   File"/usr/lib/python2.6/site-packages/eventlet/tpool.py", line 187, in doit
2014-01-1603:25:18.9923158 TRACE nova.compute.manager [instance: ab6f99d6-06f4-4ea2-b4d3-fa3dce322587]     result = proxy_call(self._autowrap, f, *args, **kwargs)
2014-01-1603:25:18.9923158 TRACE nova.compute.manager [instance: ab6f99d6-06f4-4ea2-b4d3-fa3dce322587]   File"/usr/lib/python2.6/site-packages/eventlet/tpool.py", line 147, in proxy_call
2014-01-1603:25:18.9923158 TRACE nova.compute.manager [instance: ab6f99d6-06f4-4ea2-b4d3-fa3dce322587]     rv = execute(f,*args,**kwargs)
2014-01-1603:25:18.9923158 TRACE nova.compute.manager [instance: ab6f99d6-06f4-4ea2-b4d3-fa3dce322587]   File"/usr/lib/python2.6/site-packages/eventlet/tpool.py", line 76, in tworker
2014-01-1603:25:18.9923158 TRACE nova.compute.manager [instance: ab6f99d6-06f4-4ea2-b4d3-fa3dce322587]     rv = meth(*args,**kwargs)
2014-01-1603:25:18.9923158 TRACE nova.compute.manager [instance: ab6f99d6-06f4-4ea2-b4d3-fa3dce322587]   File"/usr/lib64/python2.6/site-packages/libvirt.py", line 708, in createWithFlags
2014-01-1603:25:18.9923158 TRACE nova.compute.manager [instance: ab6f99d6-06f4-4ea2-b4d3-fa3dce322587]     ifret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
2014-01-1603:25:18.9923158 TRACE nova.compute.manager [instance: ab6f99d6-06f4-4ea2-b4d3-fa3dce322587] libvirtError: unsupported configuration: block I/O throttling not supported withthisQEMU binary
2014-01-1603:25:18.9923158 TRACE nova.compute.manager [instance: ab6f99d6-06f4-4ea2-b4d3-fa3dce322587]

可能由於qemu的bug，目前還不能正常工作。

https://www.redhat.com/archives/libvirt-users/2012-December/msg00008.html