V-REP 添加Vision Sensor與圖像獲取 | V-REP Adding Vision Sensor and Image Acquisition

在完成此文之前必須完成V-REP與Python的通信部分，參考Python與V-REP聯合仿真 | Joint simulation of Python and V-REP

V-REP端操作

1.打開scene

UR5plusRG2_PickAndPlaceDemo1.ttt

2.添加vision sensor

視圖界面右鍵add-vision sensor-Perspective projection-type

vision sensor兩個類型區別如下：

3.添加Floating View

仿真界面右鍵，add-Floating View

4.Associate Vison_sensor

操作如下

5.修改vision sensor參數

Near/far clipping plane代表檢測範圍的大小

Resolution是分辨率

6.移動與旋轉傳感器

點擊移動按鈕，可以直接拖動物體

旋轉一定角度，運行仿真可以看見如下綠點，Vison sensor V-REP端配置完成

Python端操作

代碼

#!/usr/bin/env python
# encoding: utf-8

"""
Enable the vision sensor in V-REP,Python
use the scene：VisionSensorDemo.ttt

@Author: Zane
@Contact: [email protected]
@File: VisionSensorDemo.py
@Time: 2019-07-23 15:55
"""
import vrep
import sys
import numpy as np
import math
import matplotlib.pyplot as mpl
import time

class Usage(Exception):
    def __init__(self, msg):
        self.msg = msg

def main(argv=None):
    if argv is None:
        argv = sys.argv
    
    #Python connect to the V-REP client
    vrep.simxFinish(-1)
    
    clientID = vrep.simxStart('127.0.0.1', 19999, True, True, 5000, 5)
    
    if clientID != -1:
        print("Connected to remote API server")
    else:
        print("Connection not successful")
        sys.exit("Connected failed,program ended!")
    
    #Get the handle of vision sensor
    errorCode,visionSensorHandle = vrep.simxGetObjectHandle(clientID,'Cam',vrep.simx_opmode_oneshot_wait)
    
    #Get the image of vision sensor
    errprCode,resolution,image = vrep.simxGetVisionSensorImage(clientID,visionSensorHandle,0,vrep.simx_opmode_streaming)
    time.sleep(0.1)
    errprCode,resolution,image = vrep.simxGetVisionSensorImage(clientID,visionSensorHandle,0,vrep.simx_opmode_buffer)
    
    #Process the image to the format (64,64,3)
    sensorImage = []
    sensorImage = np.array(image,dtype = np.uint8)
    sensorImage.resize([resolution[0],resolution[1],3])
    
    #Use matplotlib.imshow to show the image
    mpl.imshow(sensorImage,origin='lower')


if __name__ == "__main__":
    sys.exit(main())

測試結果

運行結果如下，可以在python中捕獲到Vison Sensor的實時圖片

註解

註解1：

vrep.simxFinish(-1)

clientID = vrep.simxStart(‘127.0.0.1’, 19999, True, True, 5000, 5)

每次重新開始仿真都需要重新運行這兩段代碼，獲取新的clientID

註解2：

errorCode,visionSensorHandle = vrep.simxGetObjectHandle(clientID,‘Cam’,vrep.simx_opmode_oneshot_wait)

這裏的Cam對應的就是V-REP中的VisionSensor命名

如果是handle名稱錯誤，會出現errorCode=8，具體見errorCode：[V-REP Constant](file:///Users/mac/Downloads/vrep/V-REP_PRO_EDU_V3_5_0_Mac/helpfiles/en/remoteApiConstants.htm)

註解3：

errprCode,resolution,image = vrep.simxGetVisionSensorImage(clientID,visionSensorHandle,0,vrep.simx_opmode_streaming)
time.sleep(0.1)
errprCode,resolution,image = vrep.simxGetVisionSensorImage(clientID,visionSensorHandle,0,vrep.simx_opmode_buffer)

resolution是之前的設置的圖像大小，這裏安裝本文設置resolution = [64，64]

獲取image，需要兩次使用simxGetVisionSensorImage，第一次用的simx_opmode_streaming，第二次是simx_opmode_buffer。第三個參數設置爲0，代表獲取RGB模式的圖片

經過測試time.sleep(0.1)必須添加。如果沒有暫停同時運行兩次simxGetVisionSensorImage，讀取不到image

註解4：

mpl.imshow(sensorImage,origin=‘lower’)

實際圖像會和V-REP裏面接收到的上下顛倒，origin='lower’恢復原圖像