再UE中是可以像我前後單一樣進行通信的
準備
先隨便建一個空白項目，按照官方文檔設置UE引擎
https://docs.unrealengine.com/4.27/zh-CN/SharingAndReleasing/PixelStreaming/PixelStreamingIntro/
找到在虛幻引擎安裝文件夾中，在 Samples/PixelStreaming/WebServers/SignallingWebServer 中找到信令服務器的路徑
例如我的就是
C:\Program Files\Epic Games\UE_4.27\Samples\PixelStreaming\WebServers\SignallingWebServer
打開可以看到是一個Node 項目，既然是node 項目那其他的先不說，先安裝一下依賴
在這裏不做贅述，參考
https://www.cnblogs.com/makalochen/p/13768288.html
也可以直接項目跟目錄執行
npm install
前提是node環境已經配好了
關於聯機調試
上面官方文檔是打包進行調試，但其實還有不用打包的，下面這樣設置
創建自定義UI
參考：
我們在項目根目錄創建
myUI.html
<!DOCTYPE html>
<html>
<head>
<!-- 該文件處理瀏覽器和虛幻引擎應用間的通信，接受並顯示來自服務器的媒體流。在非必要的情況下，請勿修改此JavaScript文件 -->
<script type="text/javascript" src="../scripts/webRtcPlayer.js"></script>
<!-- 此文件將設置處理鍵盤、鼠標和觸摸事件的事件監聽器 -->
<script type="text/javascript" src="../scripts/app.js"></script>
<link rel="shortcut icon" href="/images/favicon.ico" type="image/x-icon">
<link rel="icon" type="image/png" sizes="96x96" href="/images/favicon-96x96.png">
<link rel="icon" type="image/png" sizes="32x32" href="/images/favicon-32x32.png">
<link rel="icon" type="image/png" sizes="16x16" href="/images/favicon-16x16.png">
<link type="text/css" rel="stylesheet" href="player.css">
<title>myUI</title>
</head>
<body onload="load()">
<div id="playerUI">
<div id="player"></div>
<div id="overlay" class="overlay text-light bg-dark">
<div>
<div id="qualityStatus" class="greyStatus">●</div>
<div id="overlayButton">+</div>
</div>
<div id="overlaySettings">
<div id="kickOthers">
<div class="settings-text">Kick all other players</div>
<label class="btn-overlay">
<input type="button" id="kick-other-players-button" class="overlay-button btn-flat" value="Kick">
</label>
</div>
<div id="fillWindow">
<div class="settings-text">Enlarge Display to Fill Window</div>
<label class="tgl-switch">
<input type="checkbox" id="enlarge-display-to-fill-window-tgl" class="tgl tgl-flat" checked>
<div class="tgl-slider"></div>
</label>
</div>
<div id="qualityControlOwnership">
<div class="settings-text">Quality control ownership</div>
<label class="tgl-switch">
<input type="checkbox" id="quality-control-ownership-tgl" class="tgl tgl-flat">
<div class="tgl-slider"></div>
</label>
</div>
<br>
<!-- 編碼設置 -->
<section id="encoderSettings">
<div class="settings-text">Encoder Settings</div>
<div id="encoderParamsContainer" class="collapse">
<div class="form-group">
<label for="encoder-rate-control" class="settings-text">Rate Control</label>
<select id="encoder-rate-control">
<option value="CBR" selected>CBR</option>
<option value="VBR">VBR</option>
<option value="ConstQP">ConstQP</option>
</select>
<label for="encoder-target-bitrate-text">Target Bitrate (kbps)</label>
<input type="number" class="form-control" id="encoder-target-bitrate-text" value="0" min="0" max="100000" />
<label for="encoder-max-bitrate-text">Max Bitrate (kbps)</label>
<input type="number" class="form-control" id="encoder-max-bitrate-text" value="0" min="0" max="100000" />
<label for="encoder-min-qp-text">Min QP</label>
<input type="number" class="form-control" id="encoder-min-qp-text" value="0" min="0" max="999" />
<label for="encoder-max-qp-text">Max QP</label>
<input type="number" class="form-control" id="encoder-max-qp-text" value="0" min="0" max="999" />
<label for="encoder-multipass" class="settings-text">Multipass</label>
<select id="encoder-multipass">
<option value="DISABLED" selected>DISABLED</option>
<option value="QUARTER">QUARTER</option>
<option value="FULL">FULL</option>
</select>
<div class="settings-text">Filler Data</div>
<label class="tgl-switch">
<input type="checkbox" id="encoder-filler-data-tgl" class="tgl tgl-flat">
<div class="tgl-slider"></div>
</label>
</div>
<input id="encoder-params-submit" class="btn btn-primary btn-lg mt-3" type="button" value="Apply">
</div>
<br>
</section>
<!-- webRTC 設置 -->
<section id="webRTCSettings">
<div class="settings-text">WebRTC Settings</div>
<div id="webrtcParamsContainer" class="collapse">
<div class="form-group">
<label for="webrtc-degradation-pref">Degradation Pref</label>
<select id="webrtc-degradation-pref">
<option value="BALANCED">BALANCED</option>
<option value="MAINTAIN_FRAMERATE">MAINTAIN_FRAMERATE</option>
<option value="MAINTAIN_RESOLUTION">MAINTAIN_RESOLUTION</option>
</select>
<label for="webrtc-max-fps-text">Max FPS</label>
<input type="number" class="form-control" id="webrtc-max-fps-text" value="1" min="1" max="999" />
<label for="webrtc-min-bitrate-text">Min Bitrate (kbps)</label>
<input type="number" class="form-control" id="webrtc-min-bitrate-text" value="0" min="0" max="100000" />
<label for="webrtc-max-bitrate-text">Max Bitrate (kbps)</label>
<input type="number" class="form-control" id="webrtc-max-bitrate-text" value="0" min="0" max="100000" />
<label for="webrtc-low-qp-text">Low QP Threshold</label>
<input type="number" class="form-control" id="webrtc-low-qp-text" value="0" min="0" max="999" />
<label for="webrtc-high-qp-text">High QP Threshold</label>
<input type="number" class="form-control" id="webrtc-high-qp-text" value="0" min="0" max="999" />
</div>
<input id="webrtc-params-submit" class="btn btn-primary btn-lg mt-3" type="button" value="Apply">
</div>
</section>
<br>
<!-- 顯示FPS -->
<div id="showFPS">
<div class="settings-text">Show FPS</div>
<label class="btn-overlay">
<input type="button" id="show-fps-button" class="overlay-button btn-flat" value="Toggle">
</label>
</div>
<div id="matchViewportResolution">
<div class="settings-text">Match Viewport Resolution</div>
<label class="tgl-switch">
<input type="checkbox" id="match-viewport-res-tgl" class="tgl tgl-flat">
<div class="tgl-slider"></div>
</label>
</div>
<div id="statsPanel">
<div class="settings-text">Show Stats</div>
<label class="tgl-switch">
<input type="checkbox" id="show-stats-tgl" class="tgl tgl-flat" checked>
<div class="tgl-slider"></div>
</label>
<div id="statsContainer" class="statsContainer">
<div id="stats" class="stats"></div>
</div>
<br>
</div>
<div id="latencyTest">
<div class="settings-text">Latency Stats</div>
<label class="btn-overlay">
<input type="button" id="test-latency-button" class="overlay-button btn-flat" value="Test Latency">
</label>
<div id="latencyStatsContainer" class="statsContainer">
<div id=LatencyStats class="stats">No stats yet...</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>
player.css
/*Copyright Epic Games, Inc. All Rights Reserved.*/
:root {
/*Using colour scheme https://color.adobe.com/TD-Colors---Option-3-color-theme-10394433/*/
--colour1:#2B3A42;
--colour2:#3F5765;
--colour3:#BDD4DE;
--colour4:#EFEFEF;
--colour5:#FF5035;
--buttonFont:Helvetica;
--inputFont:Helvetica;
}
body{
margin: 0px;
background-color: black;
}
#playerUI {
width: 100%;
position: absolute;
/*top: 0;
left: 0;*/
z-index: 10;
}
.statsContainer {
background-color: rgba(0,0,0,0.8);
text-align: left;
display: block;
margin-top: 5px;
}
.stats {
font-size: 14px;
font-weight: bold;
padding: 6px;
color: lime;
}
canvas{
image-rendering: crisp-edges;
position: absolute;
}
video{
position: absolute;
width: 100%;
height: 100%;
}
#player{
width: 1280px;
height: 720px;
position: relative;
background-color: #000;
}
#overlay{
-moz-border-radius-bottomright: 5px;
-moz-border-radius-bottomleft: 5px;
-webkit-border-bottom-right-radius: 5px;
-webkit-border-bottom-left-radius: 5px;
border-bottom-right-radius: 5px; /* future proofing */
border-bottom-left-radius: 5px; /* future proofing */
-khtml-border-bottom-right-radius: 5px; /* for old Konqueror browsers */
-khtml-border-bottom-left-radius: 5px; /* for old Konqueror browsers */
-webkit-touch-callout: none; /* iOS Safari */
-webkit-user-select: none; /* Safari */
-khtml-user-select: none; /* Konqueror HTML */
-moz-user-select: none; /* Firefox */
-ms-user-select: none; /* Internet Explorer/Edge */
user-select: none; /* Non-prefixed version, currently
supported by Chrome and Opera */
position: absolute;
padding: 4px;
top: 0;
right: 2%;
z-index: 100;
border: 2px solid var(--colour4);
border-top-width: 0px;
}
.overlay {
background-color: var(--colour2);
font-family: var(--buttonFont);
font-weight: lighter;
color: var(--colour4);
}
.overlay-shown > #overlaySettings {
padding: 50px 4px 4px 4px;
display: block;
}
.overlay-shown > div > #overlayButton {
transform: rotate(-135deg);
-webkit-transform: rotate(-135deg); /* Safari */
-moz-transform: rotate(-135deg); /* Firefox */
-ms-transform: rotate(-135deg); /* IE */
-o-transform: rotate(-135deg); /* Opera */
}
#overlayButton:hover{
cursor: pointer;
}
#overlayButton{
transition-duration: 250ms;
float: right;
text-align: right;
font-size: 40px;
}
#qualityStatus{
float: left;
font-size: 37px;
padding-right: 4px;
}
#overlaySettings{
width: 400px;
display: none;
}
.greyStatus {
color: grey;
}
.limeStatus {
color: lime;
}
.orangeStatus {
color: orange;
}
.redStatus {
color: red;
}
#videoMessageOverlay{
z-index: 20;
color: var(--colour4);;
font-size: 1.8em;
position: absolute;
margin: auto;
font-family: var(--inputFont);;
width: 100%;
}
#videoPlayOverlay{
z-index: 30;
position: absolute;
color: var(--colour4);
font-size: 1.8em;
font-family: var(--inputFont);
width: 100%;
height: 100%;
background-color: rgba(100, 100, 100, 0.7);
}
/* State for element to be clickable */
.clickableState{
align-items: center;
justify-content: center;
display: flex;
cursor: pointer;
}
/* State for element to show text, this is for informational use*/
.textDisplayState{
display: flex;
}
/* State to hide overlay, WebRTC communication is in progress and or is playing */
.hiddenState{
display: none;
}
#playButton{
display: inline-block;
height: auto;
}
img#playButton{
max-width: 241px;
width: 10%;
}
#UIInteraction{
position: fixed;
}
#UIInteractionButtonBoundary{
padding: 2px;
}
#UIInteractionButton{
cursor: pointer;
}
#hiddenInput{
position: absolute;
left: -10%; /* Although invisible, push off-screen to prevent user interaction. */
width: 0px;
opacity: 0;
}
#editTextButton{
position: absolute;
height: 40px;
width: 40px;
}
.settings-text{
color: var(--colour4);
vertical-align: middle;
font-size: 18px;
font-weight: normal;
display: inline-block;
}
.overlay-button{
line-height: 1.1;
padding: 1px 6px;
}
.btn-overlay{
float: right;
vertical-align: middle;
display: inline-block;
}
.btn-flat{
background: var(--colour4);
border: 2px solid var(--colour5);
font-weight: bold;
cursor: pointer;
font-family: var(--buttonFont);
font-size: 10px;
color: var(--colour5);
border-radius: 5px;
height: 20px;
}
.btn-flat:disabled{
background: var(--colour4);
border-color: var(--colour3);
color: var(--colour3);
cursor: default;
}
.btn-flat:active{
border-color: var(--colour2);
color: var(--colour2);
}
.btn-flat:focus{
outline: none;
}
/*** Toggle Switch styles ***/
.tgl-switch {
float: right;
vertical-align: middle;
display: inline-block;
}
.tgl-switch .tgl {
display:none;
}
.tgl, .tgl:after, .tgl:before, .tgl *, .tgl *:after, .tgl *:before, .tgl + .tgl-slider {
-webkit-box-sizing: border-box;
box-sizing: border-box;
}
.tgl::-moz-selection, .tgl:after::-moz-selection, .tgl:before::-moz-selection, .tgl *::-moz-selection, .tgl *:after::-moz-selection, .tgl *:before::-moz-selection, .tgl + .tgl-slider::-moz-selection {
background: none;
}
.tgl::selection, .tgl:after::selection, .tgl:before::selection, .tgl *::selection, .tgl *:after::selection, .tgl *:before::selection, .tgl + .tgl-slider::selection {
background: none;
}
.tgl-slider {
float:right;
}
.tgl + .tgl-slider {
outline: 0;
display: block;
width: 40px;
height: 18px;
position: relative;
cursor: pointer;
-webkit-user-select: none;
-moz-user-select: none;
-ms-user-select: none;
user-select: none;
}
.tgl + .tgl-slider:after, .tgl + .tgl-slider:before {
position: relative;
display: block;
content: "";
width: 50%;
height: 100%;
}
.tgl + .tgl-slider:after {
left: 0;
}
.tgl + .tgl-slider:before {
display: none;
}
.tgl-flat + .tgl-slider {
padding: 2px;
-webkit-transition: all .2s ease;
transition: all .2s ease;
background: #fff;
border: 3px solid var(--colour4);
border-radius: 2em;
}
.tgl-flat + .tgl-slider:after {
-webkit-transition: all .2s ease;
transition: all .2s ease;
background: var(--colour4);
content: "";
border-radius: 1em;
}
.tgl-flat:checked + .tgl-slider {
border: 3px solid var(--colour5);
}
.tgl-flat:checked + .tgl-slider:after {
left: 50%;
background: var(--colour5);
}
/*** Toggle Switch styles ***/
#encoderSettings, #webRTCSettings {
margin: 10px 0px;
}
#encoderParamsContainer, #webrtcParamsContainer {
padding-left: 5%;
}
#encoder-params-submit {
float: right;
}
#webrtc-params-submit {
float: right;
}
select {
text-align: right;
}
input {
text-align: right;
}
.form-group {
display: grid;
grid-template-columns: 50% 50%;
}
.form-group label {
color: var(--colour4);
vertical-align: middle;
font-size: 18px;
font-weight: normal;
}
#latencyTest {
display: block;
}
#latencyTest button {
margin: 30px 0px;
}
#freezeFrameOverlay {
background-color: transparent;
}
.freezeframeBackground {
background-color: #000 !important;
}
更改項目首頁
找到項目目錄下的 config.json ,將項目首頁更改成自己的，然後重新啓動
可以看到
demo 測試
直接在瀏覽器輸入
http://127.0.0.1/
可以看到
點擊一下，就可以看到跟UE引擎一樣的，像素流
通信
新建遊戲模式
右鍵新建 遊戲基礎模式
新建默認pawn類，爲 Camera
雙擊打開，拖入一個Camera 攝像機
設置參數
設置遊戲模式
測試啓動
啓動之後，你會發送什麼都動不了，這是正常的，沒有寫移動的邏輯
html前端發送
我們在定義UI的html 頁面修改一點東西,下面的代碼很簡單，就是添加一個按鈕，調用發送函數，並且調用回調
<div>
<button onclick="testSend()">發送消息</button>
</div>
<script>
function api_send(proto, data , callback){
let jsonData = { command: 'event', func_name: proto, args: data};
emitUIInteraction(jsonData);
callback(jsonData);
}
function testSend(){
console.log('testSend...........');
api_send("onClickUEsend", "testSend......", (info) => {
console.log(info);
});
}
</script>
UE接收藍圖編輯
雙擊GM 遊戲模式，回到編輯界面，添加 PixelStreamerInput 事件
添加輸入事件
添加打印
運行一下看下
發送消息Demo消息測試
輸入
http://127.0.0.1/
運行點擊開始播放之後
可以看到，ue 這邊已經接收到了
ue 返回
前面我們已經實現了，從前端 到 ue，但是前面的那個回調函數是我們自己調用，那我們現在改造一下，這邊我們發給ue 的是什麼，就返回什麼
前端修改
<script>
// 發送到ue
function api_send(proto, data , callback){
let jsonData = { command: 'event', func_name: proto, args: data};
// 發送
emitUIInteraction(jsonData);
// 調用 app.js 提供的 addResponseEventListener 函數來註冊監聽器函數，註冊ue response 監聽函數
addResponseEventListener("handle_responses", callback)
}
function testSend(){
console.log('testSend...........');
api_send("onClickUEsend", "testSend......", (info) => {
console.log(info);
});
}
</script>
ue修改
主要是 Send Pixel Streaming Response 這個函數，下面的邏輯就是，接收前端的消息 並打印到ue 的像素流中，並且返回給前端
我們也可以，在接收到的字符串里加點信息，代表是ue 返回的
如
通信demo測試
將前端和ue 重新跑下，可以看到
參考
vue + ue4通信：
https://blog.csdn.net/xujl0429/article/details/125768807
ue 參考：
https://docs.unrealengine.com/4.27/zh-CN/SharingAndReleasing/PixelStreaming/
https://docs.unrealengine.com/4.27/zh-CN/SharingAndReleasing/PixelStreaming/CustomPlayer/
https://docs.unrealengine.com/4.27/zh-CN/Resources/Showcases/PixelStreamingShowcase/?