在前面:
https://www.cnblogs.com/makalochen/p/17803468.html#新建遊戲模式
我們已經實現了,從前端到UE 的通信,那能不能UE 主動給前端 發送消息呢?
就比如我在播放的像素流中點擊了某個 物體 給所有頁面發個消息,說你已經點擊了某個物體
答案是可以的,參考官網
https://docs.unrealengine.com/4.27/zh-CN/SharingAndReleasing/PixelStreaming/CustomPlayer/
前置工作
啓動信令服務器
https://docs.unrealengine.com/4.27/zh-CN/SharingAndReleasing/PixelStreaming/PixelStreamingIntro/
其實就是個node 服務,啓動就完事了
前端
聲明一下 ,這部分前端是從信令服務器摳出來的,方便前後端分離
主要就是這些東西
myUI.html
<!DOCTYPE html>
<html>
<head>
<!-- 該文件處理瀏覽器和虛幻引擎應用間的通信,接受並顯示來自服務器的媒體流。在非必要的情況下,請勿修改此JavaScript文件 -->
<script type="text/javascript" src="../scripts/webRtcPlayer.js"></script>
<!-- 此文件將設置處理鍵盤、鼠標和觸摸事件的事件監聽器 -->
<script type="text/javascript" src="../scripts/app.js"></script>
<link rel="shortcut icon" href="../images/favicon.ico" type="image/x-icon">
<link rel="icon" type="../image/png" sizes="96x96" href="/images/favicon-96x96.png">
<link rel="icon" type="../image/png" sizes="32x32" href="/images/favicon-32x32.png">
<link rel="icon" type="../image/png" sizes="16x16" href="/images/favicon-16x16.png">
<link type="text/css" rel="stylesheet" href="player.css">
<title>myUI</title>
</head>
<body onload="load()">
<div>
</div>
<div id="playerUI">
<div id="player"></div>
<div id="overlay" class="overlay text-light bg-dark">
<div>
<div id="qualityStatus" class="greyStatus">●</div>
<div id="overlayButton">+</div>
</div>
<div id="overlaySettings">
<div id="kickOthers">
<div class="settings-text">Kick all other players</div>
<label class="btn-overlay">
<input type="button" id="kick-other-players-button" class="overlay-button btn-flat" value="Kick">
</label>
</div>
<div id="fillWindow">
<div class="settings-text">Enlarge Display to Fill Window</div>
<label class="tgl-switch">
<input type="checkbox" id="enlarge-display-to-fill-window-tgl" class="tgl tgl-flat" checked>
<div class="tgl-slider"></div>
</label>
</div>
<div id="qualityControlOwnership">
<div class="settings-text">Quality control ownership</div>
<label class="tgl-switch">
<input type="checkbox" id="quality-control-ownership-tgl" class="tgl tgl-flat">
<div class="tgl-slider"></div>
</label>
</div>
<br>
<!-- 編碼設置 -->
<section id="encoderSettings">
<div class="settings-text">Encoder Settings</div>
<div id="encoderParamsContainer" class="collapse">
<div class="form-group">
<label for="encoder-rate-control" class="settings-text">Rate Control</label>
<select id="encoder-rate-control">
<option value="CBR" selected>CBR</option>
<option value="VBR">VBR</option>
<option value="ConstQP">ConstQP</option>
</select>
<label for="encoder-target-bitrate-text">Target Bitrate (kbps)</label>
<input type="number" class="form-control" id="encoder-target-bitrate-text" value="0" min="0" max="100000" />
<label for="encoder-max-bitrate-text">Max Bitrate (kbps)</label>
<input type="number" class="form-control" id="encoder-max-bitrate-text" value="0" min="0" max="100000" />
<label for="encoder-min-qp-text">Min QP</label>
<input type="number" class="form-control" id="encoder-min-qp-text" value="0" min="0" max="999" />
<label for="encoder-max-qp-text">Max QP</label>
<input type="number" class="form-control" id="encoder-max-qp-text" value="0" min="0" max="999" />
<label for="encoder-multipass" class="settings-text">Multipass</label>
<select id="encoder-multipass">
<option value="DISABLED" selected>DISABLED</option>
<option value="QUARTER">QUARTER</option>
<option value="FULL">FULL</option>
</select>
<div class="settings-text">Filler Data</div>
<label class="tgl-switch">
<input type="checkbox" id="encoder-filler-data-tgl" class="tgl tgl-flat">
<div class="tgl-slider"></div>
</label>
</div>
<input id="encoder-params-submit" class="btn btn-primary btn-lg mt-3" type="button" value="Apply">
</div>
<br>
</section>
<!-- webRTC 設置 -->
<section id="webRTCSettings">
<div class="settings-text">WebRTC Settings</div>
<div id="webrtcParamsContainer" class="collapse">
<div class="form-group">
<label for="webrtc-degradation-pref">Degradation Pref</label>
<select id="webrtc-degradation-pref">
<option value="BALANCED">BALANCED</option>
<option value="MAINTAIN_FRAMERATE">MAINTAIN_FRAMERATE</option>
<option value="MAINTAIN_RESOLUTION">MAINTAIN_RESOLUTION</option>
</select>
<label for="webrtc-max-fps-text">Max FPS</label>
<input type="number" class="form-control" id="webrtc-max-fps-text" value="1" min="1" max="999" />
<label for="webrtc-min-bitrate-text">Min Bitrate (kbps)</label>
<input type="number" class="form-control" id="webrtc-min-bitrate-text" value="0" min="0" max="100000" />
<label for="webrtc-max-bitrate-text">Max Bitrate (kbps)</label>
<input type="number" class="form-control" id="webrtc-max-bitrate-text" value="0" min="0" max="100000" />
<label for="webrtc-low-qp-text">Low QP Threshold</label>
<input type="number" class="form-control" id="webrtc-low-qp-text" value="0" min="0" max="999" />
<label for="webrtc-high-qp-text">High QP Threshold</label>
<input type="number" class="form-control" id="webrtc-high-qp-text" value="0" min="0" max="999" />
</div>
<input id="webrtc-params-submit" class="btn btn-primary btn-lg mt-3" type="button" value="Apply">
</div>
</section>
<br>
<!-- 顯示FPS -->
<div id="showFPS">
<div class="settings-text">Show FPS</div>
<label class="btn-overlay">
<input type="button" id="show-fps-button" class="overlay-button btn-flat" value="Toggle">
</label>
</div>
<div id="matchViewportResolution">
<div class="settings-text">Match Viewport Resolution</div>
<label class="tgl-switch">
<input type="checkbox" id="match-viewport-res-tgl" class="tgl tgl-flat">
<div class="tgl-slider"></div>
</label>
</div>
<div id="statsPanel">
<div class="settings-text">Show Stats</div>
<label class="tgl-switch">
<input type="checkbox" id="show-stats-tgl" class="tgl tgl-flat" checked>
<div class="tgl-slider"></div>
</label>
<div id="statsContainer" class="statsContainer">
<div id="stats" class="stats"></div>
</div>
<br>
</div>
<div id="latencyTest">
<div class="settings-text">Latency Stats</div>
<label class="btn-overlay">
<input type="button" id="test-latency-button" class="overlay-button btn-flat" value="Test Latency">
</label>
<div id="latencyStatsContainer" class="statsContainer">
<div id=LatencyStats class="stats">No stats yet...</div>
</div>
</div>
</div>
</div>
</div>
</body>
</script>
</html>
player.css
/*Copyright Epic Games, Inc. All Rights Reserved.*/
:root {
/*Using colour scheme https://color.adobe.com/TD-Colors---Option-3-color-theme-10394433/*/
--colour1:#2B3A42;
--colour2:#3F5765;
--colour3:#BDD4DE;
--colour4:#EFEFEF;
--colour5:#FF5035;
--buttonFont:Helvetica;
--inputFont:Helvetica;
}
body{
margin: 0px;
background-color: black;
}
#playerUI {
width: 100%;
position: absolute;
/*top: 0;
left: 0;*/
z-index: 10;
}
.statsContainer {
background-color: rgba(0,0,0,0.8);
text-align: left;
display: block;
margin-top: 5px;
}
.stats {
font-size: 14px;
font-weight: bold;
padding: 6px;
color: lime;
}
canvas{
image-rendering: crisp-edges;
position: absolute;
}
video{
position: absolute;
width: 100%;
height: 100%;
}
#player{
width: 1280px;
height: 720px;
position: relative;
background-color: #000;
}
#overlay{
-moz-border-radius-bottomright: 5px;
-moz-border-radius-bottomleft: 5px;
-webkit-border-bottom-right-radius: 5px;
-webkit-border-bottom-left-radius: 5px;
border-bottom-right-radius: 5px; /* future proofing */
border-bottom-left-radius: 5px; /* future proofing */
-khtml-border-bottom-right-radius: 5px; /* for old Konqueror browsers */
-khtml-border-bottom-left-radius: 5px; /* for old Konqueror browsers */
-webkit-touch-callout: none; /* iOS Safari */
-webkit-user-select: none; /* Safari */
-khtml-user-select: none; /* Konqueror HTML */
-moz-user-select: none; /* Firefox */
-ms-user-select: none; /* Internet Explorer/Edge */
user-select: none; /* Non-prefixed version, currently
supported by Chrome and Opera */
position: absolute;
padding: 4px;
top: 0;
right: 2%;
z-index: 100;
border: 2px solid var(--colour4);
border-top-width: 0px;
}
.overlay {
background-color: var(--colour2);
font-family: var(--buttonFont);
font-weight: lighter;
color: var(--colour4);
}
.overlay-shown > #overlaySettings {
padding: 50px 4px 4px 4px;
display: block;
}
.overlay-shown > div > #overlayButton {
transform: rotate(-135deg);
-webkit-transform: rotate(-135deg); /* Safari */
-moz-transform: rotate(-135deg); /* Firefox */
-ms-transform: rotate(-135deg); /* IE */
-o-transform: rotate(-135deg); /* Opera */
}
#overlayButton:hover{
cursor: pointer;
}
#overlayButton{
transition-duration: 250ms;
float: right;
text-align: right;
font-size: 40px;
}
#qualityStatus{
float: left;
font-size: 37px;
padding-right: 4px;
}
#overlaySettings{
width: 400px;
display: none;
}
.greyStatus {
color: grey;
}
.limeStatus {
color: lime;
}
.orangeStatus {
color: orange;
}
.redStatus {
color: red;
}
#videoMessageOverlay{
z-index: 20;
color: var(--colour4);;
font-size: 1.8em;
position: absolute;
margin: auto;
font-family: var(--inputFont);;
width: 100%;
}
#videoPlayOverlay{
z-index: 30;
position: absolute;
color: var(--colour4);
font-size: 1.8em;
font-family: var(--inputFont);
width: 100%;
height: 100%;
background-color: rgba(100, 100, 100, 0.7);
}
/* State for element to be clickable */
.clickableState{
align-items: center;
justify-content: center;
display: flex;
cursor: pointer;
}
/* State for element to show text, this is for informational use*/
.textDisplayState{
display: flex;
}
/* State to hide overlay, WebRTC communication is in progress and or is playing */
.hiddenState{
display: none;
}
#playButton{
display: inline-block;
height: auto;
}
img#playButton{
max-width: 241px;
width: 10%;
}
#UIInteraction{
position: fixed;
}
#UIInteractionButtonBoundary{
padding: 2px;
}
#UIInteractionButton{
cursor: pointer;
}
#hiddenInput{
position: absolute;
left: -10%; /* Although invisible, push off-screen to prevent user interaction. */
width: 0px;
opacity: 0;
}
#editTextButton{
position: absolute;
height: 40px;
width: 40px;
}
.settings-text{
color: var(--colour4);
vertical-align: middle;
font-size: 18px;
font-weight: normal;
display: inline-block;
}
.overlay-button{
line-height: 1.1;
padding: 1px 6px;
}
.btn-overlay{
float: right;
vertical-align: middle;
display: inline-block;
}
.btn-flat{
background: var(--colour4);
border: 2px solid var(--colour5);
font-weight: bold;
cursor: pointer;
font-family: var(--buttonFont);
font-size: 10px;
color: var(--colour5);
border-radius: 5px;
height: 20px;
}
.btn-flat:disabled{
background: var(--colour4);
border-color: var(--colour3);
color: var(--colour3);
cursor: default;
}
.btn-flat:active{
border-color: var(--colour2);
color: var(--colour2);
}
.btn-flat:focus{
outline: none;
}
/*** Toggle Switch styles ***/
.tgl-switch {
float: right;
vertical-align: middle;
display: inline-block;
}
.tgl-switch .tgl {
display:none;
}
.tgl, .tgl:after, .tgl:before, .tgl *, .tgl *:after, .tgl *:before, .tgl + .tgl-slider {
-webkit-box-sizing: border-box;
box-sizing: border-box;
}
.tgl::-moz-selection, .tgl:after::-moz-selection, .tgl:before::-moz-selection, .tgl *::-moz-selection, .tgl *:after::-moz-selection, .tgl *:before::-moz-selection, .tgl + .tgl-slider::-moz-selection {
background: none;
}
.tgl::selection, .tgl:after::selection, .tgl:before::selection, .tgl *::selection, .tgl *:after::selection, .tgl *:before::selection, .tgl + .tgl-slider::selection {
background: none;
}
.tgl-slider {
float:right;
}
.tgl + .tgl-slider {
outline: 0;
display: block;
width: 40px;
height: 18px;
position: relative;
cursor: pointer;
-webkit-user-select: none;
-moz-user-select: none;
-ms-user-select: none;
user-select: none;
}
.tgl + .tgl-slider:after, .tgl + .tgl-slider:before {
position: relative;
display: block;
content: "";
width: 50%;
height: 100%;
}
.tgl + .tgl-slider:after {
left: 0;
}
.tgl + .tgl-slider:before {
display: none;
}
.tgl-flat + .tgl-slider {
padding: 2px;
-webkit-transition: all .2s ease;
transition: all .2s ease;
background: #fff;
border: 3px solid var(--colour4);
border-radius: 2em;
}
.tgl-flat + .tgl-slider:after {
-webkit-transition: all .2s ease;
transition: all .2s ease;
background: var(--colour4);
content: "";
border-radius: 1em;
}
.tgl-flat:checked + .tgl-slider {
border: 3px solid var(--colour5);
}
.tgl-flat:checked + .tgl-slider:after {
left: 50%;
background: var(--colour5);
}
/*** Toggle Switch styles ***/
#encoderSettings, #webRTCSettings {
margin: 10px 0px;
}
#encoderParamsContainer, #webrtcParamsContainer {
padding-left: 5%;
}
#encoder-params-submit {
float: right;
}
#webrtc-params-submit {
float: right;
}
select {
text-align: right;
}
input {
text-align: right;
}
.form-group {
display: grid;
grid-template-columns: 50% 50%;
}
.form-group label {
color: var(--colour4);
vertical-align: middle;
font-size: 18px;
font-weight: normal;
}
#latencyTest {
display: block;
}
#latencyTest button {
margin: 30px 0px;
}
#freezeFrameOverlay {
background-color: transparent;
}
.freezeframeBackground {
background-color: #000 !important;
}
除了這兩個文件 ,還有一個文件
app.js 修改像素流流送地址
修改成對應的地址即可
ue 創建基礎項目 並 顯示鼠標
隨便創建個第三人稱遊戲項目,名稱就叫 myUEToHtml
創建完成
但是這個鼠標運行之後,鼠標沒有顯示出來,可以參考這個現實鼠標
https://www.cnblogs.com/makalochen/p/17700166.html
到現在 前置工作已完成
ue 給靜態網格體添加鼠標點擊事件
現在在ue 中,隨便添加個 靜態網格體,給它添加一個鼠標點擊事件
添加靜態網格體
添加點擊事件
選中盒體,添加點擊事件
然後再關卡藍圖中就可以看到事件
隨便打印一句話,運行看看
沒反應,將鼠標懸浮在這個事件上面,提示
啓用點擊事件
在玩家控制器中啓用了點擊事件後,鼠標懸停此actor並點擊左鍵將進行調用
那我們在玩家控制器中啓用點擊事件
運行一下看看
模型OnClick事件不生效的兩種原因
參考:
https://blog.csdn.net/qqQQqsadfj/article/details/126083284
1.玩家控制器中點擊事件沒啓用
2.模型的碰撞沒設置好
ue 向 html 發送消息
參考
https://docs.unrealengine.com/4.27/zh-CN/SharingAndReleasing/PixelStreaming/CustomPlayer/
發送之後回調函數不調的,可能的原因
如上圖,是從玩家控制器裏去拿PixelStreamerInput 組件,那必須在玩家控制器中,添加組件
myUI.html
<!DOCTYPE html>
<html>
<head>
<!-- 該文件處理瀏覽器和虛幻引擎應用間的通信,接受並顯示來自服務器的媒體流。在非必要的情況下,請勿修改此JavaScript文件 -->
<script type="text/javascript" src="../scripts/webRtcPlayer.js"></script>
<!-- 此文件將設置處理鍵盤、鼠標和觸摸事件的事件監聽器 -->
<script type="text/javascript" src="../scripts/app.js"></script>
<link rel="shortcut icon" href="../images/favicon.ico" type="image/x-icon">
<link rel="icon" type="../image/png" sizes="96x96" href="/images/favicon-96x96.png">
<link rel="icon" type="../image/png" sizes="32x32" href="/images/favicon-32x32.png">
<link rel="icon" type="../image/png" sizes="16x16" href="/images/favicon-16x16.png">
<link type="text/css" rel="stylesheet" href="player.css">
<title>myUI</title>
</head>
<body onload="load()">
<div>
<button onclick="test()">測試</button>
<button onclick="reg()">註冊</button>
</div>
<div id="playerUI">
<div id="player"></div>
<div id="overlay" class="overlay text-light bg-dark">
<div>
<div id="qualityStatus" class="greyStatus">●</div>
<div id="overlayButton">+</div>
</div>
<div id="overlaySettings">
<div id="kickOthers">
<div class="settings-text">Kick all other players</div>
<label class="btn-overlay">
<input type="button" id="kick-other-players-button" class="overlay-button btn-flat"
value="Kick">
</label>
</div>
<div id="fillWindow">
<div class="settings-text">Enlarge Display to Fill Window</div>
<label class="tgl-switch">
<input type="checkbox" id="enlarge-display-to-fill-window-tgl" class="tgl tgl-flat" checked>
<div class="tgl-slider"></div>
</label>
</div>
<div id="qualityControlOwnership">
<div class="settings-text">Quality control ownership</div>
<label class="tgl-switch">
<input type="checkbox" id="quality-control-ownership-tgl" class="tgl tgl-flat">
<div class="tgl-slider"></div>
</label>
</div>
<br>
<!-- 編碼設置 -->
<section id="encoderSettings">
<div class="settings-text">Encoder Settings</div>
<div id="encoderParamsContainer" class="collapse">
<div class="form-group">
<label for="encoder-rate-control" class="settings-text">Rate Control</label>
<select id="encoder-rate-control">
<option value="CBR" selected>CBR</option>
<option value="VBR">VBR</option>
<option value="ConstQP">ConstQP</option>
</select>
<label for="encoder-target-bitrate-text">Target Bitrate (kbps)</label>
<input type="number" class="form-control" id="encoder-target-bitrate-text" value="0" min="0"
max="100000" />
<label for="encoder-max-bitrate-text">Max Bitrate (kbps)</label>
<input type="number" class="form-control" id="encoder-max-bitrate-text" value="0" min="0"
max="100000" />
<label for="encoder-min-qp-text">Min QP</label>
<input type="number" class="form-control" id="encoder-min-qp-text" value="0" min="0"
max="999" />
<label for="encoder-max-qp-text">Max QP</label>
<input type="number" class="form-control" id="encoder-max-qp-text" value="0" min="0"
max="999" />
<label for="encoder-multipass" class="settings-text">Multipass</label>
<select id="encoder-multipass">
<option value="DISABLED" selected>DISABLED</option>
<option value="QUARTER">QUARTER</option>
<option value="FULL">FULL</option>
</select>
<div class="settings-text">Filler Data</div>
<label class="tgl-switch">
<input type="checkbox" id="encoder-filler-data-tgl" class="tgl tgl-flat">
<div class="tgl-slider"></div>
</label>
</div>
<input id="encoder-params-submit" class="btn btn-primary btn-lg mt-3" type="button"
value="Apply">
</div>
<br>
</section>
<!-- webRTC 設置 -->
<section id="webRTCSettings">
<div class="settings-text">WebRTC Settings</div>
<div id="webrtcParamsContainer" class="collapse">
<div class="form-group">
<label for="webrtc-degradation-pref">Degradation Pref</label>
<select id="webrtc-degradation-pref">
<option value="BALANCED">BALANCED</option>
<option value="MAINTAIN_FRAMERATE">MAINTAIN_FRAMERATE</option>
<option value="MAINTAIN_RESOLUTION">MAINTAIN_RESOLUTION</option>
</select>
<label for="webrtc-max-fps-text">Max FPS</label>
<input type="number" class="form-control" id="webrtc-max-fps-text" value="1" min="1"
max="999" />
<label for="webrtc-min-bitrate-text">Min Bitrate (kbps)</label>
<input type="number" class="form-control" id="webrtc-min-bitrate-text" value="0" min="0"
max="100000" />
<label for="webrtc-max-bitrate-text">Max Bitrate (kbps)</label>
<input type="number" class="form-control" id="webrtc-max-bitrate-text" value="0" min="0"
max="100000" />
<label for="webrtc-low-qp-text">Low QP Threshold</label>
<input type="number" class="form-control" id="webrtc-low-qp-text" value="0" min="0"
max="999" />
<label for="webrtc-high-qp-text">High QP Threshold</label>
<input type="number" class="form-control" id="webrtc-high-qp-text" value="0" min="0"
max="999" />
</div>
<input id="webrtc-params-submit" class="btn btn-primary btn-lg mt-3" type="button"
value="Apply">
</div>
</section>
<br>
<!-- 顯示FPS -->
<div id="showFPS">
<div class="settings-text">Show FPS</div>
<label class="btn-overlay">
<input type="button" id="show-fps-button" class="overlay-button btn-flat" value="Toggle">
</label>
</div>
<div id="matchViewportResolution">
<div class="settings-text">Match Viewport Resolution</div>
<label class="tgl-switch">
<input type="checkbox" id="match-viewport-res-tgl" class="tgl tgl-flat">
<div class="tgl-slider"></div>
</label>
</div>
<div id="statsPanel">
<div class="settings-text">Show Stats</div>
<label class="tgl-switch">
<input type="checkbox" id="show-stats-tgl" class="tgl tgl-flat" checked>
<div class="tgl-slider"></div>
</label>
<div id="statsContainer" class="statsContainer">
<div id="stats" class="stats"></div>
</div>
<br>
</div>
<div id="latencyTest">
<div class="settings-text">Latency Stats</div>
<label class="btn-overlay">
<input type="button" id="test-latency-button" class="overlay-button btn-flat"
value="Test Latency">
</label>
<div id="latencyStatsContainer" class="statsContainer">
<div id=LatencyStats class="stats">No stats yet...</div>
</div>
</div>
</div>
</div>
</div>
</body>
<script>
function test(){
// 發送
emitUIInteraction(`測試發送`);
// 註冊監聽器函數
addResponseEventListener("handle_responses", (data) => {
// ue 返回
console.log(`myHandleResponseFunction ----------> ${data}`);
});
}
function reg(){
// 註冊監聽器函數
addResponseEventListener("handle_responses", (data) => {
// ue 返回
console.log(`myHandleResponseFunction2 ----------> ${data}`);
});
}
</script>
</html>
ue 關卡藍圖
上圖最主要的 獲取玩家控制器 -> 按類獲取組件(前提是你在玩家控制器藍圖中,已經將像素流組件添加進去了)