2、WebRTC架構圖
3、WebRTC架構組件介紹
Web開發者開發的程序，Web開發者可以基於集成WebRTC的瀏覽器提供的web API開發基於視頻、音頻的實時通信應用。
面向第三方開發者的WebRTC標準API（Javascript），使開發者能夠容易地開發出類似於網絡視頻聊天的web應用，最新的標準化進程可以查看這裏。
(3) WebRTC Native C++ API
本地C++ API層，使瀏覽器廠商容易實現WebRTC標準的Web API，抽象地對數字信號過程進行處理。
a. RTP Stack協議棧
Real Time Protocol
b. STUN/ICE
可以通過STUN和ICE組件來建立不同類型網絡間的呼叫連接。
c. Session Management
一個抽象的會話層，提供會話建立和管理功能。該層協議留給應用開發者自定義實現。
音頻引擎是包含一系列音頻多媒體處理的框架，包括從視頻採集卡到網絡傳輸端等整個解決方案。
PS：VoiceEngine是WebRTC極具價值的技術之一，是Google收購GIPS公司後開源的。在VoIP上，技術業界領先，後面的文章會詳細瞭解
採樣頻率：16khz，24khz，32khz；（默認爲16khz）
自適應速率爲10kbit/s ~ 52kbit/；
自適應包大小：30~60ms；
算法延時：frame + 3ms
Internet Low Bitrate Codec
VoIP音頻流的窄帶語音編解碼器
採樣頻率：8khz；
20ms幀比特率爲15.2kbps
30ms幀比特率爲13.33kbps
標準由IETF RFC3951和RFC3952定義
c. NetEQ for Voice
回聲消除器是一個基於軟件的信號處理元件，能實時的去除mic採集到的回聲。
噪聲抑制也是一個基於軟件的信號處理元件，用於消除與相關VoIP的某些類型的背景噪聲（嘶嘶聲，風扇噪音等等… …）
WebRTC視頻處理引擎
VideoEngine是包含一系列視頻處理的整體框架，從攝像頭採集視頻到視頻信息網絡傳輸再到視頻顯示整個完整過程的解決方案。
視頻圖像編解碼器，是WebRTC視頻引擎的默認的編解碼器
VP8適合實時通信應用場景，因爲它主要是針對低延時而設計的編解碼器。
PS:VPx編解碼器是Google收購ON2公司後開源的，VPx現在是WebM項目的一部分，而WebM項目是Google致力於推動的HTML5標準之一
視頻抖動緩衝器，可以降低由於視頻抖動和視頻信息包丟失帶來的不良影響。
圖像質量增強模塊
對網絡攝像頭採集到的圖像進行處理，包括明暗度檢測、顏色增強、降噪處理等功能，用來提升視頻質量。
4、WebRTC核心模塊API
(1)、網絡傳輸模塊：libjingle
(2)、音頻、視頻圖像處理的主要數據結構
|
類、結構體、枚舉常量 |
頭文件 |
說明 |
|
Structures |
common_types.h |
Lists the structures common to the VoiceEngine & VideoEngine |
|
Enumerators |
common_types.h |
List the enumerators common to the VoiceEngine & VideoEngine |
|
Classes |
common_types.h |
List the classes common to VoiceEngine & VideoEngine |
|
class VoiceEngine |
voe_base.h |
How to allocate and release resources for the VoiceEngine using factory methods in the VoiceEngine class. It also lists the APIs which are required to enable file tracing and/or traces as callback messages |
|
class VideoEngine |
vie_base.h |
How to allocate and release resources for the VideoEngine using factory methods in the VideoEngine class. It also lists the APIs which are required to enable file tracing and/or traces as callback messages |
(3)、音頻引擎（VoiceEngine）模塊 APIs
| sub-API |
頭文件 |
說明 |
|
VoEAudioProcessing |
voe_audio_processing.h |
Adds support for Noise Suppression (NS), Automatic Gain Control (AGC) and Echo Control (EC). Receiving side VAD is also included. |
|
VoEBase |
voe_base.h |
Enables full duplex VoIP using G.711. |
|
VoECallReport |
voe_call_report.h |
Adds support for call reports which contains number of dead-or-alive detections, RTT measurements, and Echo metrics. |
|
VoECodec |
voe_codec.h |
Adds non-default codecs (e.g. iLBC, iSAC, G.722 etc.), Voice Activity Detection (VAD) support. |
|
VoEDTMF |
voe_dtmf.h |
Adds telephone event transmission, DTMF tone generation and telephone event detection. (Telephone events include DTMF.) |
|
VoEEncryption |
voe_encryption.h |
Adds external encryption/decryption support. |
|
VoEErrors |
voe_errors.h |
Error Codes for the VoiceEngine |
|
VoEExternalMedia |
voe_external_media.h |
Adds support for external media processing and enables utilization of an external audio resource. |
|
VoEFile |
voe_file.h |
Adds file playback, file recording and file conversion functions. |
|
VoEHardware |
voe_hardware.h |
Adds sound device handling, CPU load monitoring and device information functions. |
|
VoENetEqStats |
voe_neteq_stats.h |
Adds buffer statistics functions. |
|
VoENetwork |
voe_network.h |
Adds external transport, port and address filtering, Windows QoS support and packet timeout notifications. |
|
VoERTP_RTCP |
voe_rtp_rtcp.h |
Adds support for RTCP sender reports, SSRC handling, RTP/RTCP statistics, Forward Error Correction (FEC), RTCP APP, RTP capturing and RTP keepalive. |
|
VoEVideoSync |
voe_video_sync.h |
Adds RTP header modification support, playout-delay tuning and monitoring. |
|
VoEVolumeControl |
voe_volume_control.h |
Adds speaker volume controls, microphone volume controls, mute support, and additional stereo scaling methods. |
(4)、視頻引擎（VideoEngine）模塊 APIs
|
sub-API |
頭文件 |
說明 |
|
ViEBase |
vie_base.h |
Basic functionality for creating a VideoEngine instance, channels and VoiceEngine interaction. NOTE: This API must always be created. |
|
ViECapture |
vie_capture.h |
Adds support for capture device allocation as well as capture device capabilities. |
|
ViECodec |
vie_codec.h |
Adds non-default codecs, codec settings and packet loss functionality. |
|
ViEEncryption |
vie_encryption.h |
Adds external encryption/decryption support. |
|
ViEErrors |
vie_errors.h |
Error codes for the VideoEngine |
|
ViEExternalCodec |
vie_external_codec.h |
Adds support for using external codecs. |
|
ViEFile |
vie_file.h |
Adds support for file recording, file playout, background images and snapshot. |
|
ViEImageProcess |
vie_image_process.h |
Adds effect filters, deflickering, denoising and color enhancement. |
|
ViENetwork |
vie_network.h |
Adds send and receive functionality, external transport, port and address filtering, Windows QoS support, packet timeout notification and changes to network settings. |
|
ViERender |
vie_render.h |
Adds rendering functionality. |
|
ViERTP_RTCP |
vie_rtp_rtcp.h |
Adds support for RTCP reports, SSRS handling RTP/RTCP statistics, NACK/FEC, keep-alive functionality and key frame request methods. |