1. Overview
This documentation contains the resources you need to get started developing your own applications using experimental browser features for real-time voice and video communication. This includes instructions about how to install the modified WebKit library as well as an example of how to write a video conferencing application using peer-to-peer communication.
A sub-set of the current WhatWG "Web Real-Time Communication APIs" specification APIs is supported to enable real-time voice and video communication (as well as peer-to-peer data). In the WebKit library provided the following features/APIs are implemented: getUserMedia, Stream API and PeerConnection API. All APIs are JavaScript and HTML, i.e. there is no need for plug-ins.
2. Getting started
NOTE! We recommend that you use a computer dedicated for testing. The existing WebKit library will be overwritten.
Before you can start using these features/APIs you need to install a modified version of the WebKit library on your Ubuntu system. To do this just follow the instructions in downloads.
Real-time voice and video communication can then be tested using either GtkLauncher or the Epiphany browser. When the installation is ready you can try out our demo application at here.
3. Using the APIs
For a better understanding, the API examples below can be complemented by reading the full WhatWG
specification. Please remember that the entry points to the APIs are currently vendor-prefixed with the 'webkit' prefix, e.g.
navigator.webkitGetUserMedia(),
new webkitPeerConnection() etc.
3.1. Getting access to camera and microphone
The getUserMedia method is needed to make it possible to obtain local media content from either a microphone or a camera and make it addressable for a web developer.
function askForAudioAndVideoConsent() {
navigator.webkitGetUserMedia("audio, video", function(stream) {
// Do something with the stream.
});
}When this method is called the user will be prompted for permission to use the media before any data is accessible by the application. Permission is given per page and needs to be requested again on each page reload.
3.2. Creating a self-view
To make sure that the application uses the intended camera and microphone it might be a good idea to show a self-view. That can be done in the following way:
<!DOCTYPE html>
<html>
<head>
<title>Self-view example</title>
<script>
var localStream;
window.onload = function () {
//Ask for local streams to be prepared, display self view
navigator.webkitGetUserMedia("audio, video user", gotStream);
};
function gotStream(stream) {
localStream = stream;
document.getElementById("selfView").src = webkitURL.createObjectURL(localStream);
}
</script>
</head>
<body>
<video id="selfView" autoplay audio=muted></video>
</body>
</html>The getUserMedia method uses the
Stream API to return an addressable stream from the local camera. Connecting the media stream to an HTML
<video> element will display the self-view on the page.
In cases where you only want access to the microphone (or camera only) you can leave the other one out:
navigator.webkitGetUserMedia("audio", gotAudioStream);3.3. Obtaining local media content and using PeerConnection API
The PeerConnection API allows two users to communicate directly, browser-to-browser, instead of going via a server. The effect is lower latency and more efficient network utilization.
The PeerConnection object is used by attaching streams (see PeerConnection API) which helps it identify the sources from which it should stream the media. When a stream is attached, it will result in a call to the signaling callback given to the PeerConnection constructor.
A simple example is presented below showing how PeerConnection can be used. Please not that It is incomplete; however, was included for explanatory purposes. The code is split into three parts for simplicity:
1) code that is used by both clients,
2) code used by only client A which is the caller, and
3) code used only client B which is the receiver.
3.3.1. Code used by both client A and B.
createPeerConnection creates the PeerConnection object with the STUN/TURN configuration string and a signaling callback method used whenever the PeerConnection needs to send the session description to the other client.
var peerConn;
function createPeerConnection() {
if (peerConn == null) {
peerConn = new webkitPeerConnection("TURN 123.123.123.123:12345", signalingCallback);
peerConn.onopen = indicateConnected();
peerConn.onaddstream = startShowingStream;
}
}startShowingStream is called (as result of "peerConn.onAddStream = startShowingStream") when the PeerConnection object starts receiving data from a stream. When called, it puts the stream received as the source of the video element that should show it.
startShowingStream(streamEvent) {
videoElement = document.getElementById("remoteView");
video.src = webkitURL.createObjectURL(streamEvent.stream);
}signalingCallback is called when the PeerConnection needs to send a session description to the other client. It is then up to the web application to forward the message.
function signalingCallback(signal) {
somehowSendThisToPeer(signal);
}3.3.2. Code used by client A (caller)
callAudioVideo is called when the web application wants to call client B. The first step here is to get the Stream object. In this implementation, GetUserMedia will create a pop-up where where the user can select a camera and a microphone and then call the
call-back function "callWithStream" when done.
function callAudioVideo() {
navigator.webkitGetUserMedia("audio, video", callWithStream);
}callWithStream will be called when the user has selected the camera and microphone in the dialog pop-up from GetUserMedia. A PeerConnection object will be created and the stream will be added. This will result in a call-back to the "signalingCallback" function,
which will send the session description to client B.
function callWithStream(stream) {
createPeerConnection();
peerConn.addStream(stream);
}The incomingSingalMessage is called as a result of client B sending an answer back. It is the web application's responsibility to call the processSignalingMessage with the answer.
function incomingSignalingMessage(signal) {
peerConn.processSignalingMessage(signal)
}3.3.3. Code used by client B (receiver)
incomingCall is called by the web application when there is an incoming offer from client B. It then creates a PeerConnection object and call processSignalingMessage with the offer, which will result in PeerConnection calling the signalingCallback with the
answer.
function incomingCall(signal) {
createPeerConnection();
peerConn.processSignalingMessage(signal);
}addOutgoingAudioVideo is later called by the web application if it wants to add a stream to send audio and video back to client A.
function addOutgoingAudioVideo() {
navigator.webkitGetUserMedia("audio, video", addStreamToCall);
}addStreamToCall is called when the B side has selected the camera and microphone from the getUserMedia popup dialog. It then simply adds the created stream with the addStream function. This will then result in the PeerConnection calling the signalingCallback
with the offer.
function addStreamToCall(stream) {
peerConn.addStream(stream);
}3.4. Peer-to-Peer data
When the PeerConnection is created and has been connected, it is possible to send data peer-to-peer using the UDP data channel that is automatically created. This data channel is unreliable (as UDP is used for transport) and packets received out of order
are discarded.
function sendMsg(msg) {
peerConn.send(msg);
}When data is recieved an onmessage event is generated. The peer-to-peer data can be usable both in conferencing scenarios (for e.g. context data) and gaming scenarios. Since data
is transported directly between peers (without passing a server) it is especially suitable for delay sensitive data (as in real-time gaming).
4. Codecs
Currently, the browser uses codecs that are already available in Ubuntu. Voice is encoded with Speex and video is encoded using Theora. More suitable codecs will be added later, but in this experimental implementation, these codecs will be sufficient. GStreamer’s default settings have been used to configure the encoders, which means that the encoding quality parameter is 8 for the audio and 48 for the video.
5. NAT traversal
The PeerConnection API requires a direct communication channel between two browsers. To make sure such channel can be established no matter which network topology is used, NAT-traversal
techniques must be used. This browser uses the ICE methodology provided by the
libnice library to traverse NATs. When a channel between two browsers is to be established, the first task for both peers is to gather addresses where it can be reached. These are called
transport candidates and to find all candidates, STUN or TURN servers are used.
The peers exchange the list of candidates via an HTTP server and start probing for connection. Note that the HTTP server and the signaling channel to it is not provided by Ericsson Labs. When both peers have reached each other, the UDP channel is up and
the PeerConnection API can be used to send media or data peer-to-peer. If there are firewalls present between the peers, these could still prevent connectivity and cause problems for
the PeerConnection API.
5.1. Local testing
Two computers connected to the same local network can reach each other without using a STUN server. To facilitate testing we have added a non-standardized way of setting up this test case. The local host address will be used if the STUN/TURN configuration
string is set to "NONE".
peerConn = new webkitPeerConnection("NONE", signalingCallback);How to use the PeerConnection API is described
here
5.2. Remote testing
More complicated network setups, where NATs are involved, require a STUN/TURN server to work. To setup a connection include the IP-address and port of the STUN/TURN server you want to use:
peerConn = new webkitPeerConnection("TURN 123.123.123.123:12345", signalingCallback);To facilitate testing with your own TURN server we have added a non-standardized way of providing username and password.
peerConn = new webkitPeerConnection("TURN 123.123.123.123:12345 username:passwd", signalingCallback);NOTE! Ericsson Labs is currently not providing a STUN/TURN server and cannot guarantee that the included ICE implementation can be used to set up a connection for remote testing.
6. Support
The "Ericsson Labs Web-RTC" Google Group is open for anyone wanting to post a question or comment. Engineers from Ericsson Research are actively monitoring and participating in the discussions.
7. Disclaimer
This experimental browser is provided to do early experimentation and testing. It is likely to crash at any time so don't use it for any real communication. In addition, there are a number of items that are not supported, including, but not limited to:
- Not all features of the WhatWG APIs are supported. StreamRecorder is one of them.
- There is no protection or encryption. All streams could be easily intercepted.
- Ericsson Labs is currently not providing a STUN/TURN server and cannot guarantee that remote testing will work properly.
- The ICE functionality is provided by the libnice library and there may be compatibility issues for some STUN/TURN servers.
- Since only standard GStreamer components are used, the performance in terms of, for example, mouth-to-ear delay is not optimal. We have also noted a delay build up during long sessions.
- Media is using plain RTP/UDP for transport. There is no RTCP signaling and no rate adaptation. (The latter is true for the datagram channel as well).
- There is no echo control (use a headset!).
- The SDPs used to set up the connection and streams are not fully standards compliant.
- Asking the user for consent to use cam and mic is implemented in WebKit WebCore, and not part of the browser (as the spec says). This was done intentionally to be able to re-use the implementation for different WebKit (GtkLauncher, Epiphany, ...) browsers without having to work on the browser itself.
8. FAQ
When will these APIs be available in other browsers?
It all depends on how quickly the standardization progresses. There will likely be experimental implementations similar to ours made available in other browsers as well. But since the APIs are still rather unstable it will take a while before different browsers can "talk" to each other.
Why is Ericsson doing this?
It is important that Web-RTC becomes a standard adopted by all browser vendors. We believe one of the best ways of influencing the standards and specifications (see WhatWG, W3C and IETF) is to do early experimental implementations of the APIs and provide feedback based on our findings. Exposing developers to the APIs at an early stage helps to iron out potential issues.
Can I get the source code?
No, not at this stage. However, we are working hard to contribute our implementation to the WebKit Open Source project.
Will the browser be available on other platforms?
We currently don't have any plans to support other platforms than Linux (Ubuntu).
Is it possible to configure the browser to use other codecs, or other codec configurations?
No, not in this release.
How can I contribute?
Apart from building your own Web-RTC app, trying out the APIs and providing feedback you are encouraged to contribute to the WebKit Open Source project as well as join the discussion on the WhatWG and the public W3C webrtc mailing lists.
So I've built an app, now what?
We would love to see and try out your applications, please consider publishing it in the Labs showroom.
