Real-Time Communication

in a browser

Demos!

Video conference

Screen sharing

Mutliplayer games (TCP and UDP)

File sharing #1

File sharing #2

FileDrop

OS X AirDrop clone in HTML5

So what is WebRTC exactly?

3 APIs

  • MediaStream (AKA getUserMedia)
  • RTCPeerConnection
  • RTCDataChannel

MediaStream

  • Represents a stream of audio and/or video
  • Can contain multiple 'tracks'
  • Obtain a MediaStream with navigator.getUserMedia()

MediaStream

MediaStream

        var constraints = {video: true};

        function successCallback(stream) {
          var video = document.querySelector("video");
          video.src = window.URL.createObjectURL(stream);
        }

        function errorCallback(error) {
          console.log("navigator.getUserMedia error: ", error);
        }

        navigator.getUserMedia(constraints, successCallback, errorCallback);
      

MediaStream

Constraints

  • Controls the contents of the MediaStream
  • Media type, resolution, frame rate
        {
          "mandatory": {
            "width": { "min": 480 }
          },
          "optional": [
            { "width": { "min": 720 } },
            { "frameRate": 60 },
            { "facingMode": "user" }
          ]
        }
      

MediaStream

Screen capturing

        var constraints = {
          video: {
            mandatory: {
              chromeMediaSource: 'screen'
            }
          }
        };

        navigator.webkitGetUserMedia(constraints, gotStream);
      

RTCPeerConnection

  • Peer to peer communication
  • Codec handling
  • Signal processing
  • Security
  • Bandwidth management
  • ...

RTCPeerConnection

        pc = new RTCPeerConnection(configuration);
        pc.onaddstream = gotRemoteStream;
        pc.addStream(localStream); // from getUserMedia
        pc.createOffer(gotOffer);

        function gotOffer(desc) {
          pc.setLocalDescription(desc);
          sendOffer(desc);
        }

        function gotAnswer(desc) {
          pc.setRemoteDescription(desc);
        }

        function gotRemoteStream(e) {
          attachMediaStream(remoteVideo, e.stream);
        }
      

RTCDataChannel

  • Same API as WebSockets
  • Ultra-low latency
  • Unreliable or reliable (similar to UDP and TCP)
  • Secure

RTCPeerConnection

        pc = new RTCPeerConnection(configuration);

        pc.ondatachannel = function (event) {
          receiveChannel = event.channel;
          receiveChannel.onmessage = function (event) {
            document.querySelector("div#receive").innerHTML = event.data;
          };
        };

        sendChannel = pc.createDataChannel("sendDataChannel", {reliable: false});

        document.querySelector("button#send").onclick = function () {
          var data = document.querySelector("textarea#send").value;
          sendChannel.send(data);
        };
      

Peer to peer, but...

you need 3 servers

signaling

STUN

TURN

WebRTC needs four types of server-side functionality

  • User discovery and communication
  • Signaling
  • NAT/firewall traversal
  • Relay servers in case peer-to-peer communication fails

Signaling

  • Need to exchange 2 types of objects:
    • Session Description - what codecs and resolutions can be handled by my browser and the browser it wants to communicate with?
    • ICE candidates - what's my computer's IP address and port?
  • WebRTC does not specify any protocol for that - you can use anything
    • WebSockets
    • AJAX
    • ...

Signaling

JSEP architecture diagram

RTCSessionDescription

  • SDP - Session Description protocol
  • offer / answer
        pc.createOffer(gotOffer);

        function gotOffer(desc) {
          pc.setLocalDescription(desc);
          sendOffer(desc);
        }

        function gotAnswer(desc) {
          pc.setRemoteDescription(desc);
        }
      

STUN

TURN

An ideal world

Data pathways between peers if there were no NATs or firewalls

The real world

Data pathways showing firewalls

STUN

Session Traversal Utilities for NAT

  • Tell me what my public IP address is
  • Simple server, cheap to run
  • Data flows peer-to-peer

STUN

Data pathways between peers using STUN

TURN

Traversal Using Relays around NAT

  • Cloud fallback if peer-to-peer communication fails
  • Ensures that call works in almost all environments
  • Data flows through a server, uses server bandwidth

TURN

Data pathways between peers using TURN

ICE

Interactive Connectivity Establishment

  • ICE: a framework for connecting peers
  • Tries to find the best path for each call
  • Vast majority of calls can use STUN (webrtcstats.com):
Data pathways between peers using TURN

SDP

ICE

STUN

TURN

Step by step example

Peer #1: Create connection

        pc = new RTCPeerConnection(configuration);
      

Step by step example

Peer #1: Create offer (SDP)

        pc = new RTCPeerConnection(configuration);
        pc.createOffer(gotOffer)
      

Step by step example

Peer #1: Set generated offer as local description and send it to other peers

        pc = new RTCPeerConnection(configuration);
        pc.createOffer(gotOffer)

        function gotOffer(description) {
          pc.setLocalDescription(description);
          sendOffer(description);
        }
        
      

Step by step example

Peer #1: Setting local description generates local ICE candidates

        function gotOffer(description) {
          pc.setLocalDescription(description);
          sendOffer(description);
        }

        pc.onicecandidate = function (event) {
          sendCandidate(event.candidate);
        }
      

Step by step example

Peer #2: Receive an answer (SDP) from other peer and set it as remote description

        function gotAnswer(desc) {
          pc.setRemoteDescription(desc);
        }
      

Step by step example

Peer #2: Setting remote description generates local ICE candidates

        pc.onicecandidate = function (event) {
          sendCandidate(event.candidate);
        }
      

Step by step example

Peer #1 and #2: Process received ICE candidates

        function gotIceCandidate (message) {
          pc.addIceCandidate(new RTCIceCandidate({
            sdpMLineIndex: message.label,
            candidate: message.candidate
          }));
        }
      

Done :)

JavaScript frameworks

SimpleWebRTC

Easy peer-to-peer video and audio

        var webrtc = new WebRTC({
          localVideoEl: 'localVideo',
          remoteVideosEl: 'remoteVideos',
          autoRequestMedia: true
        });

        webrtc.on('readyToCall', function () {
          webrtc.joinRoom('My room name');
        });
      

PeerJS

Easy peer-to-peer data

        var peer = new Peer('someid');
        peer.on('connection', function(conn) {
          conn.on('data', function(data){
            // Will print 'hi!'
            console.log(data);
          });
        });

        // Connecting peer
        var peer = new Peer('anotherid');
        var conn = peer.connect('someid');
        conn.on('open', function(){
          conn.send('hi!');
        });
      

More information

Thank You!