Simplifying outbound bandwidth and simulcast configuration with Daily’s send settings

Daily’s Client SDK for JavaScript provides developers with many options to optimize their call experience to their needs. One powerful example of this is the ability to configure your own simulcast layers and upstream video bandwidth caps.

For those unfamiliar with simulcast, it is a technique of sending multiple encodings of a video track at varying qualities. A receiver can then consume the track that will perform the best for their own network conditions (and switch tracks on the fly as their conditions degrade or improve.) You can read more about simulcast and how it works in our introduction to simulcast.

But I’ll be the first to admit that I found that these configuration options to be a little bit fiddly when working with our API for certain kinds of custom implementations.

A prime example is my spatialization demo. In this demo, each participant’s video is displayed in square video tiles. The tiles are shown at a lower resolution when the user is traversing the world at large, and toggle to higher res when a user steps into a “focus zone”, like so:

I’ve covered how I implemented this at the time in the Daily configuration part of the series, but let’s go through a brief overview.

The original implementation using camSimulcastEncodings and setBandwidth()

In the original implementation, there were two main steps to managing a participant’s outbound video quality:

1. Toggling the video track’s dimensions and frame rate depending on where they are in the world with setBandwidth()
2. Setting a single simulcast layer to facilitate the above. setBandwidth() affects the highest simulcast layer and Daily comes configured with three layers by default. Because the tiles in this demo are already quite low-res, I defined a single layer to ensure the video quality never dropped below my chosen threshold.

Toggling the video track’s dimensions and frame rate

I toggle the user’s upstream video bandwidth between 96x96px at 15fps (in world traversal mode) and 200x200px at 30fps (in focus mode) using Daily’s setBandwidth() call object instance method:

Old implementation:

  private setBandwidth(level: BandwidthLevel) {
     switch (level) {
       case BandwidthLevel.Tile:
         this.localBandwidthLevel = level;
         this.callObject.setBandwidth({
           trackConstraints: {
             width: standardTileSize,
             height: standardTileSize,
             frameRate: 15,
           },
         });
         break;
       case BandwidthLevel.Focus:
         this.localBandwidthLevel = level;
         this.callObject.setBandwidth({
           trackConstraints: {
             width: 200,
             height: 200,
             frameRate: 30,
           },
         });
         break;
      default:
        console.warn(
          `setBandwidth() called with unrecognized level (${level}). Not modifying any constraints.`,
        );
    }
  }

Defining a single simulcast layer

To facilitate the above, I define a single simulcast layer using Daily’s camSimulcastEncodings call object property:

Old implementation:

this.callObject = DailyIframe.createCallObject({
       dailyConfig: {
         camSimulcastEncodings: [{ maxBitrate: 600000, maxFramerate: 30 }],
       },
     })

This worked, but it was a little clunky from the implementation perspective. It felt like a workaround to define a single simulcast layer with best-estimate reasonable defaults just to use setBandwidth().

Filipi’s send settings proposal to the rescue

Filipi, a software engineer on our Client SDK team, slid into Notion with an API proposal to make this feature of our JavaScript client SDK more intuitive. Thus, sendSettings was born!

Following the proposal, Daily’s Client SDK team implemented the following constructs to take the place of both camSimulcastEncodings and setBandwidth() (both of which will eventually be deprecated):

• The sendSettings call object property, which can be used to define up to three simulcast layers on call object creation.
• The updateSendSettings() call object instance method, which can be used to modify simulcast layer definitions, select a reasonable preset, or update the maximum usable layer (’low’, ’medium', or ’high’).
• The getSendSettings()call object instance method, enabling you to see what send settings are currently in effect.
• The ”send-settings-updated" event, which a client can subscribe to if needed to handle send setting modifications.

So I set about replacing my camSimulcastEncodings and setBandwidth() implementation with send settings in the spatialization demo.

Replacing the original spatialization implementation with sendSettings

You can check out a PR with a diff of the changes on GitHub, but here are the highlights of what went down:

Customizing dimensions of the sent video track

setBandwidth() allowed me to set video width and height constraints for the video track. updateSendSettings(), on the other hand, only has a scaleResolutionDownBy property. I have some logic outside of the Daily component of the demo to produce appropriately-sized video tiles, but I still wanted to send the most relevant video resolution possible from the beginning.

To do this, with the invaluable Slack guidance from our Product lead Mark, I made use of our userMediaVideoConstraints call object property to specify a default resolution of 400x400px:

New code:

    this.callObject = DailyIframe.createCallObject({
       // Other properties here...
       dailyConfig: {
         userMediaVideoConstraints: {
           width: 400,
           height: 400,
         },
       },
     })

Now, the largest tile we’ll display is 200x200px, when the user is in a focus zone. So why specify 400x400px above? I’ll let Mark himself elaborate:

Setting the value to 200x200 results in Chrome only sending a single layer. You have to send a higher resolution video so that Chrome will send two layers. We get the right video resolution by using the `scaleResolutionDownBy` value in the send settings.

- Mark (Daily’s Product Lead)

Specifying two simulcast layers with sendSettings

I then specified two relevant simulcast encodings with the sendSettings call object configuration property as follows:

New code:

  this.callObject = DailyIframe.createCallObject({
       sendSettings: {
         video: {
           encodings: {
             low: {
               maxBitrate: 75000,
               scaleResolutionDownBy: 4,
               maxFramerate: 15,
             },
             medium: {
               maxBitrate: 300000,
               scaleResolutionDownBy: 2,
               maxFramerate: 30,
             },
           },
         },
       },
      // Other properties below...
     })

The low encoding above will be used for world traversal, when the user is displayed as a small tile. The medium encoding will be used when the user steps into a focus zone. The actual encoding object is an instance of RTCRtpEncodingParameters.

Note the scaleResolutionDownBy property set for each layer above. With the original track constraints being set to 400x400px, this will result in the “low” encoding sending video with the dimensions of 100x100px and “medium” encoding sending dimensions of 200x200px.

Updating which maximum quality is sent to other participants

Now, instead of calling setBandwidth() to modify track constraints, I can call updateSendSettings() to modify the maximum layer sent to other participants in the call as follows:

New code:

  private setVideoQuality(level: BandwidthLevel) {
    switch (level) {
      case BandwidthLevel.Tile:
        this.callObject.updateSendSettings({
          video: {
            maxQuality: "low",
          },
        });
        break;
      case BandwidthLevel.Focus:
        this.callObject.updateSendSettings({
          video: {
            maxQuality: "medium",
          },
        });
        break;
      default:
        console.warn(
          `setVideoQuality() called with unrecognized level (${level}). This is a no-op.`,
        );
    }
  }

And with that, the original implementation is now fully replaced with the new send settings features. This updated approach has resulted in:

• More appropriate simulcast layers (as opposed to just one generic “best effort” layer)
• More intuitive behavior, in that you don’t need to worry about which simulcast layer is being affected when modifying track constraints like you did with setBandwidth() before.

Conclusion

If you’re currently already using camSimulcastEncodings and setBandwidth() in your own Daily implementation, I encourage you to try out the send settings approach. Eventually, send settings will replace the previous constructs completely (you might notice that camSimulcastEncodings is already marked as deprecated in our documentation and TypeScript definitions.)

If you have any questions about converting your own application, don’t hesitate to reach out to our support team or in our active WebRTC community, peerConnection.

And if you’re curious to dig more into the spatialization demo itself, check out the rest of my spatialization series. You can also take a look at our introductory spatialization guide to get started with building spatialized applications with Daily.