Implementing a participant audio level indicator with Daily’s video API

We recently introduced new methods and events to monitor local and remote participants’ audio levels in our JavaScript client SDK. This is a feature I’ve been wanting for a while, and I had the perfect demo to try it out with: Code of Daily: Modern Wordfare.

What is Code of Daily: Modern Wordfare?

If you’re not already familiar with CoD, it is a social game built with Daily that implements mechanics from the popular game Codewords (TM). You can check out my introduction to the game or the entire social gaming series.

You do not need to be familiar with any frameworks to follow along with this post. The demo is written in “vanilla” TypeScript and utilizes a client and server component. On the client, I use Daily’s Client SDK for JavaScript to instrument and manage the video call. On the server, I use Daily’s REST API to create Daily rooms. This post will focus exclusively on the client side.

For the purposes of this post, all you’ll need to know about the game itself is that the UX features participant tiles for each player. Each tile contains the player’s name and video (if their camera is on):

Code of Daily: Modern Workfare game screen

What I’m adding with Daily’s new audio level monitoring features is a subtle border to each participant, which fades in and out depending on their audio volume level:

Volume level indicator

In this post, I’ll go through how this audio level indicator is implemented. You can also check out the full diff of the implementation on GitHub.

Let’s start with a quick overview of all of Daily’s new audio level monitoring methods and events.

Daily’s new audio level monitoring constructs

For monitoring the local participant’s audio level:

For monitoring remote participants’ audio levels:

For my CoD: Modern Wordfare implementation, I’m specifically going to be using remote participants’ audio level data. With that, let’s get into the implementation.

Monitoring remote participants’ audio levels

In CoD, all the client-side Daily logic is encapsulated inside my Call class within /src/client/daily.ts.

The main consumer of this class is my Game class, which is where the whole game is managed.

Therefore, it is the game that knows what to do with the players’ audio level data. And the game needs to ask Daily to begin monitoring and providing this data.

To do that, I introduced a new exported method inside the Call class: registerRemoteParticipantsAudioLevelHandler():

registerRemoteParticipantsAudioLevelHandler(h: RemoteAudioLevelHandler) {
     // Start audio level observer only once game explicitly asks for it
this.callObject.startRemoteParticipantsAudioLevelObserver(200);
     this.callObject.on("remote-participants-audio-level", (e) => {
       if (!e) return;
       h(e);
     });
   }

The method above takes a function of type RemoteAudioLevelHandler, which I have defined as follows in /src/client/daily.js:

export type RemoteAudioLevelHandler = (
   e: DailyEventObjectRemoteParticipantsAudioLevel,
 ) => void;

The DailyEventObjectRemoteParticipantsAudioLevel type comes directly from daily-js, the entry point to Daily’s JavaScript client SDK. It will be a string-keyed object of numbers. Each key will be a Daily participant ID, and each value will be that participant’s current audio level. The audio level will be a float between 0.0 and 1.0, representing decibels (overload).

In registerRemoteParticipantsAudioLevelHandler() above, I first start the monitoring of remote participants’ audio levels at 200-millisecond intervals (this is also Daily’s default if no value is passed, but I wanted to show the parameter usage for the sake of illustration).

I then set up an event handler for the "remote-participants-audio-level" event. Once this event is emitted, I call the handler function passed in by the caller with the event payload provided by Daily.

Now, all the Game class needs to do is call this handler-registration method and provide its chosen handler. I do this in the game’s setupCall() method:

    this.call.registerRemoteParticipantsAudioLevelHandler((e) => {
       const levels = e.participantsAudioLevel;
       Object.entries(levels).forEach(([participantID, audioLevel]) => {
         updateAudioLevel(participantID, audioLevel);
       });
     });

Above, I iterate over every entry in the given event payload and update the visual representation of the audio level for each player. Let’s look at that next.

Updating the visual representation of players’ audio levels

Each player’s tile is set up within the Board class. Each instance of Game always contains one instance of Board.

I have added one new DOM element to the player’s tile div on player creation: An audio indicator div:

   const audioIndicator = document.createElement("div");
     audioIndicator.className = "audio-indicator";
     participantTile.appendChild(audioIndicator);

This indicator will be styled accordingly in style.css as follows:

.tiles .tile .audio-indicator {
   width: 100px;
   height: 100px;
   opacity: 0;
   position: absolute;
   top: 0;
   left: 0;
   border-radius: 16px;
   box-shadow: inset 0 0 1px 3px var(--gold);
   transition-duration: 200ms;
   transition-property: opacity;
 }

Note the opacity being set to 0, and the box-shadow property, which will be the primary visual behind our audio level indicator.

The above will result in a transparent div being positioned over the top of the participant div (including any video elements within it). I’ve also set up a transition for the opacity property, which as you’ll see below is what I’ll be using to display the audio level indicator for each player. I want the transition in audio level to look smooth and not jumpy, so I’ll have each opacity style change take 200ms to transition to its new state.

Now all that’s left is to update the border of this tile to match the player’s audio level. This is done through the updateAudioLevel() method:

export function updateAudioLevel(participantID: string, audioLevel: number) {
  const participantTile = getTile(participantID);
   if (!participantTile) {
     showGameError(
       new ErrGeneric(`tile for participant ID ${participantID} does not exist`),
     );
     return;
   }

   let opacity = 0;
   // Audio level will be 0 when player is muted.
   if (audioLevel !== 0) {
     // Set opacity to visually reasonable value based on audio level
     opacity = audioLevel / 0.08;
     // Clamp opacity to always be between 0 and 1
     opacity = Math.min(Math.max(opacity, 0), 1);
     opacity = +opacity.toFixed(2);
   }
   const audioIndicator = <HTMLDivElement>(
     participantTile.getElementsByClassName("audio-indicator")[0]
   );

   // Update the inner glow shown within the participant tile.
   audioIndicator.style.setProperty("opacity", `${opacity}`);
 }

Here is an overview of what’s going on above:

  1. I first retrieve the player’s participant tile (and show an error if it could not be found).
  2. Next, I set an opacity variable to 0 as the default.
  3. The audio level will be 0 if the participant is muted. So if the audio level is not 0, I set an opacity by dividing the given audio level by 0.08. I arrived at this number purely by experimenting with different visual styles and thresholds and picking whatever value produced the kind of result I thought looked best.
  4. I then clamp the opacity to ensure it’s always a number between 0 and 1 (since that is our range when styling CSS).
  5. Then, I convert the opacity value to show only two decimal points with toFixed(), because I decided there’s not much visual point to update the opacity more granularly than that.
  6. Finally, I retrieve the "audio-indicator" div (which we went through above) and set its opacity to my previously-calculated value.

And we’re done! This will result in the audio level indicator div fading in and out as the player speaks:

Volume level indicator

Conclusion

With Daily providing audio level data for local and remote participants directly to each client, the door is open for developers to implement all kinds of cool features using this information.

If you have any questions about the new audio level features, reach out to our support team or say hello in peerConnection, our WebRTC community.

Never miss a story

Get the latest direct to your inbox.

Twilio Migration Program