How to use ZEGOCLOUD SDK to implement call quality monitoring in iOS

Introduction

The ZEGOCLOUD Express-Video SDK provides the ability to monitor the streaming quality during stream publishing and stream playing. After stream publishing or stream playing is started, the SDK triggers the corresponding streaming quality callback every 3 seconds, and client applications can obtain real-time streaming quality data by listening for these callbacks.

Prerequisites

Before you monitor the streaming information, make sure you complete the following:

• Create a project in ZEGOCLOUD Admin Console, and get the AppID of your project.
• The ZEGO Express SDK has been integrated into the project. For details, see Integration and Implementation.

Monitor the quality of stream publishing

To monitor the quality of the stream publishing, listen for the onPublisherQualityUpdate callback. After stream publishing is started, this callback will be triggered every 3 seconds to provide the stream publishing quality data.

You can obtain stream publishing quality data through this callback to monitor the health of stream publishing in real time and display the upstreaming network quality on your application's UI.

// You can focus on specific quality attributes and report them to your business server, or focus on a specific field of an object for giving friendly notes for users.
// The parameter 'quality' has many attributes. If you do not want to handle every one of them, you can just focus on the 'level' attribute, a comprehensive quality indicator calculated by the SDK based on other quality attributes.
- (void)onPublisherQualityUpdate:(ZegoPublishStreamQuality *)quality streamID:(NSString *)streamID {
}

The stream publishing quality attributes in details

The stream publishing quality includes the audio and video capture frame rate, bitrate, RTT, packet loss rate, and others of capturing and encoding stages of the stream publishing process.

The ZegoPublishStreamQuality object includes quality attributes of different stages of the stream publishing process. You can find the detailed attribute definitions in this section:

Quality attributes of audio and video capturing

The following attributes reflect the stream publishing quality of audio and video capturing, which is close to the user's subjective perception of the local preview.

• audioCaptureFPS: The audio capture frame rate (fps).
• videoCaptureFPS: The video capture frame rate (fps).

Quality attributes of video encoding

The stream publishing quality attributes of video encoding of the encoding stage is as follow:

• videoEncodeFPS: The target output frame rate (fps) of video encoding.

Quality attributes of stream transmission

The stream publishing quality attributes below reflect the quality of stream transmission (sending), which is related to the encoding bitrate and the current network quality.

• audioSendFPS: The actual audio transmission frame rate (fps).
• audioKBPS: The actual audio transmission bitrate (kbps).
• videoSendFPS: The actual video transmission frame rate (fps).
• videoKBPS: The actual video transmission bitrate (kbps).
• rtt: The rount-trip time (ms) from the client device to the ZEGOCLOUD server.
• packetLostRate: The client-side upstreaming packet loss rate.

Quality attributes of the total number of bytes

The stream publishing quality attributes below reflect the total number of bytes sent.

• totalSendBytes: The total number of bytes sent, including audio, video, SEI.
• audioSendBytes: The number of audio bytes sent.
• videoSendBytes: The number of video bytes sent.

Quality attributes of encoding information

The stream publishing quality attributes below reflect the encoding information:

• videoCodecID: The video codec.
• isHardwareEncoder: Whether to enable hardware encoding.

Quality attributes of published stream quality level

The parameter quality has many attributes. If you do not want to handle every one of them, you can just focus on the level attribute, a comprehensive upstreaming network qualityindicator calculated by the ZegoExpressEngine based on other quality attributes.

The following table describes the level fields:

Monitor the quality of stream playing

To monitor the quality of the stream playing, listen for the onPlayerQualityUpdate callback. After stream playing is started, this callback will be triggered every 3 seconds to provide the stream playing quality data.

- (void)onPlayerStateUpdate:(ZegoPlayerState)state errorCode:(int)errorCode extendedData:(NSDictionary *)extendedData streamID:(NSString *)streamID {
     // You can focus on specific quality attributes and report them to your business server, or focus on a specific field of an object for giving friendly notes for users.
     // The parameter 'quality' has many attributes. If you do not want to handle every one of them, you can just focus on the 'level' attribute, a comprehensive quality indicator calculated by the SDK based on other quality attributes.
}

The stream playing quality attributes in details

The stream playing quality includes the audio and video frame rate, bitrate, delay, packet loss rate, and others of receiving, decoding, and rendering stages of the stream playing process.

The ZegoPlayStreamQuality object includes quality attributes of different stages of the stream playing process. You can find the detailed attribute definitions in this section:

Quality attributes of stream receiving

The stream playing quality attributes below reflect the quality of stream receiving, which is related to the quality of stream transmission (sending) and the current network quality.

• audioRecvFPS: The actual audio receiving frame rate (fps).
• audioDejitterFPS: The audio dejitter frame rate (f/s).
• audioKBPS: The actual audio receiving bitrate (kbps).
• audioBreakRate: The actual received audio break rate (number of breaks / every 10 seconds).
• videoRecvFPS: The actual video receiving frame rate (fps).
• videoDejitterFPS: The video dejitter frame rate (f/s).
• videoKBPS: The actual video receiving bitrate (kbps).
• videoBreakRate: The actual received video break rate (number of breaks / every 10 seconds).
• packetLostRate: The client-side downstreaming packet loss rate, in percentage, 0.0 - 1.0.
• rtt: The round-trip time (ms) from the client device to the ZEGOCLOUD server.
• avTimestampDiff: The difference between the video timestamp and the audio timestamp, used to reflect the synchronization of audio and video, in milliseconds. This value is less than 0 means the number of milliseconds that the video leads the audio, greater than 0 means the number of milliseconds that the video lags the audio, and 0 means no difference. When the absolute value is less than 200, it can basically be regarded as synchronized audio and video, when the absolute value is greater than 200 for 10 consecutive seconds, it can be regarded as abnormal.
• peerToPeerDelay: The delay from peer to peer, in milliseconds.
• peerToPeerPacketLostRate: The packet loss rate from peer to peer, in percentage, 0.0 - 1.0.

Quality attributes of stream rendering

The stream playing attributes below reflect the streaming quality at the stream rendering stage, which is close to the viewer's subjective perception. They are affected by the audio/video codec, and the values may be lower than the actual audio/video receiving frame rates.

• audioRenderFPS: The audio rendering frame rate (fps).
• videoRenderFPS: The video rendering frame rate (fps).
• delay: The delay after the data is received by the local end, in milliseconds.

Quality attributes of the total number of bytes

The stream publishing quality attributes below reflect the total number of bytes received.

• totalRecvBytes: The total number of bytes received, including audio, video, SEI.
• audioRecvBytes: The number of audio bytes received.
• videoRecvBytes: The number of video bytes received

Quality attributes of decoding information

The stream playing quality attributes below reflect the decoding information:

• videoCodecID: The video codec.
• isHardwareDecode: Whether to enable hardware decoding.

Monitor the status of stream publishing/playing

Callback for updates on stream publishing status

After stream publishing starts, if the status changes, the SDK sends out the event notification through the onPublisherStateUpdate callback. To monitor the status of stream publishing, listen for this callback.

- (void)onPublisherStateUpdate:(ZegoPublisherState)state errorCode:(int)errorCode extendedData:(NSDictionary *)extendedData streamID:(NSString *)streamID {
        // In this callback, when the [state] is [ZegoPublisherStateNoPublish] and the [errorCode] is not 0, indicates the stream publishing is failed. And the SDK won't retry to publish the stream. At this time, user can set a corresponding UI to indicate the current state.  
        // In this callback, when the [state] is [ZegoPublisherStatePublishRequesting] and the [errorCode] is not 0, indicates the stream publishing retry is ongoing. But if the stream publishing retry still fails after the default reconnection duration, it won't try again and will send out a failure notification.
}

You can determine the user's network status during stream publishing based on the state property.

The following table describes the value of the state property and the corresponding user status:

The parameter extendedData provides extended information for status updates. If you are using the ZEGOCLOUD CDN, the key of this parameter is flv_url_list, rtmp_url_list, and hls_url_list, which correspond to the stream playing URL of FLV, RTMP, and HLS protocols respectively.

Callback for updates on stream playing status

After stream playing starts, if the status changes, the SDK sends out the event notification through the onPlayerStateUpdate callback. To monitor the status of stream playing, listen for this callback.

- (void)onPlayerStateUpdate:(ZegoPlayerState)state errorCode:(int)errorCode extendedData:(NSDictionary *)extendedData streamID:(NSString *)streamID {
        // In this callback, when the [state] is [ZegoPlayerStateNoPlay] and the [errorCode] is not 0, indicates the stream playing is failed. And the SDK won't retry to play the stream. At this time, user can set a corresponding UI to indicate the current state. 
        // In this callback, when the [state] is [ZegoPlayerStatePlayRequesting] and the [errorCode] is not 0, indicates the stream playing retry is ongoing. But if the stream playing retry still fails after the default reconnection duration, it won't try again and will send out a failure notification.
}

You can determine the user's network status during stream playing based on the state property.

The following table describes the value of the state property and the corresponding user status:

Callback on capturing the first frame of audio

To receive the event notification when the first frame of audio is captured, listen for the onPublisherCapturedAudioFirstFrame callback.

After stream publishing starts, the SDK sends out the event notification through this callback when the first frame of audio is captured.

When you enable stream publishing or local preview for the first time, the SDK engine starts to capture the audio data of the local device, and the SDK sends out event notification through this callback.
You can tell whether the audio data has been captured successfully according to the returned event notification. If no notifications are received, it means the audio device is being used or the device is abnormal.

- (void)onPublisherCapturedAudioFirstFrame {

}

Callback on receiving the first frame of audio

To receive the event notification when the first frame of audio is received, listen for the onPlayerRecvAudioFirstFrame callback.

After stream playing starts, the SDK sends out the event notification through this callback when the first frame of audio is received.

- (void)onPlayerRecvAudioFirstFrame:(NSString *)streamID {
}

Monitor the status of the stream forwarding via CDN

Callback for updates on the status of CDN URL

To receive event notification when the specified CDN URL changes (adds new URL or removes existing URL), listen for the onPublisherRelayCDNStateUpdate callback.

After the ZEGOCLOUD RTC Server forwards the stream to CDN, the SDK sends out event notification when the status of streams forwarded to CDN changes, for example, the stream forwarding stops or the forwarding operation retries.

You can tell whether the audio and video streams forwarded to CDN are normal according to this callback:

• If the returned result of this callback indicates it is abnormal, locate the cause of the audio and video streams forwarded to CDN and perform a disaster recovery operation accordingly.
• If you do not know the cause of the exception, contact ZEGOCLOUD technical support for further analysis.

- (void)onPublisherRelayCDNStateUpdate:(NSArray<ZegoStreamRelayCDNInfo *> *)infoList streamID:(NSString *)streamID {
}

The stream forwarding attributes in details

The stream forwarding information includes the CDN URL, forwarding status, the cause of stream forwarding status, and when the status changes.

The ZegoStreamRelayCDNInfo object includes stream forwarding attributes. You can find the detailed attribute definitions in this section:

The following table describes the value of the state property:

The following table describes the value of the updateReason property: