Author | Ren Xuelong
Introduction
introduction
online live broadcast function as a basic Internet capability has become increasingly important, and the live broadcast function in mobile phones is becoming more and more perfect. Various live broadcast types such as e-commerce live broadcast, news live broadcast, entertainment live broadcast, etc. provide users with rich live content. With the popularity of live streaming, it is becoming increasingly important to provide users with a fast and smooth live streaming viewing experience.
full text 6657 words, and the estimated reading time is 17 minutes.
GEEK TALK
1
Background
Baidu APP As Baidu's aircraft carrier-level application provides users with complete mobile services, and live broadcast also serves as one of the necessary functions to provide users with content. As the structure and business capabilities of the live broadcast room gradually mature, the optimization of live broadcast room playback indicators is becoming more and more important. When users click on live broadcast resources, they can quickly see the live broadcast screen, which is one of the core experiences, and the starting speed has become a key indicator in the optimization of the live broadcast room.
GEEK TALK
2
Current status
Due to the package size and other reasons, the live broadcast function in the Android version of Baidu APP is connected using a plug-in method, and the live broadcast module will be loaded only when the user actually uses the live broadcast function. In order to solve the situation where users need to wait for plug-in download, installation, loading and other stages when clicking on the live broadcast function, and the failure to download compatible plug-ins, the live broadcast team has drawn the core capabilities of playback and IM to an independent, smaller first-level plug-in and built into the Baidu APP. The business capabilities of the live broadcast room are in another larger second-level plug-in. The special plug-in logic and complex business scenarios make the overall broadcast duration indicator of the Android version unsatisfactory.
022 Q1 live broadcast room overall broadcast time index 80 minutes is about 3s, among which the two jumps (sliding up and down in the live broadcast room) scene is about 1s. After the plug-in is split and launched, it is found that as the version converges, the first jump enters the live broadcast room and carries the stream address (the address will be used to pre-start and executed synchronously with the live broadcast list loading) scenes have a significant increase when the broadcast starts. From about 1.5s in the early stage of the release, it will gradually grow to 2.5s+ as the version converges. That is, when you click on the live broadcast resource online outside the live broadcast room, a large number of users need to wait 3s or even longer after clicking to truly see the live broadcast screen. This duration has a very negative impact on users' use of live broadcast functions, and the starting time indicator needs to be optimized urgently.
GEEK TALK
3
Target
△ Start link
Start process simple description is that the user clicks on the live resource, opens the live page, requests the start address, calls the kernel start, the kernel starts to complete the start of the broadcast, the kernel notifies the business, and the business starts to complete the start of the broadcast. From the perspective of monitoring the start-up time of the kernel, the start-up time of the live broadcast resource in the kernel is about 600-700ms. Considering the loss of other stages in the link and the two-hop (sliding up and down in the live broadcast room) scenes, the overall start-up time target is 1.5 seconds; considering that some places entering the live broadcast room already have a start-up stream address, the "request start-up address" stage can be omitted in some scenarios. The start-up address scene has been obtained outside this live broadcast room, and the start-up time target is set to 1.1 seconds.
GEEK TALK
4
Difficulties
Special plug-in logic and complex business scenarios make the Android version not exactly the same every time it enters live broadcast. When there is only a first-level plug-in and the second-level plug-in is not ready, the live broadcast data is requested in the first-level plug-in and the broadcast starts. When the first-level plug-in is loaded, the second-level plug-in requests live broadcast data and processes the broadcast starts. When entering the live broadcast room, the stream address is opened in seconds. After the Activity is started, the player will be created to start broadcasting using the stream address carried outside the live broadcast room.In addition to these links, there are some other situations. The complex starting link is caused by the fact that although there are timestamping between the main nodes during the starting process, and there are 80-quantile reports for two adjacent nodes at the sky level, the starting link reported in different online scenarios cannot be exhaustive, and using existing reports cannot analyze the real time-consuming position in the starting link of live disk . A new monitoring plan needs to be established to find time-consuming points before designing targeted plans to optimize each time-consuming position.
GEEK TALK
5
Solution
.1 Design a new report, locate time-consuming point
△One jump start link simple diagram
Since the existing report cannot meet the time-consuming stage positioning of the starting link, a new monitoring solution needs to be designed. Observe the flowchart of the scene with a stream address when opening the live broadcast room (above). After entering the live broadcast room, the live broadcast room list will be created simultaneously and the player will be created pre-started. When the live broadcast room list is created and the player receives the first frame notification, the broadcast process will be completed. Although the user clicks to the page Activity's onCreate, there may be multiple nodes (first-level plug-in installation, loading, etc.), the page onCreate calls the player to pre-start the player, and there may be multiple nodes in the kernel until the live broadcast service receives notification, resulting in the entire starting link being unable to be exhaustive.
But we can find that there will definitely be a path from the user clicking on onCreate, and there will definitely be a path from onCreate to creating a player. This shows that although the number of nodes and links between the two key nodes cannot be determined, the order of the two key nodes is certain and there must be certain. Therefore, we can design a query report for custom link start point and custom link end point. By finding the difference between the end point and the starting point time stamp, we can obtain the time spent between two arbitrary nodes by calculating all the differences between the two nodes on the line by 80 quantiles, and we can obtain the time spent between these two nodes in the online start time. By calculating all core key nodes in the starting link, you can find segments with abnormal time-consuming in the entire starting link.
After developing a new report according to the above idea, the time-consuming stages of the above link will be clearer. See the figure below, so that we can break it one by one for different stages.
△Time-consuming between key nodes
.2 One-hop use the first-level plug-in to start broadcasting
Time-consuming between key nodes used for statistics using new report. It is observed that the time-consuming time-consuming between live broadcast room list creation (template component creation) to the real call start broadcasting (business view ready), and this time-consuming will gradually increase as the version converges, which will increase by about 1000ms within two weeks. First, we solve the problem of increasing time-consuming between these two nodes.
After observation and analysis of the start-up link, it was found that as the version converged, this part of the start-up link changed significantly, mainly because as the version converged, the proportion of the node "business call start-up" triggered in the secondary plug-in increased. During the convergence period of the version, it is highly likely that the secondary plug-in has not been downloaded or not installed when entering the live broadcast room. At this time, the first-level plug-in can quickly create a list and create a business view. The first-level plug-in will trigger the broadcast when the item attached of the RecyclerView to the view tree. This link is mainly waiting for the kernel to complete the pulling and parsing of the first-frame data. When the secondary plug-in gradually converges, the primary plug-in no longer creates a business view after entering the live broadcast room, but has a secondary plug-in to create a business view. Since there are many business components in the secondary plug-in to load one by one, and there is also a certain amount of time-consuming calling or event distribution in the first to second level, the starting scenario of the secondary plug-in greatly increases the time-consuming between creating the live broadcast room list (template component creation) and real calling and starting (business view ready).
5.2.1 All jumps use the first-level plug-in to start broadcasting
Based on the above problem analysis, we modified the first-level jump scene start broadcasting logic, and all jumps use the first-level plug-in to start broadcasting.The player parent container id created by the first-level plug-in and the second-level plug-in are the same. In this way, after the player parent container is initialized in the first-level plug-in, the start process can be completed when the kernel first-frame calls back to . In the secondary plug-in, when initializing the player's parent container, it will also use id to determine whether it has been added to the view tree. Only when it is not added (an exception occurs during the second jump scene or the first jump) will the secondary processing be carried out. The speed can be faster when processing in the first-level plug-in. The first-level priority and second-level bottom-up logic ensures that the view can be successfully initialized after entering the live broadcast room.
5.2.2 Request interface in advance
is used to handle the starting broadcast by the same plug-in. The secondary plug-in links are optimized for many problems. Another time-consuming point is that only the room is passed into the room when entering the live broadcast room. Room_id does not carry the stream address scene. At this time, the starting broadcast data needs to be obtained through the interface request before the player and starting broadcast can be created. To optimize this part, we designed a live broadcast room data request manager, which provides cached data and timeout cleaning logic. When the page is onCreate, the manager will trigger the interface request. After the live broadcast room template is created, the requested live broadcast data will be obtained through the manager. If the manager interface request has not yet ended, the in-progress request will be reused and the data will be returned immediately after the request is completed. In this way, when we enter the live broadcast room without carrying the streaming data, we can make full use of the 300ms time in the figure to do more necessary logic.
.3 Player Activity Outside pre-start broadcast
Optimize the business link entering the live broadcast room by pre-creating, pre-start broadcast, using one-hop plug-in starting broadcast and other solutions to optimize the business link entering the live broadcast room. After the business link is time-consuming, the business link is gradually lower than the kernel part, and the player kernel time-consuming gradually becomes the optimization bottleneck of the first jump start broadcast. In addition to exploring optimization solutions within the kernel, continuing to optimize the entire starting link of the business is also an important direction. Through the time-consuming between nodes, you can find that it is time-consuming for users to click on the Activity page onCrete. When it is impossible to reduce this part of the time to a shorter time, we can try to deal with some things in parallel during this period to reduce part of the logic after the page is started. After the first-level plug-in is built into Baidu APP, the plug-in preloading function is designed and launched. After the user clicks on the live broadcast resource, 99% of the live broadcast level plug-in has been loaded. If the first-level plug-in is loaded, there is no more room for operation when the first-level plug-in is loaded here. However, the pre-start time is advanced to the user's click, and the kernel data loading and live broadcast room startup can be parallelized to a greater extent, thereby reducing the impact of kernel time consumption on the entire start time consumption.
△ Player starts the broadcast diagram outside the live broadcast room
As shown in the figure above, a new early start module is added. After the user clicks, the player starts and caches are created in parallel with the page. When creating the player after the page is started, the player will be first tried to get the started player from the cache of the early start module. If it is not obtained, the normal player will create the start logic. If the cached player is obtained and the player does not have an error, you only need to wait for the first frame of the kernel.
player starts broadcasting in advance and the first frame event is likely to arrive after Activity is started, but there is still a chance that it will arrive earlier than before setting the first frame monitoring in the live broadcast service. Therefore, when using the player with multiplexed kernels in the live broadcast room, you need to determine whether the start of the broadcast is successful. If the start of the broadcast has been successful, you must immediately distribute the start of the broadcast event (the meaning is different from the first frame event to prevent confusion with the first frame event).
Early Start Module also designed timeout recovery logic. If the early start-up fails or is not reused by the service within 5s (tentatively) (Activity startup exception or other business exception), the cached player will be actively recycled to prevent the player created in advance from taking up more memory when the live broadcast room is not reused successfully and avoid leakage; the timeout time is determined based on the online market start-up time, and a value with a slightly higher starting time of 80 minutes is used to prevent the start-up time from being reused, but it cannot be set for a longer time to prevent the memory from being consumed more when it is reused.
Through the early start function, the early start logic of hitting the early start logic during the experimental period is less advanced start logic. The overall starting time is 80 Quantile optimization mean: 450ms+.
.4 Live room task breaks down
△ kernel first frame distribution takes time
Business link and kernel link take time to consume and optimize, we continue to dismantle the time between key nodes. It takes a long time to mark the first frame notification inside the kernel to the live broadcast service actually receive the first frame notification. As shown in the figure above, the online kernel first frame distribution takes 80 quantile mean more than 1s, and this segment has a great impact on the overall starting time-consuming optimization. The kernel first frame is marked on the child thread. When notifying the business, messages will be distributed through the main thread Handler, and events will be transferred to the main thread through the system's message distribution mechanism.
checks all main thread tasks between the kernel marking the first frame time point and the service receives the first frame notification event time point. It is found that when the first frame distribution task starts to queue, there are already many other tasks in the main thread task queue, and other events are processed for a long time, resulting in the long queue time for first frame distribution, and the overall time for distribution tasks is longer. The live broadcast service is complex. If other tasks in the live broadcast room are already in the queue or are being executed when the kernel's first frame distribution task is queued, the first frame distribution task needs to wait until the live broadcast task is completed before it can be executed.
By screening all main thread tasks during the live broadcast room startup, it was found that the secondary plug-in has many business functions, and the overall loading task execution time is long. In order to verify that online is also because the secondary business tasks block the first frame distribution task, we designed an experiment that needs to wait for the first frame of the kernel to be loaded before the secondary component can be carried out. Through the comparison of the data of the experimental group and the control group, the time to first frame distribution and overall broadcasting of the first frame were significantly reduced when hitting the experiment, and the overall time to start the broadcast was optimized by about 500ms.
Through experimental verification and local pairing of business logic analysis, the number of preloads of each business component and corresponding views in the live broadcast room is large and time-consuming. This function is that in order to make full use of the time before the live broadcast room interface data return, the secondary plug-in will create a business view in advance in parallel with the interface request after loading, and lift the initial component and view as component rendering saves time after the interface is completed. If it is not pre-created, the initialization of the business component will also actively create and set the data after the interface data comes back. However, executing all pre-created tasks in serial is a long time, which will block the main thread. Execution of too many tasks in one frame of the page will also cause obvious page stuttering.
After discovering this blocking problem, we designed to split the pre-created view tasks and split the large tasks executed together into multiple small tasks. The initialization of each component is queued in the main thread task queue as a separate task. Avoid a problem that takes a very long time to do a big task. After this feature is launched, the time spent loading large tasks in the entire secondary plug-in is reduced by 40%+.
.5 The kernel subthread distributes the first frame
Since the tasks in the main thread message queue are queued, after splitting the large tasks that block the first frame distribution event into many small tasks, it is still impossible to solve the problem that these small tasks are already queued in the main thread task queue when the first frame event starts queuing. In addition to reducing the impact of live broadcast services, it can also reduce the time-consuming distribution of first frames by speeding up the kernel task distribution. It is necessary to design a solution to avoid the main thread being queued or queued quickly without affecting kernel stability and business logic.
To solve the above problem, we promoted the kernel and added a child thread to notify the business of first frame event capabilities. After receiving the first frame callback in the child thread, the service inserts a new task into the front of the main thread task queue through the postAtFrontOfQueue() method of Handler. In this way, the main thread can process the newly created task immediately after processing the current task. In this new task, the player screen-on logic can be processed immediately. There is no need to wait for the kernel's original main thread message to be played.
The main thread task is inserted before the task cannot be interrupted when the new task is queued. The task that has already started to be executed in the main thread must be executed after the task is executed. To optimize this scenario, after the kernel notifies the first frame through the child thread, the player needs to record this state. Before and after the live broadcast room business tasks in the first-level plug-in and the second-level plug-in, the logic to determine whether the first frame has been received in the player. If it has been received, it can first process the screen before continuing the current task.
determines whether the first frame notification can be processed faster by releasing the first frame message in the main thread task queue and adding business key nodes to the screen by retrieving the first frame notification on the start time.
.6 Balance of start-up and finish-up indicators
During the live broadcast optimization process, the optimization of the completion time indicator (completed time: the time when the user clicks until all the core functions of the live broadcast room appear, which goes through the page startup, live broadcast room list creation, secondary plug-in download, installation, loading, live broadcast room interface data request, initialization of live broadcast room function component view and rendering data, core business component display and other stages) is also continuing. The second-level plug-in in the live broadcast room will trigger the download, installation and loading logic when using the functions in the second-level plug-in. It is also noticed that the user clicks on the page onCreate in the loading link. This time is taken by the user to click on the page, see the figure below.
△Page startup time-consuming diagram
is to optimize the live broadcast room completion indicators. The live broadcast team considers that if the plug-in loading is parallel to the page startup, the loading time-consuming will also be optimized. The live broadcast team continued to design a secondary plug-in preloading solution, and advanced the loading position of the secondary plug-in to the time when the user clicked (this function is launched before the corresponding functions in chapters 5.4 and 5.5). After this function was launched, the data from the test group and the control group showed that the time-consuming time of the experimental group was indeed 300ms+ optimized compared to the control group. However, the starting time of broadcasting was abnormal. The starting time of the experimental group increased significantly by 500ms+ compared with the control group, and the starting deterioration is still increasing with the convergence of the version. We immediately discovered this anomaly and determined through data analysis that the data was correct. How does the starting changes occur during the optimization of the load?
After data analysis, we found that the main location of the start-up is the kernel's first frame message being distributed to the main thread segment, that is, the earlier the secondary plug-in is loaded, the greater the possibility of time-consuming task conflict between the kernel's first frame distribution and the secondary component loading. After confirming the cause of the problem, we did the functions of chapters 5.4 and 5.5 to reduce the impact of secondary component loading tasks on the starting broadcast. Since the time-consuming tasks in the secondary plug-in are completely split and broken down to alleviate the complexity of the starting broadcast deterioration caused by the pre-download of the secondary plug-in, the logic invasion of the live broadcast room is too large, and the secondary plug-in is not fully loaded online in advance. We designed other solutions to achieve the goal of completing the loading.
Although we cannot directly load the secondary plug-in when entering the live broadcast room, we can try to download the secondary plug-in before entering the live broadcast room, and load it directly when using it. This time-consuming and relatively small download time. We have optimized the plug-in pre-download module to trigger the pre-download plug-in of this module when displaying live resources outside the live broadcast room. This module will comprehensively judge the current device network, bandwidth, download frequency and other conditions, and download the matching secondary plug-in at the right time. After the plug-in is downloaded in advance, it will optimize the completion indicators. In addition to the plug-in pre-download, the live broadcast room is initialized and split by the secondary component of the live broadcast room in Chapter 5.4, and all components are initialized and optimized for the main thread blocking. In this way, after the interface data request is successful, components that affect the completion statistics can be processed first. Other components can be initialized after the completion of the load. This solution also significantly optimizes the live broadcast completion indicators.
In addition to the above two optimization solutions, the live broadcast team also optimized the loading indicators in multiple other directions, and also handled the balance between the loading time and the starting time indicators, and did not have a deterioration effect on other indicators due to the optimization of one indicator. Finally, all the starting and finishing indicators were achieved to achieve the goal.
GEEK TALK
6
Return
△2022 Android time-consuming trend
After the above optimization schemes, the latest version data on the Android side, the starting time of the market has been reduced from 3s+ to about 1.3s; the starting time of the first jump with a streaming address has been reduced from about 2.5s+ to within 1s; the starting time of the second jump has been reduced from 1s+ to within 700ms, successfully completing the predetermined goal.
GEEK TALK
7
Outlook
Start-up time As a core indicator of the live broadcast function, it still needs to be continuously polished and optimized. In addition to optimization in business architecture, there are also many directions that can be explored, including optimized stream pull protocol, optimized buffer configuration, adaptive network speed start-up, optimized gop configuration, and edge node acceleration. Baidu's live broadcast team will continue to deepen the live broadcast technology to bring users better and better live broadcast experience.
Author: Ren Xuelong
Source: WeChat public account: Baidu Geek says
Source: https://mp.weixin.qq.com/s/F6HjtfLenijurC7HNfUdKA
