" This article is the fourth article in the "Interpretation of Cube Technology" series. Previous articles "Detailed Explanation of Cube Mini Program Technology", "Summary of Alipay's New Generation Dynamic Technology Architecture and Selection", and "Interpretation of Cube Card Technology Stack" are welcome to review."
Ali is a company that focuses on operations, with most front-end developers. From 2016 to 2017, in the era of Weex or 1.0, React Native is open source not long ago and Flutter is not yet born. How to quickly spread to the Android/iOS dual platforms under the premise of fitting the front-end development environment is a hot topic. Alipay internal incubation with a dynamic cross-platform solution was born.
The first three articles introduce the current Cube architecture, Cube cards and Cube applet technology product forms respectively. This article mainly discusses Cube's rendering design and helps everyone understand the past and present of Cube card rendering technology.
Native native rendering problem
We all know that a native view needs to be rendered on the screen, taking Android as an example: create, measure, layout, and draw. These need to be completed in the main thread. When implementing the native list, even if the item is perfectly reused, it is also necessary to measure, layout, and draw when rendering different data. Moreover, as the view nesting level becomes deeper, the consumption of the main thread resources is greater. When the list flys, the frame rate drops rapidly, causing the page to be stuttered. Based on this problem, how to solve the rendering efficiency during the cube survey is an important part.
Usually, it is necessary to optimize the list scrolling frame rate, that is, view level, layout complexity, remove unnecessary background colors, solve overdrawing, lazy image loading, item reuse, etc., but it is still impossible to avoid measurement, layout, and draw. At that time, weex and RN still mapped the tags in html to the platform layer view. In some scenarios, developers could not optimize themselves like native development, and were criticized for their rendering performance. Therefore, the rendering goal during the cube survey is: optimize rendering efficiency + cross-platform.
Cross-platform asynchronous rendering scheme
Asynchronous rendering
Based on the background and requirements mentioned above, then we wonder, is there a way to remove the key steps from thread , that is, asynchronous rendering. When scrolling a list, only the system gestures and the list itself scrolling algorithm and animations need to occupy the main thread, which will greatly increase the frame rate. The product drawn by the element in the view is a pixel cache (the design used by Cube is Bitmap), and it returns to the main thread to refresh and display the view.
Cross-platform architecture
Another target cross-platform is to achieve the ability to quickly expand other platforms, and cube separates the parts involved in the platform to form a platform layer.
platform
Here, the standard c++ atomic interface is provided for each platform, which is implemented in platform language on different platforms. It has initially implemented only two platforms android and iOS. Android calls the java method through jni, and iOS mixes c++ and OC in the implementation file. If you need to expand other platforms such as macOS in the future, you only need to implement the interface defined by the platform layer, which can achieve the goal of quickly expanding other platforms.
core
library is based on the platform atomic interface and implemented in C++. It is a basic library, such as file IO, UI controls, picture downloads, message communications, etc., for use by the upper engine. Above library, it is the core implementation of cube rendering. The rendering part includes data model and rendering logic. The component library refers to some system entity controls supported by cube, or entity components that developers can externally access.
The following figure is the first version of cube rendering architecture diagram.
cube rendering architecture diagram
asynchronous rendering technology selection
As mentioned earlier, the "product" drawn asynchronously in the asynchronous rendering scheme is a bitmap handed to the "container" View. Why is it a bitmap? It seems to be very unfriendly to memory. What kind of View is the View? Is it special? Let's talk about what solutions have been studied during the cube research period and why the bitmap is selected in the end. The selection of
Android platform technology
android has been bumpy and rugged. The first thing I can think of is textView and GLSurfaceView that support independent rendering threads as containers, but it has obvious flaws and cannot be used in list scenarios for common business, and can only be applied to specific scenarios.
SurfaceView has been available since android1.0. The main feature is that its rendering can be implemented in child threads. Therefore, the problem is that although it inherits the View, it has an independent Surface, not in the View hierachy, and its display is not controlled by the View attributes. Therefore, it cannot be scaled and translated like ordinary view, and it cannot be used as an item in listView/RecycleView as a normal view, and there will be problems of out-of-sync when scrolling.
GLSurfaceView inherits SurfaceView, which comes with GLThread, which has the same problems as GLSurfaceView. In short, these two views are more suitable for single video rendering or rendering scenes like map-like rendering.
Someone may ask, just use SurfaceView/GLSurfaceView for the entire page, and even the list is implemented in the render thread? Here are two questions:
. At that time, the main goal of the cube team was to quickly verify, and the cost of implementing the list was too high, which was not the main contradiction.
TextureVIew
textureView is provided by google since android4.0. Its appearance is largely to make up for the shortcomings of the integration of SurfaceView, GLSurfaceView and native View. Based on the problem of the two views animation with native views described in the above section, the textureView seems to be more suitable for our scenario, which can not only support independent render threads, but also ensure perfect integration with native views.
However, during the actual research, it was found that the rendering mechanism of the textureView is not suitable for long lists. If the item of each list is a textureView, it involves screen out recycling and screen entry creation, otherwise it will cause memory problems. Recycling and creating SurfaceTexture is an asynchronous process, and there is a problem of flashing and black screen. In addition, it was further found that there is a certain upper limit for the number and capacity of the textureView (the cumulative size of each view), and the upper limits of different phones also vary greatly. Simply put, this is a technical route that looks beautiful but has countless compatibility pitfalls.
Bitmap + ordinary View
Finally chose the solution that bitmap looks not perfect. Although this is considered by most android developments to bring a lot of memory consumption and are regarded as unacceptable, as the application scope of cube becomes wider, this gradually proved to be the most universal solution at that time.
Each layer corresponds to a system view. The drawing content of each view is drawn asynchronously on the bitmap through the child thread through CanvasAPI. When the view is on the screen, the system onDraw draws this bitmap "product".
BitmapCache
Although Bitmap drawing scheme is used, memory overload must be considered. Here we use BitmapCache, which mainly targets list type scenarios. It depends on the system's item recovery callback notifications. Put the bitmap canvas into the cache. When the item is rendered on the screen, it is preferred to use the bitmap canvas from the cache, and the same size is preferred. If it does not exist, width and height are greater than the target width and height, so that the view canvas canvas are only drawn by b. Itmap locally, achieves the purpose of correct rendering
iOS platform technology selection
iOS implementation principle is roughly the same as Android. The difference is that the "product" drawn by iOS asynchronous threads will not be rendered using CoreGraphics in the drawRect of the UIView. This method is very inefficient and the page is obviously stuttering. In the end, it is to assign the canvas to the layer of the UIView and host it to the system rendering layer. The evolution of rendering technology of
talks about the general solution of cube asynchronous rendering and key technology selection. In fact, from the launch of the Dada Planet in early 2019, cube has been increasingly widely used in Alipay. This is also accompanied by the process of continuous exploration and optimization of the cube team based on actual business scenarios, and the rendering link has undergone two reconstructions. It should be emphasized that this evolution process is done under strict memory/performance, and compromises are required to make a compromise on Android compatibility. Some designs that look less elegant or advanced actually have to do this, such as choosing Bitmap as pixel buffering, such as designing access to three-party components. In a sense, it is not very meaningful to talk about the advantages and disadvantages of technology aside. We used to learn from flutter, but Cube finally walked along a technical route that suits its own scenario.
Common terms
LayoutTree: DomApi is constructed through yoga layout through add, update, and remove. It is used to describe the node parent-child relationship and the original tree structure containing layout information;
- RenderTree: used to describe the tree structure that draws node parent-child relationship and contains drawing information. Example of the difference between layoutTree: a layoutNode visible is gone, then the node will not appear in RenderTree;
- Layer: Generally, the root node and its child nodes are drawn on the same canvas, defined as a layer, corresponding to a view of the platform layer. When the child node has animation attributes, or exceeds the scope of the parent node, a layer needs to be independently generated;
- Laye rTree: The layer node mentioned above is a tree structure built. A layer corresponds to a platform layer and a view. We call it ContainerView;
- Entity node: The node that requires an independent layer is an entity node;
- Virtual node: Except for the entity node, other nodes will be drawn on the canvas of the parent container. These are virtual nodes.
evolution process
Data model
rendering process
bridge thread builds layoutTree through DomApi. When the main thread triggers rendering, the main thread builds RenderTree based on layoutTree. During the construction process, encounters external entity components, creates an instance and addSubView, and then switches the child thread to draw RenderTree, that is, all virtual nodes on the rootLayer, and switches the main thread map (bitmap "product") after drawing.
Disadvantages
- cannot support multi-layer structure
- Entity view is not reused, that is, how many items/cells are in the friend's dynamic list, there will be many "likes" and "rewards" entity components
However, this research verifies the feasibility of asynchronous rendering, and the frame rate is greatly improved when the list is scrolled.
productization period - 2.0 supports multiple layers 3
The feasibility was verified before. When product design, it is necessary to meet the multi-layer structure. That is, in the actual card, one or several different nodes will be set as layers. These nodes and their children are drawn on different canvases for different layers to render.
Data model
rendering process
brige thread construction layoutTree, each instruction (addNode, removeNode...) will be distributed to the main thread of the render module accordingly. The render builds the RenderTree accordingly, and uses the instruction information to generate a task to join the queue. When the VSync signal comes, the task is triggered to dequeue and deduplicate, and build a layerTree. Different layers are distributed to different draw threads to draw, and the main thread map is cut (bitmap "product").
Disadvantages
main thread has a large amount of calculation, which may cause lag
- reender node contains both drawing information, drawing objects, and logic. For example, the display: "none" node is ignored and not displayed, and the responsibilities are unclear.
optimization period - 3.0 to make up for the shortcoming
You can see above that the construction of renderTree and layerTree are all in the UI thread. When the number of nodes is large or complex, it will cause UI lag. In order to pursue the ultimate scrolling frame rate, the main thread calculation content should be reduced as much as possible, the optimization version 3.0 will change the renderObject construction layer and the drawing impact caused by the node changes. The part is completed in the child thread, forming the current online version.
data model
New PaintTree structure, which is mounted on the Layer node, and the style and attribute values are copied from RenderTree, but it does not involve any logical processing. It is simply a drawing object. Each drawing task only draws the paint node on paintTree, and there is no concurrency problem with layerTree and renderTree.
rendering process
layout thread build layoutTree, switch to render thread build renderTree, when the platform layer triggers rendering, switch to renderTree to build layerTree, calculate the impact range, etc., switch to the main thread to add the entity View corresponding to the layer on the container View, generate the drawing task to be executed on the paint thread, and switch to the main thread map (bitmap product) after the drawing is finished.
Disadvantages
above is the evolution of cube rendering from the birth to the current online solution. Currently, there are more than 20 card form access services in the Alipay side, and the number of card templates running online has reached more than 500, showing that the PV exceeds 10 billion, and has withstood the test of various business parties.
However, some problems have been found in technical support, such as when there are too many rendering tasks, the render thread blocks and queues, and the inability to consume in time leads to a higher probability of white screens. Recently, cube has continued to research optimization solutions. Problems in
Consistency problem on both ends
- cube The current drawing API uses the CanvasApi provided by the system platform layer (iOS is CoreGraphics). This leads to the two platforms that must align the manual code at the two ends in the details of drawing points, lines and surfaces, otherwise there will be differences in effect. When some new features are added, such as supporting dotted and dashed lines, the two platforms need to implement the DrawDottedLine interface respectively, but for this issue, the cube team is investigating self-drawing, that is, using skia The API sinks the drawing interface to C++, realizing cross-platform self-drawing;
- text is also a point that is easy to cause differences. It uses the platform API to layout the text, and calls the layout API to draw when drawing, so there may be product platform differences. However, the cube team has currently laid out text layout and layout algorithm on the Cube applet on the C++ layer, and does not rely on the platform API to achieve dual-platform consistency; the constraints limited to memory/performance have not yet been applied on the Cube card.
flash white problem
Because scrolling uses asynchronous rendering, it will inevitably cause flash white problem caused by the main thread card being on the screen and the asynchronous drawing has not been completed. Thread switching has a cost. This flash white theoretically must exist, but it is just a matter of time. The cube team is committed to improving rendering efficiency, minimizing the loss caused by thread switching, and improving the user's experience in list scrolling.
Future planning
In response to the currently known problems, the cube team is committed to continuous optimization. The main optimization points include but are not limited to the following:
- rendering snapshots to improve the rendering efficiency of cold start and reduce the flashing time;
- rendering strategies, such as pre-rendering, synchronous drawing adaptation, thread model optimization, component caching and preloading, etc., reduce the flashing rate and improve the rendering efficiency;
- is used for yoga layout engine optimization of Cube cards, improving layout efficiency;
- skia self-drawing implementation to achieve dual-end consistency;
cube's rendering technology application includes two technical forms: card and mini program, scenarios include Alipay's internal, external, and IOT. Team members will continue to make efforts in rendering performance, user experience, and tool chains, and strive to polish the product well, serve developers well, and grow into a competitive cross-platform dynamic rendering solution.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-