This article is shared from Huawei Cloud Community "
Overall usage process
custom operator deployed to the OPP operator library, developers can use the msopst tool to verify the correctness of the operator execution on the Ascend AI processor. The overall process is relatively simple, as shown below:
Next, we introduce each of the above steps for? How to do it?
generates test case definition file
test case definition file, as the name implies, is used to generate test cases. Its content contains the basic information required by the test case, such as: the test case name, the operator type to be tested, the operator input and output description information (shape, data type, data layout format, etc.), operator attribute related information (including attribute name, type, attribute value), data generation algorithm, etc. Taking the Add operator as an example, the command to generate the test case definition file through the msopst tool is as follows:
After the above command is executed, the test case definition file of the Add operator will be generated in the output directory of the current path: Add_case_{timestamp}.json.
Of course, you can also directly copy the test case definition file of existing operators without using the msopst create command and modify it. The advantage of using the operator information library is that it will automatically fill in the relevant fields in the test case definition file, such as name, format, type, etc. in case_name, op, input_desc and output_desc. The template of the OpType_case_xxx.json file is as follows:
configuration test case definition file
0 The configuration of the test case definition file is the core of the entire ST test process, which directly determines the number of test cases generated, data generation algorithm, test coverage, etc. The following describes the parameter meaning and configuration method of the test case definition file.
input and output description configuration
input and output description information contains the key configurations shown in the table below:
test cases are generated by orthogonal combination of the above configurations, as shown in the figure below, format values are ["ND", "NCHW", "NHWC"]; type values are ["int32", "float32"]; shape values are [32,16]; data_distribute values are ["uniform"], value_range values are [0.1, 1.0], and the number of generated test cases is 3 * 2 * 1 * 1 = 6.
operator attribute configuration
If the operator contains attributes, the parameters shown in the following table need to be configured:
large batch generation test cases
If the developer needs to generate a large number of test cases, the fuzz test parameters can be used to generate scripts for assisting generation, and the fuzz script path and the number of test cases to be generated in the test case definition file ( *.json ), and the values of the relevant parameters in the input and output description are configured as "fuzz". The operation steps are as follows:
1 and configure the fuzz test parameters to generate the script. Assume that the script is named "fuzz_shape.py", the implementation is as follows:
2 and configure the test case definition file;
3 and are equipped in the test case definition file (*.json) Set the path of the fuzz test parameter generation script, the number of test cases required to be generated, and configure the fields that need to be automatically generated as a fixed value "fuzz", as shown below:
defines the expected data generation function
If the user needs to obtain the comparison result between the actual output and the expected output, you can use Python language to customize the expected data generation function, and add this function entry to the ST test case definition file. For example, the expected data generation function of the Add operator is "/home/test/test_add.py", which is implemented as follows:
Then add the "calc_expect_func_file" parameter to the ST test case definition file, and configure it as "file of the expected data generation function: function name", as follows:
generate and execute test case
0 After configuring the relevant environment variable , you can execute the ST test case through the following command:
view the execution result
ST test case after executing, you can view the use case execution result through the screen information. As shown below, a total of 9 use cases were run, and all 9 use cases were executed successfully.
Developers can also view the generated report "st_report.json". This report records the test results and the operation status of each stage. When the ST test case runs abnormally, problem location can be based on this information.
At this point, the entire process of using the ST test tool "msopst" has been introduced. Do you have a preliminary understanding of this tool? If you need to develop custom operators, don’t forget to use this tool to perform ST tests of operators. For more examples of using this tool, please refer to the "TBE Custom Operator Development Guide" or the "AI CPU Custom Operator Development Guide" in the Ascend Community.
(https://www.hiascend.com/document?tag=community-developer )
Ascend CANN Document Center is committed to providing developers with better content and more convenient development experience, helping CANN developers jointly build an AI ecosystem. Any comments and suggestions can be given feedback in the Ascend community, and every attention you have is our driving force for progress.
Click below to learn about Huawei Cloud's new technology as soon as possible~
Huawei Cloud Blog_Big Data Blog_AI Blog_Cloud Computing Blog_Developer Center-Huawei Cloud
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-