Tool Rules

Tencent Cloud Code Analysis currently integrates numerous self-developed and well-known open-source tools, adopting a hierarchical separated architecture that enables rapid integration with internally developed tools within enterprise teams. These tools can be integrated into the platform for use by internal teams, meeting the needs of rapid self-service tool management.

Tools are categorized into platform-provided tools and team-connected tools:

• Platform Tools: A series of self-developed or well-known open-source tools provided by the platform. These tools are publicly available, allowing any team to use them and their rules for code analysis.

• Team Tools: Tools connected by teams themselves. By default, these tools can only be used within the team.

Custom Rule Integration

Team administrators can customize rules within tools that support custom rules based on business needs.

Custom Rule Permission说明

• Not all tools support custom rules; only tools with custom rule functionality enabled can have custom rules added.

• For tools with custom rule functionality enabled, only team administrators can add custom rules.

• Custom rules added by default are team-isolated, meaning they are only visible and usable within the team.

Regular Expression Tool RegexScan说明

The Regular Expression Tool RegexScan is a tool with custom rule functionality enabled. You can navigate to the tool management page, search for the tool name RegexScan, view existing rules for the tool, and add custom rules based on team business needs.

Custom Rule Steps

1. Design Regular Expressions Based on Team Business Needs

   Tips

   It is recommended to first test whether the regular expression is correct. Recommended regular expression testing website: http://tool.oschina.net/regex

   Rule Example:

   • Rule Analysis Scenario

     Analyze calls to the usleep() method in code. If the parameter is less than 100, it may easily lead to high CPU usage and performance waste, thus being identified as a defect.

   • Regular Expression

     To match the usleep() string where the content in parentheses is a 1-digit or 2-digit integer, the regular expression can be written as \busleep\s*\(\s*\d{1,2}\s*\), accounting for spaces within the string.

2. Enter the Regular Expression Tool to Add Custom Rules

   Navigate to the tool management page, locate the RegexScan regular expression tool, click to enter the custom rule list page, and click the "Add Rule" button.

3. Fill in Rule Information

   Rule Parameter Filling Instructions (Mandatory):

   The parameter format is similar to INI, i.e., key = value.

   • [Mandatory] regex parameter: Specifies the regular expression to analyze. For example: regex = \busleep\s*\(\s*\d{1,2}\s*\).

   • [Mandatory] msg parameter: Used to display the issue description. For example: msg = Function method %s is deprecated; please use the xxx method.

     The %s in msg is matched one-to-one with groups in the regular expression (parts enclosed in "()").

     • If no groups are defined in the regular expression, msg can contain at most one %s, which will be replaced by the entire string matched by the regular expression.

     • If msg does not contain %s, it will be displayed directly.

     • If msg is not provided, it defaults to "Non-standard code found: %s" (using the default format is not recommended as it is too vague).

   • [Optional] ignore_comment parameter: Specifies whether to ignore commented code. Optional values: True, true, False, false. For example, ignore_comment=True; default is False.

   • [Optional] include parameter: Specifies the file matching scope for analysis, using Unix file matching syntax. Multiple entries are separated by semicolons (;). For example, include = path/to/dir;path/to/*.cpp.

   • [Optional] exclude parameter: Specifies files to exclude from analysis. The format is the same as the include parameter.

4. Add the Custom Rule to the Project Analysis Scheme

   After adding the rule, it can be added to the analysis scheme.

Custom Tool Integration

Team administrators can integrate tools independently; by default, these tools can only be used within the team.

This requires only a few steps:

1. Write code to implement the scanning tool logic.
2. Submit the tool to a Git repository.
3. Create a new tool on the page.
4. Add rules to the tool.
5. Configure the tool to an execution node.
6. Add rules to the project analysis scheme.

Custom Tool Permission说明

• Only team administrators can create tools, add tool rules, etc., and have full permissions for the tool.

• All team members can use the tool rules. If a tool rule is added in the rule configuration, ordinary team members have read-only permissions.

Custom Tool Integration Steps

Step 1: Write Code to Implement the Analysis Tool Logic

Write the corresponding tool logic based on the target code scenarios that need to be matched. You can refer to the Python-based Demo Project.

Mandatory Requirements:

• Execution Method: Supports command-line execution, such as python run.py or run.exe. The working directory for the execution command is the root directory of the tool code.

• Runtime Environment Description:

  • It is recommended to package and compile the tool into an executable program that can be executed directly after being pulled down.
  • If the tool requires a specific environment (e.g., Python or Java), the platform provides rich tool dependency packages. These can be viewed in Tool Management - Tool Dependencies and selected when creating a tool. Dependencies will be automatically configured during execution.
  • If existing tool dependency packages do not support required dependencies, new tool dependencies can also be created.

• Predefined Environment Variables on the Platform

  • For details on how to obtain and use these variables, refer to the Demo Project.

  SOURCE_DIR: Path to the code directory to be scanned
  DIFF_FILES: Path to a JSON file containing the list of files for incremental scanning (available for incremental scans)
  SCAN_FILES: Path to a JSON file containing the list of files to be scanned (available for both incremental and full scans)
  TASK_REQUEST: Path to a JSON file containing parameters for the current scan task
  RESULT_DIR: Path to the directory where the result.json output file will be stored; results should be written to this directory
  

  Some result processing stages can be skipped by configuring the following environment variables in the tool's environment settings:

  FILTER_TYPE=NO_VERSION_FILTER
  IGNORE_TYPE=NO_ISSUE_IGNORE
  BLAME_TYPE=NO_BLAME
  

  These fields can be customized and added to the environment configuration as needed. Whether they are required depends on the tool's specific needs, and these environment variables can be customized and retrieved within the program.

• Tool Command Declaration

  Add a tool.json file in the root directory of the tool repository to declare the tool's inspection and scanning commands, for example:

  {
    "check_cmd": "python src/main.py check",
    "run_cmd": "python src/main.py scan"
  }
  

  Parameter Descriptions:

  • check_cmd:
    • Function: Determines if the current execution environment meets the tool's requirements (this command can be omitted if no check is needed). For example, some tools can only run on Linux and require checking if the current environment is Linux.
    • Output: Writes the check result to the check_result.json file, which contains either {"usable": true} or {"usable": false}.
  • run_cmd:
    • Function: Scans code and executes custom checker logic (this command must exist).
    • Output: Writes results to the result.json file in the specified format.

• Tool Output Format Requirements

  • Write scan results to the result.json file in the current working directory (Python example code):

  import json
  with open("result.json", "w") as fp:
      json.dump(result, fp, indent=2)
  
  • The result.json file format is as follows:

  [
      {
          "path": "Absolute path of the file",
          "line": "Line number (int type)",
          "column": "Column number (int type; if the tool does not output column information, 0 can be used)",
          "msg": "Prompt message",
          "rule": "Rule name (can output different rule names as needed)",
          "refs": [
              {
                  "line": "Backtrace line number",
                  "msg": "Prompt message",
                  "tag": "A brief tag for the line information (e.g., uninit_member, member_decl); if none, use the same tag for all",
                  "path": "Absolute path of the file where the backtrace line is located"
              },
              ...
          ]
      },
      ...
  ]
  

  Explanation of the refs Field:

  This is an optional field that can be omitted. It records backtrace path information for issues. For example, if a code issue in the current line is caused by the execution path of three lines of code in the context, the positions and prompt messages of these three lines can be added to the refs array in sequence.

Step 2: Submit the Tool to a Git Repository

• Create a repository and submit the tool's source code or compiled executable file to the repository (it is recommended to submit to the master branch, as TCA defaults to pulling from the master branch). Only CODING repositories are supported.

• It is recommended to include a README.md file in the repository to describe the tool's functionality and maintainers.

• Subsequent modifications to the tool's implementation logic can be made by directly updating the repository. TCA will automatically pull the latest version of the tool code when executing the tool.

Step 3: Create a Tool in the Tool Management Page

• Navigate to the tool management page and click "Create Tool".

  

• Fill in the tool information.

  

  Parameter Descriptions:

  • Tool Repository Address: The Git repository address of the tool submitted in the previous step; by default, the master branch is pulled.

  • Execution Command: The command to execute in the tool's root directory.

  • Environment Variables: Environment variables required for tool execution.

  • Supported Systems: The operating systems required for tool execution.

  • License: If it is an open-source tool, specify the open-source license it follows; otherwise, indicate self-developed or co-built.

  • Is a Compiled Tool: Indicates whether the tool requires the user's code to be compiled or executable before analysis.

  • Note: For tools used in special scanning scenarios (e.g., checking if the code repository contains certain third-party dependency directories, with results not involving individual code files), general result processing steps can be skipped by setting the following environment variables to avoid filtering out issue results:

    • BLAME_TYPE=NO_BLAME: Skips locating file/code owners for code lines/files (used when results do not involve individual files/code lines).
    • FILTER_TYPE=NO_VERSION_FILTER: Skips checking if the issue path (path field) corresponds to files committed to the repository (used when results do not involve individual files/code lines).
    • IGNORE_TYPE=NO_ISSUE_IGNORE: Skips comment-based ignore processing (used when results do not involve individual files/code lines).

Step 4: Add Rules to the Tool

• After creating the tool, navigate to the rule list to add rules for the tool.

  

• Fill in the rule information.

  Parameter Descriptions:

  • Rule Introduction: Briefly describes the issue detected by the rule; this will be displayed as the issue title in scan results.

  • Detailed Description: Provides a detailed description of the rule and its resolution methods; it is recommended to include a resolution demo case.

  • Resolution Method: Explains how to resolve the code issue based on actual circumstances; it is recommended to include a resolution demo case.

  • Rule Parameters: Can be left blank if no information needs to be passed through rule parameters.

Step 5: Configure the Tool to an Execution Node

Step 6: Use the Tool Rules in the Project After Completing the Above Steps

• Navigate to the project, go to Analysis Scheme - Code Inspection, and configure the rules.

• Click "Add Rule" to find and add the corresponding tool rule.

• After adding, start the analysis. To apply the rule to all code files, it is recommended to start a full analysis (incremental analysis only analyzes files changed since the last scan).