Exporting Sonarqube reports into Excel - based on major, minor and critical categories

Question

Is there a way to export Sonarqube reports into Excel - based on major, minor and critical categories?

Buttons

Answer 1

You can use the REST API, to query the data into JSON text and then export that JSON to a CSV file.

I used the command below to get a JSON response:

http://xxxxx.xx.xxxx.com:9000/api/issues/search?componentRoots=test_xxx_xx&statuses=OPEN,REOPENED&pageSize=500&pageIndex=1

Where componentRoots is the your sonar project name.

It gave all the issues in JSON and then I converted it in to a CSV.

Rattlebox

Okay because of paging I now have 8 JSON files, how do I convert them into a single CSV file? That is the part you left out of your answer. – Diabetes

I had to use componentKeys instead of componentRoots. Moreover, the API requires authentication, so simplest thing is to authenticate in the sonarcube app and then call API with browser. – Clown

You can use: github.com/talha2k/sonarqube-issues-export-to-excel – Scent

Answer 2

The only way is to use the api/issues/search web service

Hibbard

github.com/talha2k/sonarqube-issues-export-to-excel script based on the api you mentioned. – Scent

@Scent your script requires running 'pip install openpyxl' It worked fine later. – Senator

Answer 3

I used F12 developer tools to create the report. Here is step by step solution:

1. Before opening Sonar report, press F12 to open developer tools in the browser tab (Refer screenshot). Click on Network tab. Initially it'll be empty:

2. Open Sonar report in the same browser tab.

3. Now you have to select a request in URL column (Refer screenshot in point # 4). You've to select that request whose URL is of the format - https://sonar:8443/api/issues/search?. Base URL might be different depending upon the name of the server where you have hosted Sonar website.

4. Click on Response tab (Refer screenshot):

   Copy the entire JSON data to clipboard. First press Ctrl + A and then Ctrl + C to copy the contents.

5. JSON data that you got is in below format:

   Data of all the issues is present inside issues key. It is of array type as it contains a list of issues. Paste the entire JSON data into any notepad editor and get the value of issues node. A sample issues node data is shown below:

   [
   {
   "key":"AYMBNpviy48WWZHdsU1Z"
   ,"rule":"OWASP:UsingComponentWithKnownVulnerability"
   ,"severity":"MAJOR"
   ,"component":"B91661CE-50F8-45CB-8F54-29CD044EF32D"
   ,"project":"B91661CE-50F8-45CB-8F54-29CD044EF32D"
   ,"flows":[]
   ,"status":"OPEN"
   ,"message":"Filename: jquery-ui-1.10.3.custom.js | Reference: CVE-2021-41184 | CVSS Score: 6.1 | Category: CWE-79 | jQuery-UI is the official jQuery user interface library. Prior to version 1.13.0, accepting the value of the `of` option of the `.position()` util from untrusted sources may execute untrusted code. The issue is fixed in jQuery UI 1.13.0. Any string value passed to the `of` option is now treated as a CSS selector. A workaround is to not accept the value of the `of` option from untrusted sources."
   ,"author":""
   ,"tags":["cve","cwe","cwe-937","owasp-a9","vulnerability"]
   ,"transitions":["confirm"]
   ,"actions":["set_tags","comment","assign"]
   ,"comments":[]
   ,"creationDate":"2022-08-18T06:35:49+0100"
   ,"updateDate":"2022-08-18T06:35:49+0100"
   ,"type":"VULNERABILITY"
   ,"branch":"sonarqubepr"
   ,"scope":"MAIN"
   ,"quickFixAvailable":false
   },
   {
   "key":"AYMBNpviy48WWZHdsU1_"
   ,"rule":"OWASP:UsingComponentWithKnownVulnerability"
   ,"severity":"MAJOR"
   ,"component":"B91661CE-50F8-45CB-8F54-29CD044EF32D"
   ,"project":"B91661CE-50F8-45CB-8F54-29CD044EF32D"
   ,"flows":[]
   ,"status":"OPEN"
   ,"message":"Filename: jquery.ui.datepicker-ml.min.js | Reference: CVE-2021-41183 | CVSS Score: 6.1 | Category: CWE-79 | jQuery-UI is the official jQuery user interface library. Prior to version 1.13.0, accepting the value of various `*Text` options of the Datepicker widget from untrusted sources may execute untrusted code. The issue is fixed in jQuery UI 1.13.0. The values passed to various `*Text` options are now always treated as pure text, not HTML. A workaround is to not accept the value of the `*Text` options from untrusted sources."
   ,"author":""
   ,"tags":["cve","cwe","cwe-937","owasp-a9","vulnerability"]
   ,"transitions":["confirm"]
   ,"actions":["set_tags","comment","assign"]
   ,"comments":[]
   ,"creationDate":"2022-08-18T06:35:49+0100"
   ,"updateDate":"2022-08-18T06:35:49+0100"
   ,"type":"VULNERABILITY"
   ,"branch":"sonarqubepr"
   ,"scope":"MAIN"
   ,"quickFixAvailable":false
   }
   ]
   

6. The JSON array data obtained in previous step is of real use. Convert this JSON array data into an EXCEL file using any online converter. I used the following website - https://www.convertcsv.com/json-to-csv.htm

What if my report has more than one page?

Sonar loads 100 records per page. To get the next page you've to click on Show More button at the bottom:

Every time you click on the Show More button, a new request goes to the Sonar server which will show up in the F12 developer tool's Network tab. So for all subsequent pages, repeat same set of steps to create a new Excel sheet per page. Then merge all the Excel sheets manually to create single master report.

Note: While merging the Excel files, remember that every Excel file will have a header row at the top. While merging the files, you'll have to keep the header row from from the first Excel sheet and ignore from the rest.

Imperator

Answer 4

Yes you can use the REST APIs provided with SonarQube to query. The documentation of APIs is also embedded into every Sonar instance as different versions expose different APIs. We use Python for similar work as response will be in JSON and it will be easier to manipulate. Once you have arrived extracted issues of your liking, write them into .CSV or excel.

Link to web services will be in footer of Sonar Instance.

PS: Expanded answer to offset short-sighted answering. What is provided here is only abstract and not complete answer with query details.

Not recommended by community With DB:- If you have been using Sonar for long and if you won't upgrade SQ too often you can choose to study table structure and understand how data is organised. We have done this too, but it gets messier with every passing upgrade (more inner queries). Cost of querying will be saved on bulk process of data as ES is not involved

Hayward

@janos honestly the guy questioning didn't seem to do any home work so, the answer was incomplete. I take the blame that I didn't take broader community that will refer the answer in future – Hayward

Answer 5

Please try the following command to get all issues in JSON format. Then you may consider to parse output by using jason parser programs.

Replace "XXX:XXX" with Sonar Key defined in "sonar.projectKey" variable on "sonar-project.properties.txt" file

http://localhost:9000/api/issues/search?componentKeys=XXX:XXX

Hylophagous

Answer 6

As suggested above, you can use the api. To get all the Issues in one go, use Postman:

GET: http://localhost:9099/api/issues/search?components=<project-key>&ps=500

where ps is Page Size.

You need to pass the Authorization. Use Type as Basic Auth. Provide username password that you used at the time of login. This will reduce time of downloading 100 records and then merging them into single excel.

Entoderm

You can use: github.com/talha2k/sonarqube-issues-export-to-excel using python script. – Scent