How to get the raw content of a file through GitLab REST API?

Question

The following REST Url of GitLab API gives me the repository tree of a project.

Get Repo Tree (WORKS)

https://gitlab.gspt.net/api/v3/projects/2931/repository/tree?private_token=XXXX

Output:

[
    {
        "id": "a49d11794ed56db7f935abfd61002aef67159d10",
        "name": "src",
        "type": "tree",
        "path": "src",
        "mode": "040000"
    },
    {
        "id": "0fbd98527d4b36e3d22c164293d8fd8eee4d18cd",
        "name": ".gitignore",
        "type": "blob",
        "path": ".gitignore",
        "mode": "100644"
    },
    {
        "id": "0ef0da472176f2e6a24843ac9d4bb738c8310d23",
        "name": "pom.xml",
        "type": "blob",
        "path": "pom.xml",
        "mode": "100644"
    }
]

But I am not able to get the raw content of a file, pom.xml to be precise.

Get Raw Content of a file (DOES NOT WORK - Gives 404)

https://gitlab.gspt.net/api/v3/projects/2931/repository/files/pom%2Exml/raw?private_token=xxxx&ref_name=master

Output:

{
    "error": "404 Not Found"
}

As per the documentation here (https://docs.gitlab.com/ee/api/repository_files.html#get-raw-file-from-repository) I am specify the correct rest url. The only thing that's different however is the use of V4 instead of V3 in the rest api endpoint. I searched around but could not find the endpoint of v3 api.

Frumpy

Answer 1

First, just in case, don't percent encode the ".":

.../files/pom.xml/raw?...
            ^^

Second, you can check how the files endpoint was affected from v3 to v4 in the merge request 9637 and this comparison

v3:
GET /projects/:id/repository/raw_blobs/:sha
v4:
GET /projects/:id/repository/blobs/:sha/raw

You can see the examples (in v3) did not percent encode the dot.

curl --request GET --header 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' \
  'https://gitlab.example.com/api/v3/projects/13083/repository/files?file_path=app/models/key.rb&ref=master'

However, the v3 API only allows to get raw blobs, not a raw file.
See merge request 16834:

• Modify /projects/:id/repository/files to /projects/:id/repository/files/:filepath (:filepath should be URL-encoded)
• Move /projects/:id/repository/blobs/:sha to /projects/:id/repository/files/:filepath/raw

Only v4 API allows for a :filepath parameters.

See "Git objects SHA-1 are file contents or file names?" to decode the raw blob you get from API v3.

Randallrandan

Thanks a lot Von, I got the content of the file using this url: https://gitlab.gspt.net/api/v3/projects/2931/repository/files?file_path=pom.xml&ref=master. However, the content of the file is encoded. How to get the raw content ? – Frumpy

@user2325154 By using the /projects/:id/repository/raw_blobs/:sha syntax I mention in the answer: see gitlab.com/gitlab-org/gitlab-ce/merge_requests/9637/… – Randallrandan

Not sure what to specify in the :sha. Is :sha the commit-id of the file? I looked into the link you provided but it does not mention anything on how to get the raw content of the file. The content that I am getting from the above-mentioned url is base64 encoded. I can always decode that and get the actual raw content of the file. But is there a way to get the raw content of the file directly? – Frumpy

@user2325154 Looking once more at gitlab.com/gitlab-org/gitlab-ce/merge_requests/9637/…, I don't see a syntax for raw file content in v3. It seems it has been added for v4 only for answering gitlab.com/gitlab-org/gitlab-ce/issues/16834: I have edited the answer accordingly, including a link (https://mcmap.net/q/1166663/-git-objects-sha-1-are-file-contents-or-file-names) to decode the raw blob. So: "is there a way to get the raw content of the file directly": not in v3.. Only in v4. – Randallrandan

Answer 2

Attention dot(.) encode version: see wiki

  . == %2E

Brenneman

Answer 3

You can execute this function at your browser devtools console panel.

function getGitlabFileContentUrl(gitlabOrigin="https://gitlab.com", projectId="123", filePath="src/index.ts", ref="master", privateToken="123") {
    return `${gitlabOrigin}/api/v4/projects/${projectId}/repository/files/${encodeURIComponent(filePath)}/raw?ref=${ref}&private_token=${privateToken}`
}
// in your example
getGitlabFileContentUrl('https://gitlab.gspt.net', '2931', 'pom.xml', 'master', 'xxxx') 
// https://gitlab.gspt.net/api/v4/projects/2931/repository/files/pom.xml/raw?ref=master&private_token=xxxx

For project id you can refer to Where do I find the project ID for the GitLab API?

Flimflam