#4808 API: support listing files
Closed: Fixed by pingou. Opened by lachmanfrantisek.

We would like to use an API for listing files in the repositories.

Here is a link in the ogr repository, our Python API for pagure, github and gitlab:
https://github.com/packit-service/ogr/issues/310

We would like to use in the packit project:
- pagure (e.i. pagure.io) as an upstream: https://github.com/packit-service/packit-service/issues/556
- pagure as a centos-stream source-git

Thank you in advance.


Is there any specific information you would like or just a list of files and folders (so the equivalent to https://pagure.io/pagure/blob/master/f/pagure in JSON)?

Metadata Update from @pingou:
- Issue tagged with: RFE

lachmanfrantisek commented

Yes, that would be nice. (To be able to search through files without the need to clone a repo.)

Some relevant links:

  • github api: https://developer.github.com/v3/repos/contents
  • pygithub: https://pygithub.readthedocs.io/en/latest/github_objects/Repository.html#github.Repository.Repository.get_contents
  • our implementation: https://github.com/packit-service/ogr/blob/cddce51904617c18565effa1d1955f69486ba745/ogr/services/github/project.py#L405

  • gitlab: https://docs.gitlab.com/ee/api/repositories.html#list-repository-tree

  • python-gitlab: https://python-gitlab.readthedocs.io/en/stable/api/gitlab.v4.html#gitlab.v4.objects.Project.repository_tree
  • our implementation: https://github.com/packit-service/ogr/blob/cddce51904617c18565effa1d1955f69486ba745/ogr/services/gitlab/project.py#L344

Thanks!

lachmanfrantisek commented

Something like this:

  • directory:
{
    "type": "dir",
    "name": "dir1",
    "content": [
        {
            "type": "file",
            "name": "file1.txt",
            "content_url": "https:/....f/raw/file1.txt}"
        },
        {
            "type": "file",
            "name": "file2.txt",
            "content_url": "https:/....f/raw/file2.txt}"
        }
    ]
}
  • file:
{"type": "file", "name": "file1.txt", "content_url": "https:/....f/raw/file1.txt}"}

And ideally for any git-ref (e.g. branch, tag, commit).

lachmanfrantisek commented

Maybe it makes sense to set the level of recursion. And allow getting the whole tree with levels=-1.

Metadata Update from @ngompa:
- Issue set to the milestone: 5.11

Metadata Update from @pingou:
- Issue assigned to pingou

I have the basic code for this which use the structure you've used (except that it uses file and folder where you used file and dir :))

I have not implemented the recursion as I am a bit wondering if there is a risk of DOS with it.

Do you really need it?

lachmanfrantisek commented

I have the basic code for this which use the structure you've used (except that it uses file and folder where you used file and dir :))

Thank you so much!

I have not implemented the recursion as I am a bit wondering if there is a risk of DOS with it.

Do you really need it?

The solution without recursion is fine for start and we can still be able to load the other levels with another call. (=> We can implement it on our side.)

I'll create a new issue if we find that necessary.

Thanks!

Commit dd207ee3 fixes this issue

lachmanfrantisek commented

Thank you so much!

Do you know in which version it is going? (Just to know when we can start working on it.)

lachmanfrantisek commented

Do you know in which version it is going? (Just to know when we can start working on it.)

Sorry, I see now the 5.11 milestone.

Metadata
Related Pull Requests