How to include files in .gcloudignore that are ignored in .gitignore

C

4

32

I have certain folders that I ignore in my .gitignore as I do not want it to be published on Github.

However, these files are essential for when deploying to Google Cloud (local packages).

If I take out .gitignore from .gcloudignore then the files that I do want to ignore (like venv, .idea, .pyc) are uploaded, which is what I do not want.

How can I there include only a portion of .gitgnore in .gcloudignore?

.gcloudignore

.gcloudignore
.git
.gitignore

node_modules
#!include:.gitignore

.gitignore

# This I want to ignore in .gcloudignore
.Python
env/
build/

# This I want to keep in .gcloudignore
module_data_import
module_calendar

Counsel answered 25/6, 2019 at 9:43 Comment(1)

I just created the .gcloudignore file and it fixed my problem – Stipe 18/11, 2021 at 6:53

R

18

The 2 files are mostly orthogonal, they serve different purposes and are used by different tools.

The only relationship between them (apart from .gcloudignore inheriting heavily from the .gitignore syntax and behaviour) is the ability to include the .gitignore file inside the .gcloudignore file, which is what you do with:

#!include:.gitignore

The side effect of that is, of course, that everything in .gitignore will also be ignored during GAE deployments, as you observed. From gcloud topic gcloudignore:

This will include the contents of a .gitignore-style file at that path at that point in the file. It does not recurse (that is, the included file cannot #!include another file) and cannot be anywhere but the top-level directory to be uploaded.

But you don't have to include .gitignore. All you need to do is drop that inclusion and specify exactly just the patterns from .gitignore that you want to ignore in deployments. If I understand correctly your goal description the .gcloudignore would look something like this:

.gcloudignore
.git
.gitignore

node_modules
module_data_import
module_calendar

If duplicating those patterns in both files bothers you a potential alternative (IMHO rather complex) might be to take advantage of git's ability to ignore files based on patterns collected from multiple sources (with different scopes, though, so pay attention to that). See https://git-scm.com/docs/gitignore and Can I include other .gitignore file in a .gitignore file? (like #include in c-like languages).

If you can find a proper split fitting your application you could:

keep in .gitignore just the patterns for files that you also want to ignore at deployments (keeping, of course, the .gitignore inclusion in .gcloudignore)
place the other patterns in another file that git uses to determine ignore rules - thus they'd be ignored by git but since this file isn't included in .gcloudignore they would not be ignored at deployments.

UPDATE following comment:

In the initial answer I didn't consider the case in which the gcloud command itself executes operations that target directly or indirectly source code repository operations under the hood which, if using git as the version control system, will be, of course, impacted by the presence and content of the .gitignore file. In such case one could arguably consider that the gcloud command itself uses the .gitignore file (regardless of the .gcloudignore file).

Requisition answered 10/7, 2019 at 4:40 Comment(7)

>> The 2 files are mostly orthogonal, they serve different purposes and are used by different tools. Wrong. The 'gcloud builds' command ingores stuff in .gitignore. – Brush 14/12, 2019 at 18:51

You mean without it being included in .gcloudignore? If so I bet it's not intended ;) – Requisition 15/12, 2019 at 6:49

You can demonstrate that it does use gitignore and that it is intended. Follow the Cloud Build Quickstart at cloud.google.com/cloud-build/docs/quickstart-docker Add a .gitignore file containing the single line quickstart.sh After executing the cloud build command you will get the output Check the gcloud log... (see $ gcloud topic gcloudignore to learn more). ... Step 2/3 : COPY quickstart.sh / COPY failed: stat /var/lib/docker/tmp/docker-builder267017000/quickstart.sh: no such file or directory ERROR – Brush 16/12, 2019 at 19:49

That's not what the document says (which would, indeed, demonstrate intention). It may be indeed the description of the symptoms you observe, but that doesn't demonstrate intent, it could be accidental. – Requisition 16/12, 2019 at 23:25

@Brush I updated my answer. The gcloud build is uploading the source code from your current directory for the cloud build which will be executed remotely. I suspect git is used under the hood to manage the source code as a repo. By adding that.gitignore file you prevent uploading an essential piece for the build. – Requisition 22/12, 2019 at 8:51

Nice. Thanks. Besides GAE, I saw the same thing with Cloud Run. – Brush 23/12, 2019 at 15:34

It's probably not intentional. This is a problem for me; I hope Google will fix it soon... – Lomond 8/3, 2020 at 15:44

H

29

You can explicitly include a file if it has already been excluded by a pattern (such as inheriting your .gitignore settings, or an excluded directory) previously in the .gcloudignore file.

To include a file named foo, make sure the following comes after the rule that excluded it:

!foo

To include a folder named 'bar', make sure the following comes after the rule that excluded it:

!bar/

Source: https://cloud.google.com/sdk/gcloud/reference/topic/gcloudignore

Hendeca answered 8/10, 2019 at 19:42 Comment(2)

What about a folder – Counsel 7/11, 2019 at 12:50

Updated my answer with an example for directories. – Hendeca 11/11, 2019 at 10:49

R

18

The 2 files are mostly orthogonal, they serve different purposes and are used by different tools.

The only relationship between them (apart from .gcloudignore inheriting heavily from the .gitignore syntax and behaviour) is the ability to include the .gitignore file inside the .gcloudignore file, which is what you do with:

#!include:.gitignore

The side effect of that is, of course, that everything in .gitignore will also be ignored during GAE deployments, as you observed. From gcloud topic gcloudignore:

This will include the contents of a .gitignore-style file at that path at that point in the file. It does not recurse (that is, the included file cannot #!include another file) and cannot be anywhere but the top-level directory to be uploaded.

But you don't have to include .gitignore. All you need to do is drop that inclusion and specify exactly just the patterns from .gitignore that you want to ignore in deployments. If I understand correctly your goal description the .gcloudignore would look something like this:

.gcloudignore
.git
.gitignore

node_modules
module_data_import
module_calendar

If duplicating those patterns in both files bothers you a potential alternative (IMHO rather complex) might be to take advantage of git's ability to ignore files based on patterns collected from multiple sources (with different scopes, though, so pay attention to that). See https://git-scm.com/docs/gitignore and Can I include other .gitignore file in a .gitignore file? (like #include in c-like languages).

If you can find a proper split fitting your application you could:

keep in .gitignore just the patterns for files that you also want to ignore at deployments (keeping, of course, the .gitignore inclusion in .gcloudignore)
place the other patterns in another file that git uses to determine ignore rules - thus they'd be ignored by git but since this file isn't included in .gcloudignore they would not be ignored at deployments.

UPDATE following comment:

In the initial answer I didn't consider the case in which the gcloud command itself executes operations that target directly or indirectly source code repository operations under the hood which, if using git as the version control system, will be, of course, impacted by the presence and content of the .gitignore file. In such case one could arguably consider that the gcloud command itself uses the .gitignore file (regardless of the .gcloudignore file).

Requisition answered 10/7, 2019 at 4:40 Comment(7)

>> The 2 files are mostly orthogonal, they serve different purposes and are used by different tools. Wrong. The 'gcloud builds' command ingores stuff in .gitignore. – Brush 14/12, 2019 at 18:51

You mean without it being included in .gcloudignore? If so I bet it's not intended ;) – Requisition 15/12, 2019 at 6:49

You can demonstrate that it does use gitignore and that it is intended. Follow the Cloud Build Quickstart at cloud.google.com/cloud-build/docs/quickstart-docker Add a .gitignore file containing the single line quickstart.sh After executing the cloud build command you will get the output Check the gcloud log... (see $ gcloud topic gcloudignore to learn more). ... Step 2/3 : COPY quickstart.sh / COPY failed: stat /var/lib/docker/tmp/docker-builder267017000/quickstart.sh: no such file or directory ERROR – Brush 16/12, 2019 at 19:49

That's not what the document says (which would, indeed, demonstrate intention). It may be indeed the description of the symptoms you observe, but that doesn't demonstrate intent, it could be accidental. – Requisition 16/12, 2019 at 23:25

@Brush I updated my answer. The gcloud build is uploading the source code from your current directory for the cloud build which will be executed remotely. I suspect git is used under the hood to manage the source code as a repo. By adding that.gitignore file you prevent uploading an essential piece for the build. – Requisition 22/12, 2019 at 8:51

Nice. Thanks. Besides GAE, I saw the same thing with Cloud Run. – Brush 23/12, 2019 at 15:34

It's probably not intentional. This is a problem for me; I hope Google will fix it soon... – Lomond 8/3, 2020 at 15:44

H

2

To include a folder that is ignored in .gitignore I had to use the following syntax:

!bar/**

Without ** is was not working.

Honaker answered 16/7, 2021 at 9:34 Comment(0)

A

0

I wanted to have what is in .gitignore with one exception. This has worked for me:

#!include:.gitignore

!foo

Appolonia answered 16/1, 2024 at 9:50 Comment(0)

Recommended topics

Hot tags