Is it possible to use filters in refspec in places other than directory namespaces?

Question

Looking at this the examples show refspecs can filter directories/namespaces as in:

+refs/heads/qa/*:refs/remotes/origin/qa/*

However I'm trying to optimise a fetch and the repo I'm working with has branches named like this:

release-6.0.0

When trying to filter these I get:

fatal: Invalid refspec '+refs/heads/release*:refs/remotes/upstream/release*'

Is there any way to filter these in the fetch or do I need to fetch all the remote heads?

Slavonic

Interesting. Elsewhere in git it's OK with glob-style matching (even on refs, e.g., with git for-each-ref), so I'm not sure why this doesn't work. – Karns

Yes, the same script I'm trying to use this in has: git for-each-ref --sort='-*authordate' --format '%(refname:short)' refs/remotes/origin/release*" so I was surprised when the above failed. – Slavonic

I also tried +refs/heads/release*:refs/remotes/upstream/* thinking that it might make sense for the client side to need to be a directory but that failed too. – Slavonic

Note: this should be possible with git 2.6. See my edited answer below – Atterbury

Answer 1

Git 2.6+ (Q3 2015) will allow this kind of refspec!

See commit cd377f4, commit 53a8555 (22 Jul 2015) by Jacob Keller (jacob-keller).
^{(Merged by Junio C Hamano -- gitster -- in commit 8d3981c, 03 Aug 2015)}

refs: loosen restriction on wildcard "*" refspecs

Loosen restrictions on refspecs by allowing patterns that have a "*" within a component instead of only as the whole component.

Remove the logic to accept a single "*" as a whole component from check_refname_format(), and implement an extended form of that logic in check_refname_component().
Pass the pointer to the flags argument to the latter, as it has to clear REFNAME_REFSPEC_PATTERN bit when it sees "*".

Teach check_refname_component() function to allow an asterisk "*" only when REFNAME_REFSPEC_PATTERN is set in the flags, and drop the bit after seeing a "*", to ensure that one side of a refspec contains at most one asterisk.

This will allow us to accept refspecs such as for/bar*:foo/baz*.
Any refspec which functioned before shall continue functioning with the new logic.

---

Original answer (2013)

It doesn't seem possible, because that limitation facilitates refs matching done by remote.c.

That goes back to Git 1.5.6.5 (August 2008) and commit b2a5627 (Daniel Barkalow) which enforces the rule of "wildcard refspec must end with slash and star":

A wildcard refspec is internally parsed into a refspec structure with src and dst strings.
Many parts of the code assumed that these do not include the trailing "/*" when matching the wildcard pattern with an actual ref we see at the remote.
What this meant was that we needed to make sure not just that the prefix matched, and also that a slash followed the part that matched.

But a codepath that scans the result from ls-remote and finds matching refs forgot to check the "matching part must be followed by a slash" rule.
This resulted in "refs/heads/b1" from the remote side to mistakenly match the source side of "refs/heads/b/*:refs/remotes/b/*" refspec.

Worse, the refspec crafted internally by "git-clone", and a hardcoded preparsed refspec that is used to implement "git-fetch --tags", violated this "parsed widcard refspec does not end with slash" rule; simply adding the "matching part must be followed by a slash" rule then would have broken codepaths that use these refspecs.

This commit changes the rule to require a trailing slash to parsed wildcard refspecs.
IOW, "refs/heads/b/*:refs/remotes/b/*" is parsed as src = "refs/heads/b/" and dst = "refs/remotes/b/".
This allows us to simplify the matching logic because we only need to do a prefixcmp() to notice "refs/heads/b/one" matches and "refs/heads/b1" does not.

---

The OP Monte Goulding points out to commit 46220ca (diff) as being the origin of this rule (Git 1.5.5, April 2007)

We tightened the refspec validation code in an earlier commit ef00d15 (Tighten refspec processing, 2008-03-17) per my suggestion, but the suggestion was misguided to begin with and it broke this usage:

$ git push origin HEAD~12:master

The syntax of push refspecs and fetch refspecs are similar in that they are both colon separated LHS and RHS (possibly prefixed with a + to force), but the similarity ends there.
For example, LHS in a push refspec can be anything that evaluates to a valid object name at runtime (except when colon and RHS is missing, or it is a glob), while it must be a valid-looking refname in a fetch refspec.
To validate them correctly, the caller needs to be able to say which kind of refspecs they are.
It is unreasonable to keep a single interface that cannot tell which kind it is dealing with, and ask it to behave sensibly.

This commit separates the parsing of the two into different functions, and clarifies the code to implement the parsing proper (i.e. splitting into two parts, making sure both sides are wildcard or neither side is).

Atterbury

Hmm... I'm not sure I get this. If the refspec is refs/heads/b/* you want it to match refs/heads/b/one but not refs/heads/b1 but if it's refs/heads/b* you want it to match refs/heads/b1 and not refs/heads/b/one. I can only imaging both matching if your refspec is refs/heads/b* but in that case you should be prepared for both... – Slavonic

Looking at the commit it seems to me that if it were modified to only drop and put back the trailing slash before checking the format if the trailing slash is there it would enable what I wanted to do and still protect refspecs with trailing slashes from returning erroneous results. What would be the downsides to that change? – Slavonic

@MonteGoulding I agree, it seems feasible. It would have to be tested against all the other Git unit tests – Atterbury

do you know if there's more history to the "matching part must be followed by a slash" rule? – Slavonic

@MonteGoulding not that I know of, when I explored the history of remote.c – Atterbury

Ah... if you look at line -460,+481 which is unchanged by the commit is_glob is true only if it ends with /* which comes from 46220ca – Slavonic

I'm accepting this answer as it doesn't look like I can do what I wanted to do without sending in a patch. – Slavonic