Package management in Git for Windows (Git Bash)?
Asked Answered
F

8

105

I'm reading the github wiki for git-for-windows and it says that msys2 bundles pacman: https://github.com/git-for-windows/git/wiki/Package-management

But when I invoke it I get:

$ pacman
bash: pacman: command not found

Does anyone have an idea what is going on?

Which git version does this wiki refer to?

Is there a way to install additional packages to msys2 inside Git for windows?

Franciscafranciscan answered 22/9, 2015 at 8:31 Comment(1)
WeiHua Liu 's answer below seems to be more precise and up-to-date than my 2015 answer.Stricken
S
70

As mentioned in issue 397:

This is intended. We do not ship pacman with Git for Windows.
If you are interested in a fully fledged package manager maintained environment you have to give the Git for Windows SDK a try.

The bash that you see in the latest git for Windows (2.5.3), which is a more recent bash than the old msysgit one, is only there to execute git commands.
It is not a full-fledged linux environment to install any third-party package.


Warning: dhj reports in the comments

Do not link your existing git for windows with the msys2 main system by using a directory junction.
If you uninstall it will decide that linked directory belongs to it and DELETE YOUR ENTIRE HOME DIRECTORY including sub-directories like "Downloads".
Beware dealing with msys2.

I don't know if the same is true for the git for windows SDK, but BE CAREFUL trying to get pacman from other systems integrated with git for windows.

Stricken answered 22/9, 2015 at 9:30 Comment(13)
Yes, I don't expect to have full linux experience from the git bash, but It would be nice to install some command line utilities to make working in windows environment less painful. I tried Git for Windows SDK and it seems that this is what I wanted. Thanks for the hint.Franciscafranciscan
You can extract the Git for Windows SDK installer manually and edit the "install" script setup-git-sdk.bat to limit the packages that are installed (exclude python and the mingw build tools). This keeps installation size down, but still allows you to upgrade existing and install additional packages using pacman. You'll have to copy git-bash.exe and git-cmd.exe (and others?) from a Git for Windows installation though.Otero
@BrechtMachiels OK. I will follow your question at #25019557Stricken
It would be great to have a (supported?) version of Git for Windows which adds just what you need if you already have MSYS2. Does this exist?Piece
@Piece Not that I know of. I only know of github.com/git-for-windows/git/releasesStricken
BEWARE. Do not link your existing git for windows with the msys2 main system by using a directory junction. If you uninstall it will decide that linked directory belongs to it and DELETE YOUR ENTIRE HOME DIRECTORY including sub-directories like "Downloads". Beware dealing with msys2. Absolutely nauseated with how much I lost. I don't know if the same is true for the git for windows SDK, but BE CAREFUL trying to get pacman from other systems integrated with git for windows.Redcap
@Redcap Thank you for the feedback: I have included your comment/warning in the answer for more visibility.Stricken
I just want to run bc on a files of sum expressions, one per line. Is there a good alternative on Git Bash/Git for Windows? I like the product thus far, just missing this one thing.Psychochemical
@JohnCarlson You can install Msys2 that I mentioned here. You would then have access to Pacman. See also "What are the differences between msys/git and git-for-windows/mingw-w64-x86_64-git?" for more details.Stricken
No, I want to run Git Bash/Git for Windows right now. I am really looking for shell syntax to do the same as bc on a list of sum expressions in a file. I have no issues with generating a bash script with the sums and running it, even though it sounds like a security nightmare. I already have Cygwin if I really want to run bc. Cygwin doesn't work with my app very well currently, but it does have bc. I'm looking for a pure shell alternative to bc, something that's already present in Git Bash/Git for Windows.Psychochemical
@JohnCarlson So something like this?Stricken
@Stricken As far as I can tell, it only works with simple sums, like $((1+2)) and not $((1+2+3)). Also, I'm not sure if floats are handled. I switched to perl, but i realize that that is highly insecure and prone to injection. I would prefer bc, but I want to use Git For Windows. I'll probably switch to the Git for Windows SDK, but I'm getting additional requirements now...Psychochemical
@JohnCarlson True. You can post this as a separate question, with your current solution. Others will propose different options.Stricken
H
48

Git for Windows (https://gitforwindows.org/ or https://git-scm.com/downloads) has Git Bash but it does not include tree.

tree is available via pacman (Package Manager), but that is only available if you install "Git for Windows SDK" (scroll to the bottom of https://gitforwindows.org/ which provides a link to download installer for it from https://github.com/git-for-windows/build-extra/releases/latest)

The accepted answer was very helpful. They mention that git-for-windows was not meant to include pacman in the default install.

So I installed "Git for Windows SDK", then in its bash prompt (SDK-64) I ran the following to install current tree v1.7.0-1 (as of this posting Aug 30, 2018):

[SDK-64: Bash Terminal for Git for Windows SDK]
pacman -S tree
...
Proceed with installation? [Y/n] Y

On my system, Git for Windows SDK is installed under: C:\git-sdk-64, so from my Git for Windows Bash shell (which did not have tree installed), I copied it over tree.exe to its /usr/bin directory, e.g.

[MINGW64: Bash Terminal for Git for Windows]
cd /usr/bin
cp /c/git-sdk-64/usr/bin/tree.exe .

Now I can run tree v1.7.0 from both Git Bash shells.

To make it even easier for others and maybe myself on a future machine, I looked at where pacman was getting the tree package from by running the following in my Git for Windows SDK Bash terminal:

$ pacman -S --info tree
Repository      : msys
Name            : tree
Version         : 1.7.0-1
Description     : A directory listing program displaying a depth indented list of files
Architecture    : x86_64
...

The key thing here is that pacman is getting tree from the "msys" repository (FYI: even though it says msys, it really is using msys2), so I looked at /etc/pacman.d/mirrorlist.msys and the first mirror points to http://repo.msys2.org/msys/$arch/

So next time you want a package that is NOT in Git for Windows, you can download them from: http://repo.msys2.org/msys/x86_64/ (for 64-bit) or from http://repo.msys2.org/msys/i686/ (32-bit)

e.g. direct download link for tree v1.7.0-1

FYI: Git SCM's Window's download at https://git-scm.com/download/ pulls the latest from Git for Windows GitHub (https://github.com/git-for-windows/git from the https://github.com/git-for-windows/git/releases/ link)

Helvetic answered 30/8, 2018 at 21:3 Comment(1)
If you download the package file, you would still need pacman to install it correct? Git for windows doesn't include or support pacman ... unless you use the git for windows sdk which includes pacman but doesn't integrate with regular git for windows install.Martens
E
26

"Git for Windows SDK" is 5.33GB compared to "Git for Windows" 691MB compared to "Portable Git" 275MB. I use the lean and mean Portable Git. At first, it seems hopeless trying to restore and use pacman in the latter two flavors of Git (msys2), because Google excluded ALL metadata files in /var/lib/pacman/local. Please read this official explanation:

https://wiki.archlinux.org/index.php/Pacman#.22Failed_to_commit_transaction_.28conflicting_files.29.22_error

Without those metadata files, you don't know the exact collection and version of the msys2 packages Google selected to build a release of those 2 flavors of Git. If you force to install or copy the current version of msys2 packages, you run the risk of version mismatch with git binaries Google built and tested.

Well, that's until I discover this file: /etc/package-versions.txt, the laundry list of matching msys2 packages and versions. Now there is a definitive source in github. Here is how I restore pacman in Portable Git (commands can be copy & paste into git-bash shell all at once):

Step 1: Run these commands to download /etc/pacman.conf and 3 packages: pacman, pacman-mirrors and msys2-keyring. See my Dec 9, 2022 comment below on why you need zstd in .xz format. These packages/versions were tested on both 32 and 64-bit Portable Git 2.38.1:

if [[ "$HOSTTYPE" == "i686" ]]; then
 pacman="
pacman-6.0.0-4-i686.pkg.tar.zst
pacman-mirrors-20210703-1-any.pkg.tar.zst
msys2-keyring-1~20210213-2-any.pkg.tar.zst
"
 zstd=zstd-1.5.0-1-i686.pkg.tar.xz
else
 pacman="
pacman-6.0.1-18-x86_64.pkg.tar.zst
pacman-mirrors-20220205-1-any.pkg.tar.zst
msys2-keyring-1~20220623-1-any.pkg.tar.zst
"
 zstd=zstd-1.5.2-1-x86_64.pkg.tar.xz
fi
for f in $pacman; do curl https://repo.msys2.org/msys/$HOSTTYPE/$f -fo ~/Downloads/$f; done
curl -L https://github.com/mcgitty/pacman-for-git/raw/main/$zstd -o ~/Downloads/$zstd
curl https://raw.githubusercontent.com/msys2/MSYS2-packages/7858ee9c236402adf569ac7cff6beb1f883ab67c/pacman/pacman.conf -o /etc/pacman.conf

Step 2: Unpack them at the root then restore pacman with these commands:

cd /
tar x --xz -vf ~/Downloads/$zstd usr
for f in $pacman; do tar x --zstd -vf ~/Downloads/$f usr etc 2>/dev/nul; done
mkdir -p /var/lib/pacman
pacman-key --init
pacman-key --populate msys2
pacman -Syu

Step 3: The next set of commands restore all matching metadata (be patient). Like the zstd packages in Step 1, this relies on my public github repo pacman-for-git to provide the git-sdk commit ID of each Portable Git release, which I'll do my best to update:

t=`grep -E 'mingw-w64-[ix_0-9]+-git ' /etc/package-versions.txt`
t=`curl -sL https://github.com/mcgitty/pacman-for-git/raw/main/version-tags.txt|grep "$t"`
[[ "$t" == "" ]] && echo "ERROR: Commit ID not logged in github pacman-for-git." && read
b=64 && [[ "$t" == *-i686-* ]] && b=32
URL=https://github.com/git-for-windows/git-sdk-$b/raw/${t##* }
cat /etc/package-versions.txt | while read p v; do d=/var/lib/pacman/local/$p-$v;
 mkdir -p $d; echo $d; for f in desc files mtree; do curl -fsSL "$URL$d/$f" -o $d/$f;
 done; [[ ! -f $d/desc ]] && rmdir $d; done

Step 4: Now, is it too much to ask for 'make' and 'zip'?

pacman -S make zip

Voilà, still just a 337MB mean little environment that can expand and upgrade!

Einhorn answered 8/12, 2020 at 17:50 Comment(17)
waiting for step 3 takes like 5 minutesPembroke
You can see it going through one package after another, right? That's what matters.Einhorn
Overall this worked for me, there were two things I had to do extra: 1) The linked packages are now compressed using ZSTD which 7zip doesn't support out of the box. I installed the Modern7z codecs, unpacked them and just tar without the decompression. 2) I needed a default /etc/pacman.conf file to get it all working which I found here: raw.githubusercontent.com/msys2/MSYS2-packages/master/pacman/…Fast
When looking at "Step 3" I found the config file in the git-sdk-64 repo which seems a lot more appropiate as it also has two additional repositories for the git tooling. /etc/pacman.conf The script looks like it downloads all the meta data for the installed packages (as listed in the repo) and adds the meta data locally.Fast
to extract the files i used peazip.github.io/index.html . To find the locations I wrote in msys bash "cd /" then "start ." then opened(dragged into) a zst file in peazip then selected only folder(s) and extracted (dragged,while clicking alt tab) only the folders (without metadata) to msys locationPembroke
Andre is correct. Msys2 switched to zstd packages in December 2020, which created a chicken and egg problem: Current releases of Portable Git does not include zsdt, which prevents running the tar commands in Step 2. Adding zsdt (egg) needs pacman (chichen) that needs zstd (egg)... Therefore, I modified the 3 links in Step 1 to download the .xz packages. Hopefully, before msys2 removes all .xz pacakge, Portable Git releases would have also included zstd.Einhorn
This is my favorite answer, because it allows one to stay with the Git package but add pacman allowing other pkg installs. I was getting signer key trust issues. i solved that by instead installing the latest keyring file like thus: msys2-keyring-1_20210213-2-any.pkg.tar.zstLox
I keep coming back to this answer 😅. After making pacman aware of all the installed packages listed in /etc/package-versions.txt I occasionally have some issues with packages I've installed outside of pacman (most likely just screwed up). (re)-Installing these with pacman --sync --overwrite '*' does force the installation. Be careful though if you're going to override some dependency it might break other stuff as well. In turn enlarging the hole you shot in the your foot.Fast
This worked well, except I had to upgrade the database using pacman-db-upgrade before I could run pacman -Syu. However pacman-db-upgrade only ran once I supplied -r /../ as additional argument, not sure why. Without that argument it prepended an additional slash to the root path, i.e. //…. Finally, I had to install pacman -S --overwrite "*" msys2-keyring.Ahithophel
I tried it, but it did not work for me. Are the instructions in this answer for the 32-bit Git Bash or for the 64-bit installation? My installer is https://github.com/git-for-windows/git/releases/download/v2.29.2.windows.2/Git-2.29.2.2-32-bit.exe.Boroughenglish
However, in MSYS2 (64-bit), pacman works fine for me.Boroughenglish
Yes, you can see from package/folder names in Step 2 and 3, this was tested in 64-bit version. I may test the 32-bit Portable Git in the spirit of lean and mean.Einhorn
Should add a -f flag to command curl -sSL "$URL$d/$f" -o $d/$f x, other it will save a 404 html file as install.Proposition
As the answer provided by @MichaelChen is getting a bit outdated and I went through this process more than 3 times now, I kind of recorded a script what I was doing (for myself), to make it working better next time. I'm afraid, I'm missing at the end some steps, but they are touch at least in the troubleshooting section. See: gist.github.com/delphym/… If it can help to anyone, it's great. Feel free to leave comment on my gists.Talaria
Thank you @DelphyM, JosiahYoder. This solution is indeed facing couple new challenges: a) Step 1 is no longer possible for 64-bit Portable Git, because Msys2 removed most of the .xz packages from its repo -- see my July 8, 2021 comment above (sorry for several typos there). This could be solved by a pre-built zstd binary. b) Step 3 is unreliable, because the git-for-windows team keeps deleting older versions of packages. For example, the latest 2.38.1 comes with bash-5.1.016-1, but git-sdk-[32|64] only has /var/lib/pacman/local/bash-5.2.012-1. I have yet to find a solution for this.Einhorn
@MichaelChen any updated instructions? as of 2023 I only get a bunch of 404's response from curlsCervantes
@AlexeiSosin I just tried PortableGit-2.41.0.3-32-bit.7z.exe, and all 4 steps work by just cut-and-paste. A few 404 errors in Step 3 are harmless. The important part is Step 4 works.Einhorn
M
23

I did not want to move from my already working Git for Windows installation so I improvised a bit:

  1. Install Git for Windows SDK somewhere else. You'll need more than 3 GB of free space for that.
  2. Copy ${git-sdk}/usr/bin/pacman.exe to ${git}/usr/bin
  3. Copy ${git-sdk}/etc/pacman.conf and ${git-sdk}/etc/pacman.d to ${git}/etc
  4. Copy ${git-sdk}/var to ${git}/

That's all. You can now open your Git Bash and run pacman -S python to install packages on your existing Git for Windows setup.

You will need write access to Git for Windows directory. Also, your pacman now thinks it has a lot of packages installed (from SDK) but it did not stop me from using it.

Multiversity answered 18/2, 2019 at 4:41 Comment(2)
I got 2 errors: error: failed to init transaction (unable to lock database) error: could not lock database: Permission denied I tried to install python and rsyncTranquillity
I had to run Git Bash as admin for pacman -S to work. Not ideal...Meeks
P
14

There seems to be a documented way to do this without having to install the Git for Windows SDK (which is very large). I was given the link to this info by PhilipOakley when I asked about all this on GitHub issue #1912.

Here's the current text of the Git for Windows GitHub wiki page about it:

##Install inside MSYS2 proper

###Please note that this scenario is not officially supported by Git for Windows

(The reason this is unsupported is that there are no volunteers to support that scenario.)

This guide assumes that you want the 64-bit version of Git for Windows.

Git for Windows being based on MSYS2, it's possible to install the git package into an existing MSYS2 installation. That means that if you are already using MSYS2 on your computer, you can use Git for Windows without running the full installer or using the portable version.

Note however that there are some caveats for going this way. Git for Windows created some patches for msys2-runtime that have not been sent upstream. (This had been planned, but it was determined in issue #284 that it would probably not be happening.) This means that you have to install Git for Windows customized msys2-runtime to have a fully working git inside MSYS2.

Here the steps to take:

  1. Open an MSYS2 terminal.

  2. Edit /etc/pacman.conf and just before [mingw32] (line #71 on my machine), add the git-for-windows packages repository:

[git-for-windows] Server = https://wingit.blob.core.windows.net/x86-64

and optionally also the MINGW-only repository for the opposite architecture (i.e. MINGW32 for 64-bit SDK):

[git-for-windows-mingw32] Server = https://wingit.blob.core.windows.net/i686

  1. Authorize signing key (this step may have to be repeated occasionally until https://github.com/msys2/msys2/issues/62 is fixed)

curl -L https://raw.githubusercontent.com/git-for-windows/build-extra/master/git-for-windows-keyring/git-for-windows.gpg | pacman-key --add - && pacman-key --lsign-key 1A9F3986

  1. Then synchronize new repository

pacboy update

  1. This updates msys2-runtime and therefore will ask you to close the window (not just exit the pacman process). Don't panic, simply close all currently open MSYS2 shells and MSYS2 programs. Double-check Task Manager and kill pacman.exe it's still running after the window is closed, because it can linger. Once all are closed, start a new terminal again.

  2. Then synchronize again (updating the non-core part of the packages):

pacboy update

  1. And finally install the Git/cURL packages:

pacboy sync git:x git-doc-html:x git-doc-man:x git-extra: curl:x

  1. Finally, check that everything went well by doing git --version in a MINGW64 shell and it should output something like git version 2.14.1.windows.1 (or newer).

@VonC pointed out in the comments that there is discussion about possible additional steps needed in this approach, around half-way down the discussion at https://github.com/git-for-windows/git/issues/2688.

And now, potentially a new way to achieve the required result too:

Piece answered 4/11, 2018 at 7:32 Comment(4)
This answer assumes that one already has an existing and full MSYS2 installation. I, however, assume that the OP has Git for Windows installed. Therefore the instructions above do not apply to the OP's setup.Heterogeneous
There is no supported or documented way to simply add pacman to GfW, as far as I understand it. You can set up a new, different GfW installation which has pacman; so you do go from having GfW to having GfW with pacman as well... but yes, it would certainly replace your existing GfW installation.Piece
I see that current versions of MSYS2 actually ship a git too (installable via pacman) packages.msys2.org/base/git. But it's probably subject some (performance?) limitations... I'm guessing related to the GVFS being discussed in WeiHua Liu's answer, alas not very clearly...Kenric
This is being discussed in github.com/git-for-windows/git/issues/2688: step 4 is incomplete.Stricken
R
5

Tested on msys2 20190524 on Windows 10 x86_64 1909 10.0.18363.752 and msys2 20220128 on Windows 11 x86_64 21H2 10.0.22000.434

Using regular Git for Windows.

  1. Install msys2 (Version 20190524 is tested.) or Git for Windows SDK. (Not fully tested, but it should work.) Both include PacMan and Git.

Using VFS for Git for Windows or Scalar for Git for Windows (Aka Microsoft git). Method #1 (with some limitations)

  1. Install VFSForGit ( https://github.com/microsoft/VFSForGit ).
  2. Install Microsoft git ( https://github.com/microsoft/git ) with this option "Git from the command line and also from 3rd-party software" enabled.
  3. Create a symbolic link named "git" in msys64\usr\bin\ pointing to C:\Program Files\Git\bin\git.exe . Execute the following command in cmd.exe, not in bash. mklink git "C:\Program Files\Git\bin\git.exe"
  4. Clone a new gvfs repo. gvfs clone https://dev.azure.com/somebody/_git/somerepo. Only the gvfs command can not be executed on msys2.
  5. Use the git command as you are on msys2 normally.

Using VFS for Git for Windows or Scalar for Git for Windows. Method #2

Virtual Filesystem for Git (formerly was GVFS. Official website https://vfsforgit.org/ ) is recommended. Version 2.22 & 2.26 are tested. Scalar (Official website https://github.com/microsoft/scalar ) is NOT recommended, nor completely tested.

  1. Install GVFS and Git for Windows with GVFS patch. Or install Scalar for Git and Git for Windows with Scalar patch. NOT BOTH on the same machine. The default installation destination is C:\Program Files\Git.
  2. Install msys2 x64 somewhere other than C:\Program Files\Git. By default, it is in C:\msys64.
  3. Copy files and subfolders of msys2 (except /etc and git binaries. The msys2 comes without git from factory.) to Git for Windows VFS edition, and copy /etc/pacman.d and /etc/pacman.conf in msys64 folder to Git installation folder, overwrite existing files. It will update msys2 and MinGW runtime to the latest version. For PacMan, the necessary files are /usr/bin/pac* ; /etc/pacman.conf ; /etc/pacman.d/ ; /var ; /usr/bin/msys* ; .(Not fully tested.)
  4. Setup terminal applications. Run C:\Program Files\Git\bin\bash.exe will launch the bash of Git for Windows. Run C:\Program Files\Git\usr\bin\bash.exe will launch bash of msys2. Configure the path of bash for terminal programs, such as Hyper Terminal. Since Git is in the system folder, terminal programs should be Run as administrator.
  5. Config $PATH the environmental variable for GVFS. Run this command in Git Bash. export PATH=$PATH:/C/Program\ Files/GVFS or export PATH=$PATH:"/C/Program Files/GVFS". Or set environmental variables for GVFS in the system property of the control panel. Relogin to take effect. Sometimes this configuration does not work, but PacMan can still run.
  6. Fix PacMan. Set executable permission for binaries. Fox example. chmod +x /usr/bin/pacman ; pacman-key --init ; pacman-key --populate msys2 ; pacman-key --refresh-keys ; pacman --sync pacman --refresh --sysupgrade --sysupgrade --overwrite "*" . Use the option --overwrite \* because some packages were installed by Git for Windows instead of PacMan.
Reckon answered 10/3, 2020 at 5:14 Comment(3)
This would be a more helpful answer if it explained what "VFS for Git for Windows" actually is and also what "Scalar for Git for Windows" is...Kenric
@vonC I'm confused -- are you endorsing VFS for Git? WeiHua seems to say very little about MSYS2 or Git For Windows SDK.Asthenosphere
@JosiahYoder I was acknowledging that an alternative to Git for Windows SDK/mysgit could be used (here through VFS) could be used to use pacman. VFS did not exist when I originally wrote my answer.Stricken
V
2

pacman is compressed using zstd now. So updated @michael-chen 's scenario (i use wsl's unzstd):

for f in pacman-6.0.1-25-x86_64.pkg.tar.zst pacman-mirrors-20221016-1-any.pkg.tar.zst msys2-keyring-1~20221024-1-any.pkg.tar.zst;   do curl https://repo.msys2.org/msys/x86_64/$f -o ~/Downloads/$f; done
cd /
tar --use-compress-program=unzstd -xvf ~/Downloads/msys2-keyring-1~20221024-1-any.pkg.tar.zst usr
tar --use-compress-program=unzstd -xvf ~/Downloads/pacman-mirrors-20221016-1-any.pkg.tar.zst etc
tar --use-compress-program=unzstd -xvf ~/Downloads/pacman-6.0.1-25-x86_64.pkg.tar.zst usr
mkdir -p /var/lib/pacman
pacman-key --init
pacman-key --populate msys2
pacman -Sy
export URL=https://github.com/git-for-windows/git-sdk-64/raw/main
cat /etc/package-versions.txt | while read p v; do d=/var/lib/pacman/local/$p-$v; mkdir -p $d; echo $d; for f in desc files install mtree; do curl -sSL "$URL$d/$f" -o $d/$f; done; done
Vitreous answered 19/11, 2022 at 8:32 Comment(1)
The premise of my solution is using "Portable Git", which is not built with the zstd or unzstd program. Therefore, your 3 tar commands won't work.Einhorn
M
1

Copying pacman from msys2 works for me, at the end most people forget that you can use pacman for self update as pacman -Syy msys/pacman

  1. Run in msys pacman -Ql pacman

  2. Copy files listed that match

  • /etc/*.conf
  • /usr/bin
  • /usr/include
  • /usr/lib
  • /var/lib
  1. And copy databases as well
  • /etc/pacman
  1. At this point I already have a working pacman,I check it with pacman -Ss pacman but there are missing files so I run self refresh pacman -Syy msys/pacman
Marshall answered 1/9, 2022 at 6:46 Comment(2)
At this point, why not use MSYS2 directly?Hellkite
Aside git-bash has improvements OS specific and MSYS2 needs menu item manual configuration, It's personal preference. This answer OP question who as me has a working git-bash. A little trick for git in MSYS, you can symlink with mklink your existing .gitconfigMarshall

© 2022 - 2024 — McMap. All rights reserved.