What does repo init and repo sync actually do?

Question

I posted this question at Android Enthusiasts but figured it was the wrong place to ask, so I deleted it from there and asking it "again" here.

---

This is such a noob question, and pardon me if it is, but I just want to understand the underlying concepts clearly. Reading repo help and Google's repo command reference page doesn't really enlighten much. I understood some bits from Google's reference page, but I still need some more clarifications.

Following the instructions on how to download android source, I executed these two commands on an Ubuntu shell: (I've taken cared of all the prerequisites for the environment.)

~/android4.2.2$ repo init -u https://android.googlesource.com/platform/manifest -b android-4.2.2_r1.2
~/android4.2.2$ repo sync -j4

After waiting half a day for repo to finish downloading, I ended up with 19G of downloaded material in android4.2.2 directory. So what exactly just happened, and why did it reach 19G when Google said I should only be expecting around 8G of source files?

Selfsupport

Answer 1

repo is a python wrapper script for git, its Google Source page defines it as

repo - The Multiple Git Repository Tool

1. repo init command initializes repo in the current directory. That's, it downloads the latest repo source and a manifest.xml file that describes the directory structure of the git repositories, and store all of these in .repo sub-directory in the current directory. In your case, you have used an optional -b argument which is used to select the branch to checkout. By default (i.e., when -b argument is not used), master branch is used.

2. repo sync updates working tree to the latest revision. That's, it synchronizes local project directories with the remote repositories specified in the manifest file. If a local project does not yet exist, it will clone a new local directory from the remote repository and set up tracking branches as specified in the manifest. If the local project already exists, it will update the remote branches and rebase any new local changes on top of the new remote changes. -j argument is used to set number of parallel jobs to execute. The default value can be defined in the manifest, and also can be overridden in command line as in your case.

why did it reach 19G when Google said I should only be expecting around 8G of source files?

That should be because besides the source files, you will get all the history of Android since the beginning of the time :)

Hope this helps.

Trapeze

-j argument sets the number of parallel jobs... does that mean it's downloading 4x more (in my case)? or is it running repo in 4 different processes much like separating the entire thing into 4 subsections and running repo in each separate section yet still downloading one whole thing? – Selfsupport

19G is twice more than 8G, why do I have to download all of Android's history when I don't need it? I just want a particular version, namely 4.2.2_r1.2. – Selfsupport

-j4 means "download 4 projects simultaneously". It has nothing to do with total volume of downloads. As for your second question, I suggest you learn a little more about git and source control management in general. – Trapeze

I see... so the way I understood it is that projects in Android source refer to the different components of the entire thing, right? I have knowledge and experience using other VCS but I'm still new to git. What I don't get is why I can't just download the final version of files at the tag/commit I've specified... I guess I'll just read more on git if you say so. – Selfsupport

@PolyBug: You can use the --shallow option to repo init to create a shallow clone that only contains limited history of each git. This obviously destroys the ability to do source code history forensics and see how and when changes were introduced (and by whom). – Concubine

@MagnusBäck Thanks, I'll try that --shallow option. I just ran a du -h -d1 report and found out that .repo/ is 14G while the rest of the source is only 6G. It's a waste of space for stuff I don't need. My only intention is to compile this thing. I don't need no history of everything since 1900s. – Selfsupport

@PolyBug, if you do not need the source code change history and other functions provided by the repo/git you can delete the .repo/ folder (and .git/ folders within project folders). – Trapeze

Fine, you get the check mark. =) I've deleted the .repo/ folder and tarred the entire thing for backup. Mind you, it's only 2.6GB, and I wasted all that time downloading 19GB of unnecessary information. Currently compiling to armv7_neon and see where it goes from there. Thanks! – Selfsupport

Does having run repo sync twice or more download the same size? Hope it'd be smart enough to download only modified parts. – Onondaga

@HiI'mFrogatto No, repo sync is incremental. Except for manifest updates, non-changed files are not downloaded again. – Trapeze