Downloading Eclipse plug-in update sites for offline installation

Question

A plug-in that I want to install provides an update site for installation. However, the Eclipse installation that I want to install it to is on a machine that is not connected to the Internet. Is there a way for me to access the site (HTTP, FTP, etc.) to download the files in it for offline installation?

Bibby

Answer 1

Eclipse offers a way of mirroring these sites automatically, either through the command line or through ant tasks.

Mirror based on p2 information

$eclipse_home/eclipse -application org.eclipse.equinox.p2.artifact.repository.mirrorApplication -source $1 -destination $2
$eclipse_home/eclipse -application org.eclipse.equinox.p2.metadata.repository.mirrorApplication -source $1 -destination $2

Reference: Equinox p2 repository mirroring

Mirror based on site.xml information

java -jar $eclipse_home/plugins/org.eclipse.equinox.launcher_*.jar -application org.eclipse.update.core.standaloneUpdate -command mirror -from $from -to $to

Reference: Running the update manager from the command line

You can follow the evolution of these scripts in my script repository.

Rabelais

For Kepler use this – Lidia

@Lidia - isn't that that is found in the 'Mirror based on p2 information' heading? – Rabelais

oh sorry didnt know it is the same thing. it seems for me in Kepler i cant find this $eclipse_home/plugins/org.eclipse.equinox.launcher_*.jar – Lidia

I still have the launcher jar in my Kepler installation, but I think it's optional. So I moved the 'p2' part first, since it's the supported one and looks exactly like the one you linked. Thanks for contributing :-) – Rabelais

any way to also download the dependencies? coz i've tried doing m2e offline install and it throws an error saying it couldn't find slf4j bundle – Covalence

@Covalence You should use the b3 Aggregator tool instead to also get all of the necessary dependencies for your plugin(s). See my newly added answer: https://mcmap.net/q/259813/-downloading-eclipse-plug-in-update-sites-for-offline-installation – Ultramicrochemistry

@Lucas - yes , from is an update site URL, like example.com/update and to is a local filesystem location, like /tmp/eclipse – Rabelais

Are these methods supposed to work with a site hosted in an ObjectStore type site? I can't mirror the groovy update site with either of these methods (dist.springsource.org/snapshot/GRECLIPSE/e4.7) – Hamza

Second link (Running the update manager from the command line) seems to be broken now. – Puckett

Answer 2

After struggling with mirroring for some time, I realized that it is much easier (at least, for me) to use "wget" instead.

In short:

1. Download the site:

   wget --recursive --no-parent http://url.of/updatesite

2. Take the content of the downloaded update site and move it to your offline environment

3. Add your offline update site directory to the list of available software sites via "Local..." button.

You can read about it with more details here.

Foxing

The simpliest method. My main problem was to find an url for Jubula. download.eclipse.org hides content behind "Show Directory Contents" link. Finally i guessed it http://download.eclipse.org/jubula/release/mars/ . – Fount

This also fails for things that are hosted in S3 or other object store so don't provide a crawlable website. – Hamza

Well, it mostly worked 3 years ago. Most of plugins did not use object stores back then... – Foxing

wget --recursive --no-parent download.eclipse.org/releases/kepler but it downloads only the xml file for it. How can I download the full files. – Coheir

Answer 3

Most Eclipse plug-ins can be installed without the Eclipse updater, by copying the required JARs available at the update site, into the plugins and features directories of the Eclipse installation.

In certain cases, it is necessary to start Eclipse with a -clean switch.

Here's an example of how to do this for the m2eclipse plugin:

1. The m2eclipse update site is http://m2eclipse.sonatype.org/update. Identifying the list of JARs required is the first step. This is normally found in the site.xml file; in this case, you'll find it in http://m2eclipse.sonatype.org/update/site.xml.
2. Filter the list of JARs to be downloaded to the version of the plugin that you intend to use. This can be determined by the version attribute for each "feature". If you are lucky, you'll find the description of the JAR in the category node.
3. Note the url attribute of each JAR that needs to be downloaded. This will include the subdirectory on the server where the JAR is available, and also the directory in the Eclipse installation where they need to be placed.

PS: This method is a bit hackish, but it is based off the site-map reference. But do refer to the update

Update

I haven't attempted this, but you can create a local mirror site, from where everyone else can pickup the Eclipse plug-ins. In Galileo, this can be done by running the Eclipse updater in the standalone mode via the mirror command.

Dareen

Tried your first suggestion. Was able to find and download the JARs but they seem to contain nothing. Installed them in the "dropins" directory and restarted but nothing happened too. Will try your other suggestion. BTW, the plug-in I'm trying to install is Mylyn's Generic Web Templates Connector from download.eclipse.org/tools/mylyn/update/incubator. – Bibby

For the first suggestion, I attempted downloading download.eclipse.org/tools/mylyn/update/incubator/features/…. Had no problems downloading it. – Dareen

Looking back at it, update URL was download.eclipse.org/tools/mylyn/update/incubator and the file mentioned in site.xml was features/org.eclipse.mylyn.sandbox.ui_feature_3.2.1.v20090722-0200-e3x.jar. The effective URL for the JAR is therefore download.eclipse.org/tools/mylyn/update/incubator/features/… – Dareen

Getting the features won't help. You need the plugins. I suggest trying the Eclipse Updater to create a mirror of the site. Once you do, just package it in a zip and use the zip as an update site from the update manager. – Corporator

The same plugins are anyway available from the Mylyn download page as a zip @eclipse.org/downloads/download.php?file=/tools/mylyn/update/…. That should be easiest way for the OP to distribute the plugins locally. – Dareen

Answer 4

You can mirror p2 sites using Ant tasks:

<target name="springide">
    <echo>springide</echo> 
    <p2.mirror verbose="true">
        <repository location="${REPO_HOME}/springide" name="springide" append="true"/>
        <source>
            <repository location="http://springide.org/updatesite" />
        </source>
        <iu id="Core / Spring IDE" version="" />
        <iu id="Extensions / Spring IDE" version="" />
        <iu id="Integrations / Spring IDE" version="" />
        <iu id="Resources / Spring IDE" version="" />
    </p2.mirror>
</target>

or findbugs:

<target name="findbugs">

<echo>findbugs</echo>
    <p2.mirror verbose="true">
        <repository location="${REPO_HOME}/findbugs" name="findbugs" append="true"/>
        <source>
            <repository location="http://findbugs.cs.umd.edu/eclipse/" />
        </source>
        <iu id="edu.umd.cs.findbugs.plugin.eclipse.feature.group" version="" />
    </p2.mirror>
</target>

In order for this to work you have to run ant tasks in the same JVM as eclipse.

You can find IU ID:s by opening "Software Updates" and copy it from there. In Eclipse 3.5 there should be a More... button, in 3.4 you have to click on properties button.

Unintelligible

Answer 5

I just faced this issue and resolved it by following the instructions from this guide. In summary, run the following commands on your terminal in Eclipse folder:

1. eclipsec.exe -application org.eclipse.equinox.p2.metadata.repository.mirrorApplication -source $1 -destination $2

2. eclipsec.exe -application org.eclipse.equinox.p2.artifact.repository.mirrorApplication -source $1 -destination $2

Where $1 refers to the URL to the online repository and $2 refers to the path to the local folder. E.g. to download Vrapper on my Windows Desktop:

$1 = http://vrapper.sourceforge.net/update-site/stable/

$2 = C:/Users/foo/Desktop

Transfer the folder to the machine without Internet connection. Then, launch Eclipse -> Help > Install new software. Specify the local repository (i.e. the folder that you've just transferred) for the installation. That should work.

Erst

Same as existing answer in same question: stackoverflow.com/a/1907322. Please upvote the existing answer instead and add comments for minor contributions. – Bibby

I didn't understand the existing answer when I first read it, that's why I searched for additional information and posted my answer here. The OP of #38571701 also thought that the answer "seems out-dated". But yes, I agree that I should have posted a comment instead, thanks! – Erst

This answer provides much clearer instructions than the accepted answer and is the one I used. Thanks for providing it. – Valerio

The accepted answer on this question is too terse. This answer told me everything I needed and what each variable was. I shouldn't have to be an Eclipse developer to understand the answer. +1 – Adulterer

Answer 6

Eclipse plugins usually depend on other plugins. It's kind of hard to trace the dependencies. It's better to download all dependencies using update site once, and you can distribute to other Eclipse dropins. For Eclipse 3.4 or newer, you can use dropins which is an Eclipse feature. This way you don’t have to install plugin from update site every time you have to re-install your Eclipse. Read on http://michsan.web.id/content/how-install-eclipse-plugins-offline

If you can't see the web, I'll give you some description

Prepare directory for external plugins

Create special directory to hold our beloved plugins, e.g. in /home/ichsan/eclipse-dropins we will install Maven plugin: m2eclipse.

mkdir /home/ichsan/eclipse-dropins

For now on, we'll call this directory as DROPINS

Preparing sandbox

Next, by using Git we'll create an Eclipse sandbox. The point is to install one plugin on a fresh Eclipse. Instead of installing fresh Eclipse every time we want to install a new plugin, we'd better use Git to create new branch of fresh Eclipse.

First, extract/install new Eclipse to a directory e.g. /home/ichsan/eclipse-sandbox (so that we'll find /home/ichsan/eclipse-sandbox/eclipse.ini). We call the directory as ECLIPSE_SANDBOX.

Next, commit the fresh installation. This step should be done once only.

cd $ECLIPSE_SANDBOX
git init
git add .
git commit -am "Fresh Eclipse"

Install plugin on sandbox

Now is the interesting part. Supposed we have to install m2eclipse plugin. We will install this on new Git branch so that the master branch will stay clean or stay intact.

cd $ECLIPSE_SANDBOX
git checkout -b "m2eclipse"

Now, we start the Eclipse of the ECLIPSE_SANDBOX and download the plugin. Once we're done, we shut the Eclipse down and check what new directories or files have been created (using Git). Remember that, we only care about new plugins and features directories and the contents inside them. So, we won't copy the rest into dropins.

# Prepare the m2eclipse plugin directories
mkdir -p $DROPINS/m2eclipse/eclipse/plugins
mkdir -p $DROPINS/m2eclipse/eclipse/features

cd $ECLIPSE_SANDBOX
for f in $(git status | sed "s/#\t//g" | grep -P "^plugins" ); do cp -R $f $DROPINS/m2eclipse/eclipse/plugins; done
for f in $(git status | sed "s/#\t//g" | grep -P "^features"); do cp -R $f $DROPINS/m2eclipse/eclipse/features; done

# Make the directory read only
chmod -R -w $DROPINS/m2eclipse

# Commit changes
git add .
git add -u
git commit -am "M2Eclipse plugin installed"


# Back to master branch to make Eclipse clean again and ready for other plugin installations
git checkout master
Installing the plugin

Just copy the directory of DROPINS/m2eclipse into ECLIPSE_HOME/dropins or create a symbolic link. And we're done!

cd $ECLIPSE_HOME/dropins ln -s $DROPINS/m2eclipse

Another way is to backup differences between fresh-Eclipse commit and after-plugin-installation commit.

for i in `git diff hashFreshEclipse hashPluginInstall --name-only`;do 
  if [ -f $i ]; then
    tar -r -f m2e-android.tar $i
  fi
done
gzip m2e-android.tar

Imide

Answer 7

You can get it from here https://repository.sonatype.org/content/repositories/forge-sites/m2e/1.3.0/N/1.3.0.20121023-1108/

Download all files drilling down into plugins and features. Store in a directory on your machine keeping everything in the same directory structure. Move it to a folder on your dev machine.

In Eclipse go to Help | Intall New Software... Hit the 'Add' button Hit the 'Local...' button Browse for the directory you dropped the files into. Follow the on screen instructions.

Dietetics

Answer 8

I found that the p2 mirrorApplication did not work very well on some sites and mirrored duplicate artifacts (both the pack200 and jar version). The b3 Aggregator worked much better and made it much easier to customize my update site. See the manual at: https://wiki.eclipse.org/Eclipse_b3/aggregator/manual for install instructions and details.

The basic steps I used were:

• File > New > Other... > b3 > Repository Aggregation.
• Select the Aggregation node and set the Build Root to the folder where you want the aggregated repo to go.
• Add the platforms you care about using R-click the Aggregation node > New Child > Configuration.
• Add one Validation Set, normally set its Label to main.
• If you don't need any additional bundles from the main eclipse repository (ie: http://download.eclipse.org/releases/mars), you can add it to the Validation Set as a Validation Repository, but typically you will need to add it as a Mapped Repository under a Contribution so dependencies and/or additional eclipse features can be downloaded.
• For each plugin, add a Contribution with a Label, and under that at least one Mapped Repository with a Location.
• Add Custom Category nodes to the Aggregation if you want; give them each a unique Description, Identifier, and Label.
• Expand the Repository Browser node at the bottom > expand the repo url > expand the Installable Units > Look at the features under Categories (if present) and/or under Features.
• R-click the features you want and Add to Custom Category (if you are using those), otherwise Add to parent mapped repository as > Mapped Feature.
• R-click any node > Build Aggregation.

Important Note: if you don't map any features from a given repository, that entire repository will be mirrored (the latest versions of all bundles I think, not actually everything in the repo).

I fortunately didn't have to mess with Exclusion Rules or Valid Configuration Rules, which seem to make things more complicated. However, exclusion rules may be necessary if the repositories contain bundles that have dependency conflicts, in which case one or more conflicting bundles need to be excluded.

While b3 Aggregator normally only downloads the latest version of every feature you have mapped (and it's dependencies), if you repeatedly use Build Aggregation over time as new versions are released, the obsolete versions will accumulate in your aggregation. You could use Clean then Build Aggregation but this means you will have to redownload everything. Instead, simply add another .b3aggr aggregation file, set the Build Root, add your Configurations, and add one Contribution with a Mapped Repository using the path to your local mirror's final directory. Don't map any features or create any categories. Then Build Aggregation and only the latest versions from your mirror will be aggregated!

Ultramicrochemistry