Upload artifacts to Nexus, without Maven
Asked Answered
C

14

113

I have a non-Java project that produces a versioned build artifact, and I want to upload this to a Nexus repository. Because the project isn't Java, it doesn't use Maven for builds. And I'd rather not introduce Maven/POM files just to get files into Nexus.

The links on blogs to the Nexus REST API all end up at a sign-in wall, with no "create user" link that I can see.

So, what's the best (or any reasonable) way to upload build artifacts to a Nexus repository without Maven? "bash + curl" would be great, or even a Python script.

Confusion answered 27/10, 2010 at 2:21 Comment(1)
Note, make sure you have a settings.xml in ~/.m2 with the appropriate servers & auth defined.Confusion
H
109

Have you considering using the Maven command-line to upload files?

mvn deploy:deploy-file \
    -Durl=$REPO_URL \
    -DrepositoryId=$REPO_ID \
    -DgroupId=org.myorg \
    -DartifactId=myproj \
    -Dversion=1.2.3  \
    -Dpackaging=zip \
    -Dfile=myproj.zip

This will automatically generate the Maven POM for the artifact.

Update

The following Sonatype article states that the "deploy-file" maven plugin is the easiest solution, but it also provides some examples using curl:

https://support.sonatype.com/entries/22189106-How-can-I-programatically-upload-an-artifact-into-Nexus-

Himes answered 27/10, 2010 at 19:4 Comment(5)
If only this would allow us to download files from within this zip directly but it seem that not possible if you upload it like this.Abbasid
@Abbasid It's not possible to download files from within in a zip using Maven. It's an unusual requirement and the only dependency manager I know that can do it is ivy (and it's not simple) see the following example: #3446196Penrose
I installed Nexus to make it all simpler, but what in blazes is this?.. What if I have some home-made JAR without knowledge of its dependencies? My IDE keeps complaining about missing *.pom. I hoped that Nexus already handled that for me, but NOOOOO sh...Anaximander
Is there anyway to upload the file without specifying the version and such? Like, if you only have the url, repo, file and packaging attributes?Extinguish
I don't think so. The 3 most important pieces of metadata are Group, ArtifactId and Version. The so called GAV coordinate.Penrose
A
75

Using curl:

curl -v \
    -F "r=releases" \
    -F "g=com.acme.widgets" \
    -F "a=widget" \
    -F "v=0.1-1" \
    -F "p=tar.gz" \
    -F "file=@./widget-0.1-1.tar.gz" \
    -u myuser:mypassword \
    http://localhost:8081/nexus/service/local/artifact/maven/content

You can see what the parameters mean here: https://support.sonatype.com/entries/22189106-How-can-I-programatically-upload-an-artifact-into-Nexus-

To make the permissions for this work, I created a new role in the admin GUI and I added two privileges to that role: Artifact Download and Artifact Upload. The standard "Repo: All Maven Repositories (Full Control)"-role is not enough. You won't find this in the REST API documentation that comes bundled with the Nexus server, so these parameters might change in the future.

On a Sonatype JIRA issue, it was mentioned that they "are going to overhaul the REST API (and the way it's documentation is generated) in an upcoming release, most likely later this year".

Asteroid answered 31/10, 2013 at 5:52 Comment(2)
let's say we publish from Jenkins, and only allow build users to publish to Nexus, how do you manage the plain password issue? Does Jenkins has a plugin for the upload so we can use the Jenkins credentials?Dessau
Jenkins has a plugin that manages passwords and hides them from plain sight. But they have to end up on the command line somehow.Hindenburg
O
9

You can ABSOLUTELY do this without using anything MAVEN related. I personally use the NING HttpClient (v1.8.16, to support java6).

For whatever reason, Sonatype makes it incredibly difficulty to figure out what the correct URLs, headers, and payloads are supposed to be; and I had to sniff the traffic and guess... There are some barely useful blogs/documentation there, however it is either irrelevant to oss.sonatype.org, or it's XML based (and I found out it doesn't even work). Crap documentation on their part, IMHO, and hopefully future seekers can find this answer useful. Many thanks to https://mcmap.net/q/193538/-upload-artifacts-to-nexus-without-maven for their post, as it helped a lot.

If you release somewhere other than oss.sonatype.org, just replace it with whatever the correct host is.

Here is the (CC0 licensed) code I wrote to accomplish this. Where profile is your sonatype/nexus profileID (such as 4364f3bbaf163) and repo (such as comdorkbox-1003) are parsed from the response when you upload your initial POM/Jar.

Close repo:

/**
 * Closes the repo and (the server) will verify everything is correct.
 * @throws IOException
 */
private static
String closeRepo(final String authInfo, final String profile, final String repo, final String nameAndVersion) throws IOException {

    String repoInfo = "{'data':{'stagedRepositoryId':'" + repo + "','description':'Closing " + nameAndVersion + "'}}";
    RequestBuilder builder = new RequestBuilder("POST");
    Request request = builder.setUrl("https://oss.sonatype.org/service/local/staging/profiles/" + profile + "/finish")
                             .addHeader("Content-Type", "application/json")
                             .addHeader("Authorization", "Basic " + authInfo)

                             .setBody(repoInfo.getBytes(OS.UTF_8))

                             .build();

    return sendHttpRequest(request);
}

Promote repo:

/**
 * Promotes (ie: release) the repo. Make sure to drop when done
 * @throws IOException
 */
private static
String promoteRepo(final String authInfo, final String profile, final String repo, final String nameAndVersion) throws IOException {

    String repoInfo = "{'data':{'stagedRepositoryId':'" + repo + "','description':'Promoting " + nameAndVersion + "'}}";
    RequestBuilder builder = new RequestBuilder("POST");
    Request request = builder.setUrl("https://oss.sonatype.org/service/local/staging/profiles/" + profile + "/promote")
                     .addHeader("Content-Type", "application/json")
                     .addHeader("Authorization", "Basic " + authInfo)

                     .setBody(repoInfo.getBytes(OS.UTF_8))

                     .build();
    return sendHttpRequest(request);
}

Drop repo:

/**
 * Drops the repo
 * @throws IOException
 */
private static
String dropRepo(final String authInfo, final String profile, final String repo, final String nameAndVersion) throws IOException {

    String repoInfo = "{'data':{'stagedRepositoryId':'" + repo + "','description':'Dropping " + nameAndVersion + "'}}";
    RequestBuilder builder = new RequestBuilder("POST");
    Request request = builder.setUrl("https://oss.sonatype.org/service/local/staging/profiles/" + profile + "/drop")
                     .addHeader("Content-Type", "application/json")
                     .addHeader("Authorization", "Basic " + authInfo)

                     .setBody(repoInfo.getBytes(OS.UTF_8))

                     .build();

    return sendHttpRequest(request);
}

Delete signature turds:

/**
 * Deletes the extra .asc.md5 and .asc.sh1 'turds' that show-up when you upload the signature file. And yes, 'turds' is from sonatype
 * themselves. See: https://issues.sonatype.org/browse/NEXUS-4906
 * @throws IOException
 */
private static
void deleteSignatureTurds(final String authInfo, final String repo, final String groupId_asPath, final String name,
                          final String version, final File signatureFile)
                throws IOException {

    String delURL = "https://oss.sonatype.org/service/local/repositories/" + repo + "/content/" +
                    groupId_asPath + "/" + name + "/" + version + "/" + signatureFile.getName();

    RequestBuilder builder;
    Request request;

    builder = new RequestBuilder("DELETE");
    request = builder.setUrl(delURL + ".sha1")
                     .addHeader("Authorization", "Basic " + authInfo)
                     .build();
    sendHttpRequest(request);

    builder = new RequestBuilder("DELETE");
    request = builder.setUrl(delURL + ".md5")
                     .addHeader("Authorization", "Basic " + authInfo)
                     .build();
    sendHttpRequest(request);
}

File uploads:

    public
    String upload(final File file, final String extension, String classification) throws IOException {

        final RequestBuilder builder = new RequestBuilder("POST");
        final RequestBuilder requestBuilder = builder.setUrl(uploadURL);
        requestBuilder.addHeader("Authorization", "Basic " + authInfo)

                      .addBodyPart(new StringPart("r", repo))
                      .addBodyPart(new StringPart("g", groupId))
                      .addBodyPart(new StringPart("a", name))
                      .addBodyPart(new StringPart("v", version))
                      .addBodyPart(new StringPart("p", "jar"))
                      .addBodyPart(new StringPart("e", extension))
                      .addBodyPart(new StringPart("desc", description));


        if (classification != null) {
            requestBuilder.addBodyPart(new StringPart("c", classification));
        }

        requestBuilder.addBodyPart(new FilePart("file", file));
        final Request request = requestBuilder.build();

        return sendHttpRequest(request);
    }

EDIT1:

How to get the activity/status for a repo

/**
 * Gets the activity information for a repo. If there is a failure during verification/finish -- this will provide what it was.
 * @throws IOException
 */
private static
String activityForRepo(final String authInfo, final String repo) throws IOException {

    RequestBuilder builder = new RequestBuilder("GET");
    Request request = builder.setUrl("https://oss.sonatype.org/service/local/staging/repository/" + repo + "/activity")
                             .addHeader("Content-Type", "application/json")
                             .addHeader("Authorization", "Basic " + authInfo)

                             .build();

    return sendHttpRequest(request);
}
Orff answered 21/11, 2015 at 15:54 Comment(0)
P
8

No need to use these commands .. you can directly use the nexus web Interface in order to upload your JAR using GAV parameters.

enter image description here

So it is very simple.

Phonetics answered 22/2, 2011 at 17:36 Comment(5)
A GUI doesn't help; I need to be able to upload via a command-line script used as part of a build process.Confusion
Well, it translates to a HTTP POST request, don't you think?Castrate
@YngveSneenLindal Sure, but that doesn't mean those POST arguments are a well-defined API to use publicly.Copyboy
@KenWilliams Sure, I didn't claim that either. But they will work and represent a solution, that's my point.Castrate
At least, for our Sonatype Nexus™ 2.11.1-01 I had to grant the user the privilege Artifact Upload. Unfortunately, I couldn't find anything in the docs mentioning this... (Edit: I see, Ed I has already pointed this out )Verrazano
H
6

The calls that you need to make against Nexus are REST api calls.

The maven-nexus-plugin is a Maven plugin that you can use to make these calls. You could create a dummy pom with the necessary properties and make those calls through the Maven plugin.

Something like:

mvn -DserverAuthId=sonatype-nexus-staging -Dauto=true nexus:staging-close

Assumed things:

  1. You have defined a server in your ~/.m2/settings.xml named sonatype-nexus-staging with your sonatype user and password set up - you will probably already have done this if you are deploying snapshots. But you can find more info here.
  2. Your local settings.xml includes the nexus plugins as specified here.
  3. The pom.xml sitting in your current directory has the correct Maven coordinates in its definition. If not, you can specify the groupId, artifactId, and version on the command line.
  4. The -Dauto=true will turn off the interactive prompts so you can script this.

Ultimately, all this is doing is creating REST calls into Nexus. There is a full Nexus REST api but I have had little luck finding documentation for it that's not behind a paywall. You can turn on the debug mode for the plugin above and figure it out however by using -Dnexus.verboseDebug=true -X.

You could also theoretically go into the UI, turn on the Firebug Net panel, and watch for /service POSTs and deduce a path there as well.

Holpen answered 1/7, 2011 at 22:5 Comment(0)
W
5

You can also use the direct deploy method using curl. You don't need a pom for your file for it but it will not be generated as well so if you want one, you will have to upload it separately.

Here is the command:

version=1.2.3
artifact="myartifact"
repoId=yourrepository
groupId=org.myorg
REPO_URL=http://localhost:8081/nexus

curl -u nexususername:nexuspassword --upload-file filename.tgz $REPO_URL/content/repositories/$repoId/$groupId/$artifact/$version/$artifact-$version.tgz
Wingo answered 29/7, 2016 at 1:45 Comment(2)
"artifact" not artefactDao
@Dao artifact is just a named variable here, could be named anything. Artefact can also be spelled that way (with a e instead of i), it's the British spelling. For consistency with Nexus product, it makes sense to spell it in English(US), but not mandatory.Wingo
M
4

In ruby https://github.com/RiotGames/nexus_cli A CLI wrapper around Sonatype Nexus REST calls.

Usage Example:

nexus-cli push_artifact com.mycompany.artifacts:myartifact:tgz:1.0.0 ~/path/to/file/to/push/myartifact.tgz

Configuration is done via the .nexus_cli file.

url:            "http://my-nexus-server/nexus/"
repository:     "my-repository-id"
username:       "username"
password:       "password"
Middlebrow answered 31/10, 2013 at 21:26 Comment(0)
A
3

for those who need it in Java, using apache httpcomponents 4.0:

public class PostFile {
    protected HttpPost httppost ;
    protected MultipartEntity mpEntity; 
    protected File filePath;

    public PostFile(final String fullUrl, final String filePath){
        this.httppost = new HttpPost(fullUrl);
        this.filePath = new File(filePath);        
        this.mpEntity = new MultipartEntity();
    }

    public void authenticate(String user, String password){
        String encoding = new String(Base64.encodeBase64((user+":"+password).getBytes()));
        httppost.setHeader("Authorization", "Basic " + encoding);
    }
    private void addParts() throws UnsupportedEncodingException{
        mpEntity.addPart("r", new StringBody("repository id"));
        mpEntity.addPart("g", new StringBody("group id"));
        mpEntity.addPart("a", new StringBody("artifact id"));
        mpEntity.addPart("v", new StringBody("version"));
        mpEntity.addPart("p", new StringBody("packaging"));
        mpEntity.addPart("e", new StringBody("extension"));

        mpEntity.addPart("file", new FileBody(this.filePath));

    }

    public String post() throws ClientProtocolException, IOException {
        HttpClient httpclient = new DefaultHttpClient();
        httpclient.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1);
        addParts();
        httppost.setEntity(mpEntity);
        HttpResponse response = httpclient.execute(httppost);

        System.out.println("executing request " + httppost.getRequestLine());
        System.out.println(httppost.getEntity().getContentLength());

        HttpEntity resEntity = response.getEntity();

        String statusLine = response.getStatusLine().toString();
        System.out.println(statusLine);
        if (resEntity != null) {
            System.out.println(EntityUtils.toString(resEntity));
        }
        if (resEntity != null) {
            resEntity.consumeContent();
        }
        return statusLine;
    }
}
Adactylous answered 29/10, 2015 at 12:35 Comment(1)
first post. I've tryed to add higlighting for java but coudn't get it.Adactylous
T
1

If you need a convenient command line interface or python API, look at repositorytools

Using it, you can upload artifact to nexus with command

artifact upload foo-1.2.3.ext releases com.fooware

To make it work, you will also need to set some environment variables

export REPOSITORY_URL=https://repo.example.com
export REPOSITORY_USER=admin
export REPOSITORY_PASSWORD=mysecretpassword
Tref answered 11/5, 2016 at 15:12 Comment(0)
L
1

For recent versions of Nexus OSS (>= 3.9.0)

https://support.sonatype.com/hc/en-us/articles/115006744008-How-can-I-programmatically-upload-files-into-Nexus-3-

Example for versions 3.9.0 to 3.13.0:

curl -D - -u user:pass -X POST "https://nexus.domain/nexus/service/rest/beta/components?repository=somerepo" -H "accept: application/json" -H "Content-Type: multipart/form-data" -F "raw.directory=/test/" -F "[email protected];type=application/json" -F "raw.asset1.filename=test.txt"
Lookeron answered 8/4, 2019 at 11:59 Comment(0)
R
0

You can manually upload the artifact's by clicking on upload artifacts button in the Nexus server and provide the necessary GAV properties for uploading(it's generally the file structure for storing the artifact)

Rustice answered 24/3, 2019 at 16:46 Comment(0)
O
0

If you want to upload to your repository in Nexus in 2023, follow steps as shown below. Nexus updated their API endpoints. (The example has been tested with a tar/zip file from a Node project, like OP requested)

Note make sure your username and password are encoded with base64, else this won't work.

Transform your username and password into a base64 string.

echo -n '<your username>:<your password>' | base64

Add the output of that echo in the Authorization Header. The asset name in this case is the file you want to upload.

curl -X 'POST' \
  'http://<Nexus>/service/rest/v1/components?repository=<repo name>' \
  -H 'accept: application/json' \
  -H 'Content-Type: multipart/form-data' \
  -H 'Authorization: Basic <username:pass base64 str>=' \
  -F 'r.asset=@<assetname>'
Orit answered 17/7, 2023 at 18:37 Comment(0)
R
-1

You can use curl instead.

version=1.2.3
artifact="artifact"
repoId=repositoryId
groupId=org/myorg
REPO_URL=http://localhost:8081/nexus

curl -u username:password --upload-file filename.tgz $REPO_URL/content/repositories/$repoId/$groupId/$artifact/$version/$artifact-$version.tgz
Radiolarian answered 22/6, 2017 at 15:5 Comment(1)
this answer is not correct. With curl, the groupId should be represented as org/myorg (substitute the dot "." with slash "/")Tarantula
C
-1

@Adam Vandenberg For Java code to POST to Nexus. https://github.com/manbalagan/nexusuploader

public class NexusRepository implements RepoTargetFactory {

    String DIRECTORY_KEY= "raw.directory";
    String ASSET_KEY= "raw.asset1";
    String FILENAME_KEY= "raw.asset1.filename";

    String repoUrl;
    String userName;
    String password;

    @Override
    public void setRepoConfigurations(String repoUrl, String userName, String password) {
        this.repoUrl = repoUrl;
        this.userName = userName;
        this.password = password;
    }

    public String pushToRepository() {
        HttpClient httpclient = HttpClientBuilder.create().build();
        HttpPost postRequest = new HttpPost(repoUrl) ;
        String auth = userName + ":" + password;
        byte[] encodedAuth = Base64.encodeBase64(
                auth.getBytes(StandardCharsets.ISO_8859_1));
        String authHeader = "Basic " + new String(encodedAuth);
        postRequest.setHeader(HttpHeaders.AUTHORIZATION, authHeader);
        try
        {
            byte[] packageBytes = "Hello. This is my file content".getBytes();
            MultipartEntityBuilder multipartEntityBuilder = MultipartEntityBuilder.create();
            InputStream packageStream = new ByteArrayInputStream(packageBytes);
            InputStreamBody inputStreamBody = new InputStreamBody(packageStream, ContentType.APPLICATION_OCTET_STREAM);
            multipartEntityBuilder.addPart(DIRECTORY_KEY, new StringBody("DIRECTORY"));
            multipartEntityBuilder.addPart(FILENAME_KEY, new StringBody("MyFile.txt"));
            multipartEntityBuilder.addPart(ASSET_KEY, inputStreamBody);
            HttpEntity entity = multipartEntityBuilder.build();
            postRequest.setEntity(entity); ;

            HttpResponse response = httpclient.execute(postRequest) ;
            if (response != null)
            {
                System.out.println(response.getStatusLine().getStatusCode());
            }
        }
        catch (Exception ex)
        {
            ex.printStackTrace() ;
        }
        return null;
    }

}
Colfin answered 18/9, 2019 at 16:53 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.