I use Nexus 3 and I want to delete my old releases. In Nexus 2 there is a scheduled task called Remove Releases From Repository
. It seems that this tasks is missing in Nexus 3.
How can we delete old release from Nexus 3 ?
Thanks
I use Nexus 3 and I want to delete my old releases. In Nexus 2 there is a scheduled task called Remove Releases From Repository
. It seems that this tasks is missing in Nexus 3.
How can we delete old release from Nexus 3 ?
Thanks
Since the release of nexus 3 you can deploy groovy scripts to the nexus manager. Simply create a new execute script task and use the following script:
import org.sonatype.nexus.repository.storage.StorageFacet;
import org.sonatype.nexus.common.app.GlobalComponentLookupHelper
import org.sonatype.nexus.repository.maintenance.MaintenanceService
import org.sonatype.nexus.repository.storage.ComponentMaintenance
import org.sonatype.nexus.repository.storage.Query;
import org.sonatype.nexus.script.plugin.RepositoryApi
import org.sonatype.nexus.script.plugin.internal.provisioning.RepositoryApiImpl
import com.google.common.collect.ImmutableList
import org.joda.time.DateTime;
import org.slf4j.Logger
// ----------------------------------------------------
// delete these rows when this script is added to nexus
RepositoryApiImpl repository = null;
Logger log = null;
GlobalComponentLookupHelper container = null;
// ----------------------------------------------------
def retentionDays = 30;
def retentionCount = 10;
def repositoryName = 'maven-releases';
def whitelist = ["org.javaee7.sample/javaee7-simple-sample", "org.javaee7.next/javaee7-another-sample"].toArray();
log.info(":::Cleanup script started!");
MaintenanceService service = container.lookup("org.sonatype.nexus.repository.maintenance.MaintenanceService");
def repo = repository.repositoryManager.get(repositoryName);
def tx = repo.facet(StorageFacet.class).txSupplier().get();
def components = null;
try {
tx.begin();
components = tx.browseComponents(tx.findBucket(repo));
}catch(Exception e){
log.info("Error: "+e);
}finally{
if(tx!=null)
tx.close();
}
if(components != null) {
def retentionDate = DateTime.now().minusDays(retentionDays).dayOfMonth().roundFloorCopy();
int deletedComponentCount = 0;
int compCount = 0;
def listOfComponents = ImmutableList.copyOf(components);
def previousComp = listOfComponents.head().group() + listOfComponents.head().name();
listOfComponents.reverseEach{comp ->
log.info("Processing Component - group: ${comp.group()}, ${comp.name()}, version: ${comp.version()}");
if(!whitelist.contains(comp.group()+"/"+comp.name())){
log.info("previous: ${previousComp}");
if(previousComp.equals(comp.group() + comp.name())) {
compCount++;
log.info("ComCount: ${compCount}, ReteCount: ${retentionCount}");
if (compCount > retentionCount) {
log.info("CompDate: ${comp.lastUpdated()} RetDate: ${retentionDate}");
if(comp.lastUpdated().isBefore(retentionDate)) {
log.info("compDate after retentionDate: ${comp.lastUpdated()} isAfter ${retentionDate}");
log.info("deleting ${comp.group()}, ${comp.name()}, version: ${comp.version()}");
// ------------------------------------------------
// uncomment to delete components and their assets
// service.deleteComponent(repo, comp);
// ------------------------------------------------
log.info("component deleted");
deletedComponentCount++;
}
}
} else {
compCount = 1;
previousComp = comp.group() + comp.name();
}
}else{
log.info("Component skipped: ${comp.group()} ${comp.name()}");
}
}
log.info("Deleted Component count: ${deletedComponentCount}");
}
https://github.com/xninjaxelitex/nexus3-cleanup-release-artifact
This script will clean your nexus repository by the specified parameters at the top.
try{ tx = repo.facet(StorageFacet.class).txSupplier().get(); tx.begin(); tx.deleteComponent(comp); }finally{ if(tx!=null){ tx.close(); } }
–
Sacaton See this script as a basis to work with:
https://gist.github.com/emexelem/bcf6b504d81ea9019ad4ab2369006e66 by emexelem
import org.sonatype.nexus.repository.storage.StorageFacet;
import org.sonatype.nexus.repository.storage.Query;
import org.joda.time.DateTime;
import org.joda.time.format.DateTimeFormat;
def fmt = DateTimeFormat.forPattern('yyyy-MM-dd HH:mm:ss');
// Get a repository
def repo = repository.repositoryManager.get('nuget-releases');
// Get a database transaction
def tx = repo.facet(StorageFacet).txSupplier().get();
// Begin the transaction
tx.begin();
// Search assets that havn't been downloaded for more than three months
tx.findAssets(Query.builder().where('last_downloaded <').param(DateTime.now().minusMonths(3).toString(fmt)).build(), [repo]).each { asset ->
def component = tx.findComponent(asset.componentId());
// Check if there is newer components of the same name
def count = tx.countComponents(Query.builder().where('name').eq(component.name()).and('version >').param(component.version()).build(), [repo]);
if (count > 0) {
log.info("Delete asset ${asset.name()} as it has not been downloaded since 3 months and has a newer version")
tx.deleteAsset(asset);
tx.deleteComponent(component);
}
}
// End the transaction
tx.commit();
So, I hit a problem with running out of diskspace as our releases/builds were stacking up, so I had a bit of a dig around at creating a script to remove old builds, and got to this:
import org.sonatype.nexus.repository.storage.StorageFacet;
import org.sonatype.nexus.repository.storage.Query;
def repositoryName = 'Integration';
def maxArtifactCount = 20;
// Get a repository
def repo = repository.repositoryManager.get(repositoryName);
// Get a database transaction
def tx = repo.facet(StorageFacet).txSupplier().get();
try {
// Begin the transaction
tx.begin();
def previousComponent = null;
def uniqueComponents = [];
tx.findComponents(Query.builder().suffix(' ORDER BY group, name').build(), [repo]).each{component ->
if (previousComponent == null || (!component.group().equals(previousComponent.group()) || !component.name().equals(previousComponent.name()))) {
uniqueComponents.add(component);
}
previousComponent = component;
}
uniqueComponents.each {uniqueComponent ->
def componentVersions = tx.findComponents(Query.builder().where('group = ').param(uniqueComponent.group()).and('name = ').param(uniqueComponent.name()).suffix(' ORDER BY last_updated DESC').build(), [repo]);
log.info(uniqueComponent.group() + ", " + uniqueComponent.name() + " size " + componentVersions.size());
if (componentVersions.size() > maxArtifactCount) {
componentVersions.eachWithIndex { component, index ->
if (index > maxArtifactCount) {
log.info("Deleting Component ${component.group()} ${component.name()} ${component.version()}")
tx.deleteComponent(component);
}
}
}
}
} finally {
// End the transaction
tx.commit();
}
This works through a repository, looking for all the components. It then works through all the versions (ordered by last updated - I couldn't figure out ordering by version number, but I think this should be ok), and then removes any over the maxArtifactCount number.
Hopefully this may be of use - and if you see any issues let me know.
We do not have this scheduled task built yet, in the meantime you can use the Delete Component functionality in the UI if you need to manually remove a release.
Old Thread but still a current topic.
After upgrading from Nexus 2.x to Nexus 3.x, the build-in function for keeping the latest X releases was sadly gone. Ofcourse there is a feature-request for that: https://issues.sonatype.org/browse/NEXUS-10821
I tried the Script by Matt Harrison (see earlier in this thread), but the migration-tool in Nexus reseted all last_updated-values to the date of the migration, sad again.
I tried to sort the releases by version via ' ORDER BY version DESC', but that resulted in a mess, where a version 3.9.0 is newer than 3.11.0 and so on, not suitable in my scenario.
So I added some helper-lists, more logoutput (credits to neil201) and the version-sorter by founddrama (https://gist.github.com/founddrama/971284) to the script - And voila, I had a well working solution!
import org.sonatype.nexus.repository.storage.StorageFacet;
import org.sonatype.nexus.repository.storage.Query;
def repositoryName = 'myrepo';
def maxArtifactCount = 42;
log.info("==================================================");
log.info(":::Cleanup script started...");
log.info("==================================================");
log.info(":::Operating on Repository: ${repositoryName}");
log.info("==================================================");
// Get a repository
def repo = repository.repositoryManager.get(repositoryName);
// Get a database transaction
def tx = repo.facet(StorageFacet).txSupplier().get();
try {
// Begin the transaction
tx.begin();
int totalDelCompCount = 0;
def previousComponent = null;
def uniqueComponents = [];
tx.findComponents(Query.builder().suffix(' ORDER BY group, name').build(), [repo]).each{component ->
if (previousComponent == null || (!component.group().equals(previousComponent.group()) || !component.name().equals(previousComponent.name()))) {
uniqueComponents.add(component);
}
previousComponent = component;
}
uniqueComponents.each {uniqueComponent ->
def componentVersions = tx.findComponents(Query.builder().where('group = ').param(uniqueComponent.group()).and('name = ').param(uniqueComponent.name()).suffix(' ORDER BY last_updated DESC').build(), [repo]);
log.info("Processing Component: ${uniqueComponent.group()}, ${uniqueComponent.name()}");
def foundVersions = [];
componentVersions.eachWithIndex { component, index ->
foundVersions.add(component.version());
}
log.info("Found Versions: ${foundVersions}")
// version-sorting by founddrama
def versionComparator = { a, b ->
def VALID_TOKENS = /.-_/
a = a.tokenize(VALID_TOKENS)
b = b.tokenize(VALID_TOKENS)
for (i in 0..<Math.max(a.size(), b.size())) {
if (i == a.size()) {
return b[i].isInteger() ? -1 : 1
} else if (i == b.size()) {
return a[i].isInteger() ? 1 : -1
}
if (a[i].isInteger() && b[i].isInteger()) {
int c = (a[i] as int) <=> (b[i] as int)
if (c != 0) {
return c
}
} else if (a[i].isInteger()) {
return 1
} else if (b[i].isInteger()) {
return -1
} else {
int c = a[i] <=> b[i]
if (c != 0) {
return c
}
}
}
return 0
}
sortedVersions = foundVersions.sort(versionComparator)
log.info("Sorted Versions: ${sortedVersions}")
removeVersions = sortedVersions.dropRight(maxArtifactCount)
totalDelCompCount = totalDelCompCount + removeVersions.size();
log.info("Remove Versions: ${removeVersions}");
log.info("Component Total Count: ${componentVersions.size()}");
log.info("Component Remove Count: ${removeVersions.size()}");
if (componentVersions.size() > maxArtifactCount) {
componentVersions.eachWithIndex { component, index ->
if (component.version() in removeVersions) {
log.info("Deleting Component: ${component.group()}, ${component.name()} ${component.version()}")
// ------------------------------------------------
// uncomment to delete components and their assets
// tx.deleteComponent(component);
// ------------------------------------------------
}
}
}
log.info("==================================================");
}
log.info(" *** Total Deleted Component Count: ${totalDelCompCount} *** ");
log.info("==================================================");
} finally {
// End the transaction
tx.commit();
}
You can fork this script on github now: https://github.com/PhilSwiss/nexus-cleanup
Although this post is quite old I had a similar requirement but wanted to remove old artifacts/releases based on the versions only, so needed to find a way to sort them and keep only the latest.
This is the script I came up with that is based on some of those examples already given here.
Note this was specifically for cleaning YUM repositories but should work for other types with little modification.
import org.sonatype.nexus.repository.storage.StorageFacet;
import org.sonatype.nexus.repository.storage.Query;
import com.google.common.collect.ImmutableList
import org.joda.time.format.DateTimeFormat;
import org.joda.time.DateTime;
import org.slf4j.Logger
///////////////////////////////
def retentionCount = 3;
def repositoryName = 'repo-name';
def whitelist = [].toArray();
///////////////////////////////
log.info("==================================================");
log.info(":::Cleanup script started...");
log.info("==================================================");
log.info(":::Operating on Repository: ${repositoryName}");
log.info("==================================================");
def repo = repository.repositoryManager.get(repositoryName);
def tx = repo.facet(StorageFacet.class).txSupplier().get();
def components = null;
try {
// Begin the transaction
tx.begin();
components = tx.browseComponents(tx.findBucket(repo));
if(components != null) {
int compCount = 0;
int delCompCount = 0;
int totalDelCompCount = 0;
def listOfComponents = ImmutableList.copyOf(components);
def previousComponent = null;
def uniqueComponents = [];
////////////////////////////////////////////////////////
final Long MAX_NUMBER = 10000L;
listOfComponents.each { component ->
if(!whitelist.contains(component.name())) {
if (previousComponent == null || !component.name().equals(previousComponent.name())) {
uniqueComponents.add(component);
}
previousComponent = component;
}
}
uniqueComponents.each { uniqueComponent ->
log.info("Starting Processing on Component: ${uniqueComponent.name()}");
def artifactVersions = [];
def numberArray = new ArrayList<Long>();
tx.findComponents(Query.builder().where('name = ').param(uniqueComponent.name()).build(), [repo]).each { component ->
def artifactVersion = component.version().replaceAll('-', '.');
artifactVersions.add(artifactVersion);
}
log.info("Found Versions: ${artifactVersions}");
for(ver in artifactVersions) {
String[] vers = ver.split('\\.');
Long num = 0;
for (int i = 0; i < vers.length; ++i) {
num = num + Long.valueOf(vers[i]) * (long) Math.pow(MAX_NUMBER, vers.length - i - 1);
}
numberArray.add(num);
}
numberArray = numberArray.sort(); //.reverse();
//log.info("Sorting the Versions: ${numberArray}");
def sortedArtifactVersions = [];
for (num in numberArray) {
List<Long> parts = new ArrayList<>();
while (num > 0) {
parts.add((long) (num % MAX_NUMBER));
num = Math.floorDiv(num, MAX_NUMBER);
}
String artifact = parts.reverse().join('.');
sortedArtifactVersions.add(artifact);
}
log.info("Sorted Versions: ${sortedArtifactVersions}");
compCount = sortedArtifactVersions.size();
def toRemoveArtifactVersions = [];
if (compCount > retentionCount) {
toRemoveArtifactVersions = sortedArtifactVersions.dropRight(retentionCount);
}
delCompCount = toRemoveArtifactVersions.size();
totalDelCompCount = totalDelCompCount + delCompCount;
log.info("Remove Versions: ${toRemoveArtifactVersions}");
log.info("Component Total Count: ${compCount}");
log.info("Component Remove Count: ${delCompCount}");
for (ver in toRemoveArtifactVersions) {
StringBuilder b = new StringBuilder(ver);
b.replace(ver.lastIndexOf("."), ver.lastIndexOf(".") + 1, "-" );
ver = b.toString();
tx.findComponents(Query.builder().where('name = ').param(uniqueComponent.name()).and('version = ').param(ver).build(), [repo]).each { component ->
log.info("Deleting Component: ${uniqueComponent.name()} ${ver}");
// ------------------------------------------------
// uncomment to delete components and their assets
// tx.deleteComponent(component);
// ------------------------------------------------
}
}
log.info("==================================================");
}
log.info(" *** Total Deleted Component Count: ${totalDelCompCount} *** ");
log.info("==================================================");
}
// End the transaction
tx.commit();
} catch(Exception e) {
log.info("Error: "+e);
tx.rollback();
} finally {
tx.close();
}
I also fall in this quite old post with the need to delete old artifacts in release repository in Nexus 3.
After a migration from Nexus 2 all LastUpdated fields was overwritten with import timestamp and not all other solutions consider that. As can be seen in artifact details from repository browse, the useful information is last_modified
field contained in Attributes -> content
.
Starting from the posted solutions by @ninjaxelite, thanks to @Phil Swiss and @neil201, I tried to found a way by considering Assets and not Components (attributes are included in Assets).
A requirement was to keep at least N released versions also if the retention period has been ended.
I deal with sorting of the extracted artifacts, where myApp-war-2.12.3.war is considered more recent than myApp-war-2.2.3.war due to literal sorting, solved using @founddrama solution as in other posts.
Consider that the following solution extracts all records and takes a lot of memory and time to sort and check all items in repository each time the script is scheduled, I do not ensure it works correctly with large repos (tested with 1.5TB in 10 mins). Any enhancement in performances it is well appreciated.
import org.sonatype.nexus.repository.storage.StorageFacet;
import org.sonatype.nexus.common.app.GlobalComponentLookupHelper
import org.sonatype.nexus.repository.maintenance.MaintenanceService
import org.sonatype.nexus.repository.storage.ComponentMaintenance
import org.sonatype.nexus.repository.storage.Query;
import org.sonatype.nexus.script.plugin.RepositoryApi
import org.sonatype.nexus.script.plugin.internal.provisioning.RepositoryApiImpl
import com.google.common.collect.Lists
import com.google.common.collect.Iterables
import org.joda.time.DateTime
import org.slf4j.Logger
// ----------------------------------------------------
// delete these rows when this script is added to nexus
//RepositoryApiImpl repository = null;
//Logger log = null;
//GlobalComponentLookupHelper container = null;
// ----------------------------------------------------
// ---------------------- CONFIG ------------------------------
// ATTENTION: This script is skilled for maven repos
// 5 Years of RetentionDays
def retentionDays = 1825;
def retentionCount = 3;
def repositoryName = 'Repository-Name';
def whitelist = ["org.javaee7.sample/javaee7-simple-sample", "org.javaee7.next/javaee7-another-sample"].toArray();
// ------------------------------------------------------------
log.info(":::Cleanup script of ${repositoryName} STARTED!");
MaintenanceService service = container.lookup("org.sonatype.nexus.repository.maintenance.MaintenanceService");
def repo = repository.repositoryManager.get(repositoryName);
def tx = repo.facet(StorageFacet.class).txSupplier().get();
def assets = null;
try {
tx.begin();
//CAREFUL!! This query extracts all Assets, do filter the search where possible
assets = tx.browseAssets(tx.findBucket(repo));
}catch(Exception e){
log.info("Error: "+e);
}finally{
if(tx!=null)
tx.close();
}
if(assets != null) {
def retentionDate = DateTime.now().minusDays(retentionDays).dayOfMonth().roundFloorCopy();
int deletedAssetsCount = 0;
int assetCount = 1;
List<Iterables> listOfAssets = Lists.newArrayList(assets);
//Base Path of each single project, it will be used for retention count (for each project it will not deleted versions at retentionCount amount)
def assetBasePath = listOfAssets.head().attributes().get('maven2').get('groupId')+"."+listOfAssets.head().attributes().get('maven2').get('artifactId');
def currentAsset = null;
def assetFilename = null;
// ----> ######## Asset List Sorting ##########
// Considering version number in filename, i.e. myApp-war-2.12.3.war is more recent than myApp-war-2.2.3.war
// version-sorting by founddrama
def versionComparator = { itemA, itemB ->
def VALID_TOKENS = /.-_/
def a = itemA.name().tokenize(VALID_TOKENS)
def b = itemB.name().tokenize(VALID_TOKENS)
for (i in 0..<Math.max(a.size(), b.size())) {
if (i == a.size()) {
return b[i].isInteger() ? -1 : 1
} else if (i == b.size()) {
return a[i].isInteger() ? 1 : -1
}
if (a[i].isInteger() && b[i].isInteger()) {
int c = (a[i] as int) <=> (b[i] as int)
if (c != 0) {
return c
}
} else if (a[i].isInteger()) {
return 1
} else if (b[i].isInteger()) {
return -1
} else {
int c = a[i] <=> b[i]
if (c != 0) {
return c
}
}
}
return 0
}
log.info("Extracted Asset List Sorting ...");
listOfAssets = listOfAssets.sort(versionComparator);
log.info("Extracted Asset List Sorted");
// <---- ######## Asset List Sorting ##########
listOfAssets.reverseEach{asset ->
if (asset.attributes().get('maven2').get('asset_kind').equals("REPOSITORY_METADATA")){
//The metadata files are skipped by default
currentAsset = null;
}else if (
asset.attributes().get('maven2').get('groupId') != null
&& asset.attributes().get('maven2').get('artifactId') != null
){
// By default the asset basePath it will considered as groupId + artifactId maven attributes
currentAsset = asset.attributes().get('maven2').get('groupId')+"."+asset.attributes().get('maven2').get('artifactId');
assetFilename = asset.attributes().get('maven2').get('version')+"."+asset.attributes().get('maven2').get('extension');
}else{
// Otherwise, for raw files (and i.e. maven-metadata.xml) the same basePath it is decoded from filename
// Obvious, it can be used this way in each case avoiding previous code, but I consider it as the second chance
// Cut from penultimate occurrence of / to the end, then replace / with .
currentAsset = asset.name().replaceAll("(.*)/([^/]+)/([^/]+)", '$1').replaceAll("/", ".");
assetFilename = asset.name().replaceAll("(.*)/([^/]+)", '$2');
}
if (currentAsset != null && !whitelist.contains(currentAsset)){
log.info("Processing Asset : ${currentAsset}, filename: ${assetFilename}");
log.info("AssetBasePath: ${assetBasePath}");
if(assetBasePath.equals(currentAsset)) {
log.info("AssetCount: ${assetCount}, Retention: ${retentionCount}");
if (assetCount > retentionCount) {
def lastModifiedDate = asset.attributes().get('content').get('last_modified');
DateTime lastModifiedDateTime = lastModifiedDate==null?null:new DateTime(lastModifiedDate);
log.debug("AssetLastModified: ${lastModifiedDateTime} - RetentionDate: ${retentionDate}");
if (retentionDate.isAfter(lastModifiedDateTime)) {
log.info("AssetLastModified ${lastModifiedDateTime} isOldestThan than RetentionDate ${retentionDate}");
log.info("Deleting: ${currentAsset}, filename: ${assetFilename}");
// ------------------------------------------------
// uncomment to delete assets
// service.deleteAsset(repo, asset);
// ------------------------------------------------
log.info("Asset DELETED");
deletedAssetsCount++;
}
}
assetCount++;
} else {
assetCount = 1;
assetBasePath = currentAsset;
}
}else{
log.info("Asset skipped due to whitelisted or Format not supported: ${asset.name()}");
}
}
log.info("TASK END - TOTAL Deleted Asset count: ${deletedAssetsCount}");
}
The solution is skilled for a Maven repository, but with some changes I hope it can be useful for all types of repo.
Remember to schedule also "Compact blob store" task in order to free disk space after artifact deletion.
© 2022 - 2024 — McMap. All rights reserved.