java 8 how to get distinct list on more than one property

Question

How can one get the distinct (distinct based on two property) list from a list of objects. for example let there are list of objects with property name and price. Now how can I get a list with distinct name or price.
suppose

list<xyz> l1 = getlist(); // getlist will return the list.

Now let l1 has the following properties(name, price) :-
n1, p1
n1, p2
n2, p1
n2, p3

Now after the filter the list should be-
n1, p1
n2, p3

I tried solving like this -

public List<xyz> getFilteredList(List<xyz> l1) {

        return l1
                .stream()
                .filter(distinctByKey(xyz::getName))
                .filter(distinctByKey(xyz::getPrice))
                .collect(Collectors.toList());
    }

    private static <T> Predicate<T> distinctByKey(Function<? super T, Object> keyExtractor) {
        Map<Object,Boolean> seen = new ConcurrentHashMap<>();
        return t -> seen.putIfAbsent(keyExtractor.apply(t), Boolean.TRUE) == null;
    }

Now the problem is when i did filter on name the list return would be -
n1, p1
n2, p1

and then it would have run filter on price which return -
n1, p1

which is not the expected result.

Evalyn

Try flipping your filer statements once. – Slapstick

if i flip the statement then the case will be similar if list is - (n1, p1), (n2, p1), (n1,p2),(n3,p2) then after 1st filter on price will give- (n1, p1), (n1,p2); then on filter on name will give (n1,p1) – Evalyn

Here are some options: https://mcmap.net/q/64785/-java-lambda-stream-distinct-on-arbitrary-key-duplicate – Reborn

@Reborn I don't think the problem is to get the distinct by key, but instead is how to get first the first mapping: n1, p1 and then the second n2, p3. at least that is not clear for me – Strumpet

@Strumpet they want a list of items that are distinct by both name AND price, which the more I look at it, does not have a trivial solution. – Slapstick

@Slapstick not even that.. it looks like some more like by price or name, but not already seen... I don't even know how to phrase that – Strumpet

@Strumpet I think by price and name, both not already seen is what they're going for. And in that case, you have to look at both fields simultaneously, not one then the other, like in the OP. – Slapstick

@Slapstick besides keeping two ConcurentHashMaps, one for value and one for price, i see no other way... – Strumpet

@Strumpet I was thinking somewhere along those lines as well. Then you'd have to compare the maps against each other to get the final result. – Slapstick

Answer 1

Almost verbatim from Stuart Marks' answer:

import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.function.Function;
import java.util.function.Predicate;
import java.util.stream.Collectors;

class Class {

  public static <T> Predicate<T> distinctByKey(Function<? super T, Object> keyExtractor) {
    Map<Object, Boolean> seen = new ConcurrentHashMap<>();
    return t -> seen.putIfAbsent(keyExtractor.apply(t), Boolean.TRUE) == null;
  }

  private static List<Pojo> getList() {
    return Arrays.asList(
      new Pojo("123", 100),
      new Pojo("123", 100),
      new Pojo("123", 100),
      new Pojo("456", 200)
    );
  }

  public static void main(String[] args) {

    System.out.println(getList().stream()
      // extract a key for each Pojo in here. 
      // concatenating name and price together works as an example
      .filter(distinctByKey(p -> p.getName() + p.getPrice()))
      .collect(Collectors.toList()));
  }

}

class Pojo {
  private final String name;
  private final Integer price;

  public Pojo(final String name, final Integer price) {
    this.name = name;
    this.price = price;
  }

  public String getName() {
    return name;
  }

  public Integer getPrice() {
    return price;
  }

  @Override
  public String toString() {
    final StringBuilder sb = new StringBuilder("Pojo{");
    sb.append("name='").append(name).append('\'');
    sb.append(", price=").append(price);
    sb.append('}');
    return sb.toString();
  }
}

This main method yields:

[Pojo{name='123', price=100}, Pojo{name='456', price=200}]

Edit

Made price an int per Eugene's prompting.

Note: that you could use something more interesting as a key if you wanted to flesh it out:

import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.function.Function;
import java.util.function.Predicate;
import java.util.stream.Collectors;

class Class {

  public static <T> Predicate<T> distinctByKey(Function<? super T, Object> keyExtractor) {
    Map<Object, Boolean> seen = new ConcurrentHashMap<>();
    return t -> seen.putIfAbsent(keyExtractor.apply(t), Boolean.TRUE) == null;
  }

  private static List<Pojo> getList() {
    return Arrays.asList(
      new Pojo("123", 100),
      new Pojo("123", 100),
      new Pojo("123", 100),
      new Pojo("456", 200)
    );
  }

  private static class NameAndPricePojoKey {
    final String name;
    final int price;

    public NameAndPricePojoKey(final Pojo pojo) {
      this.name = pojo.getName();
      this.price = pojo.getPrice();
    }

    @Override
    public boolean equals(final Object o) {
      if (this == o) return true;
      if (o == null || getClass() != o.getClass()) return false;

      final NameAndPricePojoKey that = (NameAndPricePojoKey) o;

      if (price != that.price) return false;
      return name != null ? name.equals(that.name) : that.name == null;

    }

    @Override
    public int hashCode() {
      int result = name != null ? name.hashCode() : 0;
      result = 31 * result + price;
      return result;
    }
  }

  public static void main(String[] args) {

    System.out.println(getList().stream()
      // extract a key for each Pojo in here. 
      .filter(distinctByKey(NameAndPricePojoKey::new))
      .collect(Collectors.toList()));
  }

}

class Pojo {
  private String name;
  private Integer price;
  private Object otherField1;
  private Object otherField2;

  public Pojo(final String name, final Integer price) {
    this.name = name;
    this.price = price;
  }

  public String getName() {
    return name;
  }

  public void setName(final String name) {
    this.name = name;
  }

  public Integer getPrice() {
    return price;
  }

  public void setPrice(final Integer price) {
    this.price = price;
  }

  public Object getOtherField1() {
    return otherField1;
  }

  public void setOtherField1(final Object otherField1) {
    this.otherField1 = otherField1;
  }

  public Object getOtherField2() {
    return otherField2;
  }

  public void setOtherField2(final Object otherField2) {
    this.otherField2 = otherField2;
  }

  @Override
  public boolean equals(final Object o) {
    if (this == o) return true;
    if (o == null || getClass() != o.getClass()) return false;

    final Pojo pojo = (Pojo) o;

    if (name != null ? !name.equals(pojo.name) : pojo.name != null) return false;
    if (price != null ? !price.equals(pojo.price) : pojo.price != null) return false;
    if (otherField1 != null ? !otherField1.equals(pojo.otherField1) : pojo.otherField1 != null) return false;
    return otherField2 != null ? otherField2.equals(pojo.otherField2) : pojo.otherField2 == null;

  }

  @Override
  public int hashCode() {
    int result = name != null ? name.hashCode() : 0;
    result = 31 * result + (price != null ? price.hashCode() : 0);
    result = 31 * result + (otherField1 != null ? otherField1.hashCode() : 0);
    result = 31 * result + (otherField2 != null ? otherField2.hashCode() : 0);
    return result;
  }

  @Override
  public String toString() {
    final StringBuilder sb = new StringBuilder("Pojo{");
    sb.append("name='").append(name).append('\'');
    sb.append(", price=").append(price);
    sb.append(", otherField1=").append(otherField1);
    sb.append(", otherField2=").append(otherField2);
    sb.append('}');
    return sb.toString();
  }
}

Reborn

I don't think this quite does it. This gets a list of all distinct combinations of price and name together, which might be a good first step. Price is probably a double, but since it gets concatenated with the name, it shouldn't make a difference anyway. – Slapstick

Given that distinctByKey() returns a Predicate<T>, can and() / or() methods be used to compose distinctByKey with another predicate? E.g. when I use distinctByKey(e -> e.getName()).or() it tells that or() expects Predicate(? super Object) and not ? super T. – Cockup

NB if I use my type instead of the generic parameter T, e.g. Predicate<Pojo> distinctByKey(Function<? super Pojo, ? > keyExtractor), then it resolves or(Predicate <? super Pojo>) and I can compose multiple predicates using and() / or(). – Cockup

Answer 2

I'd go for something like this, which is fairly simple and flexible, and builds on your example:

public static <T> List<T> distinctList(List<T> list, Function<? super T, ?>... keyExtractors) {

    return list
        .stream()
        .filter(distinctByKeys(keyExtractors))
        .collect(Collectors.toList());
}

private static <T> Predicate<T> distinctByKeys(Function<? super T, ?>... keyExtractors) {

    final Map<List<?>, Boolean> seen = new ConcurrentHashMap<>();

    return t -> {

        final List<?> keys = Arrays.stream(keyExtractors)
            .map(ke -> ke.apply(t))
            .collect(Collectors.toList());

        return seen.putIfAbsent(keys, Boolean.TRUE) == null;

    };

}

This can then be called in the following manner:

final List<Xyz> distinct = distinctList(list, Xyz::getName, Xyz::getPrice)

Pistil

Answer 3

Here is my solution based on the class Item which defines a name and a price:

public class Item {

    public String name;
    public double price;

    Item(String name, double price) {
        this.name = name;
        this.price = price;
    }
}

The requirement is to obtain only Items from a given List<Item> which have distinct names and distinct prices, in the order in which they occur.

I catch this requirement of being distinct by a class ItemWrapper:

public class ItemWrapper {

    Item item;

    ItemWrapper(Item item) {
        this.item = item;
    }

    @Override
    public boolean equals(Object obj) {
        if (!(obj instanceof ItemWrapper)) return false;
        ItemWrapper other = (ItemWrapper) obj;
        return  Objects.equals(item.name, other.item.name) ||
                item.price == other.item.price;
    }

    @Override
    public int hashCode() {
        return 1;
    }
}

Now we have everything in place to filter a given List<Item> of items:

List<Item> items = Arrays.asList(
        new Item("name-1", 100.00),
        new Item("name-1", 555.00),
        new Item("name-2", 100.00),
        new Item("name-2", 999.99),
        new Item("name-3", 100.00),
        new Item("name-4", 555.00),
        new Item("name-5", 999.99)
);

as following:

items.stream()
     .map(item -> new ItemWrapper(item))
     .distinct()
     .map(wrapper -> wrapper.item)
     .collect(Collectors.toList());
}

The items captured are:

• name=name-1, price=100.0
• name=name-2, price=999.99
• name=name-4, price=555.0

Lepidote

what is wrapper? – Mather

The class ItemWrapper implements the method equals(Object obj) which enforces the requirement to treat two Items as equal if they share either the same name or the same price. I moved this logic into a separate class because it did not seam to be natural to the domain class Item. Does this comment answer your question? – Lepidote

Answer 4

Why not that way:

@Value
static class User {
    String firstName;
    String lastName;
    int age;
}

public static void main(String[] args) {
    var user1 = new User("firstName1", "lastName1", 20);
    var user2 = new User("firstName1", "lastName2", 20);
    var user3 = new User("firstName3", "lastName3", 30);

    var users = List.of(user1, user2, user3);

    var uniqueByFirstNameAndAge = users
        .stream()
        .collect(Collectors.toMap(usr -> Set.of(usr.getFirstName(), usr.getAge()), Function.identity(), (usr1, usr2) -> usr1))
        .values();

    System.out.println(uniqueByFirstNameAndAge);


}

Almswoman

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. – Lightfingered

Note for readers: Although this answer is not properly explained, it's correct, and does the task in a really efficient and succinct way. It creates a Set using both attributes as keys. Set does not allow duplicates, and it discards usr2 when a duplicate happens. It does not require adition functions, or modifying the object. – Colbycolbye