Storing EnumSet in a database?

Question

So in C++/C# you can create flags enums to hold multiple values, and storing a single meaningful integer in the database is, of course, trivial.

In Java you have EnumSets, which appear to be quite a nice way to pass enums around in memory, but how do you output the combined EnumSet to an integer for storage? Is there another way to approach this?

Frivol

Answer 1

// From Adamski's answer
public static <E extends Enum<E>> int encode(EnumSet<E> set) {
    int ret = 0;

    for (E val : set) {
        ret |= 1 << val.ordinal();
    }

    return ret;
}

@SuppressWarnings("unchecked")
private static <E extends Enum<E>> EnumSet<E> decode(int code,
        Class<E> enumType) {
    try {
        E[] values = (E[]) enumType.getMethod("values").invoke(null);
        EnumSet<E> result = EnumSet.noneOf(enumType);
        while (code != 0) {
            int ordinal = Integer.numberOfTrailingZeros(code);
            code ^= Integer.lowestOneBit(code);
            result.add(values[ordinal]);
        }
        return result;
    } catch (IllegalAccessException ex) {
        // Shouldn't happen
        throw new RuntimeException(ex);
    } catch (InvocationTargetException ex) {
        // Probably a NullPointerException, caused by calling this method
        // from within E's initializer.
        throw (RuntimeException) ex.getCause();
    } catch (NoSuchMethodException ex) {
        // Shouldn't happen
        throw new RuntimeException(ex);
    }
}

Lipkin

You don't need to call values() by reflection. Class has a method getEnumConstants which returns the same. – Exasperate

Answer 2

Storing the ordinal as a representation of the EnumSet is not a good idea. The ordinal numbers depend on the order of the definition in the Enum class (a related discussion is here). Your database may be easily broken by a refactoring that changes the order of Enum values or introduces new ones in the middle.

You have to introduce a stable representation of individual enum values. These can be int values again and represented in the proposed way for the EnumSet.

Your Enums can implement interfaces so the stable represenation can be directly in the enum value (adapted from Adamski):

interface Stable{
    int getStableId();
}
public enum X implements Stable {
    A(1), B(2);

    private int stableId;

    X(int id){
        this.stableId = id;
    }

    @Override public int getStableId() {
        return stableId;
    }
}

adapted from Adamski's code:

public <E extends Stable> int encode(EnumSet<E> set) {
  int ret = 0;

  for (E val : set) {
    ret |= (1 << val.getStableId());
  }

  return ret;
}

Stilton

That can be worked around by ensuring any new enum values are added to the end of the enum, not inserted in the middle (and not exceeding 32 values.) – Lipkin

@Lipkin Supposing this were a good idea (it's not but anyway) how can you remove an enum value? – Stilton

Firstly you can mark individual enum values @Deprecated. But if you have instances of it in the database then you should not remove it. If you are confident there are no instances in the database then you can reuse the ordinal later. – Lipkin

@Lipkin So there are only some simple rules and everything works perfect: 1. Dont't change the order of enum values. 2. Don't remove enum values (that are no longer used) mark them as @Deprecated. 3. Only add values at the end unless there is a @Deprecated enum value then replace this with the new one. (Ignoring the rules for legacy enum values. Sadly, the @Deprecated is already burned.) Nice! </irony> – Stilton

Even following these rules can be error prone if, for example you have multiple versions of the code out there (e.g. multiple database reader / writer processes); It may not always be feasible to redeploy your enum definition across the board if you decide to change it. – Puerilism

Answer 3

Providing your enum fits into an int (i.e. there are <= 32 values) I would roll my own implementation by using each enum's ordinal value; e.g.

public <E extends Enum<E>> int encode(EnumSet<E> set) {
  int ret = 0;

  for (E val : set) {
    // Bitwise-OR each ordinal value together to encode as single int.
    ret |= (1 << val.ordinal());
  }

  return ret;
}

public <E extends Enum<E>> EnumSet<E> decode(int encoded, Class<E> enumKlazz) {
  // First populate a look-up map of ordinal to Enum value.
  // This is fairly disgusting: Anyone know of a better approach?
  Map<Integer, E> ordinalMap = new HashMap<Integer, E>();
  for (E val : EnumSet.allOf(enumKlazz)) {
    ordinalMap.put(val.ordinal(), val);
  }

  EnumSet<E> ret= EnumSet.noneOf(enumKlazz);
  int ordinal = 0;

  // Now loop over encoded value by analysing each bit independently.
  // If the bit is set, determine which ordinal that corresponds to
  // (by also maintaining an ordinal counter) and use this to retrieve
  // the correct value from the look-up map.
  for (int i=1; i!=0; i <<= 1) {
    if ((i & encoded) != 0) {
      ret.add(ordinalMap.get(ordinal));
    }

    ++ordinal;
  }

  return ret;
}

Disclaimer: I haven't tested this!

EDIT

As Thomas mentions in the comments the ordinal numbers are unstable in that any change to your enum definition within your code will render the encodings in your database corrupt (e.g. if you insert a new enum value in the middle of your existing definition). My approach to solving this problem is to define an "Enum" table per enumeration, containing a numerical ID (not the ordinal) and the String enum value. When my Java application starts, the first thing the DAO layer does is to read each Enum table into memory and:

• Verify that all String enum values in the database match the Java definition.
• Initialise a Bi-directional map of ID to enum and vice-versa, which I then use whenever I persist an enum (In other words, all "data" tables reference the database-specific Enum ID, rather than store the String value explicitly).

This is much cleaner / more robust IMHO than the ordinal approach I describe above.

Puerilism

Interesting (+1). How would you extract those values and reconstruct enums again? – Gaut

This will also work for long or BigInteger (I don't recommend BitSet because that cannot be efficiently converted to bytes.) – Lipkin

Thanks adamski - can you just explain the syntax <E extends Enum<E>> encode(EnumSet<E> set) i havent seen that before? – Frivol

This will not give the result you are after, you probably meant to say ret |= 1 << val.ordinal() :-) – Pittance

The ordinal numbers are not stable. One change of the order of enum values and your database is corrupted. – Stilton

@Thomas: You're correct - Let me add something about that too. – Puerilism

You should correct you boundary as well 1 << 32 == 1 << 0. Yes I'm a nitpicker. – Stilton

@BalusC: I've now included a decode method, but I'm not proud of this code (!) ... and it's not an approach I usually take (see EDIT section for my preferred approach). – Puerilism

@finnw: The cast is (was) valid but there's no EnumSet.of(...) method that takes an array so I've changed to use copyOf. – Puerilism

You should use E[] values = enumKlazz.getEnumConstants() instead of Map<Integer, E>. – Exasperate

Answer 4

It struck me as a surprise that nobody was suggesting a well-maintained library instead of writing your own. The above answer is spot on and educational but it just encourages people to copy and paste code around (then mostly forget the credits).

Here's my 2 cents:

EnumSet<YourEnum> mySet = EnumSet.of(YourEnum.FIRST);
long vector = EnumUtils.generateBitVector(YourEnum.class, mySet);
EnumSet<YourEnum> sameSet = EnumUtils.processBitVector(YourEnum.class, vector);

See https://commons.apache.org/proper/commons-lang/apidocs/org/apache/commons/lang3/EnumUtils.html

Thermos

My favourite solution! – Uxorial

Answer 5

If you look in the source for RegularEnumSet, which is the implementation for Enum's <= 64 members, you will see that it contains:

/**
 * Bit vector representation of this set.  The 2^k bit indicates the
 * presence of universe[k] in this set.
 */
private long elements = 0L;

elements is a bit-mask where the bit positions equal the enum ordinals, which is exactly what you need. However this attribute is not made availlable through a getter or setter as that would not match the equivalent accessors for the JumboEnumSet.

It is not one of the nicest solutions, but if simplicity and speed is what you are after, you could create 2 static utility methods that retrieve and set the elements attribute using reflection.

For me, I would probably just setup a constants class holding the enum values as integer constants where I can be sure which enum gets assigned what bit.

Pittance

Answer 6

EnumSet implements Serializable, but there's a lot of overhead if you use that (it is written as an array of IDs, not a BitSet as you might expect, plus the object stream header.)

Lipkin

So how do you implement bit flags in java? Is there something I'm missing? – Frivol

It's represented as bit flags in memory, but there are no built-in methods to do I/O in that format or to extract the long values used to store the bits. I would roll your own as Adamski suggested. – Lipkin

enumset uses bit flags internally, it's the safe replacement for bitflags (see Effective Java) – Tannatannage

@Bishiboosh, but the OP wants those bit flags externally, to store them in the database. – Lipkin

Answer 7

This is an old post that I found helpful, but with Java 8 or newer I've adapted the solution posted by @finnw into this interface:

public interface BitMaskable {

  int getBitMaskOrdinal();

  static int bitMaskValue(Set<? extends BitMaskable> set) {
    int mask = 0;

    for (BitMaskable val : set) {
      mask |= (1 << val.getBitMaskOrdinal());
    }

    return mask;
  }

  static <E extends Enum<E> & BitMaskable> Set<E> valueOfBitMask(int mask, Class<E> enumType) {
    E[] values = enumType.getEnumConstants();
    EnumSet<E> result = EnumSet.noneOf(enumType);
    Map<Integer, E> ordinalCache = null;
    while (mask != 0) {
      int ordinal = Integer.numberOfTrailingZeros(mask);
      mask ^= Integer.lowestOneBit(mask);
      E value = null;
      if (ordinalCache != null) {
        value = ordinalCache.get(ordinal);
      }
      if (value == null) {
        for (E e : values) {
          if (e.getBitMaskOrdinal() == ordinal) {
            value = e;
            break;
          }
          // if there are more values to decode and e has a higher
          // ordinal than what we've seen, cache that for later
          if (mask != 0 && e.getBitMaskOrdinal() > ordinal) {
            if (ordinalCache == null) {
              ordinalCache = new HashMap<>(values.length);
            }
            ordinalCache.put(e.getBitMaskOrdinal(), e);
          }
        }
      }
      if (value != null) {
        result.add(value);
      }
    }
    return result;
  }

}

Usage for an enum like this (note the bmOrdinal values are out-of-order from the built-in enum ordinal values):

public enum BitMaskEnum implements BitMaskable {
  A(0),
  B(2),
  C(1),
  D(3);

  private int bmOrdinal;

  private BitMaskEnum(int bmOrdinal) {
    this.bmOrdinal = bmOrdinal;
  }

  @Override
  public int getBitMaskOrdinal() {
    return bmOrdinal;
  }
}

is then along these lines:

// encode as bit mask; result == 5
int result = BitMaskable.bitMaskValue(EnumSet.of(BitMaskEnum.A, BitMaskEnum.B));

// decode into set; result contains A & B
Set<BitMaskEnum> result = BitMaskable.valueOfBitMask(5, BitMaskEnum.class);

Rebuttal

Answer 8

With the methods given in the answers it is possible to convert a integer to an EnumSet and vice versa. But I found that this is often error prone. Especially when you get negative values as java only has signed int and long. So if you plan to do such conversions on all sets of enums you might want to use a data structure that already supports this. I have created such a data structure, that can be used just like a BitSet or an EnumSet, but it also has methods such as toLong() and toBitSet(). Note that this requires Java 8 or newer.

Here's the link: http://claude-martin.ch/enumbitset/

Ferrante

Answer 9

Without going into the debate about pros and cons of ordinal values in the database - I posted a possible answer to the given question here: JPA map collection of Enums

The idea is to create a new PersistentEnumSet which uses the implementation of java.util.RegularEnumSet, but offers the elements bitmask to JPA.

That one can than be used in an embeddable:

@Embeddable
public class InterestsSet extends PersistentEnumSet<InterestsEnum> {
  public InterestsSet() {
    super(InterestsEnum.class);
  }
}

And that set is used in the entity:

@Entity
public class MyEntity {
  // ...
  @Embedded
  private InterestsSet interests = new InterestsSet();
}

For further comments see my answer over there.

Liberec

Answer 10

I have done some changes on finnw's code, so it works with enumerations having up to 64 items.

// From Adamski's answer
public static <E extends Enum<E>> long encode(EnumSet<E> set) {
    long ret = 0;

    for (E val : set) {
        ret |= 1L << val.ordinal();
    }

    return ret;
}

@SuppressWarnings("unchecked")
public static <E extends Enum<E>> EnumSet<E> decode(long code,
                                                     Class<E> enumType) {
    try {
        E[] values = (E[]) enumType.getMethod("values").invoke(null);
        EnumSet<E> result = EnumSet.noneOf(enumType);
        while (code != 0) {
            int ordinal = Long.numberOfTrailingZeros(code);
            code ^= Long.lowestOneBit(code);
            result.add(values[ordinal]);
        }
        return result;
    } catch (IllegalAccessException ex) {
        // Shouldn't happen
        throw new RuntimeException(ex);
    } catch (InvocationTargetException ex) {
        // Probably a NullPointerException, caused by calling this method
        // from within E's initializer.
        throw (RuntimeException) ex.getCause();
    } catch (NoSuchMethodException ex) {
        // Shouldn't happen
        throw new RuntimeException(ex);
    }
}

Gaziantep