Why java.util.Optional is not Serializable, how to serialize the object with such fields

Question

The Enum class is Serializable so there is no problem to serialize object with enums. The other case is where class has fields of java.util.Optional class. In this case the following exception is thrown: java.io.NotSerializableException: java.util.Optional

How to deal with such classes, how to serialize them? Is it possible to send such objects to Remote EJB or through RMI?

This is the example:

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.ObjectOutputStream;
import java.io.Serializable;
import java.util.Optional;

import org.junit.Test;

public class SerializationTest {

    static class My implements Serializable {

        private static final long serialVersionUID = 1L;
        Optional<Integer> value = Optional.empty();

        public void setValue(Integer i) {
            this.i = Optional.of(i);
        }

        public Optional<Integer> getValue() {
            return value;
        }
    }

    //java.io.NotSerializableException is thrown

    @Test
    public void serialize() {
        My my = new My();
        byte[] bytes = toBytes(my);
    }

    public static <T extends Serializable> byte[] toBytes(T reportInfo) {
        try (ByteArrayOutputStream bstream = new ByteArrayOutputStream()) {
            try (ObjectOutputStream ostream = new ObjectOutputStream(bstream)) {
                ostream.writeObject(reportInfo);
            }
            return bstream.toByteArray();
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }
}

Ahern

If Optional was marked as Serializable, then what would happen if get() returned something that was not serializable? – Polytonality

@Polytonality You would get a NotSerializableException, of course. – Hying

@Polytonality It's just like collections. Most of the collection classes are serializable, but a collection instance can actually only be serialized if every object contained in the collection is also serializable. – Perjure

My personal take here: it isnt in to remind people to properly unit test even serialization of objects. Just ran into java.io.NotSerializableException(java.util.Optional) myself ;-( – Plovdiv

Gaaaaaaa. Lost 3 hours on this today and getting sonar to pass "Fields in a "Serializable" class should either be transient or serializable". rules.sonarsource.com/java/RSPEC-1948 – Imes

That sounds like a very good and not at all too generic error message. /s Hope you didn't have more than 30 fields to check. – Leninism

Answer 1

This answer is in response to the question in the title, "Shouldn't Optional be Serializable?" The short answer is that the Java Lambda (JSR-335) expert group considered and rejected it. That note, and this one and this one indicate that the primary design goal for Optional is to be used as the return value of functions when a return value might be absent. The intent is that the caller immediately check the Optional and extract the actual value if it's present. If the value is absent, the caller can substitute a default value, throw an exception, or apply some other policy. This is typically done by chaining fluent method calls off the end of a stream pipeline (or other methods) that return Optional values.

It was never intended for Optional to be used other ways, such as for optional method arguments or to be stored as a field in an object. And by extension, making Optional serializable would enable it to be stored persistently or transmitted across a network, both of which encourage uses far beyond its original design goal.

Usually there are better ways to organize the data than to store an Optional in a field. If a getter (such as the getValue method in the question) returns the actual Optional from the field, it forces every caller to implement some policy for dealing with an empty value. This will likely lead to inconsisent behavior across callers. It's often better to have whatever code sets that field apply some policy at the time it's set.

Sometimes people want to put Optional into collections, like List<Optional<X>> or Map<Key,Optional<Value>>. This too is usually a bad idea. It's often better to replace these usages of Optional with Null-Object values (not actual null references), or simply to omit these entries from the collection entirely.

Perjure

Well done Stuart. I must say it's an extraordinary chain if reasoning, and it's an extraordinary things do to design a class that isn't intended to be used as an instance member type. Especially when that isn't stated in the class contract in the Javadoc. Maybe they should have designed an annotation instead of a class. – Hying

This is an excellent blog post on the rationale behind Sutart's answer: blog.joda.org/2014/11/optional-in-java-se-8.html – Harquebus

Very useful information. For some reason my first instinct was to use it as a field in a class. Good to know that this should be avoided. – Shebat

Good thing I dont need to use serializable fields. This would be a disaster otherwise – Tawanda

Optional provides all means for clean and safe dealing with nullable members, but they say that "this was not our intend". Well I think supporting Optional for members would completely follow 'Optional is being added for the value it offers in "fluent" sequences of statements.' – Villiers

Interesting answer, to me that design choice was completely unacceptable and misguided. You say "Usually there are better ways to organize the data than to store an Optional in a field", sure, maybe, why not, but that should be the designer's choice, not the language's. It's another one of these cases where I deeply miss Scala optionals in Java (Scala optionals are Serializable, and they follow Monad's guidelines) – Barbirolli

"the primary design goal for Optional is to be used as the return value of functions when a return value might be absent.". Well, it seems you cannot use them for return values in a remote EJB. Great... – Marquand

This is interesting since it seems to contradict the Java 8 in Action book that I am reading where they suggest using Optional for class variables on beans. Or maybe I just haven't read far enough and they will clarify in the next chapter :) – Devious

@Devious Is that in 10.2? The non-serializability issue is discussed in a sidebar at the end of 10.3.3. See also this Twitter exchange I had with one of the authors after the book was written (follow thread forward and backward): twitter.com/stuartmarks/status/784202445102592001 – Perjure

So you hold null or instances in fields and return Optional from the getters. Fine for Object types, but less so for primitives, say int vs OptionalInt as there isn't a "not-specified" value for an int unless you resort to Integer objects. – Poncho

I agree, it should be a choice of the designer of the application. What if I want to use it across services which are implemented as EJBs, e.g. from a DAO layer to a repository layer? Since the Optional doesn't implement Serializable the code will cause a ClassCastException. So it happend to me in Websphere. – Jamarjamb

I don't really understand the reasoning here. This post is predicated on the idea that the library designer's intentions behind an introduction of a type has relevance to how the type should be used. I don't see that to be the case at all. The inventors of computers intended to speed up some arithmetic, and break some encryption, yet here we are, using it to do fluid physics simulations to optimize the aerodynamics of our cars. If I'm not interested in Java's serialization mechanism (which is broken as hell), why shouldn't I use Optional where it serves me, like in a field or parameter? – Paten

tldr; programmers use optional in useful cases outside of the original design and the original designers of optional don't like that. – Nitz

So they don't want to make Optional more useful than it was originally designed to be? That isn't a sound justification IMHO. "it forces every caller to implement some policy for dealing with an empty value" - which is exactly what I WANT. – Sporades

We used to cache results with JSR107 annotation @CacheResult and just found that the Hazelcast is not happy about unserializable Optional. Optional is used as intended as a return type, but sadly we can't cache that result. Null is not solution because it is prohibited too. – Wirewove

"This will likely lead to inconsisent behavior across callers" how? if anything it will be more consistent since you force to implement business logic for dealing with absence of values – Ambulatory

@Ambulatory because you can deal with empty value in one caller code in one way, and differently in other caller code. They meant this. – Bernardinebernardo

Answer 2

A lot of Serialization related problems can be solved by decoupling the persistent serialized form from the actual runtime implementation you operate on.

/** The class you work with in your runtime */
public class My implements Serializable {
    private static final long serialVersionUID = 1L;

    Optional<Integer> value = Optional.empty();

    public void setValue(Integer i) {
        this.value = Optional.ofNullable(i);
    }

    public Optional<Integer> getValue() {
        return value;
    }
    private Object writeReplace() throws ObjectStreamException
    {
        return new MySerialized(this);
    }
}
/** The persistent representation which exists in bytestreams only */
final class MySerialized implements Serializable {
    private final Integer value;

    MySerialized(My my) {
        value=my.getValue().orElse(null);
    }
    private Object readResolve() throws ObjectStreamException {
        My my=new My();
        my.setValue(value);
        return my;
    }
}

The class Optional implements behavior which allows to write good code when dealing with possibly absent values (compared to the use of null). But it does not add any benefit to a persistent representation of your data. It would just make your serialized data bigger…

The sketch above might look complicated but that’s because it demonstrates the pattern with one property only. The more properties your class has the more its simplicity should be revealed.

And not to forget, the possibility to change the implementation of My completely without any need to adapt the persistent form…

Muddle

+1, but with more fields there will be more boilerplate to copy them. – Questioning

@Marko Topolnik: at most, a single line per property and direction. However, by providing an appropriate constructor for class My, which you usually do as it’s handy for other uses as well, readResolve could be a single-line implementation thus reducing the boilerplate to a single line per property. Which is not much given the fact that each mutable property has at least seven lines of code in class My anyway. – Muddle

I wrote a post covering the same topic. It is essentially the long text version of this answer: Serialize Optional – Barbellate

Downside is: you need to do that for any bean/pojo that uses optionals. But still, nice idea. – Plovdiv

Answer 3

If you would like a serializable optional, consider instead using guava's optional which is serializable.

Pennyworth

Answer 4

The Vavr.io library (former Javaslang) also have the Option class which is serializable:

public interface Option<T> extends Value<T>, Serializable { ... }

Sherd

Answer 5

It's a curious omission.

You would have to mark the field as transient and provide your own custom writeObject() method that wrote the get() result itself, and a readObject() method that restored the Optional by reading that result from the stream. Not forgetting to call defaultWriteObject() and defaultReadObject() respectively.

Hying

If I own the code of a class it is more convenient to store mere object in a field. Optional class in such case would be restricted for interface of the class (get method would return Optional.ofNullable(field)). But for the internal representation it’s not possible to use Optional to clearly intent that value is optional. – Ahern

I've just shown that it is possible. If you think otherwise for some reason, what exactly is your question about? – Hying

Thank you for your answer, it adds an option for me. In my comment I wanted to show further thoughts on this topic, consider Pro and contra of each solution. In usage writeObject/readObject methods we have clear intent of Optional in state representation, but the implementation of serialization become more complicated. If field is used intensively in calculation/streams - is more convenient to use writeObject/readObject. – Ahern

Comments should be relevant to the answers they appear under. The relevance of your musings escapes me frankly. – Hying

Answer 6

Just copy Optional class to your project and create your own custom Optional that implements Serializable. I am doing it because I just realized this sh*t too late.

Ceiling

Answer 7

If you want to maintain a more consistent type list and avoid using null there's one kooky alternative.

You can store the value using an intersection of types. Coupled with a lambda, this allows something like:

private final Supplier<Optional<Integer>> suppValue;
....
List<Integer> temp = value
        .map(v -> v.map(Arrays::asList).orElseGet(ArrayList::new))
        .orElse(null);
this.suppValue = (Supplier<Optional<Integer>> & Serializable)() -> temp==null ? Optional.empty() : temp.stream().findFirst();

Having the temp variable separate avoids closing over the owner of the value member and thus serialising too much.

Jello

Answer 8

the problem is you have used variables with optional. the basic solution to avoid this, provide the variable without optional and get them as optional when you call the getter like below. Optional<Integer> value = Optional.empty(); to Integer value = null;

public class My implements Serializable {

        private static final long serialVersionUID = 1L;
        //Optional<Integer> value = Optional.empty(); //old code
        Integer value = null; //solution code without optional.

        public void setValue(Integer value ) {
           //this.value  = Optional.of(value); //old code with Optional
           this.value  = value ; //solution code without optional.
        }

        
        public Optional<Integer> getValue() {
            //solution code - return the value by using Optional.
            return Optional.ofNullable(value);
        }
}

Alienee