Should Java 8 getters return optional type?

Question

Optional type introduced in Java 8 is a new thing for many developers.

Is a getter method returning Optional<Foo> type in place of the classic Foo a good practice? Assume that the value can be null.

Panslavism

The question is whether the nullablitity is unavoidable. A component may have a property which is allowed to be null, but still, the programmer using that component might decide to strictly keep that property non-null. So the programmer shouldn’t have to deal with Optional then. Or, in other words, does null really represent the absence of a value like with the result of a search (where Optional is appropriate) or is null just one member of the set of possible values. – Helmick

See also the discussion on @NotNull annotations: https://mcmap.net/q/18328/-which-notnull-java-annotation-should-i-use/873282 – Nardi

Answer 1

Of course, people will do what they want. But we did have a clear intention when adding this feature, and it was not to be a general purpose Maybe type, as much as many people would have liked us to do so. Our intention was to provide a limited mechanism for library method return types where there needed to be a clear way to represent "no result", and using null for such was overwhelmingly likely to cause errors.

For example, you probably should never use it for something that returns an array of results, or a list of results; instead return an empty array or list. You should almost never use it as a field of something or a method parameter.

I think routinely using it as a return value for getters would definitely be over-use.

There's nothing wrong with Optional that it should be avoided, it's just not what many people wish it were, and accordingly we were fairly concerned about the risk of zealous over-use.

(Public service announcement: NEVER call Optional.get unless you can prove it will never be null; instead use one of the safe methods like orElse or ifPresent. In retrospect, we should have called get something like getOrElseThrowNoSuchElementException or something that made it far clearer that this was a highly dangerous method that undermined the whole purpose of Optional in the first place. Lesson learned. (UPDATE: Java 10 has Optional.orElseThrow(), which is semantically equivalent to get(), but whose name is more appropriate.))

Russel

(Regarding last part) …and when we are confident, that the value is never null we might use orElseThrow(AssertionError::new), ahem or orElseThrow(NullPointerException::new)… – Helmick

What would you have done differently if your intention had been to introduce a general-purpose Maybe or Some type? Is there a way in which Optional doesn't fit the bill for one, or is it just that introducing Optionals all over a new API would make it non-Java-ish? – Trabue

By "general-purpose type", I meant building it into the language's type system, rather than providing a library class that approximates it. (Some languages have types for T? (T or null) and T! (non-nullable T)) Optional is just a class; we can't make implicit conversions between Foo and Optional<Foo> as we could have with language support. – Russel

Well, it seems people adopted it more widely than you intended. Of course, one could write his own Maybe object rather easily, but using an existing class that is part of java.util is even easier. Well, at least for your comfort, I don't remember ever using Optional.get. I agree it would be really nice to have it as a part of type system. – Piselli

I've wondered for a while why the emphasis in the Java world was on Optional, and not better static analysis. Optional does have some advantages, but the huge advantage that null has is backwards compatibility; Map::get returns a nullable V, not an Optional<V>, and that's never going to change. It could have easily been annotated @Nullable, though. Now we have two ways to express lack-of-value, plus less incentive to really get static analysis happening, which seems like a worse position to be in. – Aggappora

I had no idea that using it as a return value for a property was not what y'all intended until I read this answer here on StackOverflow. In fact, I was lead to believe that it was exactly what y'all intended after reading this article on Oracle's website: oracle.com/technetwork/articles/java/… Thank you for clarifying – Coprophagous

Honestly, I don't see any grave improvements by this concept : ( I think it would have been more useful to be capable of deriving an Empty Object from any reference type (e.g. via a special constructor). You would determine one state that clearly identifies the object as unuseful (by negative IDs, etc). So, any time you can't find a result, you get the same Invalid/Unuseful/Empty Object Instance. Pretty much like empty collections. Optional<T> basically extracts your null checks into template code. And at least to me, it doesn't let APIs appear clearer as when using annotations : ( – Conscious

Brian Goetz answers 'are there changes in java 8 that you regret now?' i.stack.imgur.com/LPg1Z.png See 16':07"-17':35" of jaxenter.de/fragen-und-antworten-zu-java-8-qa-33108 – Firer

I still don't see the argument against returning Optional from the getter of a nullable member field. How is it over-use in this case? – Outspeak

@Outspeak The answer depends, to some degree, on the question "what is a getter" -- there's a fuzzy line between "getter" and "method that returns something." Obviously, there are useful cases for returning an Optional from methods. But in real-world code (for better or worse), most getters (like, 99%) are just return x. In that case, returning an Optional from a getter means you are using Optional as your state representation -- and this is definitely not what was intended (and falls squarely in the "zealous overuse" category.) – Russel

@BrianGoetz Optional is fine as a general-purpose type even if it is a library class. In fact, most languages with a general-purpose Maybe/Option type have it exactly as a library type (see Haskell, Scala, ML, ...), not counting the N versions for Java — though they also have pattern matching. – Conveyancing

"you probably should never use it for something that returns an array of results, or a list of results; instead return an empty array or list" - A none existing list and an empty list have different semantic meanings, and thus should not be considered the same. For example, a getBooksOfUser(UserId id) method would return an empty value if the user does not exist, but return an empty list if the the user exists but has no book; not the same thing. – Gastropod

I don't understand this. I want Maybe type and Optional is one. Why should I write my own, just because it is recommended not to use Optional as a field? – Timmie

Brian Goetz: Your original answer gives some examples of where we should NOT use optional, but none where they should be used. You might be able to make your point more clearly if you give an example of where an Optional return value is definitely a good way to go. – Paranymph

Optional was created for functional programming, and that's where it should be used. Here's a good example, courtesy of Brian Goetz. Outside of functional programming, Optional just add complexity to your code, which only makes bugs more likely. People are using it to avoid ever seeing the word null in their code. This leads to code like this: if(Optional.ofNullable(x).isPresent()) instead of if(x!=null). I'll stick with using null. – Paranymph

@Paranymph Nope, "created for functional programming" is neither an accurate nor helpful statement of intent. – Russel

@Brian Goetz I have since moderated my views, but I was probably overreacting to a lot of very bad use of Optional on my last project. I would now say that getters should return Optional only when the value might actually be null. The suggestion that it's intended "to help design better APIs" (from an Oracle page) has led to people putting Optional in their input parameters, and returning Optional where the value is never null. I've also seen how its misuse has led to bugs. Also, your functional-programming example was the only one I had found where Optional was clearly useful. – Paranymph

Here are some examples of bad usages of Optional, taken from production code. – Paranymph

@BrianGoetz why is it a bad practice to use Optional for a field that can have no value? It provides an extra layer of semantic information at the type system level. Definitely useful when working with the object as you don't need to look up all the places where that field was set to know if it always has a value or if it can sometimes have no value. You seem to use an argument of authority to justify this decision, but what we need is an explanation. – Cholecystotomy

Java8 seems to have orElseThrow as well, why do you say Java 10 now has orElseThrow? (docs.oracle.com/javase/8/docs/api/java/util/…) – Francinafrancine

@KorayTugay Java 8 only has the Optional.orElseThrow version that requires an argument, as in optional.orElseThrow(AssertionError::new). But Java 10 added the no-argument version of orElseThrow – Fronton

I know I'm very late to this party but there is an additional use of Optionals which most would be familiar with because its used in other languages. Using an Optional type allows you to essentially define a "third state" that a field can take. A standard java object field can either have a value or be null. With optional, you can now represent a third state, i.e. undefined. This is useful for situations where null can be ambiguous (i.e. does null mean "this value was omitted from the request" or "this value should be set to null on the server"?). – Neddy

@BrianGoetz - I wonder if we should return optional from the get methods of collections. Could you please help to get the answer for that here - stackoverflow.com/questions/61091514/… ? Thanks. – Georgeta

Your answer is not much help and it sounds more like "Because I say so..". Optional by its very name seems to be the type one should use to represent optional fields in POJOs. Of course if we are using Optional then we make the assumption that is never a null. – Kermanshah

@BrianGoetz Java did not invent Optional. Neither does Java's Optional type do anything special than its usage in other languages. Also I found oracle.com/technical-resources/articles/java/… on Oracle's website that suggests doing exactly what you are telling people not to do, i.e. use Optional for field members of classes. If "Because I say so.." is your reasoning then I am sorry but Oracle has an upper hand here. – Kermanshah

@Neddy now you have a variable which can be null, Optional.empty() or Optional.of(something), and need to remember which of those has which meaning. Better use a real alternative type here. – Craquelure

@Randomdude Java's Optional does something special compared to other languages: map(v -> null) is same as flatMap(v -> Option(null)) - returns empty instead of of(null), while Scala would return Some(null). Optional.of(null) throws an exception, unlike, again, Scala which would return Some(null) - assuming .of is equivalent to Some.apply. Which brings me to 2nd point: that article clearly has bad wording mistakes, like saying "you can create an Optional object that may hold a null value:" - you can't. Nor is it similar to features from other languages presented there. – Softfinned

@Randomdude There's also a practical reason why Optional is effective only as a return type. When a value is returned, it happens at final stages of a method logic, where there's no special logging required. On the other side we have the core of some method logic, where not unwrapping Optional ASAP leads to heavy usage of map, filter which hamper debugging, because it's impossible to diagnose at which stage the Optional becomes empty. There's no whenBecomesNull operator on Optional which would fire only once. Luckily, even the article mentions that Optional promotes unwrapping ASAP – Softfinned

@BrianGoetz with the release of Java 16 and records - how should, if at all, java.util.Optional be used with records? If used as a field it goes against this recommendation but it can't be really used as an accessor. – Aultman

After youtube.com/watch?v=9R8G8Ehzn8o where Oracle employee is on side of using Optional everywhere and after Oracle employees are using it as parameters in incubator JDK code github.com/openjdk/panama-foreign/blob/… I believe the writing is on the wall where we are most likely heading. As soon as records become primitive record/class most of arguments against it will go away - no performance penalty and no null Optionals. – Aultman

If the intention of Optional was never to use it as a field of something or a method parameter, then why wasn't it made so that the language itself doesn't allow you to do that at all, like throw a compile time error, I routinely see code where Optional fields are used in classes or as input parameters of a method, and no one will fix this messy code now, why not put guardrails around language features to ensure that it can only be used the way it's meant to be used. If you give people freedom, they will on average always write bad code. – Grimace

@Grimace Optional is a library class; it's not a language feature. Like all libraries, it had specific design goals and intents, and it can be misused and abused. It is not the job of the language to defend against misuse of specific libraries -- and I am pretty confident that you would not actually want to live in the world where the language tried to do this. – Russel

Answer 2

After doing a bit of research of my own, I've come across a number of things that might suggest when this is appropriate. The most authoritative being the following quote from an Oracle article:

"It is important to note that the intention of the Optional class is not to replace every single null reference. Instead, its purpose is to help design more-comprehensible APIs so that by just reading the signature of a method, you can tell whether you can expect an optional value. This forces you to actively unwrap an Optional to deal with the absence of a value." - Tired of Null Pointer Exceptions? Consider Using Java SE 8's Optional!

I also found this excerpt from Java 8 Optional: How to use it

"Optional is not meant to be used in these contexts, as it won't buy us anything:

• in the domain model layer (not serializable)
• in DTOs (same reason)
• in input parameters of methods
• in constructor parameters"

Which also seems to raise some valid points.

I wasn't able to find any negative connotations or red flags to suggest that Optional should be avoided. I think the general idea is, if it's helpful or improves the usability of your API, use it.

Bidwell

stackoverflow.com/questions/25693309 - it seems Jackson supports it already, so "not serializable" no longer passes as a valid reason :) – Piselli

I suggest having your answer address why using Optional in input parameters of methods (more specifically constructors) "won't buy us anything". dolszewski.com/java/java-8-optional-use-cases contains a nice explanation. – Eyelet

I have found that currying optional results which need to be transformed into other APIs requires an Optional parameter. The result is a fairly comprehensible API. See https://mcmap.net/q/18132/-why-should-java-8-39-s-optional-not-be-used-in-arguments – Nucleotidase

The link is broken. Could you please update the answer? – Apul

Just to add to this. I created an IntelliJ getter template to generate getters that return Optional<T> gist.github.com/suvodeep-pyne/4ec3a52b4b25d0026a98cf5d75d02c86 – Astri

The trouble is, it often doesn't improve the API, despite the best intentions of the developer. I have been collection examples of bad usages of Optional, all taken from production code, which you can check out. – Paranymph

@Paranymph I haven't read your link thoroughly yet, but I think I would support editing this answer with some references to that information. Particularly the last paragraph, which in hindsight was not the best choice of words. Cheers :) – Bidwell

Sorry, I also just noticed that you have referenced that information in your own answer. I upvoted it some time ago. Still, this answer could likely use some updated information and overall quality improvements. I'll support any relevant edits. – Bidwell

@Paranymph Sorry to hijack this a bit, but your Quick Take #1 seems off - the null check is for propertyAsString, not for key.parse() result. That parse() method might not like null inputs. – Goosegog

@Bidwell - what about this stackoverflow.com/questions/61091514/… ? – Georgeta

@TomJoe I guess it depends. If you are writing your own collections that you intend to be used to complement existing implementations, I would say to avoid optional for the sake of consistency. On the other hand, if these are independent from implementations in the standard library and they are unlikely to be used together, then the decision is less clear. I think there is a lot to be learned from Rust here. In Rust, optional values and results (value/error) are just how things are done. It has very clear benefits, but also means handling every possibility of what would otherwise be null. – Bidwell

Optional "wont buy us anything in input parameters of methods and in constructor parameters" is not necessarily true when you use it with dependency injection. It makes the provider method's signature more comprehensible and indirectly instructs all consumers to validate the presence before use. – Poling

The original article I referenced was written by Brian Goetz, before his answer here. Better to upvote his answer and continue further comments there if necessary. Also keep in mind these answers are from 2014! – Bidwell

Answer 3

The reason Optional was added to Java is because this:

return Arrays.asList(enclosingInfo.getEnclosingClass().getDeclaredMethods())
    .stream()
    .filter(m -> Objects.equals(m.getName(), enclosingInfo.getName())
    .filter(m ->  Arrays.equals(m.getParameterTypes(), parameterClasses))
    .filter(m -> Objects.equals(m.getReturnType(), returnType))
    .findFirst()
    .getOrThrow(() -> new InternalError(...));

is cleaner than this:

Method matching =
    Arrays.asList(enclosingInfo.getEnclosingClass().getDeclaredMethods())
    .stream()
    .filter(m -> Objects.equals(m.getName(), enclosingInfo.getName())
    .filter(m ->  Arrays.equals(m.getParameterTypes(), parameterClasses))
    .filter(m -> Objects.equals(m.getReturnType(), returnType))
    .getFirst();
if (matching == null)
  throw new InternalError("Enclosing method not found");
return matching;

My point is that Optional was written to support functional programming, which was added to Java at the same time. (The example comes courtesy of a blog by Brian Goetz. A better example might use the orElse() method, since this code will throw an exception anyway, but you get the picture.)

But now, people are using Optional for a very different reason. They're using it to address a flaw in the language design. The flaw is this: There's no way to specify which of an API's parameters and return values are allowed to be null. It may be mentioned in the javadocs, but most developers don't even write javadocs for their code, and not many will check the javadocs as they write. So this leads to a lot of code that always checks for null values before using them, even though they often can't possibly be null because they were already validated repeatedly nine or ten times up the call stack.

I think there was a real thirst to solve this flaw, because so many people who saw the new Optional class assumed its purpose was to add clarity to APIs. Which is why people ask questions like "should getters return Optionals?" No, they probably shouldn't, unless you expect the getter to be used in functional programming, which is very unlikely. In fact, if you look at where Optional is used in the Java API, it's mainly in the Stream classes, which are the core of functional programming. (I haven't checked very thoroughly, but the Stream classes might be the only place they're used.)

If you do plan to use a getter in a bit of functional code, it might be a good idea to have a standard getter and a second one that returns Optional.

Oh, and if you need your class to be serializable, you should absolutely not use Optional.

Optionals are a very bad solution to the API flaw because a) they're very verbose, and b) They were never intended to solve that problem in the first place.

A much better solution to the API flaw is the Nullness Checker. This is an annotation processor that lets you specify which parameters and return values are allowed to be null by annotating them with @Nullable. This way, the compiler can scan the code and figure out if a value that can actually be null is being passed to a value where null is not allowed. By default, it assumes nothing is allowed to be null unless it's annotated so. This way, you don't have to worry about null values. Passing a null value to a parameter will result in a compiler error. Testing an object for null that can't be null produces a compiler warning. The effect of this is to change NullPointerException from a runtime error to a compile-time error.

This changes everything.

As for your getters, don't use Optional. And try to design your classes so none of the members can possibly be null. And maybe try adding the Nullness Checker to your project and declaring your getters and setter parameters @Nullable if they need it. I've only done this with new projects. It probably produces a lot of warnings in existing projects written with lots of superfluous tests for null, so it might be tough to retrofit. But it will also catch a lot of bugs. I love it. My code is much cleaner and more reliable because of it.

(There is also a new language that addresses this. Kotlin, which compiles to Java byte code, allows you to specify if an object may be null when you declare it. It's a cleaner approach.)

Addendum to Original Post (version 2)

After giving it a lot of thought, I have reluctantly come to the conclusion that it's acceptable to return Optional on one condition: That the value retrieved might actually be null. I have seen a lot of code where people routinely return Optional from getters that can't possibly return null. I see this as a very bad coding practice that only adds complexity to the code, which makes bugs more likely. But when the returned value might actually be null, go ahead and wrap it inside an Optional.

Keep in mind that methods that are designed for functional programming, and that require a function reference, will (and should) be written in two forms, one of which uses Optional. For example, Optional.map() and Optional.flatMap() both take function references. The first takes a reference to an ordinary getter, and the second takes one that returns Optional. So you're not doing anyone a favor by return an Optional where the value can't be null.

Having said all that, I still see the approach used by the Nullness Checker is the best way to deal with nulls, since they turn NullPointerExceptions from runtime bugs to compile time errors.

Paranymph

This answer seems most appropriate to me. Optional was only added in java 8 when streams were added. And only stream functions return optional as far as I have seen. – Indelicacy

I think this is one of the best answer. People know what is Optional and how it works. But most confusing part is/was where to use it and how to use it. by reading this it clears a lot of doubts. – Zehe

Answer 4

I'd say in general its a good idea to use the optional type for return values that can be nullable. However, w.r.t. to frameworks I assume that replacing classical getters with optional types will cause a lot of trouble when working with frameworks (e.g., Hibernate) that rely on coding conventions for getters and setters.

Fluency

This advice is exactly the sort of thing I meant by "we are concerned about the risk of zealous overuse" in https://mcmap.net/q/17902/-should-java-8-getters-return-optional-type. – Russel

Answer 5

If you are using modern serializers and other frameworks that understand Optional then I have found these guidelines work well when writing Entity beans and domain layers:

1. If the serialization layer (usually a DB) allows a null value for a cell in column BAR in table FOO, then the getter Foo.getBar() can return Optional indicating to the developer that this value may reasonably be expected to be null and they should handle this. If the DB guarantees the value will not be null then the getter should not wrap this in an Optional.
2. Foo.bar should be private and not be Optional. There's really no reason for it to be Optional if it is private.
3. The setter Foo.setBar(String bar) should take the type of bar and not Optional. If it's OK to use a null argument then state this in the JavaDoc comment. If it's not OK to use null an IllegalArgumentException or some appropriate business logic is, IMHO, more appropriate.
4. Constructors don't need Optional arguments (for reasons similar to point 3). Generally I only include arguments in the constructor that must be non-null in the serialization database.

To make the above more efficient, you might want to edit your IDE templates for generating getters and corresponding templates for toString(), equals(Obj o) etc. or use fields directly for those (most IDE generators already deal with nulls).

Burgos

Using Lombok, I think is a good idea to have Optional private fields with default empty Optional value and with generated setters and getters... – Lactobacillus

But Lombok's developers disagree :) stackoverflow.com/a/31674917 – Softfinned

In addition to 4: Different static factory methods e.g. Foo.withBar(...) and Foo.withoutBar(...) could also be helpful if depending on a bar being present or not there is different validation logic. But then you could also think of making it two subclasses or two distinct classes implementing the same interface. – Lampblack

Answer 6

You have to keep in mind that the often-cited advice came from people who had little experience outside Java, with option types, or with functional programming.

So take it with a grain of salt. Instead, let's look at it from the "good practice" perspective:

Good practice not only means asking "how do we write new code?", but also "what happens to existing code?".

In the case of Optional, my environment found a good and easy to follow answer:

Optional is mandatory to indicate optional values in records:

record Pet(String name, Optional<Breed> breed,
Optional<ZonedDateTime> dateOfBirth)

This means that existing code is good as-is, but code that makes use of record (that is, "new code") causes widespread adoption of Optional around it.

The result has been a complete success in terms of readability and reliability. Just stop using null.

Lumpy

If dateOfBirth is passed in as null, and the method contract states that getDateOfBirth() will return an Optional. Then, when trying to access the value via getDateOfBirth().get(), it will result in a NullPointerException. So, this did not solve anything that a traditional null-check wouldn't? – Variant

Answer 7

TL;DR Yes, it should per need basis.

Optional being a fluent interface design pattern implementation, or a fluid programming technique, the question could be a generic one. Is implementation of design patterns a good practice?

From maintenance and extensibility perspectives it is a good practice, from a computer science perspective design patterns are less efficient. Java programming language being a high level programming language the efficiency is a concern for its running context (JVM implemented by JRE) that could change in time regardless of the design patterns implemented in applications it is used to write, thus the main concerns of Java applications by design are maintenance and extensibility.

Warren