Java records with nullable components

Question

I really like the addition of records in Java 14, at least as a preview feature, as it helps to reduce my need to use lombok for simple, immutable "data holders". But I'm having an issue with the implementation of nullable components. I'm trying to avoid returning null in my codebase to indicate that a value might not be present. Therefore I currently often use something like the following pattern with lombok.

@Value
public class MyClass {
 String id;
 @Nullable String value;

 Optional<String> getValue() { // overwrite the generated getter
  return Optional.ofNullable(this.value);
 }
}

When I try the same pattern now with records, this is not allowed stating incorrect component accessor return type.

record MyRecord (String id, @Nullable String value){
 Optional<String> value(){
  return Optional.ofNullable(this.value); 
 }
}

Since I thought the usage of Optionals as return types is now preferred, I'm really wondering why this restriction is in place. Is my understanding of the usage wrong? How can I achieve the same, without adding another accessor with another signature which does not hide the default one? Should Optional not be used in this case at all?

Mitinger

What about simply record MyRecord (String id, Optional<String> value) {}? – Coastal

Or ``` record MyRecord (String id, Optional<String> value) { public static MyRecord create(String id, @Nullable String value) { return new MyRecord(id, Optional.ofNullable(value)); } } ``` – Coastal

While possible, at least my understanding of the preferred use of Optional is not to use them as arguments or members though. – Mitinger

Agreed that a lot of people say that. I don't see a problem in this case, especially if clients of the record just use the create() method above. – Coastal

From the usage/ client perspective I think you are absolutely right, but I think the use of Optional breaks the serialization contract... – Mitinger

Yes, it does break Serialisation, but I don't think that's often a requirement today. Most 'serialisation' is to JSON, which con just omit empty optionals – Coastal

At my place, we mandated the use of Optional in records and it was the right choice from a reliability, readability and maintenance perspective. – Roarke

Answer 1

A record comprises attributes that primarily define its state. The derivation of the accessors, constructors, etc. is completely based on this state of the records.

Now in your example, the state of the attribute value is null, hence the access using the default implementation ends up providing the true state. To provide customized access to this attribute you are instead looking for an overridden API that wraps the actual state and further provides an Optional return type.

Of course, as you mentioned one of the ways to deal with it would be to have a custom implementation included in the record definition itself

record MyClass(String id, String value) {
    
    Optional<String> getValue() {
        return Optional.ofNullable(value());
    }
}

Alternatively, you could decouple the read and write APIs from the data carrier in a separate class and pass on the record instance to them for custom accesses.

The most relevant quote from JEP 384: Records that I found would be(formatting mine):

A record declares its state -- the group of variables -- and commits to an API that matches that state. This means that records give up a freedom that classes usually enjoy -- the ability to decouple a class's API from its internal representation -- but in return, records become significantly more concise.

Nuncia

While I understand the rational, I think the true state would also be reflected, when the accessors would return an Optional as the default implementation instead. Wrapping/ Accessing the record member via extra API and additional methods, defeats the concise argument and adds boilerplate or confusion on the consumer side. Maybe the real problem is the inconsistent usage of Optionals, even in the JDK. I thought returning null should be avoided but maybe I should also continue to use null and avoid the wrapping completely...but checking for null is not really convenient in Java. – Mitinger

@Mitinger the true state of the value is null, how would that be represented by Optional? The point to consider here would be to not mix of thinking that the absence of attribute is the same as assigning a null value to it. Usage of Optional is more of an API that a data carrier wouldn't expose by itself, its the layer above it (general developed as data access layer) that wraps the way a null value interpretation specific to the application. – Nuncia

Yes, you right from the definition perspective. Maybe I was hoping for a bit more than just data carriers, especially since it is possible to add accessors/ getters to records. Thus, what's your take on the usage of records in the domain model. I was hoping to reduce clutter with records, since I try to create immutable classes in the domain layer... – Mitinger

@Mitinger that’s an opportunity to rethink whether value really has to be nullable. Or you use something like interface MyRecord { String id(); Optional<String> value(); } record MyRecordNoValue(String id) implements MyRecord { public Optional<String> value(){ return Optional.empty(); } } record MyRecordWithValue(String id,String actualValue) implements MyRecord { public Optional<String> value(){ return Optional.of(actualValue); } } – Malvern

@Malvern Yes, already thought about using an EmptyObject approach, but again, in my opinion this bloats the code base. With Optional we already have a nice way of expressing the absence of certain field, thus adding more objects feels a bit overengineered. Same for the interface method. I like that thinking, but IMHO it does help to improve readability, especially when you would have more than one nullable field... But I'll try to give that style a chance...Thanks! – Mitinger

since edit wasn't possible... [...] does NOT improve readability [...] – Mitinger

@Malvern Interestingly that makes the design much cleaner and restricted to the use case(the alternative in my answer lacked the thought of using an interface). But I would then assume that it doesn't align to the @Nullable String value; contract, is that true? (I am getting to understand that the choice is now to either accepting null values OR treating value as an Optional.) – Nuncia

@Nuncia That approach assumes that you consistently use MyRecordWithValue with a non-null value and MyRecordNoValue otherwise. You may expand the constructors to enforce it and/or add factory methods which delegate automatically to the right result type. These artifacts didn’t fit into the original comment. It was food for thought anyway, not a complete solution. – Malvern

Answer 2

Due to restrictions placed on records, namely that canonical constructor type needs to match accessor type, a pragmatic way to use Optional with records would be to define it as a property type:

record MyRecord (String id, Optional<String> value){
}

A point has been made that this is problematic due to the fact that null might be passed as a value to the constructor. This can be solved by forbidding such MyRecord invariants through canonical constructor:

record MyRecord(String id, Optional<String> value) {

    MyRecord(String id, Optional<String> value) {
        this.id = id;
        this.value = Objects.requireNonNull(value);
    }
}

In practice most common libraries or frameworks (e.g. Jackson, Spring) have support for recognizing Optional type and translating null into Optional.empty() automatically so whether this is an issue that needs to be tackled in your particular instance depends on context. I recommend researching support for Optional in your codebase before cluttering your code possibly unnecessary.

Woodwaxen

Suggesting Optional to represent optional values is the way. – Alverson

> "A point has been made that this is problematic due to the fact that null might be passed as a value to the constructor" If you pass null for an Optional parameter, instead of Optional.empty(), you deserve all the problems you get. – Phasia

Answer 3

Credits go to Holger! I really like his proposed way of questioning the actual need of null. Thus with a short example, I wanted to give his approach a bit more space, even if a bit convoluted for this use-case.

interface ConversionResult<T> {
    String raw();

    default Optional<T> value(){
        return Optional.empty();
    }

    default Optional<String> error(){
        return Optional.empty();
    }

    default void ifOk(Consumer<T> okAction) {
        value().ifPresent(okAction);
    }

    default void okOrError(Consumer<T> okAction, Consumer<String> errorAction){
        value().ifPresent(okAction);
        error().ifPresent(errorAction);
    }

    static ConversionResult<LocalDate> ofDate(String raw, String pattern){
        try {
            var value = LocalDate.parse(raw, DateTimeFormatter.ofPattern(pattern));
            return new Ok<>(raw, value);  
        } catch (Exception e){
            var error = String.format("Invalid date value '%s'. Expected pattern '%s'.", raw, pattern);
            return new Error<>(raw, error);
        }
    }

    // more conversion operations

}

record Ok<T>(String raw, T actualValue) implements ConversionResult<T> {
    public Optional<T> value(){
        return Optional.of(actualValue);
    }
}

record Error<T>(String raw, String actualError) implements ConversionResult<T> {
    public Optional<String> error(){
        return Optional.of(actualError);
    }
}

Usage would be something like

var okConv = ConversionResult.ofDate("12.03.2020", "dd.MM.yyyy");
okConv.okOrError(
    v -> System.out.println("SUCCESS: "+v), 
    e -> System.err.println("FAILURE: "+e)
);
System.out.println(okConv);


System.out.println();
var failedConv = ConversionResult.ofDate("12.03.2020", "yyyy-MM-dd");
failedConv.okOrError(
    v -> System.out.println("SUCCESS: "+v), 
    e -> System.err.println("FAILURE: "+e)
);
System.out.println(failedConv);

which leads to the following output...

SUCCESS: 2020-03-12
Ok[raw=12.03.2020, actualValue=2020-03-12]

FAILURE: Invalid date value '12.03.2020'. Expected pattern 'yyyy-MM-dd'.
Error[raw=12.03.2020, actualError=Invalid date value '12.03.2020'. Expected pattern 'yyyy-MM-dd'.]

The only minor issue is that the toString prints now the actual... variants. And of course we do not NEED to use records for this.

Mitinger

Answer 4

Don't have the rep to comment, but I just wanted to point out that you've essentially reinvented the Either datatype. https://hackage.haskell.org/package/base-4.14.0.0/docs/Data-Either.html or https://www.scala-lang.org/api/2.9.3/scala/Either.html. I find Try, Either, and Validation to be incredibly useful for parsing and there are a few java libraries with this functionality that I use: https://github.com/aol/cyclops/tree/master/cyclops and https://www.vavr.io/vavr-docs/#_either.

Unfortunately, I think your main question is still open (and I'd be interested in finding an answer).

doing something like

RecordA(String a)
RecordAandB(String a, Integer b)

to deal with an immutable data carrier with a null b seems bad, but wrapping recordA(String a, Integer b) to have an Optional getB somewhere else seems contra-productive. There's almost no point to the record class then and I think the lombok @Value is still the best answer. I'm just concerned that it won't play well with deconstruction for pattern matching.

Halidom

I had the same feeling about the Either/ Try/ Validation and also use/ like them from time to time. In this case I just wanted try out the new feature and avoid dependencies. Holger pushed me again to rethink nullability in general. Maybe we just have to deal with null checks in Java (with some help Nullable annotations/ Checkerframework), since I currently also find the Optional wrapping a bit over engineered and noisy. In general Javas Optional feels very limited and not well integrated. Other languages definitely get the nullabilty handling much better (like Kotlin, Typescript...) – Mitinger