Is method reference caching a good idea in Java 8?

Question

Consider I have code like the following:

class Foo {

   Y func(X x) {...} 

   void doSomethingWithAFunc(Function<X,Y> f){...}

   void hotFunction(){
        doSomethingWithAFunc(this::func);
   }

}

Suppose that hotFunction is called very often. Would it then be advisable to cache this::func, maybe like this:

class Foo {
     Function<X,Y> f = this::func;
     ...
     void hotFunction(){
        doSomethingWithAFunc(f);
     }
}

As far as my understanding of java method references goes, the Virtual Machine creates an object of an anonymous class when a method reference is used. Thus, caching the reference would create that object only once while the first approach creates it on each function call. Is this correct?

Should method references that appear at hot positions in the code be cached or is the VM able to optimize this and make the caching superfluous? Is there a general best practice about this or is this highly VM-implemenation specific whether such caching is of any use?

Jussive

I would say that you use that function so much that this level of tuning is needed/desirable, maybe you would be better off dropping lambdas and implementing the function directly, which gives more room for other optimizations. – Alienation

@SJuan76: I am not sure about this! If a method reference is compiled to an anonyomous class, it is as fast as a usual interface call. Thus, I don't think that functional style should be avoided for hot code. – Jussive

Method references are implemented with invokedynamic. I doubt that you will see a performance improvement by caching the function object. On the contrary: It might inhibit compiler optimizations. Did you compare the performance of the two variants? – Barnabas

@nosid: No, didn't do a comparison. But I am using a very early version of OpenJDK, so my numbers might be of no importance anyway as I guess that the first version only implement the new features quick 'n' dirty and this cannot be compared to the performance when the features have matured over time. Does the spec really mandate that invokedynamic must be used? I see no reason for this here! – Jussive

It should get cached automatically (it is not equivalent to creating a new anonymous class every time) so you shouldn't have to worry about that optimisation. – Sect

Answer 1

You have to make a distinction between frequent executions of the same call-site, for stateless lambda or stateful lambdas, and frequent uses of a method-reference to the same method (by different call-sites).

Look at the following examples:

    Runnable r1=null;
    for(int i=0; i<2; i++) {
        Runnable r2=System::gc;
        if(r1==null) r1=r2;
        else System.out.println(r1==r2? "shared": "unshared");
    }

Here, the same call-site is executed two times, producing a stateless lambda and the current implementation will print "shared".

Runnable r1=null;
for(int i=0; i<2; i++) {
  Runnable r2=Runtime.getRuntime()::gc;
  if(r1==null) r1=r2;
  else {
    System.out.println(r1==r2? "shared": "unshared");
    System.out.println(
        r1.getClass()==r2.getClass()? "shared class": "unshared class");
  }
}

In this second example, the same call-site is executed two times, producing a lambda containing a reference to a Runtime instance and the current implementation will print "unshared" but "shared class".

Runnable r1=System::gc, r2=System::gc;
System.out.println(r1==r2? "shared": "unshared");
System.out.println(
    r1.getClass()==r2.getClass()? "shared class": "unshared class");

In contrast, in the last example are two different call-sites producing an equivalent method reference but as of 1.8.0_05 it will print "unshared" and "unshared class".

---

For each lambda expression or method reference the compiler will emit an invokedynamic instruction that refers to a JRE provided bootstrap method in the class LambdaMetafactory and the static arguments necessary to produce the desired lambda implementation class. It is left to the actual JRE what the meta factory produces but it is a specified behavior of the invokedynamic instruction to remember and re-use the CallSite instance created on the first invocation.

The current JRE produces a ConstantCallSite containing a MethodHandle to a constant object for stateless lambdas (and there’s no imaginable reason to do it differently). And method references to static method are always stateless. So for stateless lambdas and single call-sites the answer must be: don’t cache, the JVM will do and if it doesn’t, it must have strong reasons that you shouldn’t counteract.

For lambdas having parameters, and this::func is a lambda that has a reference to the this instance, things are a bit different. The JRE is allowed to cache them but this would imply maintaining some sort of Map between actual parameter values and the resulting lambda which could be more costly than just creating that simple structured lambda instance again. The current JRE does not cache lambda instances having a state.

But this does not mean that the lambda class is created every time. It just means that the resolved call-site will behave like an ordinary object construction instantiating the lambda class that has been generated on the first invocation.

Similar things apply to method references to the same target method created by different call-sites. The JRE is allowed to share a single lambda instance between them but in the current version it doesn’t, most probably because it is not clear whether the cache maintenance will pay off. Here, even the generated classes might differ.

---

So caching like in your example might have your program do different things than without. But not necessarily more efficient. A cached object is not always more efficient than a temporary object. Unless you really measure a performance impact caused by a lambda creation, you should not add any caching.

I think, there are only some special cases where caching might be useful:

• we are talking about lots of different call-sites referring to the same method
• the lambda is created in the constructor/class initialize because later on the use-site will
  • be called by multiple threads concurrently
  • suffer from the lower performance of the first invocation

Router

Very elaborate answer giving insight to implementation details I was hoping for, thank you! – Jussive

Clarification: the term “call-site” refers to the execution of the invokedynamic instruction that will create the lambda. It is not the place where the functional interface method will be executed. – Router

I thought that this-capturing lambdas were instance-scoped singletons (a synthetic instance variable on the object). This is not so? – Nasa

@Marko Topolnik: that would be a compliant compile strategy but no, as of Oracle’s jdk up to 1.8.0_40 it isn’t so. These lambdas are not remembered thus can be garbage collected. But remember that once an invokedynamic callsite has been linked, it might get optimized like ordinary code, i.e. Escape Analysis works for such lambda instances. – Router

So in some cases it's actually better that the instance is not cached. Similar could apply to non-capturing lambdas. BTW I also realized that it wouldn't be cool if each lambda added weight to the object. – Nasa

@Marko Topolnik: non-capturing lambdas are instantiated and remembered once per class not per instance so it makes a difference. But right, every strategy can be disputed. It’s a good thing that the specification allows different (and even changing) strategies… – Router

What do you refer to by "it makes a difference"? Since my point was just that---it does make a difference, and performance can actually suffer when EA is foiled by a singleton instance. That means a cache line will be polluted by the long fetch. I actually observed that difference on the example of cached boxed primitives. – Nasa

@Marko Topolnik: I was talking about the memory consumption only. Remembering lambdas capturing this requires storage per instance while remembering stateless lambdas requires storage per class only. In the case of stateless lambdas which don’t carry values to read, I would expect the hotspot optimizer to be smart enough to generate inlined code which doesn’t need to ever touch the associated instance. That differs from boxed primitives which carry values (and yes, making caching mandatory for certain values was a bad decision, imho). – Router

There doesn't seem to be a standard library class named MethodReference. Do you mean MethodHandle here? – Pat

@Lii: you’re right, that’s a typo. Interestingly no-one seems to have noticed before. – Router

@Router "this would imply maintaining some sort of Map..." but doesn't it imply it for resolving non-capturing lambdas? Anyway, this is guaranteed to be the same per instance, why couldn't lambda be cached per that instance? – Gristle

@Artem Novikov: lambda expression instances are associated with the byte code instruction that creates it (invokedynamic). For a stateless lambda expression, the instruction is supposed to always produce the same instance, hence, may simply have a pointer to that instance. For capturing lambdas, the same instruction has to potentially create different instances holding different state, hence, a plain pointer wouldn’t be sufficient. And the instruction is not instance specific, it can be executed for arbitrary this values, so this is not special in this regard. – Router

@Holger: instance method references and non-capturing lambdas will refer only to this, so do you know any possible situation when calling a cached object would result in a different behaviour than creating a new functional object? (I haven't found any examples yet.) – Letty

@Ogmios: the this reference is treated like any other object reference when capturing. If the lambda expression accesses this, it may exhibit different behavior for different this instances and won’t be cached. Note that this is different to inner classes, as inner classes always have a reference to their outer object, even if they don’t use it. Lambda expressions, on the other hand, only capture this, if actually needed, that’s why a lambda expression can be non-capturing in a non-static context, if it doesn’t access this (and hence can’t have different behavior in that case). – Router

@Holger: exactly, this is just a simple reference. But it is stored in the lambda object upon lambda instantiation, otherwise the jvm wouldn't create a new object for every non-capturing lambda execution. Thus a lambda object cached in the class object it was created for will always refer to a single object, so there is no different this, thus no different behaviour either if no other local variables are captured -- but this can be freely accessed from inside the lambda. (see also Rafael Winterhalter's blog) – Letty

@Ogmios: sure, caching a lambda instance in an instance variable of the owner when it captures only this, would work. As said here, it would be a compliant compile strategy. – Router

Answer 2

One situation where it is a good ideal, unfortunately, is if the lambda is passed as a listener that you want to remove at some point in the future. The cached reference will be needed as passing another this::method reference will not be seen as the same object in the removal and the original won't be removed. For example:

public class Example
{
    public void main( String[] args )
    {
        new SingleChangeListenerFail().listenForASingleChange();
        SingleChangeListenerFail.observableValue.set( "Here be a change." );
        SingleChangeListenerFail.observableValue.set( "Here be another change that you probably don't want." );

        new SingleChangeListenerCorrect().listenForASingleChange();
        SingleChangeListenerCorrect.observableValue.set( "Here be a change." );
        SingleChangeListenerCorrect.observableValue.set( "Here be another change but you'll never know." );
    }

    static class SingleChangeListenerFail
    {
        static SimpleStringProperty observableValue = new SimpleStringProperty();

        public void listenForASingleChange()
        {
            observableValue.addListener(this::changed);
        }

        private<T> void changed( ObservableValue<? extends T> observable, T oldValue, T newValue )
        {
            System.out.println( "New Value: " + newValue );
            observableValue.removeListener(this::changed);
        }
    }

    static class SingleChangeListenerCorrect
    {
        static SimpleStringProperty observableValue = new SimpleStringProperty();
        ChangeListener<String> lambdaRef = this::changed;

        public void listenForASingleChange()
        {
            observableValue.addListener(lambdaRef);
        }

        private<T> void changed( ObservableValue<? extends T> observable, T oldValue, T newValue )
        {
            System.out.println( "New Value: " + newValue );
            observableValue.removeListener(lambdaRef);
        }
    }
}

Would have been nice to not need lambdaRef in this case.

Sieber

Ah, I see the point now. Sounds reasonable, though probably not the scenario the OP speaking about. Nevertheless upvoted. – Sinuation

Answer 3

As far as I understand the language specification, it allows this kind of optimization even if it changes the observable behaviour. See the following quotes from section JSL8 §15.13.3:

§15.13.3 Run-time Evaluation of Method References

At run time, evaluation of a method reference expression is similar to evaluation of a class instance creation expression, insofar as normal completion produces a reference to an object. [..]

[..] Either a new instance of a class with the properties below is allocated and initialized, or an existing instance of a class with the properties below is referenced.

A simple test shows, that method references for static methods (can) result in the same reference for each evaluation. The following program prints three lines, of which the first two are identical:

public class Demo {
    public static void main(String... args) {
        foobar();
        foobar();
        System.out.println((Runnable) Demo::foobar);
    }
    public static void foobar() {
        System.out.println((Runnable) Demo::foobar);
    }
}

I can't reproduce the same effect for non-static functions. However, I haven't found anything in the language specification, that inhibits this optimization.

So, as long as there is no performance analysis to determine the value of this manual optimization, I strongly advise against it. The caching affects the readability of the code, and it's unclear whether it has any value. Premature optimization is the root of all evil.

Barnabas