How can I disable serialization in Wicket 1.5?

Question

I want to turn off serialization in my Wicket app and store all page/session information in RAM. My application has a very small number of users (generally 1); I do not need a cluster deployment; and I need to cache some non-serializable data between requests.

Is there a way so that Wicket will not automatically attempt to serialize my pages / session? I tried the suggestion to use use HttpSessionDataStore at https://cwiki.apache.org/confluence/display/WICKET/Page+Storage, but it had no effect. I still get stack traces like this:

SEVERE: Error serializing object class com.prosc.safetynet.Administer [object=[Page class = com.prosc.safetynet.Administer, id = 0, render count = 1]]
org.apache.wicket.util.io.SerializableChecker$WicketNotSerializableException: Unable to serialize class: com.prosc.safetynet.SafetyNetSession$1
Field hierarchy is:
  0 [class=com.prosc.safetynet.Administer, path=0]
    java.lang.Object org.apache.wicket.Component.data [class=org.apache.wicket.model.CompoundPropertyModel]
      private java.lang.Object org.apache.wicket.model.CompoundPropertyModel.target [class=com.prosc.safetynet.SafetyNetSession$2]
        final com.prosc.safetynet.SafetyNetSession com.prosc.safetynet.SafetyNetSession$2.this$0 [class=com.prosc.safetynet.SafetyNetSession]
          private java.lang.Object com.prosc.safetynet.SafetyNetSession.tryAndSerializeMeBitch [class=com.prosc.safetynet.SafetyNetSession$1] <----- field that is not serializable
    at org.apache.wicket.util.io.SerializableChecker.internalCheck(SerializableChecker.java:395)
    at org.apache.wicket.util.io.SerializableChecker.check(SerializableChecker.java:374)
    at org.apache.wicket.util.io.SerializableChecker.checkFields(SerializableChecker.java:655)
    at org.apache.wicket.util.io.SerializableChecker.internalCheck(SerializableChecker.java:578)
    at org.apache.wicket.util.io.SerializableChecker.check(SerializableChecker.java:374)
    at org.apache.wicket.util.io.SerializableChecker.checkFields(SerializableChecker.java:655)
    at org.apache.wicket.util.io.SerializableChecker.internalCheck(SerializableChecker.java:578)
    at org.apache.wicket.util.io.SerializableChecker.check(SerializableChecker.java:374)
    at org.apache.wicket.util.io.SerializableChecker.checkFields(SerializableChecker.java:655)
    at org.apache.wicket.util.io.SerializableChecker.internalCheck(SerializableChecker.java:578)
    at org.apache.wicket.util.io.SerializableChecker.check(SerializableChecker.java:374)
    at org.apache.wicket.util.io.SerializableChecker.checkFields(SerializableChecker.java:655)
    at org.apache.wicket.util.io.SerializableChecker.internalCheck(SerializableChecker.java:578)
    at org.apache.wicket.util.io.SerializableChecker.check(SerializableChecker.java:374)
    at org.apache.wicket.util.io.SerializableChecker.writeObjectOverride(SerializableChecker.java:724)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:326)
    at org.apache.wicket.serialize.java.JavaSerializer$CheckerObjectOutputStream.writeObjectOverride(JavaSerializer.java:258)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:326)
    at org.apache.wicket.serialize.java.JavaSerializer.serialize(JavaSerializer.java:77)
    at org.apache.wicket.pageStore.DefaultPageStore.serializePage(DefaultPageStore.java:368)
    at org.apache.wicket.pageStore.DefaultPageStore.storePage(DefaultPageStore.java:146)
    at org.apache.wicket.page.PageStoreManager$PersistentRequestAdapter.storeTouchedPages(PageStoreManager.java:383)
    at org.apache.wicket.page.RequestAdapter.commitRequest(RequestAdapter.java:171)
    at org.apache.wicket.page.AbstractPageManager.commitRequest(AbstractPageManager.java:94)
    at org.apache.wicket.page.PageManagerDecorator.commitRequest(PageManagerDecorator.java:68)
    at org.apache.wicket.page.PageAccessSynchronizer$2.commitRequest(PageAccessSynchronizer.java:281)
    at org.apache.wicket.Application$2.onDetach(Application.java:1598)
    at org.apache.wicket.request.cycle.RequestCycleListenerCollection$3.notify(RequestCycleListenerCollection.java:99)
    at org.apache.wicket.request.cycle.RequestCycleListenerCollection$3.notify(RequestCycleListenerCollection.java:97)
    at org.apache.wicket.util.listener.ListenerCollection$1.notify(ListenerCollection.java:119)
    at org.apache.wicket.util.listener.ListenerCollection.reversedNotify(ListenerCollection.java:143)
    at org.apache.wicket.util.listener.ListenerCollection.reversedNotifyIgnoringExceptions(ListenerCollection.java:113)
    at org.apache.wicket.request.cycle.RequestCycleListenerCollection.onDetach(RequestCycleListenerCollection.java:95)
    at org.apache.wicket.request.cycle.RequestCycle.onDetach(RequestCycle.java:603)
    at org.apache.wicket.request.cycle.RequestCycle.detach(RequestCycle.java:542)
    at org.apache.wicket.request.cycle.RequestCycle.processRequestAndDetach(RequestCycle.java:287)
    at org.apache.wicket.protocol.http.WicketFilter.processRequest(WicketFilter.java:188)
    at org.apache.wicket.protocol.http.WicketFilter.doFilter(WicketFilter.java:244)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151)
    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:870)
    at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
    at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
    at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
    at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685)
    at java.lang.Thread.run(Thread.java:680)

Shively

Answer 1

You can implement your own IPageStore that keeps pages in memory.

Moultrie

There is also an example: maciej-miklas.blogspot.de/2013/09/… – Hageman

Answer 2

I can't comment about anything specific to Wicket, but speaking generally the entire point of an Http Session is to store Serializable state between requests (and in clustered environments, to allow that state to be replicated to multiple nodes in the cluster to provide redundancy in the event of a node failure). Putting something that is not Serializable into it is generally considered an error, as shown by your stack trace. I'd be somewhat surprised if there is any sort of configuration option that would change this (though perhaps there is; as I said I can't really comment on the Wicket side of things).

A simple alternative, if you do not require true persistence and if the data is not exceptionally large/complex, is to just use hidden form fields on your page to keep track of the relevant state.

But if what you want is an in-memory cache, why not implement your own? It's simple enough to do:

public class SessionCache {
    private static final Map<String, Map<String, Object>> CACHE = Collections.synchronizedMap(new HashMap<String, Map<String, Object>>());

    public static Object getAttribute(String sessionId, String attribName) {
        Map<String, Object> attribs = CACHE.get(sessionId);
        if (attribs != null) {
            synchronized(attribs) {
                return attribs.get(attribName);
            }
        }

        return null;
    }

    public static void setAttribute(String sessionId, String attribName, Object attribValue) {
        Map<String, Object> attribs = CACHE.get(sessionId);
        if (attribs == null) {
            attribs = new HashMap<String, Object>();
            CACHE.put(sessionId, attribs);
        }

        synchronized(attribs) {
            attribs.put(attribName, attribValue);
        }
    }

    public static void destroySession(String sessionId) {
        CACHE.remove(sessionId);
    }

    public static void createSession(String sessionId, boolean force) {
        if (force || ! CACHE.containsKey(sessionId)) {
            CACHE.put(sessionId, new HashMap<String, Object>());
        }
    }
}

Note that you'll want to hook that into Wicket's session lifecycle so that old sessions are removed when they expire. Otherwise you'll have a gradual memory leak on your hands. From the docs it looks like you can accomplish this using registerUnboundListener() on the HttpSessionStore class.

Armington

I'm pretty sure that this is a Wicket-specific issue. The stack trace shows that this is not being initiated by the servlet container; it's a Wicket method that is trying to store the page in a PageMap. – Shively

To be more specific, the serialization stuff starts happening here: org.apache.wicket.Application$2.onDetach(Application.java:1598) – Shively

This specific instance of the issue may be caused by a Wicket class, but conceptually the same issue exists with Tomcat (and I suspect with any other Servlet container). – Armington

Answer 3

Here is the solution that I came up with, based on svenmeier's answers. I'm sure that this is not 100% correct, but it's working fine in my testing:

package com.prosc.wicket;

import org.apache.wicket.Application;
import org.apache.wicket.DefaultPageManagerProvider;
import org.apache.wicket.page.IManageablePage;
import org.apache.wicket.page.IPageManagerContext;
import org.apache.wicket.pageStore.IDataStore;
import org.apache.wicket.pageStore.IPageStore;
import org.apache.wicket.pageStore.memory.HttpSessionDataStore;
import org.apache.wicket.pageStore.memory.PageNumberEvictionStrategy;

import java.io.Serializable;
import java.util.HashMap;
import java.util.Map;

/**
 * This class disables Wicket's serialization behavior, while still retaining session and page data in memory (so back button will work).
 * This will run out of memory under heavy load; but it's very convenient for low volume web applications.
 * To disable serialization in your application, call this code:
 * <pre>
 *     setPageManagerProvider( new NoSerializePageManagerProvider( this, getPageManagerContext() ) );
 * </pre>
 */
public class NoSerializePageManagerProvider extends DefaultPageManagerProvider {
    private IPageManagerContext pageManagerContext;

    public NoSerializePageManagerProvider( Application application, IPageManagerContext pageManagerContext ) {
        super( application );
        this.pageManagerContext = pageManagerContext;
    }

    @Override
    protected IDataStore newDataStore() {
        return new HttpSessionDataStore( pageManagerContext, new PageNumberEvictionStrategy( 20 ) );
    }

    @Override
    protected IPageStore newPageStore( IDataStore dataStore ) {
        return new IPageStore() {
            Map<String,Map<Integer,IManageablePage>> cache = new HashMap<String, Map<Integer, IManageablePage>>();

            public void destroy() {
                cache = null;
            }

            public IManageablePage getPage( String sessionId, int pageId ) {
                Map<Integer, IManageablePage> sessionCache = getSessionCache( sessionId, false );
                IManageablePage page = sessionCache.get( pageId );
                if( page == null ) {
                    throw new IllegalArgumentException( "Found this session, but there is no page with id " + pageId );
                }
                return page;
            }

            public void removePage( String sessionId, int pageId ) {
                getSessionCache( sessionId, false ).remove( pageId );
            }

            public void storePage( String sessionId, IManageablePage page ) {
                getSessionCache( sessionId, true ).put( page.getPageId(), page );
            }

            public void unbind( String sessionId ) {
                cache.remove( sessionId );
            }

            public Serializable prepareForSerialization( String sessionId, Object page ) {
                return null;
            }

            public Object restoreAfterSerialization( Serializable serializable ) {
                return null;
            }

            public IManageablePage convertToPage( Object page ) {
                return (IManageablePage)page;
            }

            private Map<Integer, IManageablePage> getSessionCache( String sessionId, boolean create ) {
                Map<Integer, IManageablePage> sessionCache = cache.get( sessionId );
                if( sessionCache == null ) {
                    if( create ) {
                        sessionCache = new HashMap<Integer, IManageablePage>();
                        cache.put( sessionId, sessionCache );
                    } else {
                        throw new IllegalArgumentException( "There are no pages stored for session id " + sessionId );
                    }
                }
                return sessionCache;
            }
        };
    }
}

Shively

Answer 4

I want to improve on Johnny's answer who improves on Jesse's answer :)

• this is the whole IPageManagerProvider, not only the IPageStore
• frees memory by calling store.unbind(sessionId) when the session unbinds
• works also with Wicket 7.x

public class NoSerializationButCachingPageManagerProvider implements IPageManagerProvider {

    private static final Logger LOGGER = LoggerFactory.getLogger(NoSerializationButCachingPageManagerProvider.class);

    private final Application application;

    public NoSerializationButCachingPageManagerProvider(final Application application) {
        this.application = Args.notNull(application, "application");
        LOGGER.info("Pages don't get serialized, but in-memory cached.");
    }

    @Override
    public IPageManager get(IPageManagerContext pageManagerContext) {
        final IPageStore store = new NoSerializationButCachingPageStore();
        final IPageManager manager = new PageStoreManager(application.getName(), store, pageManagerContext);
        /*
         * session unbind must call store.unbind() to free memory (prevents memory leak)
         */
        application.getSessionStore().registerUnboundListener((String sessionId) -> store.unbind(sessionId));
        return manager;
    }

}

class NoSerializationButCachingPageStore implements IPageStore {

    private static final Logger LOGGER = LoggerFactory.getLogger(NoSerializationButCachingPageStore.class);

    private static final int MEDIAN_OF_NUMBER_OF_SESSIONS = 100;

    private final ConcurrentMap<String, CustomLinkedHashMap<Integer, IManageablePage>> cache = new ConcurrentHashMap<>(MEDIAN_OF_NUMBER_OF_SESSIONS);

    @Override
    public void destroy() {
        cache.clear();
    }

    @Override
    public IManageablePage getPage(final String sessionId, final int pageId) {
        LOGGER.info("getPage. SessionId: {}, pageId: {}", sessionId, pageId);
        final Map<Integer, IManageablePage> sessionCache = getSessionCache(sessionId);
        final RequestCycle requestCycle = RequestCycle.get();
        if (sessionCache == null) {
            LOGGER.warn("Missing cache. SessionId: {}, pageId: {}, URL: {}", sessionId, pageId, requestCycle == null ? "" : requestCycle.getRequest().getUrl());
            return null;
        }

        IManageablePage page;
        // noinspection SynchronizationOnLocalVariableOrMethodParameter
        synchronized (sessionCache) {
            page = sessionCache.get(pageId);
        }

        if (page == null && LOGGER.isDebugEnabled()) {
            LOGGER.debug("Missed page. SessionId: {}, pageId: {}, URL: {}", sessionId, pageId, requestCycle == null ? "" : requestCycle.getRequest().getUrl());
        }

        return page;
    }

    @Override
    public void removePage(final String sessionId, final int pageId) {
        if (LOGGER.isDebugEnabled()) {
            LOGGER.debug("removePage. SessionId: {}, pageId: {}", sessionId, pageId);
        }
        final Map<Integer, IManageablePage> sessionCache = getSessionCache(sessionId);
        if (sessionCache != null) {
            // noinspection SynchronizationOnLocalVariableOrMethodParameter
            synchronized (sessionCache) {
                sessionCache.remove(pageId);
            }
        }
    }

    @Override
    public void storePage(final String sessionId, final IManageablePage page) {
        if (LOGGER.isDebugEnabled()) {
            LOGGER.debug("storePage. SessionId: {}, pageId: {}, cache-size: {}", sessionId, page.getPageId(), cache.size());
        }
        final LinkedHashMap<Integer, IManageablePage> sessionCache = getOrCreateSessionCache(sessionId);
        final int pageId = page.getPageId();
        // noinspection SynchronizationOnLocalVariableOrMethodParameter
        synchronized (sessionCache) {
            if (sessionCache.containsKey(pageId)) {
                // do this to change insertion order and update least inserted entry
                sessionCache.remove(pageId);
                sessionCache.put(pageId, page);
            } else {
                sessionCache.put(pageId, page);
            }
        }
    }

    /**
     * @param sessionId
     */
    @Override
    public void unbind(final String sessionId) {
        if (LOGGER.isDebugEnabled()) {
            LOGGER.debug("unbind/cache-remove. SessionId: {}", sessionId);
        }
        cache.remove(sessionId);
    }

    @Override
    public Serializable prepareForSerialization(final String sessionId, final Serializable page) {
        return null;
    }

    @Override
    public Object restoreAfterSerialization(final Serializable serializable) {
        return null;
    }

    @Override
    public IManageablePage convertToPage(final Object page) {
        return (IManageablePage) page;
    }

    private Map<Integer, IManageablePage> getSessionCache(final String sessionId) {
        return cache.get(sessionId);
    }

    private CustomLinkedHashMap<Integer, IManageablePage> getOrCreateSessionCache(final String sessionId) {
        return cache.computeIfAbsent(sessionId, (final String s) -> new CustomLinkedHashMap<>());
    }

    /**
     * Mimics "least recently inserted" cache
     */
    private static class CustomLinkedHashMap<K, V> extends LinkedHashMap<K, V> {

        private static final long serialVersionUID = 1L;
        /**
         * use this parameter to control memory consumption and frequency of appearance of PageExpiredException
         */
        private static final int MAX_PAGES_PER_SESSION = 3;

        @Override
        protected boolean removeEldestEntry(final Map.Entry<K, V> eldest) {
            return size() > MAX_PAGES_PER_SESSION;
        }
    }
}

Butterandeggs

Answer 5

I want to improve Jesse's answer. Below is a thread-safe implementation of IPageStore with internal "Least Recently Inserted" cache (keeps at most 5 recently accessed stateful pages per session):

public class CustomPageStore implements IPageStore {

private static final Logger logger = LoggerFactory.getLogger(CustomPageStore.class);

private static final int MEDIAN_OF_NUMBER_OF_SESSIONS = 6000;

private ConcurrentMap<String, CustomLinkedHashMap<Integer, IManageablePage>> cache = new ConcurrentHashMap<>(MEDIAN_OF_NUMBER_OF_SESSIONS);

@Override
public void destroy() {
    cache.clear();
}

@Override
public IManageablePage getPage(final String sessionId, int pageId) {
    final Map<Integer, IManageablePage> sessionCache = getSessionCache(sessionId);
    final RequestCycle requestCycle = RequestCycle.get();
    if (sessionCache == null) {
        logger.warn("Missing cache. SessionId: {}, pageId: {}, URL: {}", sessionId, pageId, requestCycle == null ? StringUtils.EMPTY : requestCycle.getRequest().getUrl());
        return null;
    }

    final IManageablePage page;
    //noinspection SynchronizationOnLocalVariableOrMethodParameter
    synchronized (sessionCache) {
        page = sessionCache.get(pageId);
    }

    if (page == null && logger.isDebugEnabled()) {
        logger.debug("Missed page. SessionId: {}, pageId: {}, URL: {}", sessionId, pageId, requestCycle == null ? StringUtils.EMPTY : requestCycle.getRequest().getUrl());
    }

    return page;
}

@Override
public void removePage(final String sessionId, int pageId) {
    final Map<Integer, IManageablePage> sessionCache = getSessionCache(sessionId);
    if (sessionCache != null) {
        //noinspection SynchronizationOnLocalVariableOrMethodParameter
        synchronized (sessionCache) {
            sessionCache.remove(pageId);
        }
    }
}

@Override
public void storePage(final String sessionId, IManageablePage page) {
    final LinkedHashMap<Integer, IManageablePage> sessionCache = getOrCreateSessionCache(sessionId);
    final int pageId = page.getPageId();
    //noinspection SynchronizationOnLocalVariableOrMethodParameter
    synchronized (sessionCache) {
        if (sessionCache.containsKey(pageId)) {
            // do this to change insertion order and update least inserted entry
            sessionCache.remove(pageId);
            sessionCache.put(pageId, page);
        } else {
            sessionCache.put(pageId, page);
        }
    }
}

@Override
public void unbind(final String sessionId) {
    cache.remove(sessionId);
}

@Override
public Serializable prepareForSerialization(String sessionId, Object page) {
    return null;
}

@Override
public Object restoreAfterSerialization(Serializable serializable) {
    return null;
}

@Override
public IManageablePage convertToPage(final Object page) {
    return (IManageablePage) page;
}

@Nullable
private Map<Integer, IManageablePage> getSessionCache(final String sessionId) {
    return cache.get(sessionId);
}

@Nonnull
private CustomLinkedHashMap<Integer, IManageablePage> getOrCreateSessionCache(final String sessionId) {
    return cache.computeIfAbsent(sessionId, s -> new CustomLinkedHashMap<>());
}

/** Mimics "least recently inserted" cache */
private static class CustomLinkedHashMap<K, V> extends LinkedHashMap<K, V> {

    /** use this parameter to control memory consumption and frequency of appearance of PageExpiredException */
    private static final int MAX_PAGES_PER_SESSION = 5;

    @Override
    protected boolean removeEldestEntry(final Map.Entry<K, V> eldest) {
        return size() > MAX_PAGES_PER_SESSION;
    }
}
}

Geulincx

I tried this but removePage() is never called (not on Session.invalidate(), never), causing the map to increase infinitely (memory leak). Any idea why? – Butterandeggs

@marcus you need to investigate lifecycle of your web session (from creation to its removal). Wicket only wraps low-level Session which is generally handled by your servlet container (like Tomcat). – Geulincx

@marcus Also I strongly recommend you to migrate to wicket 1.6, because this version was got rid of internal memory leak problem. In my case (over 1 million lines of code) I spent about 2 weeks for migration. It worth of it. – Geulincx

I am using Wicket 7.8.0 and I meant unbind(sessionId) which is (also) never called. But I will investigate. Thank you. – Butterandeggs

unbind is called from PageStoreManager.clear() which is called from Session.clear() which is never called. So adding Session.get().clear() to my logout functionality solved my memory leak. – Butterandeggs

@marcus I think you have very specific behavior of your application and your comment (as part of your solution) is not related to question in general. Logout usually leads to session expiration, so memory leak is not possible in case of valid session expiration strategy. And customized PageStore could not lead to memory leak, because it doesn't break general contract. Mention that this implementation of PageStore keeps all pages in memory - not on your HDD which is larger (and where memory leak problem is not so obvious). – Geulincx

I do not agree. I downloaded the Wicket source and found out that only Session.clear() leads to a call to unbind() on the PageStore. And the Javadoc of Session.clear() says the method might only be useful in rare cases (it is usually not called on session expiration). So I assume that by default your solution has a memory leak like it had for me. – Butterandeggs