JavaEE6: How to safeguard web application when the database shut down

Question

First of all, my framework is Java EE 6 with JSF, managed bean, EJB and JPA. I write a simple program to query information from the database. So when I click a button, it trigger an event to a managed bean, where an event listener method will access EJB method. The EJB method will do a simple select query to an Entity. If the database is shutdown before or during the time I select, I get an exception

Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.0.1.v20100213-r6600): org.eclipse.persistence.exceptions.DatabaseException

Internal Exception: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet successfully received from the server was 51,460 milliseconds ago.  The last packet sent successfully to the server was 0 milliseconds ago.
Error Code: 0

How do I safeguard away from this Exception? Definitely a try, catch here, but not sure where to put it. When I do em.createNamedQuery or em.remove, I try to catch com.mysql.jdbc.exceptions.jdbc4.CommunicationsException, but I got an error said: Exception com.mysql.jdbc.exceptions.jdbc4.CommunicationsException is never thrown in body of corresponding try statement.

Below are my codes, where would I catch the Exception?

This is my EJB

@Stateless
@LocalBean
public class DocumentSBean {
    @PersistenceContext
    private EntityManager em;

    public List<User> listUser(){
         Query query = em.createNamedQuery("User.listUser");
         query.returnResultList();
    }
}

This is my ManagedBean

@ManagedBean(name="document")
@ViewScoped
public class DisplayListController implements Serializable {            

   @EJB
   DocumentSBean sBean;

   List<User> users = null;

   public void foo(){
       users = sBean.listUser();
   }
}

---

EDIT

I try both ways as listed below, but still return status 200 instead of 500 on firebug

<p:commandButton update="myform" actionListener="#{document.setDisplayFacility}" rendered="#{utility.admin}" value="Facilities"/>

OR

<p:commandButton update="myform" actionListener="#{document.setDisplayFacility}" rendered="#{utility.admin}" value="Facilities">
       <p:ajax actionListener="#{document.setDisplayFacility}" update="myform" event="click"/>
</p:commandButton>

Here is setDisplayFacility()

public void setDisplayFacility(){
   facilities = sBean.getAllFacilities(); //sBean is EJB
   displayFacility = true;
}

Ovenware

Answer 1

How do I safeguard away from this Exception? Definitely a try, catch here, but not sure where to put it.

Only catch the exception there where you can handle it in a reasonable manner.

---

I try to catch com.mysql.jdbc.exceptions.jdbc4.CommunicationsException, but I got an error said: Exception com.mysql.jdbc.exceptions.jdbc4.CommunicationsException is never thrown in body of corresponding try statement.

The CommunicationsException is in this case a nested exception of DatabaseException. EclipseLink is under the covers already catching the CommunicationsException and rethrowing it as DatabaseException with the CommunicationsException as root cause. Something like:

try {
     // Execute query.
} catch (Exception e) {
     throw new DatabaseException("Internal Exception: " + e, e);
}

In theory, you can only handle it as follows:

try {
     // Let EclipseLink execute query.
} catch (DatabaseException e) {
    if (e.getCause() instanceof CommunicationsException) {
        // Handle.
    }
}

Which is however ugly and not recommended in this particular case.

---

Below are my codes, where would I catch the Exception?

Depends on how you would like to handle the exception. If you want to display it in a generic error page, then you should not catch it yourself, but just let it go. The servletcontainer will then catch and handle it itself. It will lookup the best matching <error-page> in the web.xml and display it.

<error-page>
    <exception-type>java.lang.Exception</exception-type>
    <location>/generic-error.xhtml</location>
</error-page>

This one will display /generic-error.xhtml for all subclasses of java.lang.Exception.

If you want to display it in a specific error page, then you need to declare the <exception-type> more specific to match the actual exception type. E.g.:

<error-page>
    <exception-type>org.eclipse.persistence.exceptions.DatabaseException</exception-type>
    <location>/database-error.xhtml</location>
</error-page>

---

However, although not explicitly specified in your question, I know based on your question history that you're using JSF with PrimeFaces. You need to keep in mind that PrimeFaces will not display the error page when the initial request was made by ajax. Instead, its ajax view handler has already catched the exception itself and it will delegate to the <p:ajaxStatus> component in the view. Try adding ajax="false" to the PrimeFaces command component and you'll finally see the servletcontainer's default error page being displayed (or any of yours if a matching one in web.xml is found).

If you want to display some generic JSF UI when PrimeFaces received an ajax error, then use the error facet of <p:ajaxStatus>. E.g.

<p:ajaxStatus>
    <f:facet name="start"><h:graphicImage value="images/ajax-loader.gif" /></f:facet>
    <f:facet name="success"><h:outputText value="" /></f:facet>
    <f:facet name="error">
        <h:panelGroup layout="block" styleClass="ui-message-error ui-widget ui-corner-all">
            <h:outputText value="An error has occurred!" /><br />
            <h:outputLink value="#" onclick="window.location.reload(true)"><h:outputText value="Please reload page and retry" /></h:outputLink><br />
            <h:outputLink value="mailto:[email protected]?subject=Ajax%20Error"><h:outputText value="If in vain, please contact support" /></h:outputLink>
        </h:panelGroup>
    </f:facet>
</p:ajaxStatus>

(there's however some bug in PrimeFaces 2.2 RC1 which causes displaying the error facet to fail, it works correctly in PrimeFaces 2.1)

Hibernicism

It is an awesome post. Thank you. When I catch java.lang.Exception, then it work but when I catch org.eclipse.persistence.exceptions.DatabaseException, it does not work. Maybe it got nested again to a more general Exception. I have a question that is quite off the topics. I shut down the mysql server to test this, however when I turn it back on, things does not get back to normal. Meaning, when I click on component that query from the database, it still give me the error page. I have to restart both glassfish and mysql server for it to work normally again. Is this normal, BalusC? – Ovenware

You need to catch the uppermost exception in the trace, not the nested ("Caused by") parts in the trace. As to the reconnect failure: you need to configure Glassfish jdbc-connection-pool retry attempts and interval in the admin console or in sun-resources.xml file (depending on where you've definied the pool). You can find all configuration options here: docs.sun.com/app/docs/doc/820-4507/6nfvg4rbl?l=en&a=view Ctrl+F "retry". – Hibernicism

Thank you. When I said catch I mean I let the ServletContainer catch it in web.xml. java.lang.Exception work great, but it weird that org.eclipse.persistence.exceptions.DatabaseException fail. For the ajax error catch. I ran into a funny problem. When the database is down and I make an ajax request, my ajax-loader.gif start to spin, but it spin like forever, but as soon as I refresh the page, then the error show up. I would It expect that the ajax-loader.gif for a while, then the error appear. Any idea why, BalusC? – Ovenware

As to the error page fail: Is it really not wrapped in another exception according to the trace? As to the ajax loader fail: You're using primefaces-2.1.jar, right? – Hibernicism

yup. 2.1 As you said 2.2 is still buggy. About the ajax-loader spin forever, I actually take that back. It spin for a while then stop and no error message show up. But if during the time it spin, I refresh then the error show up. – Ovenware

Well just like what you said in your Primefaces bug issue, my firebug show that the response code for my request is 200 instead of 500. I gonna migrate back to the question about reconnect failure a bit if you dont mind balusC. So my connection pool is set up via glassfish, so via admin console, I have set Creation Retry Attempts: 10 and Retry Interval:10 inside JDBC - ConnectionPools - MyPool. I dont think it fix the problem. Is there a way to track to see if Glassfish actually try to reconnect, because server log show nothing at all. Thank you so much for helping me out man :D – Ovenware

Ah, it returned 200 for a DatabaseException but not for others? Interesting. Let me check that, maybe I'll update the issue report. And you're welcome. – Hibernicism

Sorry, I couldn't reproduce it. Did you define any error pages in web.xml? Are you somewhere doing a manual redirect on exception? – Hibernicism

Actually let me take that back. I try a different ajax request and it work as expect. Something wrong with the previous component that I was testing on. Investigating now ... – Ovenware

Nothing seems abnormal from my ajax component <p:commandButton>, not sure why it would return 200 for DatabaseException. I wrote some code inside my original post. Would u take a look at them, please? – Ovenware

Answer 2

In my view at least, a database becoming unavailable is a serious error that needs to be handled and usually ASAP. The general approach on safeguarding against such situation is enforcing high-avaialbility and failover mechanisms through database clustering.

If you don't have this option however nor you want the user to see a big-fat-exception, what you might try to do is route him in some user-friendly page when this particular error happens. To do so, put the following in your web.xml (assuming that your FacesServlet is mapped to *.faces):

<error-page>
    <exception-type>com.mysql.jdbc.exceptions.jdbc4.CommunicationsException</exception-type>
    <location>/errors/error.faces</location>
</error-page>

Then, in the error.xhtml page the general approach is to put some user-friendly and largely apologetic message indicating how sorry the administrator feels for the user's inconvenience...

At any rate, dealing with such situations by catching exceptions and not acting upon them, either by re-throwing them or dealing with them in the catch block-if possible, is generally considered as a bad practice at critical errors might go un-noticed.

Regarding your "is never thrown in body of corresponding try statement" question you might want to check this article.

Cheers!

Conceive

I dont think it work. I put your code in my web.xml. Load my page, turn off the Mysql Server, make a request, it did not catch the exception correct. In other word, it did not redirect me to the error page that I specify. – Ovenware

If you put the CommunicationsException within the exception-type element it sounds normal. That is because by the time your exception reaches the web layer is most likely wrapped so it will not be filtered by the rule (the exception type will be different). I would suggest checking the logs to see what's the last exception thrown or alternatively try using a superclass (i.e. java.lang.Exception). – Conceive

Do I still catch the exception in my web.xml or in my code. I wrote some codes above to demonstrate my design. Will you please take a look at them? – Ovenware

I wouldn't suggest to catch runtime exceptions in your code, unless you have a relly good reason to do so. Usually runtime exceptions signify serious errors as the loss of database connectivity in your case. What i would do is first address the problem-that is why you loose the connection. At the same time however i would still use a generic error page using the mechanism i described, so that the user doesn't see a exception in his browser. In that respect, i fully agree with James' last comment above. At any rate, your code looks fine. – Conceive

There could be many reason why database connection got lost. I just want to make sure that when it happened, the application gracefully show a nice error message said "There an error occur, please try again later". What James said above are runtime exception correct? or is he/she talking about catching exception in your web.xml? – Ovenware

Answer 3

The CommunicationsException is wrapped in a EclipseLink DatabaseException, which is a runtime Exception. If you are using JPA or JTA, then this may also get wrapped in a PersistenceException, or a TransactionRolledbackException. So, try catching one of these, or worst case RuntimeException. The CommunicationsException will be in the caused by chain.

EclipseLink will automatically try to reconnect dead connections, but if your database is down, this will fail (you may see the error logged 3 times), so you will need to report an error to your user.

Robertoroberts

I have updated my code, please take a look at them as I have more question for u. From the codes above, where would I catch PersistenceException or TransactionRolledBackException. Thank you – Ovenware