Concurrently write to XML file

Question

I have multiple processes running on different machines which are required to read/write to a shared XML file, for this I am using DOM with Java and FileLocks (While I know that a database would be a more effective approach, this is not viable due to project constraints) .

To make changes to the XML file, the relevant process first creates an exclusively locked channel which is used to read the file, it then attempts to reuse the same channel to write the new version before closing the channel; this way the lock is never down. The issue however is that I am getting a java.nio.channels.ClosedChannelException when attempting to write the result, even though I never explicitly close the channel. I have suspicions that the line of code:

doc = dBuilder.parse(Channels.newInputStream(channel));

closes the channel. If so, how could I force the channel to stay open? My code can be seen below:

[removed code after update]

UPDATE: Placing System.out.println(channel.isOpen()) before and after the suspect line of code confirms that this is where the channel is closed.

UPDATE: Having asked a separate question the code below now prevents the channel from closing during the parse operation. The issue now is that instead of replacing the original xml file, the transformer appends the changed document to the original. In the documentation I cannot find any related options for specifying the output of Transformer.transform (I have searched Transformer/Transformer factory/StreamResult). Am I missing something? Do I need to somehow clear the channel before writing? Thanks.

UPDATE: Finally solved the append issue by truncating the channel to a size of 0. Thank you @JLRishe for the advice. Have posted the working code as an answer.

Pasho

Answer 1

This is the code which finally works! See question updates for explanations of different parts.

import java.io.*;
import java.nio.channels.*;

import javax.xml.parsers.*;
import javax.xml.transform.*;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import javax.xml.xpath.*;

import org.w3c.dom.*;
import org.xml.sax.SAXException;

public class Test2{ 
    String path = "...Test 2.xml";

    public Test2(){
        Document doc = null;
        DocumentBuilderFactory dbFactory;
        DocumentBuilder dBuilder;
        NodeList itemList;
        Transformer transformer;
        FileChannel channel; 
        Element newElement;
        int prevNumber;
        TransformerFactory transformerFactory ;
        DOMSource source;
        StreamResult result;
        NonClosingInputStream ncis = null;
        try {
            channel = new RandomAccessFile(new File(path), "rw").getChannel();
            FileLock lock = channel.lock(0L, Long.MAX_VALUE, false);

            try {
                dbFactory = DocumentBuilderFactory.newInstance();
                dBuilder = dbFactory.newDocumentBuilder();
                ncis = new NonClosingInputStream(Channels.newInputStream(channel));
                doc = dBuilder.parse(ncis);
            } catch (SAXException | IOException | ParserConfigurationException e) {
                e.printStackTrace();
            }
            doc.getDocumentElement().normalize();
            itemList = doc.getElementsByTagName("Item");
            newElement = doc.createElement("Item");
            prevNumber = Integer.parseInt(((Element) itemList.item(itemList.getLength() - 1)).getAttribute("Number"));
            newElement.setAttribute("Number", (prevNumber + 1) + "");

            doc.getDocumentElement().appendChild(newElement);

            transformerFactory = TransformerFactory.newInstance();
            transformer = transformerFactory.newTransformer();
            source = new DOMSource(doc);
            channel.truncate(0);
            result = new StreamResult(Channels.newOutputStream(channel));   

            transformer.setOutputProperty("{http://xml.apache.org/xslt}indent-amount", "4");
            transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "no");
            transformer.setOutputProperty(OutputKeys.METHOD, "xml");
            transformer.setOutputProperty(OutputKeys.INDENT, "yes");
            transformer.setOutputProperty(OutputKeys.ENCODING, "UTF-8");
            transformer.transform(source, result);
            channel.close();
        } catch (IOException | TransformerException e) {
            e.printStackTrace();
        } finally {
            try {
                ncis.reallyClose();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }

    class NonClosingInputStream extends FilterInputStream {

        public NonClosingInputStream(InputStream it) {
            super(it);
        }

        @Override
        public void close() throws IOException {
            // Do nothing.
        }

        public void reallyClose() throws IOException {
            // Actually close.
            in.close();
        }
    }

    public static void main(String[] args){
        new Test2();
    }
}

Pasho

Answer 2

Try this design instead:

1. Create a new service (a process) which opens a socket and listens to "update commands".
2. All other processes don't write to the file directly but instead send "update commands" to the new service

That way, you never need to worry about locking. To make the whole thing more reliable, you may want to add buffers to the sending processes so they can continue to live for a while when the service is down.

With this approach, you never have to deal with file locks (which can be unreliable depending on your OS). The socket will also make sure that you can't start the service twice.

Embry

Hi Aaron, that does sound like a better solution, and one which I may look into in the future. For now though, the project is fairly small scale so I would prefer to get it working as it is. – Pasho

Try to rewind the channel after reading it. Or try to come up with a smaller test case where you just read a simple String; for your test, it's not necessary to add the complexity which XML brings. If the small test works, then there is a bug/problem in the DocumentBuilderFactory or maybe the InputStream returned has a finalize() method. – Embry

In that case, you're probably out of luck. AFAIK, DocumentBuilderFactory doesn't close the stream. Maybe try to set a breakpoint in Channel.close() :-/ – Embry