Calling GetOpenFileName through JNA fails for Swing app

Question

I'm trying to use the native Windows file dialog in Java, using JNA to call the comdlg32 function GetOpenFileName. I've made a static method, OpenFileDialog.display that looks like this:

 public static List<File> display(Window parent, boolean allowMultiSelect)

It should return the selected files, or null if the user canceled the dialog.

I have two simple test programs. This one works as expected:

package nativedialogs;

import java.io.File;
import java.util.List;

public class SimpleTest {

    public static void main(String[] args) {
        List<File> files = OpenFileDialog.display(null, true);
        System.out.println(files);
    }
}

This one, however, does not:

package nativedialogs;

import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.io.File;
import java.util.List;
import javax.swing.JButton;
import javax.swing.JFrame;
import javax.swing.SwingUtilities;

public class SwingTest {

    public static void main(String[] args) {

        SwingUtilities.invokeLater(new Runnable() {

            @Override
            public void run() {
                JFrame f = new JFrame();
                f.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
                JButton button = new JButton("Open file dialog");
                button.addActionListener(new ActionListener() {

                    @Override
                    public void actionPerformed(ActionEvent e) {
                        List<File> files = OpenFileDialog.display(f, true);
                        // These also fail:
                        // List<File> files = OpenFileDialog.display(f, false);
                        // List<File> files = OpenFileDialog.display(null, true);
                        // List<File> files = OpenFileDialog.display(null, false);
                        System.out.println(files);
                    }
                });
                f.add(button);
                f.pack();
                f.setVisible(true);
            }
        });
    }
}

For the latter example, CommDlgExtendedError returns 2, which according to MSDN is:

CDERR_INITIALIZATION 0x0002

The common dialog box function failed during initialization. This error often occurs when sufficient memory is not available.

...which doesn't really help me all that much. What am I doing wrong here?

---

I've put the other sources on PasteBin so I wouldn't clutter the question too much:

OpenFileDialog: http://pastebin.com/HDmu0TjX

ComDlg32JNA: http://pastebin.com/X5F5LLip

Venge

This may be a SWAG, but what if you clear your ByteBuffer before using it by calling buffer.clear();? – Gregory

That didn't make any difference, unfortunately. – Venge

Answer 1

It's better not to do any JNA code from the Swing EDT. Try using SwingWorker to run the dialog in a background thread.

I'd try to help more, but there is no comdlg32 on my Win 7 64-bit :(

Polygamous

Thanks for the suggestion, but doing it in a background thread didn't help either. :-( Funny that you don't have the DLL, 'cause I also run Win 7 64 and I have it. – Venge

@perp: what version of jna are you using? When I run your code with 3.3.0, it throws an UnsatisfiedLinkException. But it works fine with 3.2.7, event the Swing test. – Polygamous

I can't seem to make it work with either version. I've also tried both 32- and 64-bit JVMs. Next I'll try direct mapping, and maybe even skip JNA and go directly with JNI, just to see where the problem is. – Venge

@perp: you might want to give a try to this library before jni: code.google.com/p/bridj – Polygamous