Is it possible to start a zookeeper server instance in process, say for unit tests?

Question

Calling org.apache.zookeeper.server.quorum.QuorumPeerMain.main() isn't working.

Natant

Also, the curator framework ships with a in memory zookeeper implementation for use in unit tests – Baking

curator does ship with a testing zookeeper, but it is not in memory. it stores data on disk, in a temp directory. – Homogeny

Answer 1

To start ZooKeeper you have to execute ZooKeeperServerMain class.

You can use following code to start ZooKeeper in embedded mode.

Properties startupProperties = ...

QuorumPeerConfig quorumConfiguration = new QuorumPeerConfig();
try {
    quorumConfiguration.parseProperties(startupProperties);
} catch(Exception e) {
    throw new RuntimeException(e);
}

zooKeeperServer = new ZooKeeperServerMain();
final ServerConfig configuration = new ServerConfig();
configuration.readFrom(quorumConfiguration);

new Thread() {
    public void run() {
        try {
            zooKeeperServer.runFromConfig(configuration);
        } catch (IOException e) {
            log.error("ZooKeeper Failed", e);
        }
    }
}.start();

Kesterson

This is the way to go for unit tests. Using QuorumPeerMain is only for Multi Server (Clustered) Zookeeper. – Psychometry

This is useful in the context of BeforeClass, but shutting this down between tests (to clean out the nodes, for example) is non-trivial -- I gave up and switched to a Curator-based solution. (thread.stop/interrupt in After would stop the JUnitRunner) – Consequently

Any chance you know how to set this up in a C++ project? – Voltaic

serup, Zookeeper is a Java process – Kesterson

Hi, can you give an example of startup properties to set? – Anemochore

Answer 2

Netfix opensourced Curator a framework to make use of Zookeeper even more convenient. It has build in test server class. Just add this test dependency to your project descriptor be it maven, gradle or else:

org.apache.curator:curator-framework:4.0.1
org.apache.curator:curator-test:4.0.1

And here are the test essentials.

TestingServer zkTestServer;
CuratorFramework cli;

@Before
public void startZookeeper() throws Exception {
    zkTestServer = new TestingServer(2181);
    cli = CuratorFrameworkFactory.newClient(zkTestServer.getConnectString(), new RetryOneTime(2000));
    cli.start();
}

@After
public void stopZookeeper() throws IOException {
    cli.close();
    zkTestServer.stop();
}

With cli creating any test data is very easy (requires the curator-framework dependency).

cli.create()
   .creatingParentsIfNeeded()
   .forPath("/a1", "testvalue".getBytes("UTF-8"));

Gilkey

For me it didn't work without cli.start() using Curator version 2.10.0 – Yelenayelich

@MichaelBöckling thanks! I also needed to call cli.start() first then it started working – Fashionable

Hi @gertas. Ahem, could Curator be used to monitor an ensemble of zookeepers, and if 1 of the zookeeper instance went down, could it be used to restart it automatically? – Tillietillinger

Answer 3

You can use something like this.

int clientPort = 21818; // none-standard
int numConnections = 5000;
int tickTime = 2000;
String dataDirectory = System.getProperty("java.io.tmpdir");

File dir = new File(dataDirectory, "zookeeper").getAbsoluteFile();

ZooKeeperServer server = new ZooKeeperServer(dir, dir, tickTime);
NIOServerCnxn.Factory standaloneServerFactory = new NIOServerCnxn.Factory(new InetSocketAddress(clientPort), numConnections);

standaloneServerFactory.startup(server); // start the server.

And to shut it down just call standaloneServerFactory.shutdown()

Floydflss

Answer 4

Building on 1's answer by adding the use of an ephemeral port (shown by zkPort) and updated for latest ZK API:

int tickTime = 2000;
int numConnections = 5000;
String dataDirectory = System.getProperty("java.io.tmpdir");

File dir = new File(dataDirectory, "zookeeper").getAbsoluteFile();

ZooKeeperServer server = new ZooKeeperServer(dir, dir, tickTime);
standaloneServerFactory = ServerCnxnFactory.createFactory(0, numConnections);
int zkPort = standaloneServerFactory.getLocalPort();

standaloneServerFactory.startup(server);

Transnational

You probably can't start zoo keeper server this way as it will block all subsequent code from executing after standaloneServerFactory.startup(server); – Sledgehammer

Answer 5

ServerConfig config = new ServerConfig();
config.parse(new String[] {port, dir});
ZooKeeperServerMain zk = new ZooKeeperServerMain();
zk.runFromConfig(config);

Gasp

You can't run zookeeper server this way as it will block the following code i.e. code after zk.runFromConfig(config); from getting executed. – Sledgehammer

Answer 6

An updated version of GeoffBourne's answer.

    int clientPort = 2199; // not standard
    int numConnections = 5000;
    int tickTime = 2000;
    String dataDirectory = System.getProperty("java.io.tmpdir");

    File dir = new File(dataDirectory, "zookeeper").getAbsoluteFile();

    ZooKeeperServer server = new ZooKeeperServer(dir, dir, tickTime);
    ServerCnxnFactory factory = new NIOServerCnxnFactory();
    factory.configure(new InetSocketAddress(clientPort), numConnections);

    factory.startup(server); // start the server.

    // ...shutdown some time later
    factory.shutdown();

Dorwin