Start Confluent Schema Registry in windows

Question

I have windows environment and my own set of kafka and zookeeper running. To use custom objects, I started to use Avro. But I needed to get the registry started. Downloaded Confluent platform and ran this:

$ ./bin/schema-registry-start ./etc/schema-registry/schema-registry.properties
/c/Confluent/confluent-3.0.0-2.11/confluent-3.0.0/bin/schema-registry-run-class: line 103: C:\Program: No such file or directory

Then I see this on the installation page:

"Confluent does not currently support Windows. Windows users can download and use the zip and tar archives, but will have to run the jar files directly rather than use the wrapper scripts in the bin/ directory."

I was wondering how do I go about starting confluent schema registry in Windows environment ?

Looked at contents of scripts and it is difficult to decipher.

Thanks

Overcheck

Answer 1

Someone has created Windows .bat files as Ewen Cheslack-Postava suggests

https://github.com/renukaradhya/confluentplatform/tree/master/bin/windows

I saved schema-registry-run-class.bat and schema-registry-start.bat into my \confluent\bin\windows directory and then was able to run Schema Registry with

C:\confluent\bin\windows\schema-registry-start.bat C:\confluent\etc\ schema-registry\schema-registry.properties

Marsipobranch

Error: Could not find or load main class io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain Caused by: java.lang.ClassNotFoundException: io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain – Taciturnity

Answer 2

At the moment Confluent Platform does not ship with any scripts for Windows. However, you can write your own if you're comfortable with running Java applications. The schema-registry-server-start script (and the schema-registry-run-class script it depends on) do things like handle -daemon mode, set Java memory options, setup default log configuration, and more, but ultimately the key piece is that they execute Java with io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain as the main method. You might also find the kafka-run-class.bat from Kafka as a useful base: https://github.com/apache/kafka/blob/trunk/bin/windows/kafka-run-class.bat

Authorize

Answer 3

The issue is likely the presence of spaces in the JAVA_HOME environment setting for your Windows environment (as evidenced by the error message of "C:\Program" rather than "C:\Program Files..." .

You can determine the EXACT syntax of the final java invocation to launch schema_registry by adding replacing the last "exec" in the schema-registry-run-class script with the word "echo". You'll see the rather convoluted command

/Library/Java/JavaVirtualMachines/jdk1.8.0_91.jdk/Contents/Home/bin/java -Xmx512M -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+DisableExplicitGC -Djava.awt.headless=true -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dlog4j.configuration=file:/opt/confluent/bin/../etc/schema-registry/log4j.properties -cp :/opt/confluent/bin/../package-schema-registry/target/kafka-schema-registry-package-*-development/share/java/schema-registry/*:/opt/confluent/bin/../share/java/confluent-common/*:/opt/confluent/bin/../share/java/rest-utils/*:/opt/confluent/bin/../share/java/schema-registry/* io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain

That command boils down to "java [core-java-opts] [java-defines] -cp [classpath] io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain

If you replace the "/opt/confluent" references with the actual location of your Confluent install, I suspect you'll have much better luck.

NOTE: I prefer to install Java on windows to a customer location (eg "C:\java8", since many standard scripts will have problems with C:\Program Files deployment location.

Spunky

Thanks for the tip on Java8 installation location! – Overcheck

Answer 4

I've had success running the confluent tools from cmd.exe using cygwin.

C:\>c:\cygwin64\bin\bash -l /cygdrive/c/confluent/4.0.0/bin/kafka-avro-console-consumer --bootstrap-server <my_server_name>:9092 --topic <my_topic> --property schema.registry.url=http://<my_schema_registry_url>:8081 >> tmp.txt

Anikaanil

Answer 5

The code for schema registry run class bat file is : Save as schema-registry-run-class.bat

@echo off

setlocal EnableExtensions EnableDelayedExpansion
pushd %~dp0..\..
set BASE_DIR=%CD%
popd

for %%i in (%BASE_DIR%/package-schema-registry/target/kafka-schema-registry-package-*-development) do (
    call :concat %%i/share/java/schema-registry/*
)

for %%i in (confluent-common, rest-utils, schema-registry) do (
    call :concat %BASE_DIR%/share/java/%%i/*
)

rem Log4j settings
IF ["%SCHEMA_REGISTRY_LOG4J_OPTS%"] EQU [""] (
    if exist %~dp0../../etc/schema-registry/log4j.properties (
        set SCHEMA_REGISTRY_LOG4J_OPTS=-Dlog4j.configuration=file:%~dp0../../etc/schema-registry/log4j.properties
    ) else (
        set SCHEMA_REGISTRY_LOG4J_OPTS=-Dlog4j.configuration=file:%BASE_DIR%/config/log4j.properties
    )
)

rem JMX settings
IF ["%SCHEMA_REGISTRY_JMX_OPTS%"] EQU [""] (
    set SCHEMA_REGISTRY_JMX_OPTS=-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false  -Dcom.sun.management.jmxremote.ssl=false
)

rem JMX port to use
IF ["%JMX_PORT%"] NEQ [""] (
    set SCHEMA_REGISTRY_JMX_OPTS=%SCHEMA_REGISTRY_JMX_OPTS% -Dcom.sun.management.jmxremote.port=%JMX_PORT%
)

rem Which java to use
IF ["%JAVA_HOME%"] EQU [""] (
    set JAVA=java
) ELSE (
    set JAVA="%JAVA_HOME%/bin/java"
)

rem Memory options
IF ["%SCHEMA_REGISTRY_HEAP_OPTS%"] EQU [""] (
    set SCHEMA_REGISTRY_HEAP_OPTS=-Xmx512M
)

rem JVM performance options
IF ["%SCHEMA_REGISTRY_JVM_PERFORMANCE_OPTS%"] EQU [""] (
    set SCHEMA_REGISTRY_JVM_PERFORMANCE_OPTS=-server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+DisableExplicitGC -Djava.awt.headless=true
)

set COMMAND=%JAVA% %SCHEMA_REGISTRY_HEAP_OPTS% %SCHEMA_REGISTRY_JVM_PERFORMANCE_OPTS% %SCHEMA_REGISTRY_JMX_OPTS% %SCHEMA_REGISTRY_LOG4J_OPTS% -cp %CLASSPATH% %SCHEMA_REGISTRY_OPTS% %*
%COMMAND%

goto :eof
:concat
IF ["%CLASSPATH%"] EQU [""] (
  set CLASSPATH="%1"
) ELSE (
  set CLASSPATH=%CLASSPATH%;"%1"
)

Domesticate

Answer 6

The code for schema registry bat file. Save as schema-registry-start.bat

@echo off

%~dp0schema-registry-run-class.bat io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain %*

The schema registry properties file should look like this: Save as schema-registry.properties

listeners=http://10.91.31.169:8081
kafkastore.connection.url=10.91.31.169:2181
kafkastore.topic=_schemas
debug=true

Domesticate

Doesnt work for me Error: Could not find or load main class io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain Caused by: java.lang.ClassNotFoundException: io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain – Toomay