Can we use Vulkan with Java Activity on Android platform

Question

Currently, it seems all the Vulkan tutorials and samples use NativeActivity on Android platform. I would like to know whether we can use Vulkan with Java Activity on Android?

Guesstimate

Do you mean using Vulkan in a pure Java project? If so, not sure. If you mean using Vulkan in a Java Activity which calls native libraries (c/c++ code) to do the rendering, then yes it's possible - I'm doing it in some shipped titles. Unfortunately, I'm not at liberty to provide details nor am I aware of any samples that show how to set it up. – Tildi

lwjgl has a vulkan wrapper, though I'm not sure about android compat – Finable

Thanks for your advice. – Guesstimate

Answer 1

Yes, you can use Vulkan with your own Activity subclass. Because Android doesn't have Java-language bindings for Vulkan, you'll need to use either JNI or a third-party Java Vulkan library (which is just doing the JNI for you).

Your View hierarchy will need to contain a SurfaceView, and when you get the Surfaceholder.Callback#surfaceChanged callback you can get the Surface. If you're doing the JNI yourself, you can call ANativeWindow_fromSurface to get the ANativeWindow from the Surface, and use that to create your VkSurfaceKHR/VkSwapchainKHR.

The one thing to be careful of is to avoid blocking the main UI thread when calling VkAcquireNextImageKHR. Either arrange so that you only call that when it won't block for long, or put your frame loop on a separate thread.

Nonflammable

Answer 2

Say you have a C++ class which incapsulates Vulkan drawing logic:

// File: AndroidGraphicsApplication.hpp

#include <android/asset_manager.h>
#include <android/native_window.h>
#include "GraphicsApplication.h" // Base class shared with iOS/macOS/...

class AndroidGraphicsApplication : public GraphicsApplication {

public:
    AndroidGraphicsApplication(AAssetManager* assetManager, ANativeWindow* window): GraphicsApplication() {
        mAssetManager = assetManager;
        mWindow = window;
        // ... Vulkan initialisation code.
    }
    ~AndroidGraphicsApplication() {
        // ... Vulkan cleanup code.
    }

    void createSurface() {
       VkAndroidSurfaceCreateInfoKHR surface_info;
       surface_info.sType = VK_STRUCTURE_TYPE_ANDROID_SURFACE_CREATE_INFO_KHR;
       surface_info.pNext = NULL;
       surface_info.flags = 0;
       surface_info.window = mWindow;
       if(vkCreateAndroidSurfaceKHR(instance, &surface_info, NULL, &surface) != VK_SUCCESS) {
          throw std::runtime_error("failed to create window surface!");
       }
    }

    // Used to setup shaders.
    std::vector<char> readFile(const std::string& filename) {
       AAsset* file = AAssetManager_open(mAssetManager, filename.c_str(), AASSET_MODE_BUFFER);
       size_t size = AAsset_getLength(file);
       std::vector<char> data(size);
       AAsset_read(file, data.data(), size);
       AAsset_close(file);
       return data;
    }

    void setSize(uint32_t w, uint32_t h) {
       width = w;
       height = h;
    }

private:
    AAssetManager* mAssetManager;
    ANativeWindow* mWindow;
    uint32_t width;
    uint32_t height;
};

And you have JNI bridge like below:

// File: VulkanAppBridge.cpp

#include <android/log.h>
#include <android/native_window_jni.h>
#include <android/asset_manager_jni.h>
#include "AndroidGraphicsApplication.hpp"

AndroidGraphicsApplication *mApplicationInstance = NULL;

extern "C" {

   JNIEXPORT void JNICALL
   Java_com_mc_demo_vulkan_VulkanAppBridge_nativeCreate(JNIEnv *env, jobject vulkanAppBridge,
         jobject surface, jobject pAssetManager) {
      if (mApplicationInstance) {
         delete mApplicationInstance;
         mApplicationInstance = NULL;
      }
      __android_log_print(ANDROID_LOG_DEBUG, "mc-native-VulkanAppBridge", "create");
      auto window = ANativeWindow_fromSurface(env, surface);
      auto assetManager = AAssetManager_fromJava(env, pAssetManager);
      mApplicationInstance = new AndroidGraphicsApplication(assetManager, window);
   }

   JNIEXPORT void JNICALL
   Java_com_mc_demo_vulkan_VulkanAppBridge_nativeDestroy(JNIEnv *env, jobject vulkanAppBridge) {
      __android_log_print(ANDROID_LOG_DEBUG, "mc-native-VulkanAppBridge", "destroy");
      if (mApplicationInstance) {
         delete mApplicationInstance;
         mApplicationInstance = NULL;
      }
   }

   JNIEXPORT void JNICALL
   Java_com_mc_demo_vulkan_VulkanAppBridge_nativeResize(JNIEnv *env, jobject vulkanAppBridge, jint width, jint height) {
      __android_log_print(ANDROID_LOG_DEBUG, "mc-native-VulkanAppBridge", "resize: %dx%d", width, height);
      if (mApplicationInstance) {
         mApplicationInstance->setSize(width, height);
         mApplicationInstance->isResizeNeeded = true;
      }
   }

   JNIEXPORT void JNICALL
   Java_com_mc_demo_vulkan_VulkanAppBridge_nativeDraw(JNIEnv *env, jobject vulkanAppBridge) {
      __android_log_print(ANDROID_LOG_DEBUG, "mc-native-VulkanAppBridge", "draw");
      if (mApplicationInstance) {
         mApplicationInstance->drawFrame();
      }
   }
}

And you have a corresponding Java/Kotlin part of JNI bridge:

// File: VulkanAppBridge.kt

class VulkanAppBridge {

   init {
      System.loadLibrary("myApplication")
   }

   private external fun nativeCreate(surface: Surface, assetManager: AssetManager)
   private external fun nativeDestroy()
   private external fun nativeResize(width: Int, height: Int)
   private external fun nativeDraw()

   fun create(surface: Surface, assetManager: AssetManager) {
      nativeCreate(surface, assetManager)
   }

   fun destroy() {
      nativeDestroy()
   }

   fun resize(width: Int, height: Int) {
      nativeResize(width, height)
   }

   fun draw() {
      nativeDraw()
   }
}

And you have a custom subclass of the SurfaceView:

// File: VulkanSurfaceView.kt

class VulkanSurfaceView: SurfaceView, SurfaceHolder.Callback2 {

    private var vulkanApp = VulkanAppBridge()

    constructor(context: Context): super(context) {
    }

    constructor(context: Context, attrs: AttributeSet): super(context, attrs) {
    }

    constructor(context: Context, attrs: AttributeSet, defStyle: Int): super(context, attrs, defStyle) {
    }

    constructor(context: Context, attrs: AttributeSet, defStyle: Int, defStyleRes: Int): super(context, attrs, defStyle, defStyleRes) {
    }

    init {
        alpha = 1F
        holder.addCallback(this)
    }

    // ...
    // Implementation code similar to one in GLSurfaceView is skipped.
    // See: https://android.googlesource.com/platform/frameworks/base/+/master/opengl/java/android/opengl/GLSurfaceView.java
    // ...

    override fun surfaceChanged(holder: SurfaceHolder?, format: Int, width: Int, height: Int) {
        vulkanApp.resize(width, height)
    }

    override fun surfaceDestroyed(holder: SurfaceHolder?) {
        vulkanApp.destroy()
    }

    override fun surfaceCreated(holder: SurfaceHolder?) {
        holder?.let { h ->
            vulkanApp.create(h.surface, resources.assets)
        }
    }

    override fun surfaceRedrawNeeded(holder: SurfaceHolder?) {
        vulkanApp.draw()
    }
}

Then you can use your custom VulkanSurfaceView inside layout with a custom size together with other views:

<?xml version="1.0" encoding="utf-8"?>

<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:orientation="vertical"
    android:id= "@+id/linearlayout1" >

    <Button
        android:id="@+id/mcButtonTop"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="A Button" />

    <com.mc.demo.vulkan.MyGLSurfaceView
        android:id="@+id/mcSurfaceView"
        android:layout_width="fill_parent"
        android:layout_height="wrap_content"
        android:layout_weight="0.23" />

    <!-- Custom SurfaceView -->
    <com.mc.demo.vulkan.VulkanSurfaceView
        android:id="@+id/mcVulkanSurfaceView"
        android:layout_width="fill_parent"
        android:layout_height="wrap_content"
        android:layout_weight="0.23" />

    <Button
        android:id="@+id/mcButtonBottom"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="A Button" />

</LinearLayout>

Result:

Here is a link to "Vulkan Case Study" which has Android usage example: https://www.khronos.org/assets/uploads/developers/library/2016-vulkan-devu-seoul/2-Vulkan-Case-Study.pdf

Gader

This looks good, can you share the source code please? – Tendinous

Answer 3

If anyone lands here looking for a working implementation of VulkanSurfaceView, there is one here:

https://github.com/google-ar/arcore-android-sdk/tree/main/samples/hello_ar_vulkan_c

Calamitous

A link to a solution is welcome, but please ensure your answer is useful without it: You need to provide at least a technical summary of how the problem is solved, so that it can be reproduced even without the link. It is not enough to advertise what it achieves. Also please add context around the link so your fellow users will have some idea what it is and why it is there. Answers that are little more than a link may be deleted. – Louisiana

While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. - From Review – Absalom