How to read pixels from SubViewport image?

Question

Hi! I'm making a code for reading the image of a SubViewport node and then running a for loop to get all pixels colors values and averaging each channel (red, green and blue) to get each pixel`s lightness value. Next I'm appeding these value to an array.

The SubViewport image is my by getting the image of an orthogonal camera pointed at a sphere.

For some reason when I'm debugging I found that all pixel color values were (0,0,0).

(by the way how do I insert code properly here?)

This is the code I'm using:

extends Node3D

var LightLevel : float

Called when the node enters the scene tree for the first time.

func _ready() -> void:
pass # Replace with function body.

Called every frame. 'delta' is the elapsed time since the previous frame.

func _process(delta: float) -> void:

# The following code will take a screenshot of our camera and create an array of floats
# that will be used to average the float values so we can determine our general lightness
# values. Obs: An image is like an array of pixels
#var image : Image = get_node("SubViewportContainer/SubViewport").get_texture().get_image()

var LightDetectSV : SubViewport = get_tree().get_nodes_in_group("LightDetectSubViewport")[0]

var image : Image = LightDetectSV.get_texture().get_image()

var floats : Array[float]

for y in range(0, image.get_height()):
	for x in range (0, image.get_width()):
		var pixel = image.get_pixel(x,y)
		var lightValue = (pixel.r + pixel.g + pixel.b) / 3
		floats.append(lightValue)
return average(floats)
pass

func average(numbers: Array[float]) :
var sum = 0.0
for n in numbers:
sum += n
return sum / numbers.size()

Here is the link for this project folder:
https://drive.google.com/drive/folders/1C2btE2fwocHtXslp1rv5zKpiYL_rWnyl?usp=sharing

Vellum

Answer 1

Try setting viewport's update mode to Always
For proper code formatting enclose the block between lines containing only ~~~ or ```

Titanesque

Answer 2

Hey, sorry for the delay I had a pretty busy day.

Where can I find this Viewport's update mode option? I looked on google and couldn't find it

Vellum

Answer 3

It's in Render Target section in the Inspector.

Titanesque

Answer 4

Philippopolis If it is the sub-menu under the render target for the SubViewport Node it already set like this (and won't work as I want)

Vellum

Answer 5

You have no lights in the scene so all pixels end up being fully black which results in final value of 0.0

Titanesque

Answer 6

That's because I'm instancing this LightDetect packed scene into my Character scene, and the character packed scene is being instanced in my testLevel scene. So I was expecting it would create images using the testLevel scene light.

Vellum

Answer 7

If you run it in testLevel it won't give all zeros. Test it out by printing the entire array from the function that creates it.

Titanesque

Answer 8

I am running it on my testLevel scene. And lightValue gets equals to zero and the floats array is [0,0,0,....]. I put a print(lightValue) line and also inserted a breakpoint to debug and see each variable value. Those are the values I get

Vellum

Answer 9

You need to manually update global position of Camera3D that's parented to SubViewport, every frame. Best to do it in _process() in LightDetect.gd. It won't automatically inherit any 3D transforms from ancestor nodes because 3D transformation chain is broken by inserting non-3D nodes (SubViewport and SubViewportContainer) inbetween.

Alternatively you can use RemoteTransform3D node to drive camera's position.

Titanesque

Answer 10

I tried both ways. I'm not sure I did it correctly. First I tried the RemoteTransform3D node (didn't know this node before) (How do I highlight these names?). I set the Remote Path to the Light Detect (Node 3D parent) and it did not work.

Then I tried the following code on _process() in LightDetect.gd and also got the same result.

extends Node3D

var LightLevel : float

@onready var camera_3d: Camera3D = $SubViewportContainer/SubViewport/Camera3D
@onready var light_detect: Node3D = $"."

func _process(delta: float) -> void:
	
	# The following code will take a screenshot of our camera and create an array of floats
	# that will be used to average the float values so we can determine our general lightness
	# values. Obs: An image is like an array of pixels
	#var image : Image = get_node("SubViewportContainer/SubViewport").get_texture().get_image()
	
	camera_3d.global_position = light_detect.global_position + Vector3(0,1.3,0)
	
	var LightDetectSV : SubViewport = get_tree().get_nodes_in_group("LightDetectSubViewport")[0]
	
	var image : Image = LightDetectSV.get_texture().get_image()
	
	var floats : Array[float]
	
	for y in range(0, image.get_height()):
		for x in range (0, image.get_width()):
			var pixel = image.get_pixel(x,y)
			var lightValue = (pixel.r + pixel.g + pixel.b) / 3
			floats.append(lightValue)
	return average(floats)
	pass

func average(numbers: Array[float]) :
	var sum = 0.0
	for n in numbers:
		sum += n
	return sum / numbers.size()

Vellum

Answer 11

it did not work

In what way it didn't work?

Also why do you return something from _process() whose return value is declared void?

Titanesque

Answer 12

In what way it didn't work?

The lightValue is always zero and the floats array is always [0,0,0,...0]. There values should be in a 0 to 1 interval.

Also why do you return something from _process() whose return value is declared void?

I should probably have made the floats array a "public" global variable instead of trying to return something. It needs to be read by other scripts.

Vellum

Answer 13

Instead of return average(float) put print(floats) and run the main scene. What gets printed?

Titanesque

Answer 14

Now it worked!
I had been missing 2 thing:

1. I completely forgot to assign LightLevel = average(floats)

2. At some point I hid the LightDetect scene nodes i-i

Thanks for pointing out I needed to move the camera LightDetect scene camera by code. I had no idea. I just checked and you're completely right.

Vellum