How do I convert screen space to world space coords in Bevy using Camera2dComponents?

Question

Using Res<Events<CursorMoved>> I can get the mouse position change in screen space coordinates (zero in bottom-left corner), e.g.

#[derive(Default)]
struct State {
    cursor_moved_reader: EventReader<CursorMoved>,
}

fn set_mouse_pos(mut state: ResMut<State>, cursor_moved: Res<Events<CursorMoved>>) {
    for event in state.cursor_moved_reader.iter(&cursor_moved) {
        println!("cursor: {:?}", event.position);
    }
}

The question now is if I set SpriteComponents' transform.translation to the cursor position when using a camera from Camera2dComponents::default() a sprite with position 0, 0 is rendered in the center of the screen. What's the idiomatic way of converting between the screen space mouse coords and the camera world space coords?

Juttajutty

Don't think there's something idiomatic. Seems like you'll have to interpolate 0 to 1 interval of screen space (or 0 to max_pixel_idx, depending on convention) to -1 to 1 interval of camera space. That's just a linear interpolation, e.g. camera_x = screen_x/max_screen_x * 2.0 - 1.0 – Santalaceous

I believe this may help – Greyso

The cheat book has moved to here. I've also added an answer for that link, because I think this is the best reference, and it's kind of hidden in this comment here. – Benny

Answer 1

The previous answer seems to be on the right track, however the transformation is not fully implemented and actually overengineered. Stumbled on the same problem and figured that's what the camera transform is for! This works for me:

fn window_to_world(
    position: Vec2,
    window: &Window,
    camera: &Transform,
) -> Vec3 {

    // Center in screen space
    let norm = Vec3::new(
        position.x - window.width() / 2.,
        position.y - window.height() / 2.,
        0.,
    );

    // Apply camera transform
    *camera * norm

    // Alternatively:
    //camera.mul_vec3(norm)
}

Perfectible

How do I obtain the camera Transform? Suppose I use OrthographicCameraBundle::new_2d(). – Frenulum

You use a query like transform: Query<&Transform, With<Camera>>. – Benny

Answer 2

I had a very similar confusion when I first started hacking with bevy. Here's what I am currently using to translate from window to world coordinates (for a 2D game):

fn window_to_world(
    window: &Window,
    camera: &Transform,
    position: &Vec2,
) -> Vec3 {
    let center = camera.translation.truncate();
    let half_width = (window.width() / 2.0) * camera.scale.x;
    let half_height = (window.height() / 2.0) * camera.scale.y;
    let left = center.x - half_width;
    let bottom = center.y - half_height;
    Vec3::new(
        left + position.x * camera.scale.x,
        bottom + position.y * camera.scale.y,
        0.0,  // I'm working in 2D
    )
}

In bevy, the Window coordinate system has the origin (0, 0) at the bottom left, measured in pixels, and increasing as you go up the screen. The Camera's coordinates default to (0, 0) in the center of the window, measured in whichever unit makes the most sense for your game.

This function is limited because it doesn't take rotation of the camera into account, only the scale.

Naif