What is an acceptable approach to dragging sprites with Bevy 0.4?

Question

While trying out Bevy, I had the need for dragging and dropping sprites. Unfortunately, this does not seem to come ready made or I did not find it in the documentation.

What would be the most idiomatic way to achieve this goal?

What I have tried so far is in my answer, but I'll gladly accept another solution that is better/faster/more idiomatic.

Snafu

@Shepmaster, thanks for the comments. I moved my solution into an answer, but am not sure how to add a minimal reproducible example that does not work, since I do not have one to start with. – Snafu

That comment was largely about the link to a complete example that is now part of your answer. To improve the question, you could consider if there is any missing information or limitations you might have in your brain but not explicitly listed here that would prevent someone from being able to answer. – Nydianye

Check this example from bevy_mod_picking for Bevy 0.8: github.com/aevyrie/bevy_mod_picking/blob/main/examples/… – Multitudinous

Answer 1

I'm not experienced enough to know what's idiomatic unfortunately, however, here's an overview of how I've implemented sprite dragging in my application, and it feels like a good way to me:

• I have a "cursor location" entity with a transform component (and a Cursor component for identification) that I update in a system each frame to the location of the cursor.
• Every draggable object has a Hoverable and Draggable component. I iterate over those objects in one system each where I add/remove a Hovered and Dragged component to the entities to indicate if they are hovered or dragged.
• I have a system that checks if an object is getting dropped, and if so gives it a Dropped component.
• I have a system that runs when an entity gets the 'Dragged' component (using the Added<C> filter), which sets the objects parent to the "cursor location" entity.
• And another system for when an entity gets the 'Dropped' component, which clears the parent.

To me, having many systems with small areas of responsibility feels good. I would be interested to hear opposing views as I lack experience.

There are of course many things I've left out in this overview, so here's my code for reference. There are some oddities and unnecessary code for a minimal example since this is adapted from my actual code:

#![allow(clippy::type_complexity)]

use bevy::{prelude::*, render::camera::Camera};

fn main() {
    App::build()
        .init_resource::<State>()
        .add_resource(WindowDescriptor {
            title: "Bevy".to_string(),
            width: 1024.0,
            height: 768.0,
            vsync: true,
            ..Default::default()
        })
        .add_plugins(DefaultPlugins)
        .add_plugin(MyPlugin)
        .run();
}

pub struct MyPlugin;

impl Plugin for MyPlugin {
    fn build(&self, app: &mut AppBuilder) {
        app.add_startup_system(setup.system())
            .add_system_to_stage(stage::PRE_UPDATE, cursor_state.system())
            .add_system_to_stage(stage::UPDATE, cursor_transform.system())
            .add_system_to_stage(stage::UPDATE, draggable.system())
            .add_system_to_stage(stage::UPDATE, hoverable.system())
            .add_system_to_stage(stage::POST_UPDATE, drag.system())
            .add_system_to_stage(stage::POST_UPDATE, drop.system())
            .add_system_to_stage(stage::POST_UPDATE, material.system());
    }
}

const SPRITE_SIZE: f32 = 55.0;

fn setup(
    commands: &mut Commands,
    asset_server: Res<AssetServer>,
    mut materials: ResMut<Assets<ColorMaterial>>,
) {
    let bevy_texture = asset_server.load("sprites/bevy-icon.png");

    commands
        .spawn(Camera2dBundle::default())
        .spawn(())
        .with(CursorState::default())
        .spawn((Transform::default(), GlobalTransform::default(), Cursor));

    for _ in 0..4 {
        commands
            .spawn(SpriteBundle {
                material: materials.add(bevy_texture.clone().into()),
                sprite: Sprite::new(Vec2::new(SPRITE_SIZE, SPRITE_SIZE)),
                ..Default::default()
            })
            .with(Hoverable)
            .with(Draggable);
    }
}

#[derive(Default)]
struct CursorState {
    cursor_world: Vec2,
    cursor_moved: bool,
}

struct Cursor;

struct Draggable;
struct Dragged;
struct Dropped;

struct Hoverable;
struct Hovered;

fn cursor_state(
    mut state: ResMut<State>,
    e_cursor_moved: Res<Events<CursorMoved>>,
    windows: Res<Windows>,
    mut q_cursor_state: Query<&mut CursorState>,
    q_camera: Query<&Transform, With<Camera>>,
) {
    let event_cursor_screen = state.er_cursor_moved.latest(&e_cursor_moved);

    for mut cursor_state in q_cursor_state.iter_mut() {
        if let Some(event_cursor_screen) = event_cursor_screen {
            let window = windows.get_primary().unwrap();
            let cam_transform = q_camera.iter().last().unwrap();
            cursor_state.cursor_world =
                cursor_to_world(window, cam_transform, event_cursor_screen.position);

            cursor_state.cursor_moved = true;
        } else {
            cursor_state.cursor_moved = false;
        }
    }
}

fn cursor_transform(
    commands: &mut Commands,
    q_cursor_state: Query<&CursorState>,
    mut q_cursor: Query<(Entity, &mut Transform), With<Cursor>>,
) {
    let cursor_state = q_cursor_state.iter().next().unwrap();

    for (cursor_e, mut transform) in q_cursor.iter_mut() {
        transform.translation.x = cursor_state.cursor_world.x;
        transform.translation.y = cursor_state.cursor_world.y;
        commands.remove_one::<Parent>(cursor_e);
    }
}

fn hoverable(
    commands: &mut Commands,
    q_cursor_state: Query<&CursorState>,
    q_hoverable: Query<(Entity, &Transform, &Sprite), (With<Hoverable>, Without<Dragged>)>,
) {
    let cursor_state = q_cursor_state.iter().next().unwrap();

    if cursor_state.cursor_moved {
        for (entity, transform, sprite) in q_hoverable.iter() {
            let half_width = sprite.size.x / 2.0;
            let half_height = sprite.size.y / 2.0;

            if transform.translation.x - half_width < cursor_state.cursor_world.x
                && transform.translation.x + half_width > cursor_state.cursor_world.x
                && transform.translation.y - half_height < cursor_state.cursor_world.y
                && transform.translation.y + half_height > cursor_state.cursor_world.y
            {
                commands.insert_one(entity, Hovered);
            } else {
                commands.remove_one::<Hovered>(entity);
            }
        }
    }
}

fn material(
    mut materials: ResMut<Assets<ColorMaterial>>,
    q_hoverable: Query<
        (&Handle<ColorMaterial>, Option<&Hovered>, Option<&Dragged>),
        With<Hoverable>,
    >,
) {
    let mut first = true;

    for (material, hovered, dragged) in q_hoverable.iter() {
        let (red, green, alpha) = if dragged.is_some() {
            (0.0, 1.0, 1.0)
        } else if first && hovered.is_some() {
            first = false;
            (1.0, 0.0, 1.0)
        } else if hovered.is_some() {
            (1.0, 1.0, 0.5)
        } else {
            (1.0, 1.0, 1.0)
        };

        materials.get_mut(material).unwrap().color.set_r(red);
        materials.get_mut(material).unwrap().color.set_g(green);
        materials.get_mut(material).unwrap().color.set_a(alpha);
    }
}

fn cursor_to_world(window: &Window, cam_transform: &Transform, cursor_pos: Vec2) -> Vec2 {
    // get the size of the window
    let size = Vec2::new(window.width() as f32, window.height() as f32);

    // the default orthographic projection is in pixels from the center;
    // just undo the translation
    let screen_pos = cursor_pos - size / 2.0;

    // apply the camera transform
    let out = cam_transform.compute_matrix() * screen_pos.extend(0.0).extend(1.0);
    Vec2::new(out.x, out.y)
}

fn draggable(
    commands: &mut Commands,
    i_mouse_button: Res<Input<MouseButton>>,
    q_pressed: Query<Entity, (With<Hovered>, With<Draggable>)>,
    q_released: Query<Entity, With<Dragged>>,
) {
    if i_mouse_button.just_pressed(MouseButton::Left) {
        if let Some(entity) = q_pressed.iter().next() {
            commands.insert_one(entity, Dragged);
        }
    } else if i_mouse_button.just_released(MouseButton::Left) {
        for entity in q_released.iter() {
            commands.remove_one::<Dragged>(entity);

            commands.insert_one(entity, Dropped);
        }
    }
}

fn drag(
    commands: &mut Commands,
    mut q_dragged: Query<(Entity, &mut Transform, &GlobalTransform), Added<Dragged>>,
    q_cursor: Query<(Entity, &GlobalTransform), With<Cursor>>,
) {
    if let Some((cursor_e, cursor_transform)) = q_cursor.iter().next() {
        for (entity, mut transform, global_transform) in q_dragged.iter_mut() {
            let global_pos = global_transform.translation - cursor_transform.translation;

            commands.insert_one(entity, Parent(cursor_e));

            transform.translation.x = global_pos.x;
            transform.translation.y = global_pos.y;
        }
    }
}

fn drop(
    commands: &mut Commands,
    mut q_dropped: Query<(Entity, &mut Transform, &GlobalTransform), Added<Dropped>>,
) {
    for (entity, mut transform, global_transform) in q_dropped.iter_mut() {
        let global_pos = global_transform.translation;

        transform.translation.x = global_pos.x;
        transform.translation.y = global_pos.y;

        commands.remove_one::<Parent>(entity);
        commands.remove_one::<Dropped>(entity);
    }
}

#[derive(Default)]
struct State {
    er_cursor_moved: EventReader<CursorMoved>,
}

This code is for bevy 0.4.

Unsteel

thanks for your example, which works great so far. I'll check how I can adapt this to my current task. – Snafu

Answer 2

This is the solution I came up with. Complete example

main.rs

use bevy::prelude::*;
use bevy::render::pass::ClearColor;
use bevy::window::CursorMoved;

const SPRITE_SIZE: f32 = 55.0;

fn main() {
    App::build()
        .add_resource(WindowDescriptor {
            width: 1000.0,
            height: 1000.0,
            resizable: false,
            title: "Bevy: drag sprite".to_string(),
            ..Default::default()
        })
        .add_resource(Msaa { samples: 4 })
        .add_resource(ClearColor(Color::rgb(0.9, 0.9, 0.9)))
        .add_plugins(DefaultPlugins)
        .add_startup_system(setup.system())
        .add_system(sprite_system.system())
        .add_system(bevy::input::system::exit_on_esc_system.system())
        .run();
}

fn setup(
    commands: &mut Commands,
    asset_server: Res<AssetServer>,
    mut materials: ResMut<Assets<ColorMaterial>>,
) {
    commands.spawn(Camera2dBundle::default());

    // show sprite in the middle of the screen
    let bevy_texture = asset_server.load("sprites/bevy-icon.png");
    commands.spawn(SpriteBundle {
        sprite: Sprite::new(Vec2::new(SPRITE_SIZE, SPRITE_SIZE)),
        material: materials.add(bevy_texture.clone().into()),
        ..Default::default()
    });
}

#[derive(Default)]
struct State {
    cursor_moved_event_reader: EventReader<CursorMoved>,
    // store current cursor/mouse position
    cursor_pos: Vec2,
    // store entity ID and the difference between sprite center and mouse click location
    sprite: Option<(Entity, Vec3)>,
}

fn sprite_system(
    mut state: Local<State>,
    windows: Res<Windows>,
    mouse_button_input: Res<Input<MouseButton>>,
    cursor_moved_events: Res<Events<CursorMoved>>,
    mut sprites: Query<(Entity, &Sprite)>,
    mut transforms: Query<&mut Transform>,
) {
    let window = windows.get_primary().unwrap();
    let half_window = Vec2::new(window.width() / 2.0, window.height() / 2.0);

    // if cursor has moved, transform to graphics coordinates and store in state.curser_pos
    if let Some(cursor_event) = state.cursor_moved_event_reader.latest(&cursor_moved_events) {
        state.cursor_pos = cursor_event.position - half_window;
        state.cursor_pos.x = state.cursor_pos.x;
    };

    // stop dragging if mouse button was released
    if mouse_button_input.just_released(MouseButton::Left) {
        state.sprite = None;
        return;
    }

    // set new sprite position, if mouse button is pressed and a sprite was clicked on
    // take previous click difference into account, to avoid sprite jumps on first move
    if mouse_button_input.pressed(MouseButton::Left) && state.sprite.is_some() {
        let sprite = state.sprite.unwrap();

        let mut sprite_pos = transforms.get_mut(sprite.0).unwrap();

        trace!("Sprite position old: {:?}", sprite_pos.translation);
        sprite_pos.translation.x = state.cursor_pos.x + sprite.1.x;
        sprite_pos.translation.y = state.cursor_pos.y + sprite.1.y;
        trace!("Sprite position new: {:?}", sprite_pos.translation);
        // position clamping was left out intentionally
    }

    // store sprite ID and mouse distance from sprite center, if sprite was clicked
    if mouse_button_input.just_pressed(MouseButton::Left) {
        for (entity, sprite) in sprites.iter_mut() {
            let sprite_pos = transforms.get_mut(entity).unwrap().translation;
            let diff = cursor_to_sprite_diff(&state.cursor_pos, &sprite_pos);
            // sprite is a circle, so check distance from center < sprite radius
            if diff.length() < (sprite.size.x / 2.0) {
                state.sprite = Some((entity, diff));
            }
        }
    }
}

fn cursor_to_sprite_diff(cursor_pos: &Vec2, sprite_pos: &Vec3) -> Vec3 {
    Vec3::new(
        sprite_pos.x - cursor_pos.x,
        sprite_pos.y - cursor_pos.y,
        0.0,
    )
}

Cargo.toml

[package]
name = "bevy-drag-sprite"
version = "0.1.0"
authors = ["Me"]
edition = "2018"

[dependencies]
bevy = "0.4"

Snafu

Answer 3

There is now a crate to handle picking with drag and drop if folks don't want to roll their own. https://crates.io/crates/bevy_mod_picking

Holladay