The camera has a viewport that you can imagine as a movable overlay above the contents (with some background being displayed in areas where no contents are placed). For the sake of simplicity, I would separate scrolling (i.e. moving the viewport) from content transformations (e.g. zooming).
Based on this mental model, you can define the scrollable bounds to be the bounds of your contents as well as a possibly empty portion of the current viewport (e.g. in case of contents smaller than viewport). The scrollable bounds needs to be recomputed after every scroll operation (increasing/reducing empty space within the current viewport) or content manipulation (transformations and bounds changes). If you restrict scrolling to the scrollable bounds, then you can ensure that empty space within the viewport is never increased by a scroll operation.
You can create an ObjectBinding scrollableBounds that is bound to the contents' bounds-in-local and local-to-parent-transform properties, as well the viewport-bounds. Then you can create a scollableBoundsProperty that is bound to the binding. That property can be accessed when scrolling to restrict the translation before applying it, thus preventing an increase of empty space within the viewport.
ObjectBinding<Bounds> scrollableBoundsBinding = new ObjectBinding<>() {
{
// TODO: bind to dependencies: viewport bounds and content bounds
// TODO: (transformed to the same coordinate system)
bind(camera.boundsInParentProperty(),
contentPane.boundsInLocalProperty(),
contentPane.localToParentTransformProperty());
}
@Override protected Bounds computeValue() {
// TODO: compute union of viewport and content bounds
return unionBounds(viewportBounds, contentBounds);
}
};
ObjectProperty<Bounds> scrollableBoundsProperty = new SimpleObjectProperty<>(
scrollableBoundsBinding);
// ...
// on mouse drag:
// dx, dy: relative mouse movement
// tx, ty: current scrolling
// mintx, maxtx, minty, maxty: translation range
// (taken from scrollable bounds and viewport size)
if (dx < 0) { tx = max(mintx, tx + dx); }
else { tx = min(maxtx, tx + dx); }
if (dy < 0) { ty = max(minty, ty + dy); }
else { ty = min(maxty, ty + dy); }
You might want to further restrict scrolling when the contents fully fit within the viewport, e.g. by placing the contents at the top left corner. You could also restrict the minimal zoom level in that case so that the contents are displayed as big as possible.
Note on usability: As already pointed out by another answer, you might want to consider allowing to drag over the contents by a bit, possibly with decreasing efficiency the further away one tries to scroll from the contents, comparable to the behavior of scrolling via touchpad in Safari. Then, when the interaction finishes, you could transition back instead of snapping in order to restrict the viewport to the contents again.