As far as I can see, there is no way to make the fill pattern origin follow the map or geometry as it is panned or moved when using it with vector layers. Openlayers just uses the pattern as the fill without translating it, so it will always be statically positioned relative to the the canvas.
However, PR #5854 introduces an example where it works. It's not live on the examples page yet, but can be viewed on http://jsfiddle.net/61b7szjn/.
The difference between this and your example is that it uses an ImageVector source (making it an Image layer). Doing that would make it work, such as in your example:
var imageVectorLayer = new ol.layer.Image({
source: new ol.source.ImageVector({
source: vectorSource,
style: styleFunction
}),
})
Using Image layers for vector data might not be suitable for all situations, so adding support for translating the fill pattern seems like great PR material.
Edit:
Upon further testing, it seems like the ImageVector
solutions is not a guaranteed fix. The pattern generally follows along as I pan, but is sometimes moves to different offsets in unpredictable ways. I assume it's because of how the rendered source canvas is clipped: