Clearly you need to generate your Voronoi diagram to the constraints of the greater polygon. Although you refer to it as a polygon, I notice that your example diagram has spline-based edges. Let's forget that for now.
What you want to do is to ensure that you start out with the containing polygon (whether generated by you or from another source) having edges of fairly equal length; a variance factor would make this look more natural. I would probably go for a variance of 10-20%.
Now that you have your containing polygon bounded by lines segments of approximately equal length, you have a basis from which to begin generating your Voronoi diagram. For each edge on your container:
- Determine the edge normal (perp line jutting inward from centre of that segment).
- Use the edge normal as a sliding scale on which to place a new Voronoi node centre. The distance away from the edge itself would be determined by what you want your average Voronoi cell "diameter" to be, if they were all taken as circles. In your example that looks like maybe 30 pixels (or whatever the equivalent in your world units would be). Again, you should apply a variance factor to this so that not every cell centre is placed equidistant from its source edge.
- Generate the Voronoi cell for your newly placed centre.
- Store your Voronoi cell source point in a list.
As you incrementally generate each point, you should begin to see that the algorithm subdivides each convex "constituent area" of your concave container in a radial fashion.
You may be wondering what the list is for. Well, obviously, you're not done yet, you've only generated a fraction of the total Voronoi tesselation you want. Once you have created these "boundary" cells of your concave space, you don't want new cells to be generated closer to the boundary than the boundary cells already are, you only want them inside that area. By maintaining a list of the boundary cell source points, you can then ensure that any further points you create are inside that area. It's a little bit like taking an internal Minkowski sum to ensure you have a buffer zone. Now you can randomise the rest of your cells in this derived concave space, to completion.
(Caveat emptor: You will have to be careful with this previous step. If any "passage" areas are too narrow, then the boundaries of this derived space will overlap, you will have a non-simple polygon, and you may find yourself placing points in the wrong places in spite of your efforts. The solution is to ensure that either your maximum placement distance from edges is never more than half of your minimum passage width... or use some other geometric means, including Minkowski summation as one possibility, to ensure you do not wind up with a degenerate derived polygon. It is quite possible that you will end with a multipolygon, i.e. fragments.)
I've not applied this method myself yet, but although there will certainly be bugs to work out, I think the general idea will get you started in the right direction.