Incidentally, an easy way to select everything in a layer (and thus make them act even more like a group and muddy the waters of how these two things differ even more!) is to click on the small circle icon on the right of the layers panel. This selects everything, including groups, within that layer. Additionally, holding down alt and then clicking selects everything in all the layers.
A highly useful aspect of using layers is the ability to lock and hide elements while you're working. These are found on the left hand side of the layers panel. It should be clarified that you need to keep your drawing in good order as you go along to maximise the usefulness of this feature.
It is also worth mentioning that groups cannot be made up of elements from different layers. Attempting to do so will move everything to the currently selected layer and grouping it there.
The main use for groups I have found is for ease of selection if I have certain elements within a layer that will use the same appearance settings as each other, but that are different to the rest of the layer. Doing so allows me to simply click once to select the whole group and then apply the desired appearance settings.
Ultimately, how you use layers and groups is entirely a case of personal preference and what makes sense to you as to how to use the strengths of each e.g. as mentioned in the previous answer, grouping allows the same scale factor be applied to each element within, and also prevents them from losing their positions relative to each other if you use the arrange / align buttons on the top menu bar. Personally, I tend to use layers to break down what each type of element is representing; for instance I had to make a fair few maps studying architecture, so I would have a layer named contours, buildings etc. which would broadly use the same appearance so I could alter these quickly if I needed to by using the select method I mentioned above. Occasionally there would be groups within these layers, which admittedly I could have placed in additional layers instead.
Really, if it makes sense to you then run with it.