Git can see that you changed your non-text files, but you won't be able to get the best of git in that case. With text files you can see what is the actual difference between different versions / commits.
That being said, you could try this solution for image diffs in git. I am sure that there should be software to display differences between other file types, that you may need and that would make sens to check for differences.
Compared to dropbox, git should be better, because you can use commit messages that will say what was done in that particular change, and you can create feature branches; but it is a bit more complicated, due to its purpose, namely keeping track of source code differences between versions.
EDIT:
A̶n̶d̶ ̶n̶o̶,̶ ̶G̶i̶t̶ ̶d̶o̶e̶s̶ ̶n̶o̶t̶ ̶s̶a̶v̶e̶ ̶i̶n̶c̶r̶e̶m̶e̶n̶t̶a̶l̶ ̶c̶h̶a̶n̶g̶e̶s̶ ̶f̶o̶r̶ ̶n̶o̶n̶-̶t̶e̶x̶t̶ ̶f̶i̶l̶e̶s̶,̶ ̶b̶u̶t̶ ̶n̶e̶i̶t̶h̶e̶r̶ ̶d̶o̶e̶s̶ ̶d̶r̶o̶p̶b̶o̶x̶,̶ ̶a̶s̶ ̶f̶a̶r̶ ̶a̶s̶ ̶I̶ ̶k̶n̶o̶w̶.̶
It looks like git is storing non-text files as character strings, so yes it should only keep track of differences. Therefore, any good difftool like meld or Beyond Compare should be able to tell the difference between two images, for instance. For instance, I was able to see the differences between two png images with Beyond Compare.
It also seems to do a good job with PDF files, but, like exe files, you should not track those file types with version control. Instead of PDFs, keep track of their source code - for instance LaTeX files (which are plain text). Due to their nature, compiled files, like exe files are not suited for version control. The reason for this is that even if you edit directly into the character string of the file, you won't be able to accomplish much - you are supposed to edit the source code.
