I read an awesome post on application server vs. webserver at What is the difference between application server and web server?. Moreover, Difference between proxy server and reverse proxy server nicely explains what a proxy server is.
I also learned that some web servers, such as Apache, have reverse proxy built-in. (Source). Also, Wikipedia (https://en.wikipedia.org/wiki/Reverse_proxy) has an image that shows webserver and reverse proxy as separate entities.
Source: https://en.wikipedia.org/wiki/Reverse_proxy (image originally via Privacy Canada, now CC0, license info)
So, I am not sure about the difference between webserver and reverse proxy. Can someone please shed the light?