According to this post (thanks Muhammad), I should be able to get the runtime error on Azure by editing the server's web.config (quite correct, Celt).
Unfortunately this did not work - no detailed exception.
I did some digging around and found these "DetailedError" logs:
This is what they contained:
It appears that something may have been going wrong when trying to resolve favicon.ico
at D:\home\site\wwwroot\favicon.ico
.
There indeed was no favicon at that location. I rectified this, but still the same problem. In fact, I have never had a favicon, and this used to work.
In the end, I deleted the entire Web App in Azure Portal and republished... TADA, it works again.