I've been looking into this myself for the same reasons and from I can gather you need to use log4j-1.2-api-2.15.0.jar based on the following read:
Perhaps the simplest way to convert to using Log4j 2 is to replace the log4j 1.x jar file with Log4j 2's log4j-1.2-api.jar.
source
I would like to point out that this shouldn't be the accepted answer but I wanted to give an answer for those who are trying to grasp the situation.
[Edit]
It would seem that log4j-1.2-api-2.15.0.jar is not enough. It helps with migrating Geoserver to the newer log4j version whilst still using the previous main package but it fails to reference to the newer version. As such I also added log4j-api-2.15.0.jar to the mix to complete the pathing.
After this it seems to work again and the version has been updated.
I could validate this by using the api of Geoserver (/geoserver/rest/about/manifest.xml) specifically. If you search for log4j before upgrading you'll see 1.2.17 and afterwards 2.15.