As you mentioned, GPS returns the altitude as an offset from the WGS84 reference ellipsoid, but most people want to see mean sea level (MSL), and the two frequently don't agree. The way this is most frequently done is by looking up the delta in a table and using that to compute MSL based on the height from GPS and the delta in the table.
There's some java code here: https://github.com/NASAWorldWind/WorldWindJava/blob/develop/src/gov/nasa/worldwind/util/EGM96.java. The other functions that it uses from Worldwind aren't that complicated, so you could probably use most of the code unmodified, and the rest you could adapt if you're working in Java and their license meets your needs.
It uses information from the EGM 96 data set (link here if you're interested -- not strictly necessary though), which you can download here: https://github.com/jleppert/egm96/blob/master/WW15MGH.DAC. You will want the WW15MGH.DAC file. It's a binary file full of 16-bit signed integers. You can use the Java example to show you how to access the data in the file. They also provide a Fortran example if that's your thing. :-)
Here's the information on the file from their readme.
Data Description for 15 minute worldwide binary geoid height file:
---- FILE: WW15MGH.DAC
The total size of the file is 2,076,480 bytes. This file was created
using an INTEGER2 data type format and is an unformatted direct access
file. The data on the file is arranged in records from north to south.
There are 721 records on the file starting with record 1 at 90 N. The
last record on the file is at latitude 90 S. For each record, there
are 1,440 15 arc-minute geoid heights arranged by longitude from west to
east starting at the Prime Meridian (0 E) and ending 15 arc-minutes west
of the Prime Meridian (359.75 E). On file, the geoid heights are in units
of centimeters. While retrieving the Integer2 values on file, divide by
100 and this will produce a geoid height in meters.