Load the page with just a direct download to the audio file. Detect if the browser supports MP3s. If it does, progressively enhance the direct download into an AUDIO tag. Here's a sample:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Audio demo</title>
<script src="http://cdnjs.cloudflare.com/ajax/libs/modernizr/2.0.6/modernizr.min.js"></script>
</head>
<body>
<p id="audio">
<a href="BadAppleEnglish.mp3">Listen »</a>
</p>
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js"></script>
<script src="http://ajax.aspnetcdn.com/ajax/jquery.templates/beta1/jquery.tmpl.min.js"></script>
<script src="audio.js"></script>
</body>
</html>
And the Javascript:
$(function () {
var audioElement = $('#audio'),
href = audioElement.children('a').attr('href');
$.template('audioTemplate', '<audio src="${src}" controls>');
if (Modernizr.audio.mp3) {
audioElement.empty();
$.tmpl('audioTemplate', {src: href}).appendTo(audioElement);
}
});
I would imagine that most of the zillions of prebuilt MP3 playing plugins work like this.
UPDATE:
Since you've edited the question to specify that you'd prefer a solution that doesn't use jQuery, I will point out that rewriting this to not use jQuery is trivial. It's just longer and less elegant. Do keep in mind that Javascript libraries loaded from CDNs are usually cached by the browser.