Analyze currently playing audio with Python

Question

I'd like to build a small Python program that can listen to and analyze currently playing audio on a computer, for example, from any media player.

I know that this is possible with DirectShow on Windows, but I'm not sure how to use it from Python. However, I'd ideally like a cross-platform way that does not use DirectX.

Metzger

This is not going to be possible in a cross-platform fashion, and it's hard to imagine you'll be able to do it in Python either. – Solidstate

Answer 1

In general, to "listen to something" from your sound card you are going to have to use some audio toolkit / module and usually, you will end up setting up a record-process-play routine (you can ommit play of course)

If your application is not a hard real-time one (i.e. you can afford to miss a few samples from the input) you could start off with PyAudio's "Record a few seconds of audio and save it to a file" example from their website.
So in your case, you would:

1. Record a buffer
2. Process it
3. If the terminating condition is not satisfied, Go back to (1).

But, in this case,
(You may have noticed) You would be missing samples from the input while you are doing the processing because during that time, you are not recording anything.
Depending on your application, you could get away with that...This is especially true for PyAudio because for the moment it only supports blocking-mode so if you want real-time (ish) operation you would have to use threads.

If your real-time specifications are more strict (i.e. you can't afford to lose even a few samples from your input) you would still use the "record-process-[play]" routine but this time you would need to do it in a Thread and have it communicating with your main process through a LIFO stack (Last In First Out or Deque).

It would go something like this:

Recording Thread:

1. Record a buffer
2. Push the data on the Deque
3. Repeat from (1)

Main Process:

1. If the Deque has buffers then
   1. Pull a buffer from the Deque
   2. Process it
2. Repeat from (1)

In this way, your processing can go on at its own pace while the recording thread keeps filling up buffers and pushing them on the Deque.

The good news in the case of Python is that the Deque is thread safe, so you will not have any sync problems when your main process and thread try to access the Deque simultaneously.

Again, Depending on your application you might also need to move towards faster hardware such as those that are based on the ASIO protocol.

Eventually,
You will also need to modify your processing algorithms a little bit to take into account that you are now working with frames instead of one buffer...Therefore, to keep things smooth you would have to save the state of your operations from one frame to the next. For more information you can see the "overlap-add" method

All the best

Shakedown

Thanks for the very detailed explanation! I ended up using PyAudio as you suggested and enabled my sound card's stereo mix (howtogeek.com/howto/39532/…) to listen to the current audio. Here's the result: julianapena.com/2012/03/arduino-python-soundlight – Aufmann