A working example for splitting audio on silences

[aalekhpatel07]

Thanks for all your work in maintaining this awesome project!

Is your feature request related to a problem? Please describe.
I'm wondering if it'd be possible to have a working example of an API for splitting audio on silences. I tried to put something together using a concoction of the existing APIs but I couldn't really get it to work. :(

Describe the solution you'd like
The intended usage could be along the lines of:

use wavy::Microphone;
use fon::Audio;

#[tokio::main]
pub async fn main() {
    let mut microphone = Microphone::default();

    let stream = microphone.record().await;
    let threshold = ...;
    let noise = stream.split_on_silence(threshold);

    while let Ok(audio) = noise.next().await {
        // here `audio` is an Audio<...> instance.
        process_audio(audio)
    }
}

fn process_audio(audio: Audio<...>) {
    // do something with the samples.
}

(aside) Would it be possible to make these APIs compatible with tokio (and if it already is could we please add a couple of examples)?

[AldaronLau]

Thanks for the issue @aalekhpatel07!

Yeah, it would be possible to create an example for splitting on silences.

I wrote down the algorithm for decibel calculations in the twang readme which you'd want to calculate on chunks of audio (or otherwise just base it off mean amplitude).

Wavy is designed to be compatible with tokio. I've had it on my to-do list to add tokio examples for a while 😅

Until I can get around to it, for getting it working, I suggest:

• Take existing record example
• Introduce filtering on chunks as they are returned before passing them to processing
• Switch out for #[tokio::main] (you may need to use tokio's LocalSet and/or #[tokio::main(flavor = "current_thread")])

Also, I can help if you can send me an error you're running into and code giving that error.

[aalekhpatel07]

I appreciate the quick response, @AldaronLau! Lemme try to play around with these ideas and come up with an implementation.

Additionally, as I dig a little bit deeper, adding a couple of tokio examples shouldn't be completely out of reach for me. I'd love to try to help out with that if you're accepting contributions.

Thanks!

[aalekhpatel07]

Not sure if I'm running it incorrectly but the record example seems kinda broken right now. I did a fresh checkout of the stable branch and ran cargo run --example record and there was no output at all. So I thought of adding a print statement to track any progress using the following patch:

diff --git a/examples/record.rs b/examples/record.rs
index 78525b4..ca345f8 100644
--- a/examples/record.rs
+++ b/examples/record.rs
@@ -3,7 +3,7 @@
 // Setup async main
 include!(concat!(env!("OUT_DIR"), "/main.rs"));

-use fon::{mono::Mono32, Audio, Frame};
+use fon::{mono::Mono32, Audio, Frame, Stream};
 use pasts::{prelude::*, Join};
 use wavy::{Microphone, MicrophoneStream};

@@ -18,6 +18,7 @@ struct App {
 impl App {
     /// Event loop.  Return false to stop program.
     fn record(&mut self, stream: MicrophoneStream<Mono32>) -> Poll<()> {
+        eprintln!("Extending with {:?} bytes", stream.len());
         self.buffer.extend(stream);
         if self.buffer.len() >= 48_000 * 10 {
             return Ready(());

After this, the program logs that statement exactly once before idling indefinitely. So I put this under gdb but the backtrace didn't reveal much except that the last call before idling was pasts::exec::MainExec::sleep. Could this be related to #32?

Whenever you get some time, could you please guide me towards a way to adapt the Microphone capture API that relies on pasts into a tokio channel-like API?

For example,

use fon::{Audio, mono::Mono32};
use wavy::{Microphone};

#[tokio::main]
pub async fn main() {
    let mut microphone = Microphone::default();

    let microphone_handle = tokio::task::spawn(async move {
        while let Ok(audio) = microphone.record().await {
            println!("received: {:?}", audio);
        }
    });

    // spawn more tasks.    
    let handles = tokio::join!(microphone_handle, ...);
    for handle in handles {
        handle.unwrap();
    }
}

[AldaronLau]

@aalekhpatel07 Yes, I think it's related to that issue. I am open to accepting contributions, keeping in mind that things are kind of a mess right now in this repository (shouldn't affect examples as much). From what I remember since I last looked at it, I thought it would be fixed with update of the dependencies smelling_salts and pasts, which was possibly blocked on GATs in Rust. I think it's time I revisit that, and close out the dependabot PRs.

I would think something like this would work on tokio for the current version of wavy (I didn't test it), given a workaround or fix of the microphone issue:

use fon::{Audio, mono::Mono32};
use tokio::task::LocalSet;
use wavy::Microphone;

#[tokio::main]
pub async fn main() {
    let microphone_handle = task::LocalSet::new();

    microphone_handle.spawn_local(async {
        let mut microphone = Microphone::default();

        while let Ok(audio) = microphone.record().await {
            println!("received: {:?}", audio);
        }
    }).await;

    // spawn more tasks.    
    let handles = tokio::join!(microphone_handle, ...);
    for handle in handles {
        handle.unwrap();
    }
}