How to create an rtsp stream from raw mic audio?

[cbright99]

Hello,
I'm working on an application where we send raw audio data (in UDP format) from a mic output over a socket to a GO app. (16bit, stereo, 48000hz)

We want to use aacEncoder to encode the raw audio data and send it to an rtsp stream ... playable in VLC

The stream gets to VLC fine, but the audio is obviously formatted wrong and we see errors like;
"faad warning: Maximum number of bitstream elements exceeded"
"faad warning: Gain control not yet implemented"
"faad warning: Bitstream value not allowed by specification"

I assume we are missing headers/formatting data that needs to be added to the raw audio data?

Below is a simplified example showing what we are trying to do.

Thanks for an help/suggestions on how to make this work!

package main

import (
	"fmt"
	"os"
	"os/signal"
	"time"

	"github.com/aler9/gortsplib"
	"github.com/aler9/gortsplib/pkg/rtpaac"
	"github.com/gordonklaus/portaudio"
	)
func main() {
	// Create AAC Track
	track, err := gortsplib.NewTrackAAC(97, &gortsplib.TrackConfigAAC{Type: 2, SampleRate: 44100, ChannelCount: 1})
	if err != nil {
		panic(err)
	}

	// sequenceNumber := uint16(0x44ed)
	// ssrc := uint32(0x9dbb7812)
	// initialTs := uint32(0x88776655)
	aacEncoder := rtpaac.NewEncoder(97, 44100, nil, nil, nil)
	fmt.Print(track.Media)

	// connect to the server and start publishing the track
	conn, err := gortsplib.DialPublish("rtsp://localhost:8554/audio",
		gortsplib.Tracks{track})
	if err != nil {
		panic(err)
	}
	defer conn.Close()

	fmt.Println("Recording.  Press Ctrl-C to stop.")

	sig := make(chan os.Signal, 1)
	signal.Notify(sig, os.Interrupt, os.Kill)

	fileName := "test.pcm" // os.Args[1]
	f, err := os.Create(fileName)
	chk(err)

	defer func() {
		chk(f.Close())
	}()

	t := 0

	portaudio.Initialize()
	time.Sleep(1)
	defer portaudio.Terminate()
	in := make([]byte, 32)
	stream, err := portaudio.OpenDefaultStream(1, 0, 44100, len(in), in)
	chk(err)
	defer stream.Close()

	chk(stream.Start())
loop:
	for {

		t++

		chk(stream.Read())

		fmt.Println(in)

		fmt.Println(len(in), [][]byte{in})
		pkts, err := aacEncoder.Encode([][]byte{in}, time.Microsecond*10*time.Duration(t))
		bytss := make([][]byte, len(pkts))
		for i, pkt := range pkts {
			byts, err := pkt.Marshal()
			if err != nil {
				fmt.Errorf("error while encoding AAC: %v", err)
			}
			bytss[i] = byts
		}

		for _, byts := range bytss {
			fmt.Println(byts)
			err = conn.WriteFrame(0, gortsplib.StreamTypeRTP, byts)
		}

		if err != nil {
			panic(err)
		}

		select {
		case <-sig:
			break loop
		default:
		}
	}
	chk(stream.Stop())
}

func chk(err error) {
	if err != nil {
		panic(err)
	}
}

[aler9]

You are missing the conversion from the raw audio to the AAC codec.

A good pipeline could be:

1. take the audio in raw format
2. encode the audio into AAC units
3. wrap AAC units into RTP packets
4. send these RTP packets with UDP
5. receive these RTP packets with Go and UDP
6. send these RTP packets with RTSP and gortsplib

Steps 1-4 can be done with Gstreamer and a very simple command line, steps 5-6 are done with Go and gortsplib.
I've added a working example here:

https://github.com/aler9/gortsplib/blob/main/examples/client-publish-aac/main.go

Find a way to send the raw audio to Gstreamer (i think it can be done with udpsrc), and the example will work.