Dropping samples?

Discussion and support for general JUCE issues

Re: Dropping samples?

Postby TheVinn » Mon Apr 11, 2011 3:13 am

jordanh wrote:...I call ScopedLock sl (audioDeviceManager.getAudioCallbackLock() before entering a for loop to start/stop recording on all recording tracks.


Just my opinion but using this type of synchronization for concurrency in your architecture, seems to be begging for trouble. Instead, you should have a thread safe queue with asynchronous functors.

Have a look at this thread:

viewtopic.php?f=8&t=6959

with specific attention to my lengthy post towards the middle, regarding manipulation of states shared with the audio I/O callback.

Of course, such a change will be a radical departure from what you have currently but it will let you go a lot farther.
Open Source: LayerEffects, VFLib, SimpleDJ, DSP Filters, LuaBridge, JUCE, FreeType, TagLib
"This isn't a big project, it shouldn't take long." - Jules
User avatar
TheVinn
JUCE UberWeenie
 
Posts: 2975
Joined: Sat Aug 29, 2009 11:31 am
Location: Marina del Rey, California

Re: Dropping samples?

Postby jordanh » Mon Apr 11, 2011 4:12 am

Thanks a lot for the reply Vinn, coincidentally I was actually in the middle of PMing you if you had any ideas when you responded here, ha! I'll keep this in the forums so that if someone else is in my shoes at some point, they might find this for reference. I have a lot to learn but the idea of the asynchronous thread-safe queue makes sense. I'm having a bit of a mind-f*** thinking about how to fit it into my programs architecture however. In any case, before I start gutting my project, is this way off base?

Goal: Synchronous start/stopping of recording on multiple recorders

1) Lets say I instantiate two instances of my recorder class in some main component, "a" and "b"-- they register with my AudioDeviceManager to receive audio callbacks, and handle their writing in audioDeviceIOCallback as currently set up above (using juce's FIFO ThreadedWriter...)
2) Create a thread queue (lets call it m_queue) in my main component and call
Code: Select all
m_queue.open()

3) Here's where I am a bit foggy. I want to start/stop recording on recorders a and b synchronously so that the resulting files output with the same length / amount of samples. So in my maincomponent, when instructed to start/stop, I say something like (replace the startRecording function with stopRecording if we're supposed to be stopping).
Code: Select all
m_queue.call (bond (a.startRecording, ?, true));
m_queue.call (bond (b:startRecording, ?, true));


Once I work through my questions with using bond (see "?" above), this seems like it would still exhibit the same behavior I am currently seeing? It seems like somehow I need the to also be calling m_queue.call on ThreadedWriter::write? Thanks for your time guys.
jordanh
JUCE UberWeenie
 
Posts: 128
Joined: Sun Mar 14, 2010 5:16 am

Re: Dropping samples?

Postby jules » Mon Apr 11, 2011 8:59 am

I recently did something just like this - recording incoming each channel to a separate mono file.

My approach was pretty simple - I created a bunch of separate recorder objects for each channel, and each one just added itself to the device manager as a callback. Initalliy they're all in a 'stopped' state so don't write any data yet. Then, once they're all ready to go, a master startRecording() method locks the getCallbackLock() method, and flips them all into the 'recording' state. Not complicated, and works just fine.
User avatar
jules
Fearless Leader
 
Posts: 17193
Joined: Mon Sep 06, 2004 9:03 am
Location: London, UK

Re: Dropping samples?

Postby jordanh » Mon Apr 11, 2011 9:31 am

@Jules - Thats pretty much how I have it set up... In my main component I have a method (startStopRecording) which just locks the getAudioCallbackLock() and then calls either start or stop recording in all my AudioRecorder objects--
Code: Select all
ScopedLock sl (audioDeviceManager.getAudioCallbackLock() );
for(int i=0; i< audioRecorderManager.size(); i++){
      //check to see if the recorder is currently recording, if so stop, otherwise start...
      if(audioRecorderManager[i]->isRecording()){   
         audioRecorderManager[i]->stop();
      }else{
         audioRecorderManager[i]->startRecording();
      }
   }

Each AudioRecorder has an instance of a RecOut object (which is almost identical to the JUCE demo, only using the single channel array as described above), and so when AudioRecorder::startRecording is called on each AudioRecorder, it checks to see if a record enable button on the track is selected and if the user selected a channel to record (channel != -1), and if so it creates a file and passes it into the RecOut::startRecording method as so

Code: Select all
void AudioRecorder::startRecording()
{
   //create file for recording and name it according to set name
   if (inputEnable->getToggleState() == true){
      if (channel != -1){
         if (savePath != "default"){
            fileToRecord = File(savePath).getNonexistentChildFile (fileName, ".wav");
         }else{
            fileToRecord = File(File::getSpecialLocation (File::userDocumentsDirectory)
                           .getNonexistentChildFile (fileName, ".wav"));
         }
         recorder->startRecording(fileToRecord); //starts the actual recording in RecOut class...
      }
   }
}


RecOut::startRecording is identical to the startRecording method in the JUCE Demo and same thing with stop(). I really can't see whats going wrong... is there some additional step you did for that project that I'm missing? Thanks again for your time!
jordanh
JUCE UberWeenie
 
Posts: 128
Joined: Sun Mar 14, 2010 5:16 am

Re: Dropping samples?

Postby jules » Mon Apr 11, 2011 10:00 am

You're doing a huge amount of work while you're holding that lock.. In my code, I only hold the lock very very briefly, during which I just set a boolean flag in all the recorders to tell them it's ok to start.
User avatar
jules
Fearless Leader
 
Posts: 17193
Joined: Mon Sep 06, 2004 9:03 am
Location: London, UK

Re: Dropping samples?

Postby jordanh » Mon Apr 11, 2011 10:42 am

hmm, I just took all the file stuff out of AudioRecorder::startRecording method (I pre-create the file which is passed into the AudioRecorders RecOut::startRecording method when the channel you wish to record is selected) and it didn't seem to get any better. Are you suggesting that I also somehow break out the code in RecOut::startRecording and RecOut::stop? There are maybe certain things I can preallocate, like the WavAudioFormat, but the rest seems that it needs to be created/deleted each time right...? Unless its fine to create the ThreadedWriter and everything (basically everything in RecOut::startRecording) once when the class is instantiated and just have a global bool in my audio callbacks that gets checked if its true or not before calling ThreadedWriter::write?

I'm refering to these methods...

Code: Select all
void RecOut::startRecording (const File& file)
{
   stop();
   if (sampleRate > 0)
   {
      // Create an OutputStream to write to our destination file...
      file.deleteFile();
      ScopedPointer<FileOutputStream> fileStream (file.createOutputStream());
      
      if (fileStream != 0)
      {
         // Now create a WAV writer object that writes to our output stream...
         WavAudioFormat wavFormat;
         AudioFormatWriter* writer = wavFormat.createWriterFor (fileStream, sampleRate, 1, 16, StringPairArray(), 0);
         
         if (writer != 0)
         {
            fileStream.release(); // (passes responsibility for deleting the stream to the writer object that is now using it)
            
            // Now we'll create one of these helper objects which will act as a FIFO buffer, and will
            // write the data to disk on our background thread.
            threadedWriter = new AudioFormatWriter::ThreadedWriter (writer, backgroundThread, 32768);
            
            // And now, swap over our active writer pointer so that the audio callback will start using it..
            const ScopedLock sl (writerLock);
            activeWriter = threadedWriter;
         }
      }
   }
}

and
Code: Select all
void RecOut::stop()
{
   // First, clear this pointer to stop the audio callback from using our writer object..
   {
      const ScopedLock sl (writerLock);
      activeWriter = 0;
   }
   
   // Now we can delete the writer object. It's done in this order because the deletion could
   // take a little time while remaining data gets flushed to disk, so it's best to avoid blocking
   // the audio callback while this happens.
   threadedWriter = 0;
}
jordanh
JUCE UberWeenie
 
Posts: 128
Joined: Sun Mar 14, 2010 5:16 am

Re: Dropping samples?

Postby TheVinn » Mon Apr 11, 2011 3:10 pm

jordanh wrote:Goal: Synchronous start/stopping of recording on multiple recorders


I would do the following:

- Each recorder would have its own thread and thread queue (I would not use the ThreadedWriter)
- Create a reference counted AudioSampleBuffer
- From inside the audio callback, copy the audio data into a reference counted AudioSampleBuffer
- From inside the audio callback, queue a call to each recorder's thread queue with a reference to the AudioSampleBuffer
- Each recorder, upon receiving a buffer, simply writes it synchronously (since its getting called on its own thread).

It would be easy enough to add some kind of start/stop calls into this model.
Open Source: LayerEffects, VFLib, SimpleDJ, DSP Filters, LuaBridge, JUCE, FreeType, TagLib
"This isn't a big project, it shouldn't take long." - Jules
User avatar
TheVinn
JUCE UberWeenie
 
Posts: 2975
Joined: Sat Aug 29, 2009 11:31 am
Location: Marina del Rey, California

Re: Dropping samples?

Postby TheVinn » Mon Apr 11, 2011 3:26 pm

jordanh wrote:Once I work through my questions with using bond (see "?" above)


'bond' is just a simplified version of boost::bind that I wrote to avoid making my DspDemo depend on boost.

You would want to use boost::bind, or depending on your build environment it might be available as std::bind or std::tr1::bind.

To bind a class member you would write

bind (&Class::member, classPtr, param1, param2, ...)

where classPtr is a Class*
Open Source: LayerEffects, VFLib, SimpleDJ, DSP Filters, LuaBridge, JUCE, FreeType, TagLib
"This isn't a big project, it shouldn't take long." - Jules
User avatar
TheVinn
JUCE UberWeenie
 
Posts: 2975
Joined: Sat Aug 29, 2009 11:31 am
Location: Marina del Rey, California

Re: Dropping samples?

Postby jordanh » Tue Apr 12, 2011 5:59 am

I'm thinking I should try to get it working both ways and here's why-- the way Jules is doing it is pretty much identical to how I had it structured, and so if it recently worked for him on a project, I'd really like to know why it is that mine is exhibiting this behavior. Vinn - the way you propose looks very interesting to me on many fronts, especially about learning about threads and concurrency, something I'd really like to school myself on as I become a better and more efficient programmer (been reading the websites you linked to in the other thread). As this is for a piece of software I need to have finished for my PhD research, the most important part is really that I have it finished as soon as possible, so I can use the software for data collection which I need to collect and stay analyzing soon. Hopefully I can have this all finished by the end of this week. Id be interested to get both models working, and compare the results. As I am not only recording Audio data, but upsampling (sorta) and doing the same recording with data from analog/digital sensors (on instruments/performers) and via OSC, it is really important that everything is synchronous and accurate as possible...

1) I guess I'll see what Jules says about reducing the workload during the audio thread lock
2) Vinn, as I try to conceptualize your method (apologize in advance if some of this is brutally simple):
*Make my RecOut class extend AudioIODeviceCallback and Thread
- Create a reference counted AudioSampleBuffer
- From inside the audio callback, copy the audio data into a reference counted AudioSampleBuffer
- From inside the audio callback, queue a call to each recorder's thread queue with a reference to the AudioSampleBuffer

*These are all referring to the same buffer right? So I just create one (ref counted) AudioSample buffer (in my RecOut class) when the class is instantiated, and each audio callback I clear the buffer, copy in the new data and then pass it to an AudioFormatWriter* (lets call it myWriter) via writeFromAudioSampleBuffer using the ThreadQueue call:
e.g. m_queue.call (bond (&AudioFormatWriter::writeFromAudioSampleBuffer, myWriter, AudioSampleBuffer &source, int startSample, int numSamples));
* Next I call m_queue.process()

Thanks!
jordanh
JUCE UberWeenie
 
Posts: 128
Joined: Sun Mar 14, 2010 5:16 am

Re: Dropping samples?

Postby TheVinn » Tue Apr 12, 2011 7:38 am

jordanh wrote:*These are all referring to the same buffer right? So I just create one (ref counted) AudioSample buffer (in my RecOut class) when the class is instantiated, and each audio callback I clear the buffer, copy in the new data and then pass it to an AudioFormatWriter* (lets call it myWriter) via writeFromAudioSampleBuffer using the ThreadQueue call:
e.g. m_queue.call (bond (&AudioFormatWriter::writeFromAudioSampleBuffer, myWriter, AudioSampleBuffer &source, int startSample, int numSamples));
* Next I call m_queue.process()


They refer to the same buffer but you need to create a new reference counted buffer each time the audio I/O callback executes. The old one might still be getting written out (this would be the situation if the I/O cannot keep up with the stream of generated buffers).

AudioSampleBuffer cannot be passed as a reference through the thread queue. It needs to be passed by value. You want something like this:

Code: Select all
struct ReferenceCountedBuffer : public ReferenceCountedObject
{
  typedef ReferenceCountedObjectPtr <ReferenceCountedBuffer> Ptr;

  AudioSampleBuffer buffer;
};


You would need to do new ReferenceCountedBuffer in the audio I/O callback and assign it to a variable of type ReferenceCountedBuffer::Ptr. The argument passed to the function used in bind (or bond) needs to be of type ReferenceCountedBuffer::Ptr. Since writeFromAudioSampleBuffer() does not take the Ptr as a parameter, you would have to write a wrapper function.

I didn't realize this was just a school project. Getting a thread queue implementation up and running is a non-trivial task, and definitely has a higher "start up" cost than a straight critical section, although the benefits definitely pay off as the system grows in size (although this might not be your use-case).

You might be misunderstanding how the queue works. m_queue.process() has to be called from the destination thread, not the thread that is doing the m_queue.call(). There are two players at work here:

1) Audio I/O Callback.
- Acts as the "Producer"
- Puts a sequence of reference counted sample buffers into each thread queue

2) Writer thread
- Acts as the "Consumer"
- Blocks until there is work in it's associated thread queue
- Calls m_queue.process() when there is work
- Receives reference counted sample buffers as an argument to a callback function

What is missing from the DspFilters demo is the "signaling" mechanism that tells the consuming thread when to wake up and call process(). In my demo, process() is simply called once at the beginning of every audio I/O callback. However, the ThreadQueue implementation provided in the demo contains overridable virtual methods (signal and reset) which allow you to implement your own signaling.

In your case, you would need to create a subclass of ThreadQueue and override the signal() and/or reset() functions. You could use the juce::Thread::notify() and juce::Thread::wait() functions - the implementation of these is done using a juce::WaitableEvent which is perfectly suited.
Open Source: LayerEffects, VFLib, SimpleDJ, DSP Filters, LuaBridge, JUCE, FreeType, TagLib
"This isn't a big project, it shouldn't take long." - Jules
User avatar
TheVinn
JUCE UberWeenie
 
Posts: 2975
Joined: Sat Aug 29, 2009 11:31 am
Location: Marina del Rey, California

Re: Dropping samples?

Postby jules » Tue Apr 12, 2011 8:43 am

1) I guess I'll see what Jules says about reducing the workload during the audio thread lock


Well, I say: "reduce the workload during the audio thread lock"

Like I said, in my version all I do is lock it, set a couple of boolean flags, and then unlock it. Don't do any more than that.
User avatar
jules
Fearless Leader
 
Posts: 17193
Joined: Mon Sep 06, 2004 9:03 am
Location: London, UK

Re: Dropping samples?

Postby jordanh » Fri Apr 29, 2011 1:46 am

jules wrote:Like I said, in my version all I do is lock it, set a couple of boolean flags, and then unlock it. Don't do any more than that.


If you don't mind me asking, when do you create your fileoutputstream and ThreadedWriter? Do you do it upon creating an instance of your recorder and then reuse them somehow? I thought they needed to be deleted / created each time; That's the one thing I can't conceptualize how to take out of the equation-- perhaps I just do all that (creating the FileOutputStream, ThreadedWriter, etc) before calling my lock, and then set my shared bool which gets checked in each recorder audio callback to write or not? Thanks!
jordanh
JUCE UberWeenie
 
Posts: 128
Joined: Sun Mar 14, 2010 5:16 am

Re: Dropping samples?

Postby jules » Fri Apr 29, 2011 9:37 am

Yeah, just set up all your threads and file stuff, and then when it's all ready to go, lock it and start it all running.
User avatar
jules
Fearless Leader
 
Posts: 17193
Joined: Mon Sep 06, 2004 9:03 am
Location: London, UK

Re: Dropping samples?

Postby jordanh » Sat Apr 30, 2011 1:40 pm

So I think I have it all set up per your suggestions, but the problem still pops up after a few recordings... I have it set up like so:

When recording is enabled and the user presses play, it loops through my collection of AudioRecorders (tracks) and calls a method which handles creating a file (FileOutputStream) and the threadedWriter; that all happens pretty much identically to the juce audio recording demo. Once it has looped through all recorders, it calls the audio lock, and sets a flag shared by all recorders either true or false. Inside each AudioRecorders audio callback, it checks this flag to either write the samples to its buffer, or to call stop(), which deletes the ThreadedWriter as per the demo. I also tried taking out my channel selection, so it just writes inputChannelData from the audio callback just like in the demo, but the error persists.

Could there be something else thats the culprit? I'll keep having a go... thanks
jordanh
JUCE UberWeenie
 
Posts: 128
Joined: Sun Mar 14, 2010 5:16 am

Re: Dropping samples?

Postby jordanh » Sun May 01, 2011 3:03 am

Been testing all morning, and these are the things I've been able to reproducibly verify:

[*] The first recording / take always seems to be fine, and then the problem arises, sometimes on the second take, but usually some later take (4, 5, etc)
[*] If I record the same input on 2+ tracks at once, the resulting .wav files are identical-- including glitches
[*] Removing the shared bool and audio callback lock, the problem disappears. In this way, recording is set up identically to the demo. Clicking in the gui or pressing spacebar to call startRecording() and stop()
[*] Adding the shared bool seems to bring the problem back. I tried this both with and without calling the audio lock before setting the shared bool. Currently the bool is checked in the recorders audio callbacks to either write or call stop() depending on the state of the bool like so

hmm...

Code: Select all
void RecOut::audioDeviceIOCallback (const float** inputChannelData, int numInputChannels, float** outputChannelData, int numOutputChannels, int numSamples)
{      
   const ScopedLock sl (writerLock);
   if (activeWriter != 0 )
   {
      if(*finishWriting == true)
      {
         activeWriter->write(inputChannelData, numSamples);
      }else {
         stop();
      }
   }
   // We need to clear the output buffers, in case they're full of junk..
}


[EDIT] I did more testing tonight, and got the same behaviour in the JUCE Demo. All I did was modify AudioDemoRecordPage so that it modifies a shared bool instead of calling start stop directly on the recorder-- when the record button is pressed, if a shared bool (writingState) is false, it creates a new file and passes it into the recorders startRecording, then it briefly calls the audio lock and flips the shared writingState bool. Then in the AudioRecorder class I changed it per above (if the shared bool writingState is true, it should write the new data, else it should call stop() ).

@Jules - Can you think of anything else that could be causing this problem or that different that you can remember between the project you had this working on the way I've set it up?
Best, J
jordanh
JUCE UberWeenie
 
Posts: 128
Joined: Sun Mar 14, 2010 5:16 am

PreviousNext

Return to General JUCE discussion

Who is online

Users browsing this forum: Google [Bot] and 4 guests