Skip to content

GPUImageMovieWriter - occasional black frames at either ends of the recorded video #1255

New issue
New issue
Open
Open
GPUImageMovieWriter - occasional black frames at either ends of the recorded video#1255
@zakinaeem

Description

@zakinaeem
zakinaeem
opened on Oct 14, 2013

I have recording app implementation where user can tap the "record" button to start/stop recording. I achieve this with a basic GPUImageVideoCamera with output set to a GPUImageView as well as a GPUImageMovieWriter.

50% of the time, the recorded clip ends up with a couple (or a single) black frame at either ends, sometimes both. The implementation is fairly straightforward, but here is it anyway.

gpuImageView = [[GPUImageView alloc] initWithFrame:cameraView.frame];
gpuImageView.fillMode = kGPUImageFillModePreserveAspectRatioAndFill;

[cameraView addSubview:gpuImageView];

videoCamera = [[GPUImageVideoCamera alloc] initWithSessionPreset:AVCaptureSessionPresetHigh cameraPosition:(usingBackCamera) ? AVCaptureDevicePositionBack : AVCaptureDevicePositionFront];

[videoCamera addTarget:gpuImageView];
[videoCamera addTarget:movieWriter];
videoCamera.audioEncodingTarget = movieWriter;
[videoCamera startCameraCapture];

double delayToStartRecording = 0.5;
dispatch_time_t startTime = dispatch_time(DISPATCH_TIME_NOW, delayToStartRecording * NSEC_PER_SEC);
        dispatch_after(startTime, dispatch_get_main_queue(), ^(void){
            NSLog(@"Start recording");
            [movieWriter startRecording];
        });

And then stop the recording with the following (while the live camera continues to show up on gpuimageview.

[movieWriter finishRecording];

Has anyone else experienced this and/or found a solution to avoid black frames? I cannot pause/resume camera capture so to ensure seamless user experience.

Activity

staykids

staykids commented on Oct 15, 2013

@staykids
staykids
on Oct 15, 2013

I've definitely encountered this issue before. It's annoying - I spent some time trying to investigate why it occurs but it's sporadic enough that I just looked to mitigate it some other way.

Here's a low-tech solution: after the movieWriter is finished recording, start an AVAssetExportSession with the newly completely video URL, chop some "buffer time" off the video, then save it back to disk. I find that removing a CMTime duration of CMTimeMake(1,10) off both the start and end of the movieWriter video removes all black frame artifacts. YMMV. I'd recommend maybe using movieWriter finishRecordingWithCompletionHandler, and then in the completion block make a call to whatever method does the "buffer time" trimming.

If you notice, Apple's stock Camera app (when taking a video) chops off a bit of time from the beginning of a video after you finish recording. I'm not saying it's necessarily because of an issue like this, but just wanted to mention it to illustrate that it can indeed be a lot of data to deal with.

I leave it to more interested minds to discover the "why" of this issue and engineer a better solution.

zakinaeem

zakinaeem commented on Oct 15, 2013

@zakinaeem
zakinaeem
on Oct 15, 2013
Author

Thanks for your input mate - what you suggest (chopping the frames off) is definitely in for Plan B. I never noticed the chopping off on Apple's stock camera app, interesting!

Is definitely worth finding out the reason behind this behavior and a possible solution. Will continue to look into it myself.

m1entus

m1entus commented on Oct 20, 2013

@m1entus
m1entus
on Oct 20, 2013

videoCamera.audioEncodingTarget = movieWriter - this is your problem. It's very expensive operation.

followmyheart-Bao

followmyheart-Bao commented on Nov 21, 2013

@followmyheart-Bao
followmyheart-Bao
on Nov 21, 2013

I experienced the same issue, anyone else has solution?

zmzhuai

zmzhuai commented on Nov 27, 2013

@zmzhuai
zmzhuai
on Nov 27, 2013
Contributor

I think I found the reason, I look into the file GPUImageVideoCamera.m, the GPUImageVideoCamera process frames, There is a semaphore named frameRenderingSemaphore, at line 951. It will wait this semaphore, when the process is not finished,It will drop the frames. and I think GPUImage does not change CMTime of CMSampleBufferRef. I do not have some solution, To solute this problem, You may rewrite this class.

zakinaeem

zakinaeem commented on Dec 25, 2013

@zakinaeem
zakinaeem
on Dec 25, 2013
Author

No luck with this and as much as I have tried to push it back, its quite incorrect user-experience wise to have these black frames you cannot control. Anyone with a solution, please do let know.

barrault01

barrault01 commented on Feb 12, 2014

@barrault01
barrault01
on Feb 12, 2014

I do have the same problem and can't resolved it.

xujingzhou

xujingzhou commented on Oct 7, 2014

@xujingzhou
xujingzhou
on Oct 7, 2014

I met the same issue. Any solution now?

KageDev

KageDev commented on Dec 26, 2014

@KageDev
KageDev
on Dec 26, 2014

The problem is that the audio buffer is written before the video buffer.
Fix:

Inside GPUImageMovieWriter.m
add code:

static BOOL allowWriteAudio = NO;

- (void)startRecording;
{
  ...
  allowWriteAudio = NO;
}

- (void)processAudioBuffer:(CMSampleBufferRef)audioBuffer;
{
  if (!allowWriteAudio) {
    return;
  }
  ...
}

- (void)newFrameReadyAtTime:(CMTime)frameTime atIndex:(NSInteger)textureIndex;
{
  ...
  if (![assetWriterPixelBufferInput appendPixelBuffer:pixel_buffer withPresentationTime:frameTime])
    NSLog(@"Problem appending pixel buffer at time: %@", CFBridgingRelease(CMTimeCopyDescription(kCFAllocatorDefault, frameTime)));

  allowWriteAudio = YES;   //< add this
  ...
}
zakinaeem
added a commit that references this issue on Dec 26, 2014

- fix for black frames as per BradLarson#1255 (comment)

98bff65
protez

protez commented on Jul 15, 2015

@protez
protez
on Jul 15, 2015

Thank you so much KageDev! You're the hero.

yumincode

yumincode commented on Jan 14, 2016

@yumincode
yumincode
on Jan 14, 2016

hello,can anyOne help me?When i add this code,my video is silent,there is no sound, please help me...

hk41
added a commit that references this issue on Apr 20, 2016

add patch BradLarson#1255

e292465
mingyue1991

mingyue1991 commented on Nov 24, 2016

@mingyue1991
mingyue1991
on Nov 24, 2016

@KageDev e~,it can avoid the black frame at the beginning, but the black frame at the end still occur sometimes.do you know how to solve it?

jiliszc

jiliszc commented on Nov 30, 2016

@jiliszc
jiliszc
on Nov 30, 2016

5c and 4s the black frame at the end still occur

15 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @KageDev@followmyheart-Bao@barrault01@protez@zakinaeem

        Issue actions
          GPUImageMovieWriter - occasional black frames at either ends of the recorded video · Issue #1255 · BradLarson/GPUImage