feat: implement enterPictureInPictureOnLeave prop for both platform(Android, iOS)

[YangJonghun]

Update the changelog

feat(android): implement pictureInPicture prop and onPictureInPictureStatusChanged event

Describe the changes

• (Breaking Change) Remove pictureInPicture boolean prop (iOS)

• Add enterPictureInPictureOnLeave boolean prop (Android, iOS)

• Add enterPictureInPicture method (Android, iOS)

• Add exitPictureInPicture method (Android, iOS)

• Add onPictureInPictureStatusChanged event listener (Android)

• Resolve issue React Native Video Picture In Picture for Android #2621

memo)
As I know Activity's onUserLeaveHint is acceptable lifecycle method for enter PIP. but it's impossible to detect that method within this library. so, I implement enterPicturInPicture within onHostPause.
If someone doesn't like this approach, please leave comment in this PR, I'll give you intent style code.

[freeboub]

Thank you for the PR, can you open a PR in draft, it will allow to comment on it.

public void onHostPause() {
isInBackground = true;
if (pictureInPictureEnabled) {
enterPictureInPictureMode();
return;
}
You cannot do like this, when another application will go over the app, it will automatically switch to Pip ...

[freeboub]

I don't understand why you use a dedicated fragment : ReactExoplayerFragment can you explain why you had to use it please ? (point to doc or other sample is enough)

[freeboub]

can you enable pip in sample to be able to test easily please

[YangJonghun]

can you enable pip in sample to be able to test easily please

      <Video
        style={{width: '100%', height: 300, backgroundColor: 'black'}}
        resizeMode={ResizeMode.CONTAIN}
        source={{
          uri: 'https://demo.unified-streaming.com/k8s/features/stable/video/tears-of-steel/tears-of-steel-en.ism/.mpd',
        }}
        selectedTextTrack={{
          type: SelectedTrackType.INDEX,
          value: 0,
        }}
        fullscreen={false}
        controls
        pictureInPicture
        playWhenInactive
        playInBackground={false}
        debug={{enable: true, thread: true}}
        onPictureInPictureStatusChanged={({isActive}) => {
          console.log(isActive, 'isActive');
        }}
      />

[YangJonghun]

@freeboub

I don't understand why you use a dedicated fragment : ReactExoplayerFragment can you explain why you had to use it please ? (point to doc or other sample is enough)

To implement PIP, we need to use the onUserLeaveHint and onPictureInPictureModeChanged methods of Activity. However, this requires forcing the user to modify their native code.
To avoid this, I tricky replaced onUserLeaveHint with onPause in the Fragment and onPictureInPictureModeChanged with Fragment's onPictureInPictureModeChanged in the Fragment.

[freeboub]

can you enable pip in sample to be able to test easily please

      <Video
        style={{width: '100%', height: 300, backgroundColor: 'black'}}
        resizeMode={ResizeMode.CONTAIN}
        source={{
          uri: 'https://demo.unified-streaming.com/k8s/features/stable/video/tears-of-steel/tears-of-steel-en.ism/.mpd',
        }}
        selectedTextTrack={{
          type: SelectedTrackType.INDEX,
          value: 0,
        }}
        fullscreen={false}
        controls
        pictureInPicture
        playWhenInactive
        playInBackground={false}
        debug={{enable: true, thread: true}}
        onPictureInPictureStatusChanged={({isActive}) => {
          console.log(isActive, 'isActive');
        }}
      />

Ok, but will it work without native controls ?

[YangJonghun]

can you enable pip in sample to be able to test easily please

      <Video
        style={{width: '100%', height: 300, backgroundColor: 'black'}}
        resizeMode={ResizeMode.CONTAIN}
        source={{
          uri: 'https://demo.unified-streaming.com/k8s/features/stable/video/tears-of-steel/tears-of-steel-en.ism/.mpd',
        }}
        selectedTextTrack={{
          type: SelectedTrackType.INDEX,
          value: 0,
        }}
        fullscreen={false}
        controls
        pictureInPicture
        playWhenInactive
        playInBackground={false}
        debug={{enable: true, thread: true}}
        onPictureInPictureStatusChanged={({isActive}) => {
          console.log(isActive, 'isActive');
        }}
      />

Ok, but will it work without native controls ?

Yes, it's same like iOS and there are no restrictions.

[YangJonghun]

Thank you for the PR, can you open a PR in draft, it will allow to comment on it.

public void onHostPause() { isInBackground = true; if (pictureInPictureEnabled) { enterPictureInPictureMode(); return; } You cannot do like this, when another application will go over the app, it will automatically switch to Pip ...

I'm not sure, but as I know onUserLeaveHint is also called when another app is turned on.
Actually I wanted to detect and implement the home key event, but the API was deprecated and I replaced it with onPause.

[YangJonghun]

@nick-mccomb-easygo
Unfortunately, I've been so busy lately that I haven't been able to keep up with this work, but I'm going to finish it.
You're welcome to implement it if you'd like.

[YangJonghun]

@freeboub @KrzysztofMoch

Hi. I'd like to let you know this PR is ready for review 🙌
please review when you possible. :)

FYI,
this PR depends on PR #3489
If doesn't merge, PIP's action button doesn't work properly

I think this code is too mush complicate to understand.
but I couldn't rewrite FullScreenPlayerView code in this PR.
It would be nice if you(or someone) could refactor it later.

[KrzysztofMoch]

LGTM 👍, thank you for the PR!
Tested it on Android 14, and 10 (simulators) and it worked for me. Will test it on device later.
Lets @freeboub review and for me we can merge

[tiplefbxx]

It seems one more issue for ios
When you change source being in background with PiP, it will close the pip and do not notify onPictureInPictureStatusChanged, closing manually also do not help videoRef.current?.exitPictureInPicture

[YangJonghun]

@tiplefbxx
Thank you for providing sample code.
Previous code does not assume source or mount changes from entering PIP to leaving PIP, as it is common to utilize the existing view when playing other videos.

Fixed Code has been applied with 362459d to force PIP to exit when the video component is unmounted within PIP

BTW: Don't forget to update doc! It needs to add android:supportsPictureInPicture="true" to main activity, otherwise it will not work

I've already stated this in this PR. Thanks for letting me know. :)

It seems one more issue for ios
When you change source being in background with PiP, it will close the pip and do not notify onPictureInPictureStatusChanged, closing manually also do not help videoRef.current?.exitPictureInPicture

onPictureInPictureStatusChanged is an API provided by iOS native, and it's not clear what the intent of the prop is. To implement the feature, I have to forced call onPictureInPictureStatusChanged for JS, which may be outside the intent of the existing prop.🤔

[tiplefbxx]

Might this ios thing is not related to you pr, but if you started changes for PiP it could be interesting for you
Not crucial issue but
When you go to PiP and return to foreground by clicking app icon from HomeScreen it will return to app, but the PiP will be not resolved, player will be minimised in PiP, so it needs to manually call method to close PiP

It's might be not issue at all as PiP usually works for ios in that way always, but could be surprising behaviour for developers who are used to see closed pip after returning to foreground as it usually works for cross button and app selector

[YangJonghun]

@tiplefbxx
Thanks for suggestion, but for now I don't want this PR to grow any bigger, so I'll figure out it when this PR is merged🥲

[KrzysztofMoch]

Anyone interested in this feature, please see this discussion

• Picture in Picture on Android - We need your help! #4228

[freeboub]

I just fix conflict. The only comments I have are in the discussion

[KrzysztofMoch]

I have created 6.7.0-rc.0 release that contain this PR - as we will get feedback and do some changes I will push more RCs. For now let's see how it will go 🙌

[KrzysztofMoch]

I have released next rc version 6.8.0-rc.0 - if there will be no reported issues, this PR should be merged 🙌

[tungmin97]

Hi @YangJonghun, I'm encountering the same issue as @tiplefbxx mentioned where my app would always crash upon return from PIP.

For a little bit more context, I have a separated Video component outside my navigator stack, and in my testing unload the video component below when having a floating player eliminates this behavior.

Your app just crashed. See the error below.
java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
  java.util.ArrayList.get(ArrayList.java:437)
  com.brentvatne.exoplayer.ReactExoplayerView.setIsInPictureInPicture(ReactExoplayerView.java:2305)
  com.brentvatne.exoplayer.ReactExoplayerFragment.onPictureInPictureModeChanged(ReactExoplayerFragment.kt:35)
  androidx.fragment.app.Fragment.performPictureInPictureModeChanged(Fragment.java:3221)
  androidx.fragment.app.FragmentManager.dispatchPictureInPictureModeChanged(FragmentManager.java:3017)
  androidx.fragment.app.FragmentManager.lambda$new$3$androidx-fragment-app-FragmentManager(FragmentManager.java:468)
  androidx.fragment.app.FragmentManager$$ExternalSyntheticLambda3.accept(Unknown Source:4)
  androidx.activity.ComponentActivity.onPictureInPictureModeChanged(ComponentActivity.java:1097)
  android.app.Activity.dispatchPictureInPictureModeChanged(Unknown Source:23)
  android.app.ActivityThread.handleWindowingModeChangeIfNeeded(Unknown Source:40)
  android.app.ActivityThread.performActivityConfigurationChanged(Unknown Source:12)
  android.app.ActivityThread.performConfigurationChangedForActivity(Unknown Source:28)
  android.app.ActivityThread.handleActivityConfigurationChanged(Unknown Source:273)
  android.app.ActivityThread$ActivityClientRecord$1.onConfigurationChanged(Unknown Source:14)
  android.view.ViewRootImpl.performConfigurationChange(Unknown Source:81)
  android.view.ViewRootImpl.handleResized(Unknown Source:112)
  android.view.ViewRootImpl.-$$Nest$mhandleResized(Unknown Source:0)
  android.view.ViewRootImpl$ViewRootHandler.handleMessageImpl(Unknown Source:658)
  android.view.ViewRootImpl$ViewRootHandler.handleMessage(Unknown Source:15)
  android.os.Handler.dispatchMessage(Unknown Source:19)
  android.os.Looper.loopOnce(Unknown Source:182)
  android.os.Looper.loop(Unknown Source:82)
  android.app.ActivityThread.main(Unknown Source:123)
  java.lang.reflect.Method.invoke(Native Method)
  com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(Unknown Source:11)
  com.android.internal.os.ZygoteInit.main(Unknown Source:312)

[YangJonghun]

@tungmin97
Thanks for reporting me. but it's hard to me to reproduce what you're describing🥲
If you can, please provide me any small reproduction code🙏

[tungmin97]

@YangJonghun here is the crash example. I've found some issues only happen on android.

1 - We can still swipe home to enter pip even after the video outside navigation is paused, the other has enterPictureInPictureOnLeave set to false.

2 - Has both video playing, pause then play the floated video, swipe home or press the button to enter PIP mode, notice the source is set to the other video with enterPictureInPictureOnLeave set to false.

3 - Resume the app while in PIP will always crash the application

[YangJonghun]

@tungmin97
Sorry for late reply.
Multiplayer issue is a difficult problem to solve, so fix will be delayed.
I'm working on this PR personally, so I can't guarantee any due date, but I'll get to it as soon as I can. 🙏

FYI. @KrzysztofMoch @freeboub