You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
So far Gluten's native shuffle implementations, including VL, CH, VL + RSS are activated by specifying corresponding shuffle manager types in Spark configuration.
However if we want to mix different native shuffles in the same Spark session even in the same query plan, as Spark only allows one shuffle manager to be registered (see also a related SPIP) yet, we should enhance out current approach by providing a registry of native shuffles internally under an umbrella shuffle manager. Then query planner could choose an optimal one among all the candidates.
The text was updated successfully, but these errors were encountered:
Part of #6920.
So far Gluten's native shuffle implementations, including VL, CH, VL + RSS are activated by specifying corresponding shuffle manager types in Spark configuration.
However if we want to mix different native shuffles in the same Spark session even in the same query plan, as Spark only allows one shuffle manager to be registered (see also a related SPIP) yet, we should enhance out current approach by providing a registry of native shuffles internally under an umbrella shuffle manager. Then query planner could choose an optimal one among all the candidates.
The text was updated successfully, but these errors were encountered: