Proposal: a way of tracking pointer position on pan/zoom touch-action

[samuelmtimbo]

I understand one of the main purposes of "pointercancel" is to signal the user agent will handle panning/zooming/etc; and will no longer fire pointer events, which makes sense, but there are genuine situations where this might be limiting. Let me try to exemplify.

This might be a very particular scenario, but I would like to bring it to the table: some applications rely solely on Pointer Events to track which pointers are "visible" on the screen and such tracking might be visually represented, say by rendering an element on at the current pointers(s) positions(s). This might be done for an aesthetic and/or usability reason. This will work fine most of the time but not when scrolling an overflown element (a list, for instance) on a touch device. Moreover, basically it won't work whenever the browser "handles a touch-action".

Full disclosure: I am developing a game-like web app where I envisioned a "bot", an eye-like creature that basically follows all the pointers on the screen like he is omniscient of what the user is doing... There is this expectation break when the user performs panning on a list, say a common usage search list, which intuitively makes no sense because the finger is clearly still touching the screen.

I can think of some workarounds, but all of them feel hardcore:

1. Use CSS touch-action: none; on absolutely all elements and implement scrolling manually in JavaScript using Pointer Events - this is a complex and error-prone task and probably will come with a performance penalty. I am also not sure if it would be possible to correctly replicate the "scroll momentum" browser behavior;

2. Use Pointer Events alongside Touch Events (say by registering "touchmove"/"touchend" after "pointercancel") - this would lead to complex and duplicated logic and repeated state since it would be hard to associate the canceled pointer with a subsequent touch (there is no direct correspondence between Pointer Events pointerId and Touch identifier, for instance);

3. Do not use Pointer Events and use a combination of Mouse Events and Touch Events instead - this would totally go against the point of Pointer Events IMHO (and there is a chance it would also be unfeasible all things considered);

Am I missing something?

I would like your help thinking about a non-breaking alternative. I am sorry for my lack of understanding of the implementation details, but here are some possibilities that come to my mind:

1. Adding a flag on "pointerdown" signaling to the user agent "I still want to receive pointer events after you start handling a touch action". That could make the user agent not automatically fire "pointercancel"/"pointerup" at all in that case, but still move on handling the touch action. I am imagining something syntactically similar to addEventListener passive property;

2. Firing additional events such as "touchactionstart", "touchactionmove", "touchactionend" associated with any of the previously canceled pointers (by pointerId);

I recognize it might be hard to feel empathetic with this need, but I don't see any other way around. I hope I can help further.

[yacinehmito]

I have a similar use case.
I am trying to implement on the browser something I can only call "natural panning on overscroll".

This a delightful behaviour for scrollable modals: when reaching the top of the scrolling area, panning the element moves it along the finger. If it crosses a threshold, the modal is closed. If the finger is released before reaching the threshold, the modal snaps back into place. Here is an example from Apple Maps (native iOS application), that illustrates the kind of behaviour I am trying to implement.

natural_panning_on_overscroll.mp4

I've spent a lot of time trying to build this, and came to conclusions very similar to yours. I can either use touch-action: none; and reimplement the scrolling logic in JavaScript (which would be terrible from a UX and performance standpoints; scrolling behaviour is quite sophisticated), or try to hack my way around with the deprecated touch events.

The core issue is that, once the user agent decides to take over, we become blind to any types of changes. Use cases that rely on the default behaviour from the user agent but still needs to react on pointer changes are, at the moment, not implementable with pointer events. The exception is with pointerrawupdate, which keep firing even when the default behaviour from the user agent takes effect. However, this 1) even alone is not sufficient to reflect the pointer state 2) this does not seem in line with the event type's intended purpose.