LlamaVision: Move xattn cache generation to text prefill forward

[cglagovichTT]

Ticket

#15008

Problem description

From the ticket:

Currently, xattn caches are generated before running text prefill. This is an issue for vLLM integration since vLLM will provide page tables for the xattn caches which we must respect. If xattn caches are generated before text prefill, then prefill attention SDPA will need to support paged KV.

In order to allow vLLM integration without changing prefill SDPA to support paged KV, we will modify the model to compute xattn caches during text prefill. That way, text prefill will generate an unpaged cache, use it locally in attention, then do a paged fill cache to store the cache for decode iterations.

What's changed

I moved xattn cache generation to text model prefill. This is a change to the plumbing, nothing drastic.

Checklist

• Post commit CI https://github.com/tenstorrent/tt-metal/actions/runs/11843122221
  • Note: My changes should have no effect on the post commit tests
• T3K unit, frequent, demo https://github.com/tenstorrent/tt-metal/actions/runs/11843107464