Feature Request: More Fine-Grained Control Over Texture Units

[fweth]

Hi, I often have the situation where there is a texture which should be available to all programs at all times and then I have two framebuffers between which I play ping-pong. My understanding is that in vanilla WebGL I would bind the texture to the first texture unit and the framebuffers to the second and third texture unit and then only call uniform1i to tell the programs which framebuffers they should be using. Or, if I have program A which reads from FB 0 and writes to FB 1 and then program B which reads from FB 1 and writes to FB 0, I wouldn't even need to change any uniforms in the render loop. Yet with TWGL, I haven't found a method to achieve this, as it will always put the first texture I add as a uniform in the first texture unit, and so on, with this process repeated for each program I set the uniforms, so I need to rebind textures in the render loop instead of just resetting uniforms. Again, that's just my understanding, maybe I'm wrong though.

[greggman]

I'm not really sure this can be handled in a good way in twgl. You're free to mix twgl and raw WebGL as twgl doesn't keep any state. I do that often.

As for this case I can think of 2 ways, both with issues.

1. Provide an object to twgl.setUniforms as in

   twgl.setUniforms(programInfo, {
       someUniformSampler: { texture: someTex, unit: 3 },  // use unit 3
   });
   

   The problem with this solution is twgl can't know which units you used and which you didn't. Meaning if you did this

   twgl.setUniforms(programInfo, {
       someUniformSampler1: { texture: someTex1, unit: 2 },  // use unit 2
       someUniformSampler2: someTex2,
       someUniformSampler3: someTex3,
       someUniformSampler4: someTex4,
   });
   

   I'd expect someTex2 to go to unit 0, someTex3 to unit 1, someTex3 to unit 4 (skipped 3)

   But that doesn't work because in the very next call could specify conflicting results. Or you could specify them separately

   twgl.setUniforms(programInfo, {
       someUniformSampler1: { texture: someTex1, unit: 2 },  // use unit 2
   });
   twgl.setUniforms(programInfo, {
       someUniformSampler2: someTex2,
       someUniformSampler3: someTex3,
       someUniformSampler4: someTex4,
   });
   

   The above is valid twgl. The second call would need to know not to use unit 2 but can't unless state is saved from the first call. But if you do that you'd now need a new call to set "reset the used flags". Further, there's now tracking overhead that wasn't there before make it's slower to set uniforms.

   Even in a single call

   twgl.setUniforms(programInfo, {
       someUniformSampler2: someTex2,
       someUniformSampler3: someTex3,
       someUniformSampler4: someTex4,
       someUniformSampler1: { texture: someTex1, unit: 2 },  // use unit 2
   });
   

   You'd have to scan the list of uniforms twice, once to find the ones that have a uint specified and then another that finally does the remaining uinforms

2. Set them in twgl.createProgramInfo

   This might work? You'd do this

   const pInfo = twgl.createProgramInfo(gl, [vs, fs], {
       texUnitMapping: {
           someUniformSampler: 3,  // make someUniformSampler always use unit 3
        },
   });
   

   But, then I went to implement it and it just seems like overkill. It's not as simple as it seems because it means createUniformSetters needs to do some non insignificant tracking of available texture units when it's allocating units. It's not a huge amount of code but it's not trivial either. It would have to handle arrays of sampler uniforms which mean you basically need this array of unused uniform spans so that you can find the first span that has enough uniforms to satisfy the need. That's not a ton of work but it seems like too much work for something no one has asked for after this library existing for 6 yrs

   Example: You specify like above so you have this

   assigned = {someUniformSampler: 3}
   unused = [0, 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
   

   If the first uniform sampler that comes in needs 4 units (an array of 4) it can't use 0, 1, 2, it has to start a 4,5,6,7. This means you probably don't want to have unused be a list. You want it to be spans so that you can more easily check if there are enough consecutive units. So,

   assigned = {someUniformSampler: 3}
   unused = [[0, 3], [4, 12]];  // where this is [indexOfFirstUnitInSpan, numUnitsInSpan]
   

   But now you need code that splits spans (in other words more code, yes, tiny but still).

   If there are 4 unassigned uniform samplers that appear before someUniformSampler they'd use units 0, 1, 2, 4, but then when you actually get to someUniformSampler, if it's an array it's already too late. So now you need to change the code so that it assigns specified uniforms before unspecified (more code).

Further, what are you doing that this level of optimization matters? Yes, you could make less calls into WebGL by organizing which texture units are used but are you doing this operation 5000 times per frame or just 1, 2 or 3 times per frame? If it's like normal, just a few times a frame, then if it was me I wouldn't worry about it and just set all the uniforms every time.

[fweth]

Thanks so much for the quick and comprehensive answer! I didn't know about the someUniformSampler: { texture: someTex, unit: 3 } pattern, that's basically all I was asking for! Still, it seems that I didn't fully understand the WebGL mechanism, apparently you can't render to a buffer if the buffer is still bound to some texture unit, even if this unit is not bound to the current program (I got some errors), is that correct? If so, there isn't really much space for improvement. Also, I don't do 5000 operations per frame, I am a bit neurotic with trying to eliminate unnecessary computation steps but if you say they don't really matter, I totally believe this :)

[greggman]

Just to clarify

I didn't know about the someUniformSampler: { texture: someTex, unit: 3 } pattern

It's not a pattern. I was a possible solution but has issues so I didn't implement it

you can't render to a buffer if the buffer is still bound to some texture unit, even if this unit is not bound to the current program (I got some errors)

I'm not sure what this means. You can render just fine if the current program is not using the texture that is in the current framebuffer.

Example: https://jsgist.org/?src=ac918fbac37047b5170bac55361eaa8d

Note: branching in a program so that the the texture is not accessed does not count as "not using".

[fweth]

Oh, misunderstanding then, I thought this pattern is already implemented. So I got some errors and thought it's because I can't have the framebuffer I'm rendering to bound to some texture unit at the same time, but probably the errors were just because I used an unsupported pattern.

Regarding a solution, maybe I don't understand WebGL well enough, but would it be possible to have some optional parameter in createTexture specifying a unit to which the texture should be bound, so that I can later set this integer manually as a uniform, e.g. instead of twgl.setUniforms(gl, {u_Texture: myTex}); I do twgl.setUniforms(gl, {u_Texture: 3}); when I specified that myTex should be bound to texture unit 3?