# Read/write Stencil buffer and pass to glsl filter

**URL:** <https://openrndr.discourse.group/t/read-write-stencil-buffer-and-pass-to-glsl-filter/548>\
**Category:** How to?\
**Created:** [April 16, 2023, 12:57pm UTC](https://openrndr.discourse.group/t/read-write-stencil-buffer-and-pass-to-glsl-filter/548 "2023-04-16T12:57:31Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![LH99](https://yyz2.discourse-cdn.com/free1/user_avatar/openrndr.discourse.group/lh99/32/575_2.png) [@LH99](https://openrndr.discourse.group/u/LH99)\
**Post date:** [April 16, 2023, 12:57pm UTC](https://openrndr.discourse.group/t/read-write-stencil-buffer-and-pass-to-glsl-filter/548/1 "2023-04-16T12:57:31Z")

</div>

I am currently working on a stencil buffer to hold shape references 1,2,3…

The idea is to draw a shape like a rect or circle with `drawer.isolatedWithRenderTarget(rt)` and draw the shapes at different z-values (maybe ?) of 1 or 2 each.  
On draw, the stencil buffer gets updated to now hold the unsigned integer reference of that shape.

This would be the uniform input to a glsl filter, something like `uniform sampler2D stencil;`.  
There, I want to update each pixel based on that stencil value.

I have only found `depthBuffer(format = DepthFormat.STENCIL8)`so far, but most access functions are private and this seems to be “walled off” from me.

Any ideas on how to realize such a stencil buffer with write access ? 🤔

Kind regards,  
Lukas

---

<div class="post-metadata">

**Author:** ![LH99](https://yyz2.discourse-cdn.com/free1/user_avatar/openrndr.discourse.group/lh99/32/575_2.png) [@LH99](https://openrndr.discourse.group/u/LH99)\
**Post date:** [April 16, 2023, 1:04pm UTC](https://openrndr.discourse.group/t/read-write-stencil-buffer-and-pass-to-glsl-filter/548/2 "2023-04-16T13:04:53Z")

</div>

This sketch illustrates what i try to achieve:

 ![Stencil_Buffer_Sketch](https://global.discourse-cdn.com/free1/uploads/openrndr/original/1X/c783e0455486ec7dd9d8bbb8642be68fbfb07f26.jpeg)

Start with cleared buffer (0 everywhere), draw shape 1 on top, draw shape 2 on top.

---

<div class="post-metadata">

**Author:** ![LH99](https://yyz2.discourse-cdn.com/free1/user_avatar/openrndr.discourse.group/lh99/32/575_2.png) [@LH99](https://openrndr.discourse.group/u/LH99)\
**Post date:** [April 18, 2023, 7:17pm UTC](https://openrndr.discourse.group/t/read-write-stencil-buffer-and-pass-to-glsl-filter/548/3 "2023-04-18T19:17:13Z")

</div>

I found a decent workaround, which I’d like to share:

I set up a new renderTarget with a single specific colorBuffer holding an 8-bit uint per pixel:

```auto
val stencilTarget = renderTarget(width, height) {
    colorBuffer(format = ColorFormat.R, type = ColorType.UINT8)
}
val stencilBuffer = stencilTarget.colorBuffer(0)

```

On that stencilTarget I draw like on any other renderTarget, … but I cast Ints with a helper function `Int.toR()` shown below:

```auto
drawer.isolatedWithTarget(stencilTarget) {
    clear(0.toR())

    fill = 1.toR()
    stroke = null
    circle(Circle(200.0, 100.0, radius = 50.0))

     fill = 2.toR()
     stroke = 2.toR()
     rectangle(Rectangle(width*0.5, 0.0, width*0.5, height * 1.0))
}

```

using the mentioned cast:

```auto
fun Int.toR(): ColorRGBa = ColorRGBa(r = this / 256.0, g = 0.0, b = 0.0)

```

I can pass this to my glsl filter just as I wanted, where I define `uniform usampler2D stencil;`. In my filter main I use it via `uint stencilValue = texture(stencil, v_texCoord0).x;` with typical control flow `if (stencilValue == 1u) ... else if (stencilValue == 2u) ...` and so on.

---

<div class="post-metadata">

**Author:** ![abe](https://yyz2.discourse-cdn.com/free1/user_avatar/openrndr.discourse.group/abe/32/700_2.png) [@abe](https://openrndr.discourse.group/u/abe)\
**Post date:** [April 19, 2023, 7:18am UTC](https://openrndr.discourse.group/t/read-write-stencil-buffer-and-pass-to-glsl-filter/548/4 "2023-04-19T07:18:16Z")

</div>

Great that you found a workaround, and thank you for sharing it! 🙂

I thought about using renderTargets but didn’t want to suggest that because I was curious about actually using stencils as described in [learnopengl](https://learnopengl.com/Advanced-OpenGL/Stencil-testing). I guess one could directly access the [OPENGL driver](https://guide.openrndr.org/advancedTopics/lowLevelDrawing.html) but that is discouraged as maybe in the future it is switched to a different backend.

With your approach, could you also use `ColorType.FLOAT32` and then just round to integers? Then you wouldn’t be limited to 256 levels (in case that’s an issue). Maybe faster to use ints in the shader?

---

<div class="post-metadata">

**Author:** ![LH99](https://yyz2.discourse-cdn.com/free1/user_avatar/openrndr.discourse.group/lh99/32/575_2.png) [@LH99](https://openrndr.discourse.group/u/LH99)\
**Post date:** [April 20, 2023, 7:45am UTC](https://openrndr.discourse.group/t/read-write-stencil-buffer-and-pass-to-glsl-filter/548/5 "2023-04-20T07:45:09Z")

</div>

I dabbled with some variations before and found `ColorType.FLOAT32` to be tricky.

In my case I want to check for equality (in an iterative procedure so ~100 times per pixel) and writing floats to a buffer where the precision on the glsl end may change (lowp/mediump/highp precision) seemed wasteful.  
Also, I have not yet found a robust way to get the _exact_ same floats to the filter 🤔

As the picture above suggests, I just need a handful of ints. Should I need to scale, I’d opt in for `ColorType.SINT16_INT` or `ColorType.SINT32_INT`and go for an int sampler on the glsl side.