Use samplerCube and sampler2D in a single glsl Shader

Stephcraft

How can I use a samplerCube and sampler2D in a single glsl Shader ?

I get the following error when I try to use both at the same time : Cannot validate shader program:Validation Failed: Sampler error: Sampler of different types use the same texture image unit. -or- A sampler's texture unit is out of range (grater than max allowed or negative)

I dont think my unit is out of range, I think the problem is because of the same unit problem.

Exemple code :

Sketch

import queasycam.*;
import java.nio.IntBuffer;

QueasyCam cam;

PShader shader;
PShape sphere;

PImage[] skybox = new PImage[6];
PImage cubeMap;
PImage texture;

void setup() {
  fullScreen(P3D);
  noStroke();

  //Create FPS Camera
  cam = new QueasyCam(this);

  //Load Shader
  shader = loadShader("frag.glsl", "vert.glsl");

  //Load Skybox Image
  cubeMap = loadImage("skybox.jpg");
  skybox = sliceCubeMap(cubeMap);

  //Load texture
  texture = loadImage("texture.png");

  //Cube Map
  glslCubeMap(2, cubeMap);
  shader.set("cubemap", 2);

  //Diffuse Map
  shader.set("diffuseMap", texture);

  sphere = createShape(SPHERE, 150);
  sphere.setFill(color(-1, 50));
} 

void draw() {
  background(0);

  //Update Camera Position
  shader.set("camPosition", cam.position);

  //Render Shape
  shader(shader);
  shape(sphere);
}

PImage[] sliceCubeMap(PImage cubeMap) {
  PImage[] textures = new PImage[6];

  int ux = cubeMap.width/4;
  int uy = cubeMap.height/3;

  //X
  textures[0] = cubeMap.get(ux*2, uy, ux, uy);
  textures[1] = cubeMap.get(0, uy, ux, uy);

  //Y
  textures[2] = cubeMap.get(ux, 0, ux, uy);
  textures[3] = cubeMap.get(ux, uy*2, ux, uy);

  //Z
  textures[4] = cubeMap.get(ux, uy, ux, uy);
  textures[5] = cubeMap.get(ux*3, uy, ux, uy);

  return textures;
}

void glslCubeMap(int unit, PImage cubeMap) {
  glslCubeMap(unit, sliceCubeMap(cubeMap));
}

void glslCubeMap(int unit, PImage[] textures) {
  glslCubeMap(unit, textures[0], textures[1], textures[2], textures[3], textures[4], textures[5]);
}

void glslCubeMap(int unit, PImage posX, PImage negX, PImage posY, PImage negY, PImage posZ, PImage negZ) {

  PGL pgl = beginPGL();
  // create the OpenGL-based cubeMap
  IntBuffer envMapTextureID = IntBuffer.allocate(1);
  pgl.genTextures(1, envMapTextureID);
  pgl.activeTexture(PGL.TEXTURE0 + unit); // Change texture unit
  pgl.enable(PGL.TEXTURE_CUBE_MAP);  
  pgl.bindTexture(PGL.TEXTURE_CUBE_MAP, envMapTextureID.get(0));
  pgl.texParameteri(PGL.TEXTURE_CUBE_MAP, PGL.TEXTURE_WRAP_S, PGL.CLAMP_TO_EDGE);
  pgl.texParameteri(PGL.TEXTURE_CUBE_MAP, PGL.TEXTURE_WRAP_T, PGL.CLAMP_TO_EDGE);
  pgl.texParameteri(PGL.TEXTURE_CUBE_MAP, PGL.TEXTURE_WRAP_R, PGL.CLAMP_TO_EDGE);
  pgl.texParameteri(PGL.TEXTURE_CUBE_MAP, PGL.TEXTURE_MIN_FILTER, PGL.LINEAR);
  pgl.texParameteri(PGL.TEXTURE_CUBE_MAP, PGL.TEXTURE_MAG_FILTER, PGL.LINEAR);

  //Load in textures
  PImage[] textures = { posX, negX, posY, negY, posZ, negZ };
  for (int i=0; i<textures.length; i++) {
    PImage texture = textures[i];
    int w = texture.width;
    int h = texture.height;
    texture.loadPixels();
    pgl.texImage2D(PGL.TEXTURE_CUBE_MAP_POSITIVE_X + i, 0, PGL.RGBA, w, h, 0, PGL.RGBA, PGL.UNSIGNED_BYTE, IntBuffer.wrap(texture.pixels));
  }

  endPGL();
}

vert.glsl

uniform mat4 transform;
uniform mat4 modelview;
uniform mat3 normalMatrix;
uniform mat4 texMatrix;

uniform vec3 camPosition;

varying vec3 reflectVector;
varying vec3 refractVector;
varying vec4 vertTexCoord;

attribute vec4 position;
attribute vec4 vertex;
attribute vec3 normal;
attribute vec2 texCoord;

void main(void) {
    gl_Position = transform * vertex;

    vec3 unitNormal = normalize(normal);

    vec3 viewVector = normalize(vertex.xyz - camPosition);

    reflectVector = reflect(viewVector, unitNormal);
    refractVector = refract(viewVector, unitNormal, 1.0/1.33);

    //2D Texture Coordinates
    vertTexCoord = texMatrix * vec4(texCoord, 1.0, 1.0);
}

frag.glsl

#ifdef GL_ES
precision mediump float;
precision mediump int;
#endif

//Cube texture
uniform samplerCube cubemap;

//2D texture
uniform sampler2D diffuseMap;

varying vec3 reflectVector;
varying vec3 refractVector;

//Texture Coordinates
varying vec4 vertTexCoord;

void main(void) {
    vec4 reflectColor = texture(cubemap, -reflectVector);
    vec4 refractColor = texture(cubemap, -refractVector);

    vec4 diffuse = texture2D(diffuseMap, vertTexCoord.st);

    //Switch Red and Blue Channel (ARGB != RGBA)
    reflectColor = vec4(reflectColor.b, reflectColor.g, reflectColor.r, reflectColor.a);
    refractColor = vec4(refractColor.b, refractColor.g, refractColor.r, refractColor.a);

    vec4 materialColor = mix(reflectColor, refractColor, 1.0);

    gl_FragColor = mix(diffuse, materialColor, 0.5);
}

Here I change the unit : pgl.activeTexture(PGL.TEXTURE0 + unit); // Change texture unit

thank you in advance !

nabr

Hello

probably the error will be solved:
frag.glsl texture >= GLSL #version 130

with texture2D(cubemap ...

your scetch runs fine on my x64 windows machine with processing version 3.3.7

For debug: make sure that loadImage("skybox.jpg"); loadImage("texture.png"); have the same dimensions.

i used this one
https://learnopengl.com/img/advanced/cubemaps_skybox.png
Further reading:
https://www.khronos.org/opengl/wiki/Cubemap_Texture
https://en.wikipedia.org/wiki/Cube_mapping

Stephcraft

Well :/ it still doesn't work for me after updating Processing from 3.3.6 to 3.3.7. I tried using texture2D(cube map, ...) but it still doesn't work.

The problem happens as soon as I add this line : uniform sampler2D diffuseMap;

The code is not working on my macOS Sierra version 10.12.6

Also I get this warning in the console : OpenGL error 1280 at top endDraw(): invalid enumerant

nabr

@Stephcraft sound like a bug, can you try run processing windowed

did you use this texture for both loadimages ?
https://learnopengl.com/img/advanced/cubemaps_skybox.png

what does this mean : ) ? (seriously idk )
shader.set("cubemap", 2);

https://processing.org/reference/PShader_set_.html

Stephcraft

I used the skybox for both images, tried Windowed, but still not working.

: ) means : Happy Face, the ':' are the eyes and the ')' is the mouth

nabr

funny

what happens if you just

pgl.activeTexture(PGL.TEXTURE0); // without unit

?

Stephcraft

Still the same error :/

Weird, because TEXTURE4 doesn't exist, maybe its not where the unit goes ? I took the + unit from the PShader class where it binds the Textures here : https://github.com/processing/processing/blob/master/core/src/processing/opengl/PShader.java#L728

nabr

i was wrong

https://thebookofshaders.com/glossary/?search=textureCube
https://thebookofshaders.com/glossary/?search=texture2D

so in modern jogl you can pass a samplerCube to a texture(), but in older version the function was textureCube

frag.glsl

vec4 reflectColor = textureCube(cubemap, -reflectVector);
vec4 refractColor = textureCube(cubemap, -refractVector);

Stephcraft

Yeah you are right, not getting errors for unknown function textureCube when using it but still the same image unit error :

Cannot validate shader program:Validation Failed: Sampler error: Sampler of different types use the same texture image unit. -or- A sampler's texture unit is out of range (grater than max allowed or negative)

nabr

idk can you try it with different jpg without the for loop ?

https://stackoverflow.com/questions/36444975/trying-to-port-a-glsl-glass-shader-to-processing-3-0

nabr

pgl.texImage2D(PGL.TEXTURE_CUBE_MAP_POSITIVE_X + i
shader
is an Array cubeSampler

pgl.texImage2D(PGL.GL_TEXTURE_2D
shader
is a sampler2d

can you also use .bind and .unbind() in a shader ??
https://github.com/processing/processing/wiki/Advanced-OpenGL#processing-3x

its nicer.

nabr

this feels weird

shader.set("cubemap", 2);
it should be
shader.set("cubemap", textureArray);

Stephcraft

Maybe, but I think the int value, in this case '2' is the location of the texture or something

shader.set("cubemap", 2);

also,

glslCubeMap(int unit, PImage[] textures)

is the same to

glslCubeMap(int unit, PImage posX, PImage negX, PImage posY, PImage negY, PImage posZ, PImage negZ)

because its calling :

glslCubeMap(unit, textures[0], textures[1], textures[2], textures[3], textures[4], textures[5]);

I also tried shader.bind() and shader.unbind() around the glslCubeMap function, but again, same error :

void glslCubeMap(int unit, PImage posX, PImage negX, PImage posY, PImage negY, PImage posZ, PImage negZ) {

  shader.bind(); // new
  PGL pgl = beginPGL();
  // create the OpenGL-based cubeMap
  IntBuffer envMapTextureID = IntBuffer.allocate(1);
  pgl.genTextures(1, envMapTextureID);
  pgl.activeTexture(PGL.TEXTURE0 + unit); // Change texture unit
  pgl.enable(PGL.TEXTURE_CUBE_MAP);  
  pgl.bindTexture(PGL.TEXTURE_CUBE_MAP, envMapTextureID.get(0));
  pgl.texParameteri(PGL.TEXTURE_CUBE_MAP, PGL.TEXTURE_WRAP_S, PGL.CLAMP_TO_EDGE);
  pgl.texParameteri(PGL.TEXTURE_CUBE_MAP, PGL.TEXTURE_WRAP_T, PGL.CLAMP_TO_EDGE);
  pgl.texParameteri(PGL.TEXTURE_CUBE_MAP, PGL.TEXTURE_WRAP_R, PGL.CLAMP_TO_EDGE);
  pgl.texParameteri(PGL.TEXTURE_CUBE_MAP, PGL.TEXTURE_MIN_FILTER, PGL.LINEAR);
  pgl.texParameteri(PGL.TEXTURE_CUBE_MAP, PGL.TEXTURE_MAG_FILTER, PGL.LINEAR);

  //Load in textures
  PImage[] textures = { posX, negX, posY, negY, posZ, negZ };
  for (int i=0; i<textures.length; i++) {
    PImage texture = textures[i];
    int w = texture.width;
    int h = texture.height;
    texture.loadPixels();
    pgl.texImage2D(PGL.TEXTURE_CUBE_MAP_POSITIVE_X + i, 0, PGL.RGBA, w, h, 0, PGL.RGBA, PGL.UNSIGNED_BYTE, IntBuffer.wrap(texture.pixels));
  }

  endPGL();
  shader.unbind(); // new
}

nabr

i think processings
shader.set() is waiting for a GL_TEXTURE_2D
and you pass TEXTURE_CUBE_MAP_POSITIVE_X to pgl.activeTexture(PGL.TEXTURE0 + unit);

and the shader gets confused

best if you would set the active Unit on
//2D texture
uniform sampler2D diffuseMap;

by yourself so its not mixed, i think the default location is 0
in #shader version >=330 you have layout( binding=TEXTUREUNITLOCATION )

so the first 6 locations are for a cubeMap sampler and the diffuseMap is on location 7
pgl.activeTexture(PGL.TEXTURE6 --> diffuseMap

nabr

when you reading this http://antongerdelan.net/opengl/cubemaps.html

// generate a cube-map texture to hold all the sides
glActiveTexture(GL_TEXTURE0);

GL_TEXTURE0 is holding the hole CUBEMAP

so the diffesemap has to be on location = 1

also:
https://www.khronos.org/opengl/wiki/Cubemap_Texture To allocate mutable storage for the 6 faces of a cubemap's mipmap chain, bind the texture to GL_TEXTURE_CUBE_MAP. Then call glTexImage2D 6 times, using the same size, mipmap level​, and internalformat​. The target​ parameter is not GL_TEXTURE_CUBE_MAP; instead, it specifies which of the 6 faces of the cubemap to specify. These faces are:

nabr

i cant repuduce the bug on my machine is hard to say, where the problem is

what you can also try

change the order of execution, becourse glsl is C the order of fuctioncalls matter

//2D texture will be on location = 0
uniform sampler2D diffuseMap;

//Cube texture location = +1 
uniform samplerCube cubemap;

set the cubemap on location 1
pgl.activeTexture(PGL.TEXTURE1); // Change texture unit

Stephcraft

Still the same error :/

nabr

yes

BUT what happens if you try to load a texture ? i still think the 2 is just an int

//Load Skybox Image
  cubeMap = loadImage("skybox.jpg");
  skybox = sliceCubeMap(cubeMap);

  //Load texture
  texture = loadImage("texture.png");

  //Cube Map
  glslCubeMap(2, cubeMap);
  // shader.set("cubemap", 2);
  shader.set("cubemap", cubeMap );

nabr

the problem here

https://github.com/processing/processing/blob/master/core/src/processing/opengl/PShader.java#L728

pgl.activeTexture(PGL.TEXTURE0 + unit);
pgl.bindTexture(PGL.TEXTURE_CUBE_MAP, envMapTextureID.get(0));

and later with diffuse it becomes

pgl.activeTexture(PGL.TEXTURE0 + unit); // unit++
pgl.bindTexture(PGL.TEXTURE_2D, someprocessing value);

Cannot validate shader program:Validation Failed: Sampler error: Sampler of different types use the same texture image unit. CUBE_MAP vs TEXTURE2D -or-...

or also possible
pgl.activeTexture(PGL.TEXTURE0 + unit);
pgl.bindTexture(PGL.TEXTURE_CUBE_MAP, envMapTextureID.get(0));
vs
shader.set("cubemap", 2 INT TYPE )

that would explain the error message

Stephcraft

The 2 is 100% not only an int, we are setting a samplerCube as an int, so in that case, Processing doesn't assign the samplerCube as an int, Processing uses the 2 as a location to assign the cube map texture to the uniform.

The cubemap works fine without using any sampler2D and if the unit in glslCubeMap is not the same than the shader.set("cubemap", /*number here*/) than the cube map will not show.

pgl.activeTexture(PGL.TEXTURE0 + unit); Could be the problem, but if our unit is bigger than the amount of samplers2D that Processing uses, than we should be fine, but we are not, and I dont understand why.

nabr

hm.

unit = 0
pgl.activeTexture(PGL.TEXTURE0 + unit) == TEXTURE0
unit = 1
pgl.activeTexture(PGL.TEXTURE0 + unit) == TEXTURE1
unit = 2
pgl.activeTexture(PGL.TEXTURE0 + unit) == TEXTURE2

glslCubeMap(2, cubeMap);

so where the 2 comes from ?

nabr

idk

their are 10 different shader.set methods

i think https://github.com/processing/processing/blob/master/core/src/processing/opengl/PShader.java#L724

this will be only executed if the second argument is a PImage.

nabr

pgl.activeTexture(PGL.TEXTURE0 + unit)

is pointing to the first time a texture is set. meas with
shader.set("diffuseMap", texture); == PGL.TEXTURE0

nabr

i was also wrong
in lower opengl/jogl shader versions you can not use NOT a power of two textures
so this won't work : ) https://learnopengl.com/img/advanced/cubemaps_skybox.png

Stephcraft

because we "slice" it into an array of 6 squares with : sliceCubeMap(cubeMap), it should work. But the image width/4 and the image height/3 must be the same

Stephcraft

Still not working with multiple different attemps,

I tried using Windows but I get the same error, only this time it specifies only : Cannot validate shader program:Validation Failed: Sampler error: Sampler of different types use the same texture image unit

Please help

Dawars

I am also interested in the solution, using a mac. I'll probably test this on Windows on Monday.

This looks to be the problem to me but I haven't gotten deep into it yet: https://opengl.org/discussion_boards/showthread.php/176366-Samplers-of-different-types-use-the-same-textur?p=1231274&viewfull=1#post1231274

ps: Seems to be an apple only issue

nabr

@Stephcraft @Dawars

an well known -and documented, also an widely used OpenGL Graphic feature

my recommandation is to open a processing issue https://github.com/processing/processing/issues

Dawars

I created the issue: https://github.com/processing/processing/issues/5507