How to change the textures of a cubemapshader?

HBo

Hello. I created a cubemapshader with skybox textures as explained by TioPepe here:

https://forum.processing.org/two/discussion/7039/samplercube-and-ambient-reflection.

Now I would like to change the textures of the shader by using the images of a webcam. I've already tried to recreate the cubemapshader frame by frame with beginPGL() ..... endPGL(), but the time for the calculations is too long, so that's the wrong way. Does anybody know how a fast method to change the shadertextures?

Thanks :)

nabr

The Question is probaly already solved:

The following code works for me, i took it as personal challenge for learning processing,so it's a boilerplate code, works on PC, it has to be optimised, something to get started i'm too lazy at the moment to write any notes.

Here is a good tutorial https://processing.org/tutorials/pshader/

String[] vertSource={"#version 150"
  , "in vec4 position;"
  , "in vec2 texCoord;"
  , "uniform mat4 transform ;"
  , "out vec4 vertTexCoord;"
  , "void main() {"
  , "gl_Position =transform*position;"
  , "vertTexCoord.xy = texCoord;"
  , "}"
};
String[] fragSource={"#version 150"
  , "out vec4 fragColor;"
  , "uniform sampler2D textur;"
  , "uniform float time;"
  , "in vec4 vertTexCoord;"
  , "void main() {"
  , "fragColor =mix(texture(textur,vertTexCoord.xy),"
                    ,"vec4(1.,0.,0.,1),sin(time)*.5+.5);"
  , "}"
};
import processing.video.*;
Capture cam;
PShader shader;

void setup() {
  size(800, 600, P3D);
  cam = new Capture(this, 640, 360);
  cam.start();  
  shader=new PShader(this, vertSource, fragSource);
  shader(shader);
}
void draw() {
  float t = millis()*.001f;
  if (cam.available() == true) {
    cam.read();
    shader.set("textur", cam);
     }
    shader.set("time", t);

    translate(width/2, height/2);
    rotateY(t);
    rotateX(t);


    background(0);
    /**/
    scale(80);
    shape(createWebCube());
    //you dont need push(vertices) also the predef shape/cube will work
    /*
    noStroke();
    box(height/2);*/
}

PShape createWebCube() {
  textureMode(NORMAL);
  PShape sh = createShape();

  sh.beginShape(QUAD);
  sh.noStroke();
  sh.vertex(-1, -1, 1, 0, 0);
  sh.vertex( 1, -1, 1, 1, 0);
  sh.vertex( 1, 1, 1, 1, 1);
  sh.vertex(-1, 1, 1, 0, 1);

  sh.vertex( 1, -1, -1, 0, 0);
  sh.vertex(-1, -1, -1, 1, 0);
  sh.vertex(-1, 1, -1, 1, 1);
  sh.vertex( 1, 1, -1, 0, 1);

  sh.vertex(-1, 1, 1, 0, 0);
  sh.vertex( 1, 1, 1, 1, 0);
  sh.vertex( 1, 1, -1, 1, 1);
  sh.vertex(-1, 1, -1, 0, 1);

  sh.vertex(-1, -1, -1, 0, 0);
  sh.vertex( 1, -1, -1, 1, 0);
  sh.vertex( 1, -1, 1, 1, 1);
  sh.vertex(-1, -1, 1, 0, 1);

  sh.vertex( 1, -1, 1, 0, 0);
  sh.vertex( 1, -1, -1, 1, 0 );
  sh.vertex( 1, 1, -1, 1, 1);
  sh.vertex( 1, 1, 1, 0, 1);

  sh.vertex(-1, -1, -1, 0, 0);
  sh.vertex(-1, -1, 1, 1, 0);
  sh.vertex(-1, 1, 1, 1, 1);
  sh.vertex(-1, 1, -1, 0, 1);
  sh.endShape();
  return sh;
}

HBo

hi nabr,

thanks for your reply and your example. i've already had a short look at the shadertutorial but not enough time to dig deeper in the OPENGL-stuff

so far i've only modified the code mentioned in the thread above to update the cubemap in the draw() loop. currently i work with six cubemaptextures to create an environment which is then reflected by the loaded 3D geometry:

import java.nio.IntBuffer;

IntBuffer envMapTextureID = IntBuffer.allocate(1);

PShader cubemapShader;

PGL pgl;

PShape s;   

void setup(){

  //load 3d-Geomety
  s = loadShape("test.obj");    

  // create the OpenGL-based cubeMap
  pgl = beginPGL();
  pgl.genTextures(1, envMapTextureID);
  pgl.activeTexture(PGL.TEXTURE1);
  pgl.enable(PGL.TEXTURE_CUBE_MAP);  
  pgl.bindTexture(PGL.TEXTURE_CUBE_MAP, envMapTextureID.get(0));
  pgl.texParameteri(PGL.TEXTURE_CUBE_MAP, PGL.TEXTURE_WRAP_S, PGL.CLAMP_TO_EDGE);
  pgl.texParameteri(PGL.TEXTURE_CUBE_MAP, PGL.TEXTURE_WRAP_T, PGL.CLAMP_TO_EDGE);
  pgl.texParameteri(PGL.TEXTURE_CUBE_MAP, PGL.TEXTURE_WRAP_R, PGL.CLAMP_TO_EDGE);
  pgl.texParameteri(PGL.TEXTURE_CUBE_MAP, PGL.TEXTURE_MIN_FILTER, PGL.LINEAR);
  pgl.texParameteri(PGL.TEXTURE_CUBE_MAP, PGL.TEXTURE_MAG_FILTER, PGL.LINEAR);    

  //load in six textures

  // put the textures in the cubeMap
  for (int i=0; i<textures.length; i++) {
    int w = textures[i].width;
    int h = textures[i].height;
    textures[i].loadPixels();
    int[] pix = textures[i].pixels;
    pgl.texImage2D(PGL.TEXTURE_CUBE_MAP_POSITIVE_X + i, 0, PGL.RGBA, w, h, 0, PGL.RGBA, PGL.UNSIGNED_BYTE, java.nio.IntBuffer.wrap(pix));
  }

  endPGL();

  // Load cubemap shader
  cubemapShader = loadShader("cubemapfrag.glsl", "cubemapvert.glsl");
  cubemapShader.set("cubemap", 1);
}    

void draw() {  

  shader(cubemapShader);

  lights();
  noStroke();
  translate(width/2,height/2);

  shape(s,0,0);

  resetShader();        

  //update textures    

  //update OpenGL Cubemap
  pgl = beginPGL();  

  envMapTextureID = IntBuffer.allocate(1);

  pgl.activeTexture(PGL.TEXTURE1);
  pgl.enable(PGL.TEXTURE_CUBE_MAP);  
  pgl.bindTexture(PGL.TEXTURE_CUBE_MAP, envMapTextureID.get(0));
  pgl.texParameteri(PGL.TEXTURE_CUBE_MAP, PGL.TEXTURE_WRAP_S, PGL.CLAMP_TO_EDGE);
  pgl.texParameteri(PGL.TEXTURE_CUBE_MAP, PGL.TEXTURE_WRAP_T, PGL.CLAMP_TO_EDGE);
  pgl.texParameteri(PGL.TEXTURE_CUBE_MAP, PGL.TEXTURE_WRAP_R, PGL.CLAMP_TO_EDGE);
  pgl.texParameteri(PGL.TEXTURE_CUBE_MAP, PGL.TEXTURE_MIN_FILTER, PGL.LINEAR);
  pgl.texParameteri(PGL.TEXTURE_CUBE_MAP, PGL.TEXTURE_MAG_FILTER, PGL.LINEAR);

  for (int i=0; i<textures.length; i++) {
    int w = textures[i].width;
    int h = textures[i].height;
    textures[i].loadPixels();
    int[] pix = textures[i].pixels;
    pgl.texImage2D(PGL.TEXTURE_CUBE_MAP_POSITIVE_X + i, 0, PGL.RGBA, w, h, 0, PGL.RGBA, PGL.UNSIGNED_BYTE, java.nio.IntBuffer.wrap(pix));
  }

  endPGL();

}

cubemap vertexshader:

uniform mat4 transform;
uniform mat4 modelview;
uniform mat3 normalMatrix;

attribute vec4 vertex;
attribute vec3 normal;

varying vec3 reflectDir;

void main() {
  gl_Position = transform * vertex;

  vec3 ecNormal = normalize(normalMatrix * normal); // Vertex in eye coordinates
  vec3 ecVertex = vec3(modelview * vertex); // Normal vector in eye coordinates
  vec3 eyeDir = ecVertex.xyz;
  reflectDir = reflect(eyeDir, ecNormal);
}

cubemap fragmentshader:

uniform samplerCube cubemap;
varying vec3 reflectDir;

void main() {
  vec3 color = vec3(textureCube(cubemap, reflectDir));
  gl_FragColor = vec4(color, 1.0);      
}

is it possible to pass a frame of a 360 degree video to a shader? i'm looking for something like a spheremap shader instead of a cubemap shader...

nabr

@HBo I see some guys just "drag and drop" their textures on a sphere() and some do the hard math.

someone did already a lib for processing. i think you are more advanced user and can just put stuff together.

https://forum.processing.org/two/discussion/21471/p5video360-360videos-in-processing#latest

When I heard of it:

https://github.com/mrdoob/three.js/issues/1621

the latest comment, is work in progress shader: (the perspective should be fixed, when you are going for reflective obj) the bgShader works fine. just plug your video in.

https://forum.processing.org/two/discussion/22443/image-texture-mode-strange-mapping-360#latest

in any other case just filp the normals of a sphere and map your 360 texture to it, i try it with other engines. with a shader you just have more control.

@GoToLoop: HBo is the best answer. :)

HBo

thank you nabr, i'll give it a try. if it doesn't work, i'll meditate - i've always known (somehow) that i'm the best answer but stil have to find out