› ›

Troubles with the p5.Image.pixels "array"

pixelformpixelform
edited April 2015 in p5.js Programming Questions

I can't get array operations to work properly with instances of the p5.Image.pixels array.

I had been working with image buffers by first loading a source image into a p5.Image, then setting up a buffer, and finally a display as p5.Image objects - why is not important as I'm sure I could be doing this more effectively with p5.Graphics objects.

In copying the p5.Image.pixels arrays between each other, I had no problems using for loops, but when I tried to use the arrayCopy() method, I encountered an error.

here's the code that breaks: `

var imgSrc; var imgDest;

    function preload(){
        imgSrc = loadImage("https://upload.wikimedia.org/wikipedia/commons/5/54/Landscape_of_Shadegan.jpg");
    }

    function setup() {
        imgDest = createImage(imgSrc.width,imgSrc.height);
        createCanvas(imgSrc.width,imgSrc.height);
        noLoop();


    }

    function draw() {

        imgSrc.loadPixels();
        imgDest.loadPixels();

        arrayCopy(imgSrc.pixels, 0, imgDest.pixels, 0, imgSrc.pixels.length);

        imgSrc.updatePixels();
        imgDest.updatePixels();

        image(imgDest,0,0);

    }`

and here's the console error: Uncaught TypeError: src.slice is not a function

which occurs in line 2244 of the p5.js code (p5.js v0.4.4 April 12, 2015) p5.prototype.arrayCopy = function (src, srcPosition, dst, dstPosition, length) { var start, end; if (typeof length !== 'undefined') { end = Math.min(length, src.length); start = dstPosition; src = src.slice(srcPosition, end + srcPosition); // <----- line 2244 } else { if (typeof dst !== 'undefined') { end = dst; end = Math.min(end, src.length); } else { end = src.length; } start = 0; dst = srcPosition; src = src.slice(0, end); } Array.prototype.splice.apply(dst, [ start, end ].concat(src)); };

Turns out that I can't even use the sort function on a p5.Image.pixels array. Calling .sort() on an ordinary array, say var foo = [9,2,4,0,1,3,5,7,8,6]; foo.sort(); gives me [0,1,2,3,4,5,6,7,8,9] no problem, but if I call .sort() on a p5.Image.pixels array say imgSrc.pixels, I get an error saying " imgSrc.pixels.sort is not a function."

Anyone know what's going on here? is there someother way I could be more efficiently copying pixels arrays for either Image or Graphics objects?

Tagged:

Best Answer

Answers

  • GoToLoopGoToLoop
    edited April 2015 Answer ✓

    p5.Color, and thus p5.Image, doesn't use a regular array for storage.
    Instead it relies on a not-so-well-known typed array container: >-)
    https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray#Syntax

    More specifically an Uint8ClampedArray: :-B
    https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Uint8ClampedArray

    Typed arrays don't have everything that a regular array offers. For example the slice() method! :-&
    Neither can be dynamically resized; it's just like a Java array!

    But I believe most of the problems shows up b/c the "array_functions.js" module wasn't coded w/ typed arrays in mind.
    I daresay they don't care about typed arrays at all, possibly thinking no regular p5*js coder would ever use them!

    But as you just found out now, the fact that p5.Color objects rely on Uint8ClampedArray, the whole "array_functions.js" module should be revisited:
    https://github.com/processing/p5.js/blob/master/src/data/array_functions.js

    Newest included method shuffle() there has some typed array compatibility b/c it was me who wrote the implementation btW! O:-)

    Perhaps I'll see if there's a way to include typed array compatibility for arrayCopy(), sort() and others, just like I did for shuffle()... (:|

    P.S.: Typed arrays got slice() (and many others) method now.
    However, only in Firefox 38 and no 1 else! 3:-O

    Who knows your sketch might end up working in most recent Firefox versions? ... ^#(^
    I recommend Firefox Development Edition btW: :D
    https://www.mozilla.org/en-US/firefox/developer/

  • PhiLhoPhiLho

    "p5.Color, and thus p5.Image, [...] relies on a not-so-well-known typed array container"
    Really? Where have you seen that? (you underscore names which are not links, which is confusing...)

    Looks like an odd design choice, as it limits operations, as you point out, and it excludes Internet Explorer versions below 11, which is very restrictive!

    Of course, it is very memory efficient, but the cost in usability is high.

  • GoToLoopGoToLoop
    edited April 2015

    ... and it excludes Internet Explorer versions below 11, which is very restrictive!

    Typed arrays are ECMA 5, so browsers which support HTML 5 should have support for it as well.
    According to the typed array's reference link, basic support for it is available since IE10.

    Looks like an odd design choice, as it limits operations, ...

    Exotic maybe, but it doesn't limit anything which JS language can already deal with!
    Like I've mentioned, merely lack of vision on the p5*js's "array_functions.js" module for not taking typed array support for consideration! :-q

    ECMA 6 specification will just make typed array's available methods match of that of regular arrays! [-O<
    And Firefox is almost done w/ it. Pity the other browsers are such lazy slowpokes! [-(

  • PhiLhoPhiLho

    You mention specifically Uint8ClampedArray which is listed as supported by IE11 only.
    Still, I haven't found reference to this type in my quick search on the only GitHub repository, and cloning it takes forever! We should be able to do a shallow clone, we are not necessarily interested by the whole history of the project!

  • pixelformpixelform

    GoToLoop, Thanks for the thorough explanation. Love Firefox, but I need maximum browser compatibility.

    I'm not quite sure how best to proceed though. I'm not advanced enough to tamper with the p5.js modules to build in what I need.

    If I want to use the array methods to process pixel values, it seems like I have to copy the p5.Image.pixels array over to an untyped array using a for loop, apply the methods I want, then copy them back using another for loop. Seems really laborious. Are there better ways to do this?

    I wonder why is p5.Color using a typed array anyways? What's the advantage? I suspect it's memory as PhiLo pointed out, but then a single pixel value is stored as four values instead of one. Processing uses a 32bit int type for color. Why not p5.js?

  • GoToLoopGoToLoop
    edited April 2015

    Indeed, Uint8ClampedArray is no where to be found there but in some comments inside: https://github.com/processing/p5.js/blob/master/src/image/filters.js

    Seems like Uint8ClampedArray was not a conscious choice but inherited from browser's DOM.
    And after frenetically perusing the framework, just found out that each single p5.Image is actually a whole fatty canvas element: @-)
    https://github.com/processing/p5.js/blob/master/src/objects/p5.Image.js#L54

    This simple sample proves that pixels property doesn't hold a regular array but rather an Uint8ClampedArray container: :-B

    const img = createImage(2, 2);

img.loadPixels();
print(img.pixels);
print(img.pixels instanceof Uint8ClampedArray);
print('\n');

const pix = img.drawingContext.getImageData(0, 0, img.width, img.height).data;
print(pix);
print(pix instanceof Uint8ClampedArray);
print('\n');

print(ArrayBuffer.isView(img.pixels)); // typed array
print(ArrayBuffer.isView(pix));        // typed array
print(ArrayBuffer.isView([]));         // regular array
  • pixelformpixelform

    Thanks for hunting this down. I'm not exactly sure what this all means in terms of answering the application questions from my last post.

  • GoToLoopGoToLoop
    edited June 2015

    P.S.: If you don't have "p5.js" nor "p5.min.js", replace this line below:
    <script src="p5.min.js" defer></script>
    with this 1 just here: :D
    <script src="http://cdnjs.cloudflare.com/ajax/libs/p5.js/0.4.4/p5.min.js" defer></script>

    "index.html":

    <!DOCTYPE html>
<head>

  <meta charset="utf-8">
  <title>ArrayCopy</title>

  <script src="p5.min.js" defer></script>
  <script src="sketch.js" defer></script>

</head>

<body>

  <h1>☺ ArrayCopy Fix ☻</h1>
  <h2>by GoToLoop (2015/Apr/24)</h2>
  <h3>forum.processing.org/two/discussion/10485/
  <br>troubles-with-the-p5-image-pixels-array</h3>

</body>

    "sketch.js":

    p5.prototype.arrayCopy = function (src) {
  const args = arguments.length;

  var sIdx = 0, dst = arguments[1], dIdx = 0,
      len  = args == 3? ~~Math.abs(arguments[2]) : src.length;

  if (args > 3) {
    sIdx = ~~Math.abs(dst);
    dst  = arguments[2];
    dIdx = ~~Math.abs(arguments[3]);
    len  = args > 4? ~~Math.abs(arguments[4]) : len;
  }

  const sLen = src.length, dLen = dst.length,
        end  = Math.min(len + sIdx, sLen);

  if (!sIdx && sLen <= len && sLen + dIdx <= dLen && ArrayBuffer.isView(dst))
    dst.set(src, dIdx);
  else
    for (var i = sIdx, j = dIdx; i < end & j < dLen; dst[j++] = src[i++]);

  return dst;
};

var img, im1, im2;

function setup() {
  createCanvas(600, 400);
  background(0);
  noLoop();

  img = createGraphics(width/3 | 0, height>>1); // half height
  im1 = createGraphics(width/3 | 0, height);    // full height
  im2 = createGraphics(width/3 | 0, height);    // full height
}

function draw() {
  img.background(0xFF, 0, 0), img.loadPixels(); // all red
  im1.background(0, 0, 0xFF), im1.loadPixels(); // all blue
  im2.background(0, 0x80, 0), im2.loadPixels(); // all green

  image(img, 0, height>>2); // display red block

  arrayCopy(img.pixels, im1.pixels, im1.pixels.length>>2);    // 3 parameters
  im1.updatePixels();
  image(im1, width/3); // display blue block

  arrayCopy(img.pixels, 0, im2.pixels, im2.pixels.length>>1); // 4 parameters
  im2.updatePixels();
  image(im2, width - width/3); // display green block
}
  • PhiLhoPhiLho

    I just made the same finding (Filters._toPixels). Actually, it is documented as Uint8ClampedArray but it might be something else in IE (or even in other browsers). For example, I found a reference in the MSDN site: https://msdn.microsoft.com/en-us/library/ie/ff974957(v=vs.85).aspx They don't mention a specific type, but the data can be manipulated with bit mask / shifting.

Sign In or Register to comment.