Need help with sending p5.Image to Tesseract.js

MarcoHeleno

After correctly loading an image I get the below error message when trying to recognize text on an image with Tesseract.js.

tesseract.js:356 Uncaught DOMException: Failed to execute 'postMessage' on 'Worker': HTMLCanvasElement object could not be cloned. at https://cdn.rawgit.com/naptha/tesseract.js/1.0.10/dist/tesseract.js:356:25 at loadImage (https://cdn.rawgit.com/naptha/tesseract.js/1.0.10/dist/tesseract.js:415:16) at Object.sendPacket (https://cdn.rawgit.com/naptha/tesseract.js/1.0.10/dist/tesseract.js:354:5) at TesseractJob._send (https://cdn.rawgit.com/naptha/tesseract.js/1.0.10/dist/tesseract.js:549:21) at https://cdn.rawgit.com/naptha/tesseract.js/1.0.10/dist/tesseract.js:643:9 at Array. (https://cdn.rawgit.com/naptha/tesseract.js/1.0.10/dist/tesseract.js:673:5) at TesseractWorker._dequeue (https://cdn.rawgit.com/naptha/tesseract.js/1.0.10/dist/tesseract.js:683:19) at TesseractWorker._delay (https://cdn.rawgit.com/naptha/tesseract.js/1.0.10/dist/tesseract.js:675:32) at TesseractWorker.recognize (https://cdn.rawgit.com/naptha/tesseract.js/1.0.10/dist/tesseract.js:635:16) at recognizeFile (http://localhost:8080/:41:19) (anonymous) @ tesseract.js:356 loadImage @ tesseract.js:415 sendPacket @ tesseract.js:354 _send @ tesseract.js:549 (anonymous) @ tesseract.js:643 (anonymous) @ tesseract.js:673 _dequeue @ tesseract.js:683 _delay @ tesseract.js:675 recognize @ tesseract.js:635 recognizeFile @ (index):41 keyPressed @ (index):35 e._onkeydown @ p5.min.js:9

This is my code:

<title>Test</title>

<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.5.12/p5.min.js" type="text/javascript"></script>
<script src='https://cdn.rawgit.com/naptha/tesseract.js/1.0.10/dist/tesseract.js' type="text/javascript"></script>

<script>

  var img;

  function preload() 
  {
    // http://cheesiemack.com/wp/wp-content/uploads/2012/06/CT-872NED.jpg
    img = loadImage("licenseplate.jpg");
  }

  function setup() 
  {
    canvas = createCanvas (windowWidth, windowHeight);
  }

  function draw() 
  {
    background (255, 255, 0);

    image (img, 0, 0);
  }

  function keyPressed() 
  {
    recognizeFile (img);
  }


  function recognizeFile (img)
  {
    Tesseract.recognize (img, {lang: 'eng'})
    .then (function(data) {console.log (data.text)})
  }


  function windowResized() 
  {
    resizeCanvas (windowWidth, windowHeight);
  }

</script>

GoToLoop

• According to https://GitHub.com/naptha/tesseract.js#imagelike, Tesseract::recognize()'s myImage parameter is restricted to some very specific datatypes.
• However, you're passing to it a p5.Image object: https://p5js.org/reference/#/p5.Image
• Which was created via p5::loadImage() method: https://p5js.org/reference/#/p5/loadImage
• And as you may realize now, p5.Image isn't listed as 1 of the valid ImageLike datatypes.
• BtW, a p5.Image object is merely a wrapper for an HTMLCanvasElement: https://Developer.Mozilla.org/en-US/docs/Web/API/HTMLCanvasElement
• Which in turn represents a <canvas> tag element: https://Developer.Mozilla.org/en-US/docs/Web/HTML/Element/canvas
• And guess what, <canvas> is cited as 1 of the valid ImageLike datatypes! \m/
• But how can we access p5.Image's underlying <canvas>?
• Well, it isn't documented but, a p5.Image has a property called canvas, which is indeed of datatype HTMLCanvasElement! $-)
• Additionally, it's got another property named drawingContext of datatype CanvasRenderingContext2D:
  https://Developer.Mozilla.org/en-US/docs/Web/API/CanvasRenderingContext2D
• Which is another valid ImageLike datatype too! :)>-
• Being variable img of datatype p5.Image, you've got 2 options now:
• Pass either img.canvas or img.drawingContext to Tesseract::recognize() as its argument. :\">

GoToLoop

• I've refactored your sketch to use my approach. B-)
• And it's available online here: http://Bl.ocks.org/GoSubRoutine/241b4070e20a13f1c3dd8f852390baa0
• In order to ignite the OCR job over the license plate image, just click on it.
• You're gonna need to hit F12 to open browser's console tab in order to see the entire progress and the final results.
• And below's the file pair "index.html" & "sketch.js", so you can run the program locally as well:

index.html:

<script async src=http://CDN.JSDelivr.net/npm/p5></script>
<script defer src=http://CDN.JSDelivr.net/gh/naptha/tesseract.js/dist/tesseract.min.js></script>
<script defer src=sketch.js></script>

sketch.js:

/**
 * Tesseract License Plate OCR (v1.0.2)
 * MarcoHeleno & GoToLoop (2017-Feb-23)
 *
 * Forum.Processing.org/two/discussion/23878/
 * need-help-with-sending-p5-image-to-tesseract-js#Item_2
 *
 * Bl.ocks.org/GoSubRoutine/241b4070e20a13f1c3dd8f852390baa0
 */

"use strict";

const HTTP = 'http:' + '//',
      PROX = 'CORS-Anywhere.HerokuApp.com/',
      SITE = 'CheesieMack.com/',
      FOLD = 'wp/wp-content/uploads/2012/06/',
      FILE = 'CT-872NED.jpg',
      PATH = HTTP + PROX + SITE + FOLD + FILE,
      LOCAL = false;

let img;

function preload() {
  img = loadImage(LOCAL && FILE || PATH, console.info);
}

function setup() {
  createCanvas(img.width, img.height).mousePressed(() => ocr(img.canvas));
  background(img);
}

function ocr(imageLike) {
  Tesseract.recognize(imageLike).progress(print).then(logFoundWords);
}

function logFoundWords(json) {
  console.table(json.lines);
  print(json);
}

P.S.: I had to use http://CORS-Anywhere.HerokuApp.com in order to load the remote image "CT-872NED.jpg" from http://CheesieMack.com, b/c it wasn't CORS-enabled there! :-O

MarcoHeleno

@GotoLoop, thanks for taking the time to look into this and giving me a such detailed explanation! Much appreciated!