threads blocking draw loop or vice versa ? [background processes slowed]

AndrewS

Hi all, I have some processing code running on a remote system, so very difficult to diagnose. It has some problems (see end of this long post). At home the software works fine (of course).

The purpose of the code is to read a json file located on my home server every minute and write it to a local file that is then read by the main artwork processing sketch which drives a large inflatable sculpture. The value in the json file is live share market prices and the artwork is : https://andrewstyan.com/portfolio/life-support-system/

The code has a draw loop which simply draws a performance chart of various monitoring variables and some stats. This is emailed to me when requested (hourly as triggered by a flag in the json file). As far as I can tell there is no code in the draw loop that should take any time. The frameRate() is set to 2 as the draw loop does very little and does not need to run any faster.

Four threads do the work, each based on a while(true) { .. code ... delay() } loop:

1) internet quality checker:

• 3 times per minute (at 10, 30, 50 sec) check within a try/catch what is returned by: Process ping = Runtime.getRuntime().exec("ping " +" -c "+" 1 "+"www.google.com" );

• maintain an internetOK boolean

• maintain an error count for consecutive failures

• maintain a ping time

2) read the server file:

• at 20 seconds into the minute read the server json file if internetOK == true;

• maintain some monitor flags and variables - eg. loadFailed, loadTime

3) write the local data file

• at 40 seconds into the minute write the data read from the server file to the local data file (if the "read the server" thread has not completed it's task then the previous data will be written)

4) an error loop which monitors all the status variables and escalates compounding issues, such as bad internet quality or repeated read errors.

A log file is maintained (and emailed on request) that logs the 'write local data' event and errors that can be detected by various logic tests, such as a file load taking too long or escalating errors.

Two problems are occurring which are probably related.

1) Every 5 or 6 minutes the server file is not communicated through to the next sketch. This was much worse (every 2 minutes) until I realised the file was frequently taking 10 seconds to be read from the server (the wifi quality is quite poor in the gallery where the work is installed). The sketch should be able to deal with this and I reduced the problem by extending the time I allowed for reading a file.

2) Although the framerate of the draw loop is set at 2 fps it actually runs MUCH slower, frequently running less than 7 frames per minute rather than 120. I can tell this from the event log which includes a time stamp and frame count for each entry, and there is an entry about every minute.

Perhaps problem 1 is caused by problem 2 but I do not understand what is slowing the draw loop. I can probably recode the sketch to better handle the poor wifi, but still need to understand the slow frame rate as they may be related. It shouldn't be a computer power issue as the sketch for the main artwork is far more computer intensive but is operating correctly at the set rate of 17 fps.

Help please:

• Can anyone suggest what would be slowing the draw loop? It does very little and I understood that threads do not/cannot block the draw loop. Does the slow draw loop impact on the performance/timing of the threads or vice versa? Each thread uses the second() function to trigger its action, does this depend on the draw loop? I have made a simple sketch that confirms that none of these interactions should occur.

• Any suggestions for diagnosing this from afar? The local staff are not computer literate at all. I can add more sophisticated remote diagnostics to the sketch (especially around actual durations to read and save files) and get them to compile it locally but that may be a stretch.

• Any ideas for simulating the behaviour locally? What could I try to slow the draw loop like this ?

I can post code or segments here, but it is massive and for obvious reasons I can't reduce it down to the minimum size that causes the problem. The draw() loop is below. The booleans are set in the threads. The calls are to simple functions that shouldn't block.

The remote machine is a 2014 mac-mini, definitely not latest OS and probably not latest Java. Processing 3.3.6 (I think).

Thanks in advance, Andrew

    void draw() {
      background(50);
      fill(160);
      stroke(200);

      if (frameCount == 150)  // start the status writing thread once the system stabilises
      {
        thread("statusLoop");
        statusRecord = "starting status loop";
      }

      getTime();

      if(hour()==0 && minute()==0 && second()<2) resetChart();

      // end of draw

      // internet
      img.beginDraw();
      if (internetOK) fl = green;
      else fl = red;
      if (inChecking) fl = blue;
      stroke(fl);
      fill(fl);
      rect(10, 10, 10, 10);
      float pt = map(pingTime, 0, 300, 75, 0);
      img.fill(fl);
      img.stroke(fl);
      if (!inChecking) img.line(scrapeMins, pt, scrapeMins+3, pt);

      if (internetFailed) fl = red;
      else fl = green;
      if (inChecking) fl = blue;
      stroke(fl);
      fill(fl);
      rect(22, 10, 10, 10); 
      img.fill(fl);
      img.stroke(fl);
      pt = map(timeDiff, 0, 270000, 150, 75);
      if (!loadingPackage) img.ellipse(scrapeMins, pt, 2, 2);

      img.endDraw();
      image(img, 0, 30);
      stroke(170);
      line(scrapeMins, 30+50, scrapeMins, 30+100);

      fill(150);
      text(pingTime, 50, 20);
      text("Fails: "+internetFails, 150, 20);

      // time
      text(scrapeTimeHMS(now), 400, 20);

      // reader
      if (!loadFailed) fill(0, 255, 0);
      else fill(255, 0, 0);
      if (loadingPackage) fill(0, 0, 255);
      rect(10, 200, 10, 10);
      fill(150);
      text("Load time: "+nf(loadTime, 2, 0)+"   Time stamp difference: "+timeDiff, 50, 200);
      text(packageTxt, 50, 220);

      text(statusRecord, 50, 300);

      text(getRowSummary(), 50, 350);

      reportLoop();
    }

Chrisir

  if(hour()==0 && minute()==0 && second()<2) resetChart();  

fhis seems suspicious; it's not some time that has passed but the time your clock shows like 22:02.33

instead use millis() and a timer based on it

koogs

Four threads do the work, each based on a while(true) { .. code ... delay() } loop:

I'd watch out for delay(), from the reference it's not obvious that it only stops the thread it's in.

koogs

Put some print statements in there, see where the holdup is.

AndrewS

@chrisir that bit of code SHOULD only run at midnight and just clears the monitor chart

@koogs I just ran some test code with 3 threads, each with lengthy delays and they didn't seem to affect the draw loop at all. As far as print statements go that is where I am headed but will have to do some smart event logging equivalents as the device is on the other side of the country so I can't watch things happen real time.

Chrisir

I see. my bad.

AndrewS

@chrisir now worries. every comment is a good comment :)

AndrewS

@koogs & @chrisir I discovered the problem. The program was running behind another program, so the OS was treating it as a background process and throttling it's speed dramatically. There may be other solutions but I have decided to combine the two programs into one. Makes life more difficult for later modifications due to the way the programs interact, but I couldn't see an easy java or processing way of avoiding the throttling. Thanks for your help.