lot of nonsense "unexpected token" errors with old code that used to work in older versions

matteosis..

lot of nonsense "unexpected token" errors with old code that used to work in older versions

in Integration and Hardware • 2 years ago

Hi,

I have an old sketch that used to work perfectly on an older version of Processing (it was around the end of 2009 - beginning of 2010 when I last run it). At that time I used Windows. I ran that sketch a lot of times on different machines and platforms and it was pretty robuts.

Now I've opened it with Processing 1.2.1 on Ubuntu, and I get a lot of error messages such as:

line 13:14: unexpected token: boolean
line 13:14: unexpected token: float

etc, (a few more of them). They all reference lines of code such as:

static final boolean CALIBRATE=false, READCALIBRATION=false;

static final float VIRTUAL_FRAMERATE=8.0;
static final int NSENSORS=4;

and the "unexpected" tokens as you can see are not that unexpected.

After a lot of errors like these, at the end I get a Null Pointer Exception which seems to be caused by the preprocessor itself, since my code doesn't even compile so it never gets to run:

Uncaught exception type:class java.lang.NullPointerException
java.lang.NullPointerException
    at processing.app.antlr.PdeRecognizer.constructorCast(PdeRecognizer.java:1263)
    at processing.app.antlr.PdeRecognizer.primaryExpression(PdeRecognizer.java:1568)
    at processing.app.antlr.PdeRecognizer.postfixExpression(PdeRecognizer.java:7304)
    at processing.app.antlr.PdeRecognizer.unaryExpressionNotPlusMinus(PdeRecognizer.java:7267)
    at processing.app.antlr.PdeRecognizer.unaryExpression(PdeRecognizer.java:7125)
    at processing.app.antlr.PdeRecognizer.multiplicativeExpression(PdeRecognizer.java:6988)
....... (much much longer than this)

I guess there has been some change in processing that is not backward compatible and my old code is not valid any more, but the problem cannot be in the lines where the "unexpected token" error points to, as they are perfectly valid, so the error must be somewhere else and must be driving the preprocessor crazy.

As I said my code used to work perfectly on older versions, though I don't know exactly which version of processing it was where it used to work (it was 1.something).

Does anybody know of some big non-backward-compatible changes that have happened in the last year that may cause this kind of issue? Wher should I start looking at?

The structure of my code is like this:

- variable declarations such as those listed above
- void setup() { ... }
- void draw() { ... }
- other event handlers such as void serialEvent(Serial p) {...}
- A COUPLE OF CLASS DEFINITIONS

May it be because of the class definitions? Should I put them in external files? It wasn't necessary before....

Thanks in advance
m.

Replies(10)

PhiLho

Re: lot of nonsense "unexpected token" errors with old code that used to work in older versions

2 years ago

I suggest to read the release notes, perhaps you can get an information there...
For example, Java libraries like java.awt are no longer imported by default, they must be declared explicitly.
Processing now understands Java 5 syntax, but that's supposed to be more comprehensive, not more restrictive. But it means the Java parser has changed a lot.
Perhaps show the 15 first lines?

matteosis..

Re: Re: lot of nonsense "unexpected token" errors with old code that used to work in older versions

2 years ago

by the way thank you for your help :)

matteosis..

Re: lot of nonsense "unexpected token" errors with old code that used to work in older versions

2 years ago

Hi,

Yes I have read the change notes, can't find any clue.
It says static final can cause problems if you don't have a setup() method, because code is automatically wrapped inside such a method (and static final is not legal there); but it is not my case since I do have a setup() method.

Also says not to use final in methods, which is not my case either.

I moved all the classes to one file per class, that didn't help. Now I commented out everything until the problem disappears, then introduced one thing at a time, and now, I get:
- all the bogus "unexpected token" errors, and also:
- an error which is correct ("cannot find a class or type named Dna" which is ok because I have removed the file with the class definition) but when it shows that error it highights the wrong line of code in the wrong file: it highlights a line of code where there is no reference whatsoever to the Dna class and in a file where that class is not referenced at all.

So the situation seems to be complicated by the fact that the preprocessor and/or IDE environment are buggy and give error messages regarding a line of code while highlighting some other random line of code in some other random file.

Anyway, regarding the first 15 lines of code, here they are (sorry I didn't post them before):

import hypermedia.net.*;
import processing.net.*;
import processing.serial.*;
//import pitaru.sonia_v2_9.*;
import processing.opengl.*;
import java.util.StringTokenizer;
static final boolean CALIBRATE=false, READCALIBRATION=false;
static final float VIRTUAL_FRAMERATE=8.0;
static final int NSENSORS=4;
static final int LINE=0, POLYGONS=1, CIRCLES=2, SQUARES=3;
static final int MODE=SQUARES;
static final boolean VIEWRADIATION=false, USESERIAL=false, USESOUND=false, SENDSOUND=false, USESOUNDSENSORS=false;
static final boolean DEBUGCELLSLIST=false, DEBUGZONES=false, DEBUGINDICES=false, VERBOSEIMAGEMAPINIT=false;
static final boolean DEBUGWHILES=false, DEBUGCPU=true, DEBUGSERIAL=false, DEBUGCHANGEIMAGE=false, DEBUGSOUNDSENSORS=true;
static final boolean CELLTEST=false, TESTLINE=false, DEBUGSQUARE=false, SQUAREVERBOSE=false, DRAW1ONLY=false;
static final boolean MULTIREFLECT=false;
static final String INFOFILE="img2006.txt", DELIMITERS=" ,:;";
static final String CALIBRATIONFILE="calibration.txt";
static final int IMAGEMAP_DOWNSAMPLING_FACTOR=4;
static final int NVOICES=8, NSAMPLES=8;
String[] SOUNDFILENAMESPREFIX_BORN={"provasin", "provabuzz"};
String[] SOUNDFILENAMESPREFIX_DIE={"provasin", "provabuzz"};
static final String SOUNDIPADDRESS="localhost";
static final int SOUNDTCPPORT=3001;
//float LEN_AVG[]={70.0, 60.0}, LEN_VAR[]={0.4, 0.2};
float LEN_AVG[]={70.0, 70.0}, LEN_VAR[]={0.2, 0.2}; //2009
int NVERTS=4;
//static final float LEN_RADIUS_RATIO=1;
float GAMMA=2*PI/3, BETA=(PI-GAMMA)/2;
float[] LIFE_AVG={60.0,5.0}, LIFE_VAR={0.1, 0.5};
//float[] RANDOM_ROTATION_AMPL={1.0,0.0};//8
float[] RANDOM_ROTATION_AMPL={1.0,0.1}; //2009
float[] ROTATION_FORCE_RATE={0.15,0.07}; //2
//float[] MAX_REPRODUCTION_RATE={0.7,2.0}, REPRODUCTION_DECAY_START={1.0,0.0}, REPRODUCTION_DECAY_AGE={2.0,0.1}; //2.0, 0.2
float[] MAX_REPRODUCTION_RATE={2,2.0}, REPRODUCTION_DECAY_START={3.0,0.0}, REPRODUCTION_DECAY_AGE={2.0,0.1}; //2.0, 0.2
float[] REPRPROBDECWITHCHLDN={0.5,0.8};
float MAX_DEATH_RATE=0.1, DEATH_RATE_EXPONENT=2;
//float[] RADIATION_INCREASES_REPULSION={1.0,25.0};
float[] RADIATION_INCREASES_REPULSION={1.0,20.0}; //2009
float RADIATION_SHORTENS_LIFE=1.0, RADIATION_VULNERABILITY_AGE=20;
float[] RADIATION_INCREASES_REPRODUCTION={0,5};
float MAX_MUTATION_RATE=0.4, MUTATION_RATE_FUNC_EXPONENT=1.0, MUTATION_DECAY_AGE=10;
//float RADIATION_MANTAINS_MCS_ALIVE=0.5; //the lower, the more effective
float RADIATION_MANTAINS_MCS_ALIVE=0.8; //2009
float[] CALIB_FACTOR={1.0/256, 1.0/256, 1.0/256, 1.0/256, 1.0/256, 1.0/256, 1.0/256, 1.0/256};
float[] CALIB_OFFSET={0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0};
Serial serialport;
static final byte terminator='*';
static final int NBYTES=21;
float[] values={0,0,0,0,0,0,0,0};
boolean comp1=false, comp2=false;
static final int SS_NONE=0;
static final int SS_STARTED=1;
static final int SS_COMPLETE=2;
int readbytevalue=0;
int currentreadbyte=0;
int lastidx=-1;
int synchrostatus=SS_NONE;
PFont times;
UDP udpserver;
World theworld;
float realtime;
long newtime, previoustime;
int nframes;
Client soundclient;
boolean recording=false;
void setup() {
// everything commented out here
}
void draw() {
// everything commented out here
}

I think the error is not even in this file, since the errors had disappeared when I had commented everything out of other files and started to reappear when I reintroduced code that is in other files......

matteosis..

Re: Re: lot of nonsense "unexpected token" errors with old code that used to work in older versions

2 years ago

I mean, the error it complains about _is_ here (if I enter an empty line the error is detected one line later), but I suspect the _real_ cause of the error is somewhere else

matteosis..

Re: lot of nonsense "unexpected token" errors with old code that used to work in older versions

2 years ago

Now, with a lot of comment commented out, I've come to the absurd situation in which I get all (and only) the "unexpected token" errors, but the applet compiles and runs!!!! How can it be???

Here's the errors:

line 14:14: unexpected token: boolean
line 16:14: unexpected token: float
line 17:14: unexpected token: int
line 19:14: unexpected token: int
line 20:14: unexpected token: int
line 21:14: unexpected token: boolean
line 22:14: unexpected token: boolean
line 23:14: unexpected token: boolean
line 24:14: unexpected token: boolean
line 25:14: unexpected token: boolean
line 27:14: unexpected token: String
line 28:14: unexpected token: String
line 29:14: unexpected token: int
line 32:14: unexpected token: int

matteosis..

Re: lot of nonsense "unexpected token" errors with old code that used to work in older versions

2 years ago

I THINK I HAVE FOUND IT (still have to complete the process of verifying it)

It seems the cause is that in my class definitions I had for example:

// this is file World.pde
public class World {
}

instead of

class World {
}

By removing the public modifier the errors disappear (I still have to uncomment a lot of things).

Isn't the public modifier for a class legal?

Anyway, even if it is not, the way the error is treated definitely sucks (said with all respect)

PhiLho

Re: lot of nonsense "unexpected token" errors with old code that used to work in older versions

2 years ago

" the way the error is treated definitely sucks"
Don't forget that writing a compiler is hard, and reporting sensible errors is even harder! And Processing authors are not compiler writer (to my knowledge), they just added a thin wrapper to make Java easier for newbies. And as I wrote, they went from a well tested Java 2 parser to a Java 5 one, so mistakes and bugs are to be expected.

Now, the public declaration of classes in PDE files is legal, although no really useful within most sketches: don't forget that these are nested classes, so they have access to everything in the main class and reciprocally.
I just tried an old sketch with static global declaration and a class file in a separate PDE file: I made it public, and it ran without problem. So I guess you found an unfortunate combination of code that trips the preprocessor. Perhaps you can package and send your sketch (the original one) to the bug tracker so the authors can debug the issue.

matteosis..

Re: lot of nonsense "unexpected token" errors with old code that used to work in older versions

2 years ago

Thanks phi.lho for you reply.

I sometimes have a rather rude way of expressing myself, however the expression "sucks" was not meant as a criticism to the work of people who wrote the preprocessor but rather to the undesirable situation itself.

I will certainly package the sketch and send it to the bugtracker, but I'll try to reduce the sketch to a minimum that triggers the issue, so as to help narrowing down the problem.

thanks again
m.

rbrauer

Re: lot of nonsense "unexpected token" errors with old code that used to work in older versions

2 years ago

I don't think your global variables need to be static because they belong to PApplet which you will only have one instance of. I did a little test and got the same results of the original code using static final variables while removing static fixed the problem.

PhiLho

Re: lot of nonsense "unexpected token" errors with old code that used to work in older versions

2 years ago

" I don't think your global variables need to be static because they belong to PApplet which you will only have one instance of"
Sure, but that's still valid Java code, which is the point of the thread: it shouldn't crash the compiler!
Note that as I wrote, the 'public' declarations aren't needed either.

Top Reply