Loading large dataset from *.txt and string questions & problem //

Programming Questions

voskull

Loading large dataset from *.txt and string questions & problem //

in Programming Questions • 2 years ago

Hi there,

I'm trying to load a txt file with a million entry / lines, wondering if anyone as tried that before, and if it will be a problem ? or if there is better practices... ?

also i have started developing some simple code, but cain't seem to get past split()... problem with my string declaration or simple syntax... 'm a little lost...

thanks heaps in advance,,, sorry it might and probably is really obvious...

code below:

void setup() {

size(550,550);

}

// load text file in data folder inside the sketch folde

// String[] data = loadStrings("data.txt"); // load text file data into datatxt as string

void draw (){



String[] data = loadStrings("SmsValues_01.txt");

//String data = "1,2,3,4,5,6,7,4,5,6,7,7,7,8,3,3,34,6,5,45,6,45,3,46,4";

String[] smstxt = splitTokens (data); // breaks the datatxt string into smstxt[0]string for each values

int [] smstxtINT = int(smstxt);

// debug

println ("Raw"+ data + " ---- " + " Extracted ** " + smstxtINT [0] + " ** " + smstxtINT [1] + " ** " + smstxtINT [2] + " ** ");

}

background(0);

noStroke();

smooth();

For every value draw an ellipse

for (int i = 0; i < values.length; i++) {

ellipseMode(CENTER);

  fill(values[i]*2,126);

  ellipse(width/2,i*3,values[i],8);

}

data set (part of)

-12,-16,236

-15,-13,235

-17,-20,237

-11,-17,235

-13,-16,235

-13,-17,236

-13,-17,237

-13,-18,236

-15,-20,237

-15,-18,236

-13,-17,235

-15,-17,237

-15,-17,238

-13,-16,236

-13,-17,237

-15,-17,236

-15,-17,235

-16,-18,237

-15,-17,236

-15,-18,237

-12,-17,236

-13,-18,236

-13,-18,237

-13,-17,237

-15,-17,236

-15,-17,237

-13,-16,236

Replies(2)

PhiLho

Re: Loading large dataset from *.txt and string questions & problem //

2 years ago

Ouch, if you try and load a text file of a million of lines 30 times per second, since you have put the code in draw() without using a noLoop() directive, you might indeed have some issues...

Such code loading and processing data is better put in setup().
Beside, you don't tell us what " problem with my string declaration or simple syntax" you have. As I often say, reporting the exact error messages help us solving your problems, as often we cannot run the given code.
Well, I think I can guess the issue...
You try to split the data which is an array of strings, one per line. You must loop over the data, ie. the lines, and split each line one by one.

voskull

Re: Loading large dataset from *.txt and string questions & problem //

2 years ago

Thanks a lot for your input and the ouch !

The error message i get is :

The method splitTokens(string) in the type PApplet is not applicable for the arguments (String[])

I'll up-grade the code based on your suggestion and re-post it here, whether i'm successful or not..

thanks.

Top Reply