Read text files from directory into an array of arrays

typ0

Hi All--

I'm trying to write some code that will scan a directory containing an unknown number of txt files, create an array for each txt file and then an array of these arrays. Any ideas on how to do this without being able to declare the size of the arrays? The number of txt files in the directory and the number of lines in each txt file are variables?

I think the File class will do what I need for scanning the directory but I'm having trouble using loadStrings() and a two dimensional String array since the lengths are variables.

GoToLoop

I believe my answer in this recent post is a start:
http://forum.processing.org/two/discussion/10183/about-data-base#latest

Write your own attempt and we'll try to fix it! ;)

typ0

The following reads a directory and populates the filenames string with the txt file names:

import java.io.FilenameFilter;
static final FilenameFilter TXTFILES = new FilenameFilter() {
  boolean accept(File f, String s) {
    return s.endsWith(".txt");
  }
};

void setup() {
  File dir = dataFile("C:/Users/XXX/Desktop/Files");
  String[] filenames = dir.list(TXTFILES);
  int len = filenames.length;

  println("Found", len, "object model(s) in folder:\n" + dir + '\n');
  printArray(filenames);
}

Now I want to create a two dimensional array files[][] and populate one dimension with the filenames and the other with the lines from the text files obtained by using loadStrings(). I tried the following:

import java.io.FilenameFilter;
static final FilenameFilter TXTFILES = new FilenameFilter() {
  boolean accept(File f, String s) {
    return s.endsWith(".txt");
  }
};

String[][] files;

void setup() {
  File dir = dataFile("C:/Users/XXX/Desktop/Files");
  String[] filenames = dir.list(TXTFILES);
  int len = filenames.length;

  println("Found", len, "object model(s) in folder:\n" + dir + '\n');
  printArray(filenames);

  for (int x = 0; x < len; x ++ ) {
  files[x][] = loadStrings("C:/Users/XXX/Desktop/Files/" + filenames[x]);
 } 
  printArray(files);
}

I can't iterate through one dimension of the array while populating the other dimension with loadStrings(). Any ideas?

GoToLoop

Let's try to fix it by parts: ~:>

• I didn't expect that undocumented function dataFile() would work on absolute paths.
• It's supposed to point to a File inside sketch's "/data/" subfolder only.
• And not some arbitrary 1 from the computer! 8-}
• Even though dataFile() worked, for the sake of clarity, let's replace it w/ a more formal File instantiation in order to get an arbitrary folder path:

http://docs.oracle.com/javase/8/docs/api/java/io/File.html#File-java.lang.String-
File dir = new File("C:/Users/XXX/Desktop/Files");

P.S.: Yet it's good practice placing files used by a sketch inside its "/data/" subfolder!

• Now we've got another problem: method list() gets filenames, but w/o their paths!
• If you were using "/data/" subfolder, that wouldn't be a problem.
• But the full path is needed for an arbitrary file located somewhere else!
• In order to fix that, let's replace list() w/ listFiles(), which returns a File[] instead, preserving the whole absolute path:

http://docs.oracle.com/javase/8/docs/api/java/io/File.html#listFiles-java.io.FileFilter-
File[] filenames = dir.listFiles(TXTFILES);

• Before accessing an array, we gotta instantiate it 1st!
• In the original example, that happened this way: models = new OBJModel[len];
• But seems like you forgot about doing the same to your String[][] files!
• The 1st outer dimension represents the future loadStrings() files.
• While the 2nd inner dimension is each line obtained from a loadStrings().
• Although we can't know how many String lines we're gonna get from each loadStrings(),
  the # of necessary loadStrings() is already known at this point.
• It's the filenames.length we've got from listFiles() of course: :>

files = new String[len][];

Here's what I've come up w/ after modifying your latest attempt: O:-)

// forum.processing.org/two/discussion/10216/
// read-text-files-from-directory-into-an-array-of-arrays

import java.io.FilenameFilter;
static final FilenameFilter TXTFILES = new FilenameFilter() {
  @ Override boolean accept(File f, String s) {
    return s.endsWith(".txt");
  }
};

String[][] txts;

void setup() {
  File dir = new File("C:/Users/XXX/Desktop/Files");
  File[] filenames = dir.listFiles(TXTFILES);
  int len = filenames.length;

  println("Found", len, "text file(s) in folder:\n" + dir + '\n');
  printArray(filenames);
  println();

  txts = new String[len][];
  for (int i = 0; i != len; txts[i] = loadStrings(filenames[i++]));

  printArray(txts[0]);
  exit();
}

typ0

Thanks GoToLoop for all the detail. I tried your code and it works perfectly. That helps A LOT!!!

typ0

One question though....why do you use "!=" in your for loop instead of "<". Is it just a style choice or is there a reason why it is necessary?

GoToLoop

Style choice and a "dangerous" 1! Only for increments or decrements by 1! :-\"