" without using Java libraries"
Uh? What do you want to use, then?
I was about to tell you to turn to java.io.* package, if you want a finer control of your loading.
Beside, how many megabytes do you plan to load such way? Because reading a file of 1 or 2 MB probably will not leave the time to display some progress feedback to the user.

As I stated, I want Processing to have proper file abilities. Processing should be able to load any portion of a file, a few bytes, integers, and reals at a time. You know, read and write data, like other programing languages?

Amongst other things I would like to build file headers, or something as simple as:

1. Int count, data;
2. data = 12; // Whatever, blah, blah...
   
3. myFile = New Openfile[];
4. for (count = 0 ; count < 128 ; count++)
5. {
6.     myfile.Read (data);
7.     myfile.Write (data);
8. }
9. myfile.Close;

I have not understood it was a wish list for Processing evolution.
Basically, why not, but you know, Java is not "another language". Or, rather, Processing isn't a "real" language, it is Java with some little syntax sugar above it to simplify its usage. And some functions to ease usage of some of its libraries. So if you use a Java library, like java.io.*, you are not "switching to another language", you just use the immense capabilities of this language, at the cost, indeed, of some added complexity.

So, of course, one can improve Processing with easy to use, more advanced file handling that it currently have, at least the often asked capability to append data to files, but it is unlikely to come (at least in the short term) as the Processing creators concentrate more on different stuff (like getting out a stable 2.0 version...).
This could go in a library, too.

But Processing already has createWriter(), createOutput(), and the equivalent readers.


About the buggy board: yes, it is... And currently, it is very slow and I often see threads marked as new with lot of new topics, but I see much less, actually. Eg. a topic listed with 14 answers and showing only 2. Puzzling.
For example:
https://forum.processing.org/topic/gif-lib-position
I see two answers, finishing with mine, 5 days ago, while the forum displays:

Very annoying.
People can delete their messages, so can moderators, but I doubt that many messages were suppressed here!

Hi Moderator

I added a post here that does not show, yet it shows in the index. You have not moderated me yet, have you?

I am asking because I have seen a post reappear, as well.
http://forum.processing.org/topic/this-board-has-issues

Thanks,
BEM

No, actually I even answered in this thread, and reported the issue at Zoho.
Looks they modified a bit the forum, and hopefully the issues are gone. Are they?

Hi,

Is not But createWriter(), createOutput(), for text?

I still want to save integers, reals, etc.

Yes for createWriter(), no for createOutput(), which returns an OutputStream, allowing to write bytes if you want.

See also http://stackoverflow.com/questions/13071777/convert-double-to-byte-array and similar.
Or use Java...

I appreciate the effort and reply, but half-of a solution is not a solution.

I am sorry, but the file handling is weak. Not being able to save other data types is quite a deficiency in Processing as a language. I implore the language architects to please add the rest of the functionality to load and save data.

Reading and saving files isn't indeed the strongest point of Processing, concentrating more on graphics than on this aspect.
Although the recent betas of 2.0 added a data package (not yet documented, AFAIK) improving the XML handling and adding a Json support, along with improved CSV support.
Modern usage for self-use (read data you wrote) is to use textual file formats, easier to parse, although a bit more verbose (but they can be compressed).

If you need to write binary data, eg. to match an existing file format, you can use saveBytes() after building the corresponding byte array, or use the DataOutputStream as I suggested.
It is quite easy to use (despite the need to add try / catch blocks hidden in other parts of Processing), so there is little point to add a whole set of functions to mimic its usage.
Example:

1. import java.io.*;
2.  
3. String fileName = "H:/Temp/SomeFile.dat";
4.  
5. void setup()
6. {
7.   OutputStream os = createOutput(fileName);
8.   DataOutputStream dos = new DataOutputStream(os);
9.   try
10.   {
11.   dos.writeInt(42);
12.   dos.writeFloat(3.14159);
13.   dos.writeBoolean(true);
14.   dos.close();
15.   }
16.   catch (IOException e) { println("Error: " + e); }
17.  
18.   println("Done");
19.  
20.   InputStream is = createInput(fileName);
21.   DataInputStream dis = new DataInputStream(is);
22.   try
23.   {
24.   println(dis.readInt());
25.   println(dis.readFloat());
26.   println(dis.readBoolean());
27.   }
28.   catch (IOException e) { println("Error: " + e); }
29.  
30.   exit();
31. }

Note that I only express my opinion here, it doesn't invalidate or reject your proposal, I only give hints why it might be ignored. That, and the fact the authors might just don't see this thread: for official requests, you have to open an issue on GitHub... Here you will see an official answer (rejected / to do, soon or in a far future, etc.).

Hi,

It is not practical to use XML or text data for my purpose; I will be working with non-textual binary data, and quite a bit of it. I also need random file access.

I appreciate the effort. I will try to adapt the code you, but I have had series usability problems with Processing's other file IO, such as the file chooser dumping you right back into the start of draw. I hope that has been fixed, but I am pessimistic.

I feel that processing strength are that it allows me to code at a high enough level that I can code quickly. Obviously, its graphics routines are rich. Processing has two weaknesses: File I/O and sound. When those two are addressed, it will make an excellent graphics-centric hardware abstracted programming language.

To form a file header, I need to save a string. Are these calls documented somewhere?

Is it possible to have more than one file open at a time?

" Are these calls documented somewhere?"
Not sure what are "these calls"...
You have:
http://processing.org/reference/
http://processing.org/reference/javadoc/core/
http://docs.oracle.com/javase/6/docs/api/
among other things.

And yes to the second question.

Thank you.

I seem to have it working with one file. I suppose I will have to turn it into a class to use more than one, such as the conversion program I will need to even load the data, to begin programming its visualization, but I can save out the header, at least.

1. import java.io.*;
2. // Convert File
3. int count;
4. // File Header - 56 Bytes
5. String fileName = "converted.tmv";
6. char filemeta_name[] = {'T', 'e', 'm', 'v', 'i', 'e', 'w'}; // This appears to be saved as Unicode.
7. byte filemeta_version = 1; // Is this an oldfile, newfile?
8. int filemeta_layer = 0; // Just in case we want to compare data
9. int filemeta_sizex = 1024; // How big?
10. int filemeta_sizey = 1024; // How big?
11. int filemeta_locationx = 0; // Upper corner of data
12. int filemeta_locattony = 0;  // Upper corner of data
13. int filemeta_lookatx = 0;  // Are we looking at center, or where?
14. int filemeta_lookaty = 0;  // Are we looking at center, or where?
15. int filemeta_visualizer = 0; // Last used visualization mode
16. long filemeta_crc = 0xffffff; // Reserved
17. boolean lock = false; // Reserved
18. void setup()
19. {
20.   OutputStream os = createOutput(fileName);
21.   DataOutputStream dos = new DataOutputStream(os);
22.   try
23.   {
24.     for (count = 0 ; count < 7 ; count++)
25.     {
26.       dos.writeChar (filemeta_name [ count] );
27.     }
28.    
29.     dos.writeByte(filemeta_version);
30.     dos.writeInt (filemeta_layer);
31.     dos.writeInt (filemeta_sizex);
32.     dos.writeInt (filemeta_sizey);
33.     dos.writeInt (filemeta_locationx);
34.     dos.writeInt (filemeta_locattony);
35.     dos.writeInt (filemeta_lookatx);
36.     dos.writeInt (filemeta_lookatx);
37.     dos.writeInt (filemeta_visualizer);
38.     dos.writeLong (filemeta_crc); // Reserved
39.     dos.writeBoolean (lock); // Reserved
40.     dos.close();
41.   }
42.   catch (IOException e) {
43.   println("Error: " + e);
44.   }
45.   println("File Written");
46.   exit();
47. }

For my project, I wrote a sample file converter, with both input and output. At this point, the goal is just to test my understanding of the file IO, and have some sample data, to drag, zoom, and do visualizer modes on, for my project.

The file converter is supposed to create a file, write a sample header, and in this case, copy RAW data, 16 bits at a time. (The source file appears to have the correct amount of bytes, as well. It is a 16 bit 2048x2048 grayscale raw file)

Oddly, my sketch seems to create the file, add the header, and copy the data, BUT, strangely, it does it in half the lines it is supposed to, as if it is copying data 4 integer bytes at a time, instead of 2.

The result is an apparently good file, half the reads, ending with Error: java.io.EOFException

Perhaps it will be a simple forehead-smaker, I will see tomorrow, but....

1. import java.io.*;
2. // Convert File
3. int count, x, y, data;
4. // Filenames
5. String filename_input = "input.raw";
6. String filename_output = "output.tmv";
7. // File Header - 56 Bytes
8. char filemeta_name[] = {  'T', 'e', 'm', 'v', 'i', 'e', 'w'}; // There appear to be saved as Unicode.
9. byte filemeta_version = 1; // Is this an oldfile, newfile?
10. int filemeta_layer = 0; // Just in case we want to compare data
11. int filemeta_sizex = 2048; // How big?
12. int filemeta_sizey = 2048; // How big?
13. int filemeta_locationx = 0; // Upper corner of data
14. int filemeta_locattony = 0;  // Upper corner of data
15. int filemeta_lookatx = 0;  // Are we looking at center, or where?
16. int filemeta_lookaty = 0;  // Are we looking at center, or where?
17. int filemeta_visualizer = 0; // Last used visualization mode
18. long filemeta_crc = 0xffffff; // Reserved
19. boolean lock = false; // Reserved
20. void setup()
21. {
22.   // Open Source
23.   InputStream is = createInput(filename_input);
24.   DataInputStream source = new DataInputStream(is);
25.   // Create Destination
26.   OutputStream os = createOutput(filename_output);
27.   DataOutputStream destination = new DataOutputStream(os);
28.   // Write Header
29.   try
30.   {
31.     for (count = 0 ; count < 7 ; count++)
32.     {
33.       destination.writeChar (filemeta_name [ count] );
34.     }
35.     destination.writeByte(filemeta_version);
36.     destination.writeInt (filemeta_layer);
37.     destination.writeInt (filemeta_sizex);
38.     destination.writeInt (filemeta_sizey);
39.     destination.writeInt (filemeta_locationx);
40.     destination.writeInt (filemeta_locattony);
41.     destination.writeInt (filemeta_lookatx);
42.     destination.writeInt (filemeta_lookatx);
43.     destination.writeInt (filemeta_visualizer);
44.     destination.writeLong (filemeta_crc); // Reserved, for perhaps someday
45.     destination.writeBoolean (lock); // Reserved, as it might also help determine good file write
46.   }
47.   catch (IOException e)
48.   {
49.     println("Error: " + e);
50.   }
51.   // Copy Data
52.   try
53.   {
54.     for (y = 0 ; y < 2047 ; y++)
55.     {
56.       print("Row:");
57.       println(y);
58.       for (x = 0 ; x < 2047 ; x++)
59.       {
60.         data = source.readInt ();
61.         destination.writeInt (data);
62.       }
63.     }
64.     source.close();
65.     destination.close();   
66.     println("File Converted");
67.     exit();
68.   }
69.   catch (IOException e)
70.   {
71.     println("Error: " + e);
72.   }
73. }

Grr! I mean, ah-hah. That was the problem; they needed to be writeShort's

http://docs.oracle.com/javase/6/docs/api/java/io/RandomAccessFile.html#readInt%28%29

Unrelated to your questions, since you seem to have solved them yourself (good job!), but re-reading my code, I saw I was sloppy in I/O handling. The kind of code seen everywhere on the Internet, bad examples. OK for such small program, quickly made to illustrate a point, and where everything is closed at the end of the sketch, but not acceptable in a larger program that can run for a while and this can leak resources.
I show here a better version, leaving the bad one so that people can compare...

1. import java.io.*;
2.  
3. String fileName = "H:/Temp/SomeFile.dat";
4.  
5. void setup()
6. {
7.   OutputStream os = createOutput(fileName);
8.   DataOutputStream dos = new DataOutputStream(os);
9.   try
10.   {
11.     dos.writeInt(42);
12.     dos.writeFloat(3.14159);
13.     dos.writeBoolean(true);
14.   }
15.   catch (IOException e) { println("Error: " + e); } // A real program would do a real exception handling... (log, recovery, rethrow, etc.)
16.   finally // Close the stream even if an exception happened
17.   {
18.     if (dos != null)
19.     {
20.       try { dos.close(); } // The joys of Java I/O... With Java 7, auto-close is nicer!
21.       catch (IOException e) { println("Error when closing: " + e); }
22.     }
23.   }
24.  
25.   println("Done");
26.  
27.   InputStream is = createInput(fileName);
28.   DataInputStream dis = new DataInputStream(is);
29.   try
30.   {
31.     println(dis.readInt());
32.     println(dis.readFloat());
33.     println(dis.readBoolean());
34.   }
35.   catch (IOException e) { println("Error: " + e); }
36.   finally
37.   {
38.     if (dis != null)
39.     {
40.       try { dis.close(); }
41.       catch (IOException e) { println("Error when closing: " + e); }
42.     }
43.   }
44.  
45.   exit();
46. }

Thank you for pointing out the IO issues. I need to read about "finally." I will modify my programs, but I was just explicitly copying the above example. I also had to raise the count from <2047 to <2048.

Perhaps these library calls, and perhaps a seek and rewind, if applicable, could be included in Processing's documentation under a heading such as "Random Access File I/O" or "Advanced File Operations," or something of that ilk.

After reading your example, and this page: http://docs.oracle.com/javase/tutorial/essential/exceptions/finally.html

I still have an error if I use finally. I'm sorry, but this is now my reader prototype, but its is largely similar. I only have one finally block, as I can't seem to get them working. The data seems to load as long as it is commented.

Still, I was quite please to see the program kick out lines as the array filled up, with what I believe is my sample file data. Thank you : )

[The data array will oddly be largely global because several parts of the program will called on to create, manipulate and visualize, zoom, pan, etc, the data.]

1. // With file help from PhiLho
2. import java.io.*;
3. 
   
4. // Read File
5. int count, x, y;
6. int [][] data;
7. int datachunk;
8. char character;
9. 
   
10. // Filenames
11. String filename_input = "sampledata.tmv";
12. 
    
13. // File Header is 56 Bytes
14. char filemeta_name[] = new char [8];
15. byte filemeta_version;
16. int filemeta_layer;
17. int filemeta_sizex;
18. int filemeta_sizey;
19. int filemeta_locationx;
20. int filemeta_locationy;
21. int filemeta_lookatx;
22. int filemeta_lookaty;
23. int filemeta_visualizer;
24. long filemeta_crc;
25. boolean lock = false;
26. void setup()
27. {
28.   // Input File
29.   InputStream is = createInput(filename_input);
30.   DataInputStream source = new DataInputStream(is);
31.   // Read Header
32.   try
33.   {
34.     for (count = 0 ; count < 7 ; count++)
35.     {
36.       character = ( source.readChar( ) ); 
37.       filemeta_name[count] = character;
38.     }
39.     filemeta_version = ( source.readByte() );
40.     filemeta_layer = ( source.readShort () );
41.     filemeta_sizex = ( source.readShort () );
42.     filemeta_sizey = ( source.readShort () );
43.     filemeta_locationx = ( source.readShort () );
44.     filemeta_locationy = ( source.readShort () );
45.     filemeta_lookatx = ( source.readShort () );
46.     filemeta_lookatx = ( source.readShort () );
47.     filemeta_visualizer =( source.readShort () );
48.     filemeta_crc = ( source.readLong () ) ; // Reserved, for perhaps someday
49.     lock = ( source.readBoolean () ) ; // Reserved, as it might also help determine good file write
50.   }
51.   catch (IOException e)
52.   {
53.     println("Error: " + e);
54.   }
55.  
56. //  finally // Uncommenting this block causes "java.io.IOException: Read error" for some reason.
57. //  {
58. //    if (source != null)
59. //    {
60. //      try {
61. //        source.close();
62. //      }
63. //      catch (IOException e) {
64. //        println("Error when closing: " + e);
65. //      }
66. //    }
67. //  }
68. 
    
69.   // Init Dynamically Sized Data Array
70.   int [] [] data = new int [filemeta_sizex] [filemeta_sizex];
71. 
    
72.   // Read Data
73.   try
74.   {
75.     for (y = 0 ; y < filemeta_sizey ; y++)
76.     {
77.       // Update Status Bar Here
78.       print("Loading Row:");
79.       println(y);
80.       for (x = 0 ; x < filemeta_sizex ; x++)
81.       {
82.         datachunk = source.readShort ();
83.         data [x][y] = datachunk;
84.       }
85.     }
86.     source.close();    
87.     println("File Loaded");
88.     exit();
89.   }
90.   catch (IOException e)
91.   {
92.     println("Error: " + e);
93.   }
94. }

The problem is that you close source, then you try to read on it.
A perhaps better version is:

1. void setup()
2. {
3.   // Input File
4.   InputStream is = createInput(filename_input);
5.   DataInputStream source = new DataInputStream(is);
6.   // Read Header
7.   try
8.   {
9.     for (count = 0 ; count < 7 ; count++)
10.     {
11.       character = source.readChar(); 
12.       filemeta_name[count] = character;
13.     }
14.     filemeta_version = source.readByte();
15.     filemeta_layer = source.readShort();
16.     filemeta_sizex = source.readShort();
17.     filemeta_sizey = source.readShort();
18.     filemeta_locationx = source.readShort();
19.     filemeta_locationy = source.readShort();
20.     filemeta_lookatx = source.readShort();
21.     filemeta_lookatx = source.readShort();
22.     filemeta_visualizer = source.readShort();
23.     filemeta_crc = source.readLong() ; // Reserved, for perhaps someday
24.     lock = source.readBoolean() ; // Reserved, as it might also help determine good file write
25.     // Init Dynamically Sized Data Array
26.     int [] [] data = new int [filemeta_sizex] [filemeta_sizex];
27.     for (y = 0 ; y < filemeta_sizey ; y++)
28.     {
29.       // Update Status Bar Here
30.       print("Loading Row:");
31.       println(y);
32.       for (x = 0 ; x < filemeta_sizex ; x++)
33.       {
34.         datachunk = source.readShort ();
35.         data [x][y] = datachunk;
36.       }
37.     }
38.     println("File Loaded");
39.     exit();
40.   }
41.   catch (IOException e)
42.   {
43.     println("Error: " + e);
44.   }
45.   //*
46.   finally
47.   {
48.     if (source != null)
49.     {
50.       try {
51.         source.close();
52.       }
53.       catch (IOException e) {
54.         println("Error when closing: " + e);
55.       }
56.     }
57.   }
58.   //*/
59. }

Thank you, that did the trick, and it is proper not to have bad file handles kicking around, if things go bad.

I found that I was able to eliminate the intermediate variables: character, and datachunk. I have seen strangness, which I wanted to rule out, but it appears to run fine.

" it is proper not to have bad file handles kicking around"
Sure!
Particularly if you make a sketch opening and closing regularly a file, for example.
Better have good habits / practices from the start.

Is there something about using a try, which would prevent me from calling/updating other classes and functions?

Try seem to be the kiss of death for my program.

No, beside the usual scope issue: variables defined in the try section are not accessible in the catch nor finally sections.

I found part of the problem: There is a nasty bug in Processing's validation; commenting out the "try" and "Catch" statements in one class (and tab) caused errors to reported in others. I believe this bug to exist in the last few versions of processing, but it will hard to distill a simple test case without uploaded my whole program.

Still, I am left with my classes not being called within the try statements. I am looking into it.

The code in tabs is seen as just the concatenation of the content of all the tabs. As usual with compiler errors, if you forget to close a brace, for example, the compiler see it only when it finds out an incoherence, for example trying to close a brace when none is opened. This can happen lot of lines later, so in another tab.
It is an old limitation in Processing, one that is hard to fix, I suppose.

This Board is acting up again, and not adding posts, but I will try again, so if you you see a double post, please ignore it. Also, once invoked, the emphasis won't shut off, anyway.. .

It appears that there is something innately powerful about "try." In my version of Processing, calling external classes does not work. Not only that, having try in a a glass will keep calls from executing in its parent. This seems like either a serious bug or deficiency.

In Win64, in the following code, the marked lines will not work, yet it will still read the file, and I presume: also load my data, while printing out the y values.

1. class widget_fileread
2. {
3.   // With file help from PhiLho
4.   // Needs Version Checking, and Error Returns
5.   DataInputStream source;
6.   InputStream is;
7.   String filename_input;
8.   int count, x, y;
9.   // File Header is 56 Bytes
10.   char filemeta_name[] = new char [8];
11.   byte filemeta_version;
12.   int filemeta_layer;
13.   int filemeta_sizex;
14.   int filemeta_sizey;
15.   int filemeta_locationx;
16.   int filemeta_locationy;
17.   int filemeta_lookatx;
18.   int filemeta_lookaty;
19.   int filemeta_visualizer;
20.   long filemeta_crc;
21.   boolean lock = false;
22.   widget_fileread (String filename)
23.   {
24.     filename_input= filename;
25.     InputStream is = createInput(filename_input);
26.     DataInputStream source = new DataInputStream(is);
27.     try
28.     {
29.       // Read Header
30.       for (count = 0 ; count < 7 ; count++)
31.       {
32.         filemeta_name[count]  = ( source.readChar( ) );
33.       }
34.       filemeta_version = ( source.readByte() );
35.       filemeta_layer = ( source.readShort () );
36.       filemeta_sizex = ( source.readShort () );
37.       filemeta_sizey = ( source.readShort () );
38.       filemeta_locationx = ( source.readShort () );
39.       filemeta_locationy = ( source.readShort () );
40.       filemeta_lookatx = ( source.readShort () );
41.       filemeta_lookatx = ( source.readShort () );
42.       filemeta_visualizer =( source.readShort () );
43.       filemeta_crc = ( source.readLong () ) ; // Reserved
44.       lock = ( source.readBoolean () ) ; // Reserved
45.       // Init Dynamically Sized Line Array
46.       int [] [] [] data = new int [filemeta_sizex] [filemeta_sizey] [1];
47.       // Read Data - These following 4 calls are not executed!
48.       status0.update("Loading...");
49.       status1.update(" ");
50.       status2.update(" ");
51.       status3.update(" ");
52.       for (y = 0 ; y < filemeta_sizey ; y++)
53.       {
54.         progressbar.update(y, filemeta_sizey); // This also does not execute!
55.         print("Loading Row:");  
56.         println(y);
57.         for (x = 0 ; x < filemeta_sizex ; x++)
58.         {
59.           data [x][y][0] = source.readShort();
60.         }
61.       }
62.       source.close();    
63.       progressbar.update(0, 0);
64.       status0.update("File Loaded");
65.       println("File Loaded");
66.       exit();
67.     }
68.     catch (IOException e)
69.     {
70.       println("Error: " + e);
71.     }
72.     finally
73.     {
74.       if (source != null)
75.       {
76.         try {
77.           source.close();
78.         }
79.         catch (IOException e) {
80.           println("Error when closing: " + e);
81.         }
82.       }
83.     }
84.   }
85. }

If some lines are not executed in a try block, it probably means you got an exception and the code following the line where the exception happened is not run, as the process jumps directly in the catch block.
If you have no debugger, I suggest to put a println() between each line: thus you can spot at which point the I/O process fails.
If you have no error trace in the console from your current code, the problem is elsewhere. I see you call exit() at the end of the code. Are you sure your status has the time to be updated before the end?

try is not magic: it doesn't prevent code to be run, nor classes to be called. The problem is probably elsewhere, but without being able to run your code, it is hard to diagnose.