SimpleDateFormat conversion

setup · May 2015

Hi,

I'd like to convert a Java date time() value (milliseconds that have elapsed since January 1, 1970) from a 'long' to a 'float' where the integer part of the float is days and the fraction part is time. What I've tried appears to work, as in I get a number where the days are correct, but I don't understand the time value. The result for the time is 0.167 - I would have expected 0.25 since 6am is a quarter of the way through the day.

This is my code:

import java.text.SimpleDateFormat;
import java.util.Date;

void setup(){
  int ONE_DAY = 1000*24*60*60;
  Date pdate = new Date();

  String date = "24JUN14:06:00";
   try {pdate = new SimpleDateFormat("ddMMMyy:HH:mm").parse(date);}
   catch (Exception e) {}

   println(pdate);
   long x = pdate.getTime();
   println(pdate.getTime());
   println(new SimpleDateFormat("HH:mm").format(pdate));

   println((float) pdate.getTime()/ONE_DAY);
}

The code gives the following result:

Tue Jun 24 06:00:00 CEST 2014 1403582400000 06:00 16245.167

If there is an easier way to achieve this I'd love to know.

Many thanks.

setup · May 2015

I've delved deeper but still don't have a solid solution. The following code gives a workable solution for my current need. For some reason I need to add 2 hours (0.083) but I need it to be accurate.

If anyone knows the proper way to do this I would love to hear:

import java.text.SimpleDateFormat;
import java.util.Date;
int ONE_DAY = 1000*24*60*60;

void setup() {
  Date pdate = new Date();

  dateTimeY("24JUN14:00:00", "ddMMMyy:HH:mm");
  dateTimeY("24JUN14:06:00", "ddMMMyy:HH:mm");
  dateTimeY("24JUN14:12:00", "ddMMMyy:HH:mm");
  dateTimeY("24JUN14:18:00", "ddMMMyy:HH:mm");
  dateTimeY("25JUN14:00:00", "ddMMMyy:HH:mm");

  dateTimeY("24JUN14", "ddMMMyy");
  dateTimeY("25JUN14", "ddMMMyy");
}

float dateTimeY(String d, String f) {
  float pdate = 0.0;
  try {
    pdate = (float) new SimpleDateFormat(f).parse(d).getTime()/ONE_DAY + 0.083;
    println(d + " " + pdate);
    return pdate;
  }
  catch (Exception e) {
    return 0.0;
  }
}

quark · May 2015

You need to take into account daylight savings like this

import java.text.SimpleDateFormat;
import java.util.*;

void setup() {
  int ONE_DAY = 1000*24*60*60;
  Date pdate = new Date();

  String date = "24JUN14:06:00";
  try {
    pdate = new SimpleDateFormat("ddMMMyy:HH:mm").parse(date);
  }
  catch (Exception e) {
  }

  Calendar c = Calendar.getInstance();
  c.setTime(pdate);
  println(c.getTimeInMillis() + c.getTimeZone().getDSTSavings());

  long x = c.getTimeInMillis() + c.getTimeZone().getDSTSavings();
  println(new SimpleDateFormat("HH:mm").format(pdate));

  println(((float) x) / ONE_DAY);
}

setup · May 2015

Thanks quark,

I guess this answers my question about the spurious fix I needed before, but I'm still not getting integer numbers for whole days and accurate 'time' fractions.

The results for 24JUN14:06:00 from your code on my computer gives:

1403586000000 06:00 16245.208

I don't understand the fraction part. Since 6:00 is a quarter of the way through the day shouldn't it be 0.25? Is this normal?

quark · May 2015

I added the following line at position 16:17

println(c.getTimeZone());

to print the time zone

This is the output I got on my computer

sun.util.calendar.ZoneInfo[id="Europe/London",offset=0,dstSavings=3600000,useDaylight=true,transitions=242,lastRule=java.util.SimpleTimeZone[id=Europe/London,offset=0,dstSavings=3600000,useDaylight=true,startYear=0,startMode=2,startMonth=2,startDay=-1,startDayOfWeek=1,startTime=3600000,startTimeMode=2,endMode=2,endMonth=9,endDay=-1,endDayOfWeek=1,endTime=3600000,endTimeMode=2]]
1403589600000
06:00
16245.25

Check the time zone on your computer.

PhiLho · May 2015

Note that using floating point numbers to represent time isn't a good idea, as these have rounding errors, so the accuracy will not be optimal.

setup · May 2015

I get the following:

sun.util.calendar.ZoneInfo[id="Europe/Zurich",offset=3600000,dstSavings=3600000,useDaylight=true,transitions=119,lastRule=java.util.SimpleTimeZone[id=Europe/Zurich,offset=3600000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=2,startMonth=2,startDay=-1,startDayOfWeek=1,startTime=3600000,startTimeMode=2,endMode=2,endMonth=9,endDay=-1,endDayOfWeek=1,endTime=3600000,endTimeMode=2]]

and tried to apply the information in the code as follows:

println(c.getTimeInMillis() + c.getTimeZone("Europe/Zurich").getDSTSavings(3600000));

but it didn't work.

PhilLho, I agree with you and I will have to revisit this code to do it properly (I'm just not sure what 'properly' is yet). I'm still learning how dates work in Java as it's not what I'm used to. I didn't expect the time zone of the computer to affect a static date. Also, the dates I will be using aren't recorded to the millisecond, so I hope a floating point error won't affect the accuracy.

quark · May 2015

I suggest that you use the Calendar and GregorianCalendar classes since the Date class has been deprecated.

setup · May 2015

Hi Quark,

I'm sorry, but I've lost you. Are you suggesting a completely new method to handle the dates or is this a solution to the way it was programmed in my original example. Either way I'm not sure what I should do next.

Ideally I would like to stick to the original way for the moment because if I don't then I'll have to change other parts of my program that rely on the float values. Perhaps if I show you the other part of my program you'll see what I mean. Basically, I'm creating a flexible scale where you can zoom in on the dates and the scale expands to produce times. That part works fine until I plot points based on time values as that's where I get these shift in placements.

I would like to program it correctly but I need a little more guidance.

Once again, many thanks for the advice so far - I'm learning slowly.

import java.text.SimpleDateFormat;
import java.util.Calendar;
import java.util.Date;
import org.gicentre.utils.move.*;

long min = 13500, max = 13550;
float zoom, axisY;
ZoomPan zoomer; 
int gStt = 50, gEnd = 1200, gBot = 150;
long ONE_DAY = 1000*24*60*60;
PFont f2;

void setup() {
  size(1000, 100);
  zoomer = new ZoomPan(this);
  f2 = createFont("Georgia", 60);
  textFont(f2);
  fill(0);
}

void draw() {
  background(255);
  frame.setTitle("Zoom : " + zoom + " FPS : " + int(frameRate));

  zoom = (float) zoomer.getZoomScale();
  axisY   = zoomer.getDispToCoord(new PVector(0, 50)).y;
  zoomer.transform();

  textAlign(CENTER, BOTTOM);
  // Position on the y axis adjusting for zoom:
  float textPos  = axisY - 15/zoom;                   //   Text 
  float pointPos = axisY - 5/zoom;                    //   points 

  for (long x = min; x < max; x+=1) {                 // Loop from screening date to max days
    Float xPos = map(x, min, max, gStt, gEnd);        //    Calculate x position of day

    if (zoom <= 8) zoomer.text(String.valueOf('.'), xPos, pointPos); //    Position points
    textSize(8/zoom); 
    if (zoom >= 10) {
      zoomer.text(String.valueOf("0"), map(x-0.5, min, max, gStt, gEnd), pointPos);
      zoomer.text(String.valueOf("6"), map(x-0.25, min, max, gStt, gEnd), pointPos);
      zoomer.text(String.valueOf("12"), xPos, pointPos);
      zoomer.text(String.valueOf("18"), map(x+0.25, min, max, gStt, gEnd), pointPos);
    }

    textSize(10/zoom);       
    if (x == min) zoomer.text(date(x), xPos, textPos);            
    else {
      int major, minor;
      if (zoom <= 2) major = 8;
      else if (zoom <= 3) major = 4;  
      else if (zoom <= 5) major = 2;  
      else major = 1;  

      if ((x-min)%major == 0) zoomer.text(date(x), xPos, textPos);
    }
  }
}


String date(long num){
  Date d = new Date(num * ONE_DAY);
  String date = new java.text.SimpleDateFormat("ddMMMyy").format(d); 
  //println("date : " + num  + " " + d +"\t" + date  +"\t   " + num);
  return date; 
}

quark · May 2015

Working with dates is very difficult because there is more than one calendar used in the world. The date class was deprecated because it couldn't handle these well and the Calendar class was introduced instead. The code below has a function that will convert a date-time string to a Date simply so we can easily initialise a calendar object. It then uses this object to get the hours (am/pm), minutes and seconds and calculates the fractional part.

I got the following output when I ran this program on my machine.

16245.25

16245.75

import java.text.SimpleDateFormat;
import java.util.*;

int ONE_DAY = 1000*24*60*60;

void setup() {
  Date pdate = new Date();

  String date = "24JUN14:06:00";
  println(fromEPOCH("24JUN14:06:00"));
  println(fromEPOCH("24JUN14:18:00"));
}


// Get the number of days since the epoch as a float. The
// decimal part represents part of the day
float fromEPOCH(String date) {
  Date pdate;
  try {
    pdate = new SimpleDateFormat("ddMMMyy:HH:mm").parse(date);
  }
  catch (Exception e) {
    return 0;
  }
  Calendar c = Calendar.getInstance();
  c.setTime(pdate);
  long nbrDays =  c.getTimeInMillis() / ONE_DAY;
  int hour = c.get(Calendar.HOUR);
  if (c.get(Calendar.AM_PM) == Calendar.PM) hour += 12;
  int minute = c.get(Calendar.MINUTE);
  int second = c.get(Calendar.SECOND);
  float fraction = (hour + minute / 60.0 + second / 3600.0)/24.0;
  return nbrDays + fraction;
}

setup · May 2015

Hi Quark,

This is perfect - I get the same results on my computer too. Thank you very much.

Although ... ... I'm still a little confused by all the ways to record & derive java datetime values and I would like to learn more. If I would rewrite the sliding scale again in a more Java oriented way how would you propose I program it?

Thanks again

quark · May 2015

in a more Java oriented way

Java is an object orientated (OO) language so if I was to program this myself then I would do it the OO way.

I would design and create a class that would encapsulate ALL the information and behaviour (actions) required for what is effectively a 'date zoom slider'

I don't know if you are have any experience with creating your own classes or how much you know about OO. If you let us know then I can give a more informative answer.

setup · May 2015

I understand inheritance and have written a few classes (I think I've even extended one), but never libraries, packages or interfaces. My experience of OO comes from following Ben Fry's book on Visualising Data and just playing with the examples in Processing. It's certainly true that the more I learn the more I realize how little I know. As I'm self taught, I'm in no doubt that there are large holes or even misunderstandings in my knowledge. I come from a data analysis background using SAS so my programming style is based around the data rather than an OO style, but I'm trying to adapt.

GoToLoop · May 2015

quark · May 2015

Ben Fry's book is good as far as it goes but does not discuss the object orientated paradigm.

Many self taught programmers and programmers who originally learned a procedural language like BASIC simply view OO as introducing new syntax in the same way as the if-else, for loop and switch statements are syntactically different. Nothing could be further from the truth, OO is a programming paradigm. For instance inheritance and polymorphism are part of the OO paradigm, but when should they be used? To answer that goes beyond syntax and requires the programmer to study OO concepts.

One of the key concepts of OO is the relationship between classes and objects. The class is a blueprint for creating objects, in the same way an architects drawing can be used repeatably to create multiple buildings.

A class encapsulates both data (things that describe the object e.g. position, colour etc.) and methods (aka functions) which are used to set, get and utilise that data.

Another key concept is autonomy, by that I mean

1) the objects data values are ONLY modified by the methods inside the class.

2) the methods inside the class ONLY use the class data or data passed through the methods parameters.

_The advantage of ensuring your classes are autonomous is that they can be used without code changes in different programs. _

Java does not enforce this concept so it is easy to create classes that are not autonomous. This is NOT a design fault in Java because there are occasions where you might not want autonomy, but this is rarely the case when you first start creating our own classes.

The Processing language is based on Java but it was designed to enable quite sophisticated sketches to be easily created without the burden of having to learn OO. It means that it is much easier to create non-autonomous classes in Processing and some effort is required by the programmer to make autonomous classes. If you think that the class might be useful in other programming projects then it is worth the effort to make it autonomous.

OK having said that what about your time slider. You could consider it to be a 'GUI control' in the same way as a button, text field, scrollbar etc. In which case it would be worth creating a class to represent this control.

The code below could be used as a starting point for your class.

public class TimeSlider {
  // The class fields (aka attributes)
  private PApplet app;
  private int x, y; // Position of top left corner of slider in pixels
  private int w, h; // Width and height of slider in pixels
  private float start; // represents date/time of left hnnd side  
  private float duration; // represents the data/time range of the slider

  // Other fields needed e.g. colours etc

  public TimeSlider(PApplet app, int x, int y, int w, int h) {
    this.app = app;
    this.x = x;
    this.y = y;
    // same for w & h
    start = 0; // 1st Jan 1970
    duration = 365 * 24 * 60 * 60 *1000; // 1 year in milliseconds
  }

  public void pan(float panBy) {
    // panBy = number of milliseconds to pan slider
    start += panBy;
  }

  public void display() {
    app.pushMatrix(); // remember current transofrmation matrix
    app.translate(x, y);
    // Use ONLY the information in the fields
    // to draw the slider. Note the translate statement
    // means that the top left corner of the slider has
    // the coordinates 0,0


    app.popMatrix(); // restore the transformation matrix
    // The push/popMatrix statements 'isolate' any 
    // transformations you might use to draw the slider
  }
}

setup · May 2015

Thanks for the links to PhilLho's detailed explanations. I forgot, but I should have mentioned I've also read Daniel Shiffman's Learning Processing book too. In his book he goes into the OO concepts in more detail. I've learnt a lot from the books and the articles were helpful and all brilliantly written, but I still feel I am missing something. As you mentioned above, it's more than the syntax, it's the design or the framework of the program that's important too. I can see the clear advantages of classes, but I struggle with determining what classes should be created in the first place.

My program now works in the way I'd like it to, so there's no real problems in the functionality, but it lacks the scalability of a correctly designed OO program. So far I've been limiting my questions to specific programming issues, but I'm revisiting the code and now have fundamental design issue that need resolving. I see what you're saying about the time slider, but the more I think about what I'm actually trying to achieve the more I get lost, since the time slider is only part of the final goal.

Let me explain a little more ....

I have data coming from multiple sources - 6 so far, but I want to increase this to around 30. Each dataset has a different structure, but common elements with a unique key variable to link across all datasets. For each key I create a consolidated graphical representation of all data from the multiple datasets. With the program I've written I can zoom in on the timeline to look at a more granular level and I can flip between graphs associated with each key. I can select the data type displayed (for the key) and choose the scale and even put labels on important time points.

My intention is to rewrite the code so that it's easier to add more datasets. I'd also like the option to view the data by data type summarised by key as opposed to key summarising the data, with the ability to drill down to the key summary from a data point and vice versa.

Currently my program contains only one class for the Graph. To create a Graph object I instantiate the object by passing all the data associated with a key and draw the graph within the class with separate methods that I call from the draw(). It's a real mishmash of logic with duplicated code and doesn't fit into any programming paradigm, but it works :-). Clearly it's not scalable because I don't want to have to add 30 datasets to the constructor and then have specific code for each dataset within the class.

So, when I think about redesigning the code ... ...

From a 'high level' view, I know I want to view the data by both data type and by key with the ability to traverse the two. This leads me to think I need a class for either the data type or the key. I don't think I can have both because the two are too interconnected. I can have up to 30 datatypes, but several thousand keys.

If I plan from a 'low level' view I guess I need classes to contain the:

position of point on timeline

shape associated with a point (bar, triangle, line, etc)

data to group the shapes

key to group the data

At this point I think the classes should be based around the 'key' rather than the datatype. Then I go back to my original intention to improve the scalability of the program by having a class for the data and it's no different to what I've written already.

I'm sure I'm over complicating things as what I'm trying to achieve has been done before in practically all graphing applications. I guess I'm asking the wrong questions about the design, but hopefully I've explained to you my confusion rather than confused you with my explanation and you may be able point me in the right direction.

GoToLoop · May 2015

I'm not on par about the subject, but I guess you should consider creating some Key class in addition to Graph.
This way, a Graph object can add() & remove() Key objects as the need arises.

quark · May 2015

Unfortunately I find your explanation confusing. The reason is that it is all too abstract.

What is a dataset? What are the sources? What are the differences between datasets in terms of structure?

a unique key variable to link across all datasets

Does this mean that there is some 'link' between datasets or that every data set has a 'key' attribute which identifies a particular dataset?

Need to make the description less abstract. :)

setup · May 2015

Yes, you're right to be confused as I am too :-). I'm probably mixing up my terminologies.

I use the term dataset as it's SAS terminology for a set of data containing data values that are organized as a table of observations (rows) and variables (columns). In my program I just read the data into processing from csv files.

When say 'data coming from multiple sources' I mean: I don't have one csv file with the data neatly collated and ready to use - I have many. I've started with displaying 6 files, but will have up to 30 that all have a similar (but different) structures.

If I give an example in banking terms it may be less abstract:

Let's say I have a file of all transactions, debits and credits, listed by date and time for many customers. The key here is the customer.

So, my file by transaction has the following structure:

                              Credit/
Customer             ADTM      Debit

   1           14JUN13:18:00        C
   1           15JUN13:21:00        D
   1           16JUN13:07:00        C
   1           17JUN13:06:00        D
   1           17JUN13:10:00        D
   1           18JUN13:10:00        D
   1           18JUN13:16:00        D
   1           19JUN13:07:00        D
   1           19JUN13:11:00        D
   1           20JUN13:14:00        D
   2           07MAY13:11:15       C
   2           07MAY13:21:35       C
   2            .                   D
   2           09MAY13:09:55       D
   2           09MAY13:19:15       D
   2           09MAY13:22:47       C
   2           10MAY13:12:00       C
   2           10MAY13:13:15       C
etc

I have another file containing summary of transaction data by day

Customer          ADT     Credit    Transaction
   1            14JUN13      1          1
   1            15JUN13      0          1
   1            16JUN13      1          1
   1            17JUN13      0          2
   1            18JUN13      0          2
   1            19JUN13      0          2
   1            20JUN13      0          1
   2            07MAY13     2          2
   2            09MAY13     1          3
   2            10MAY13     2          2
etc

And I have other files containing single activities such as 'VisitBankAdvisor':

Customer           ADT
   1            17JUN13
   2            14MAY13

Now, I have created a graph that displays the information from all these files by 'Customer' on one page and it looks like this with the date along the top small black and red ticks representing each transaction. The summary file is displayed as a bar graph and odd single activities represented by lines or triangles:

Capture

It works well, but is not easily scalable since I have only one graph class and instantiate the object by feeding each Table subset by customer in the constructor and draw the data within methods in the class:

custG = new Graph(transactions.matchRow(cust[index], "Customer"), 
                    summary.matchRows(cust[index], "Customer"), 
                    VisitBankAdvisor.matchRows(cust[index], "Customer"), 
                    etc4.matchRows(cust[index], "Customer"),
                    etc5.matchRows(cust[index], "Customer"),
                    etc6.matchRows(cust[index], "Customer"),
                    );              // Get data

I now want to scale up the information graphed and look into other way of displaying the data too. Currently I only display data by customer, but I'd like to have the option to summarise the customers by datatype too. For example, Individal transactions by customer could look like this:

final

As such, I want to convert the program structure using a more dynamic OO approach this time, but I don't know quite where to start with the redesign of the program.

Is MVC the way to go?

Create a new 'customer class' that represents the 'Model'. Keep the 'Graph class', but remove all reference to the data which represents the 'View' Create an 'Controller Class' that communicates between the Model and the View.

Howdy, Stranger!

Categories

In this Discussion

SimpleDateFormat conversion

Best Answer

Answers