Java and Memory — Yes, it Can Leak

July 29, 2008

Note: I had previously posted this on my SourceForge Diary on 6-18-08 , but after two posts to the diary I decided the blog was a better place :).

I have been keeping up work on Spheriosity lately as I have devoted my summer to it. Normally I spent around 7 or 8 hours a day writing code for it. Then on the weekend I try to give myself a bit of a break from staring at the computer screen 😉

Before I get in to the main point of this entry I will just list a few things I have done with Spheriosity recently:

  • Added the ability to place a point on a line
  • Moved the move point button to the draw section
  • Wrote unit tests for the new code (putting a point on a line)
  • Added a little information section to the bottom of the window which displays area, radius, etc.  so the user has some idea what is going on

There are other small changes and if you really want you can investigate the CVS logs. When I release the next version (hopefully the end of the month) I will introduce a changelog file where I will try to sum up all the new features! As a simple FYI the next feature I plan to work on is parallel transport 🙂  I can’t wait to get this implemented, but I do not think this will be any trivial task so please be patient with me as I work on this.

Anyway the reason I started this journal entry was because I just found a huge memory leak today in Spheriosity. A memory leak!? In Java!? YES! You CAN have memory leaks in Java. This realization hit me pretty hard today when I found out that I was wasting a lot of memory. Before today I had let the lore of Java convince me it was impossible to have memory leaks, but it’s true… you CAN have them.

How is this possible you ask? Well it has to do with how garbage collection works. Yes, in the traditional sense of C/C++ you can not have a memory leak where you personally forget to deallocate a chuck of memory. This still does not explain what I mean by memory leak though. I am talking about what happens when you hold on to a reference of an object long after it is needed. This happens very easily with private class variables and static variables. Lets talk a bit about garbage collection:

(This is simplified and I am sure the Java garbage collector is far from trivial, but I will simply explain what I know) In a nut shell the garbage collector knows it’s time to get rid of an object when there are no more references to it in code. So if you are storing an object in a private class variable and no longer need it you should set that variable to null. Especially if the class is going to be around for most of the life of the program. This is quite plausible with things such as user interfaces. It is also worth mentioning that you can think of this as a tree like structure. If you stop referencing a class at the top of the tree the entire branch will be collected as long as the only references to the sub objects occur within that branch.

In Spheriosity the problem I had was that after a user selected ‘New’ to clear the sphere I simply removed the highest node in the scene graph from the scene graph and did not clear all of the objects out of it. This caused everything in the frontend and backend to be kept around! All it took was a single line to clear out the scene graph. There were issues where I would get rid of some frontend objects but it was still an observer to something in the backend. As such the garbage collector would never collect it.

Now that I have learned all of this I will be extremely careful to avoid any future memory leaks. I am also going to be more careful when I explain memory management in java to people. I hear people say all the time that it is so amazing and they make it sound completely fail proof, but that is NOT the truth. You still need to be careful with how you manage memory. There is an entirely separate issue here known as heap smashing and in short it is caused by calling ‘new’ too much and forcing the garbage collector to work overtime! This is a conversation for another day, however.

I hope that I have enlightened at least one person about some of the pitfalls of Java. I still like the language and the library support is awesome. However I will never again say the memory management is easier than C++. The reality is that you have to be just as careful! Why do I care so much about memory usage? Well, I don’t like programs that are memory hogs so I try to create ones that don’t hog memory in their own right.

Sometimes I start to think Java itself is a memory hog by default because of the virtual machine and I do get concerned about that. However, without Java3D Spheriosity wouldn’t be anything that amazing so for now I will be sticking with Java 🙂

PS: The cross platform support is nice too 😀


Update Slowdown

July 28, 2008

For the next week or two I won’t be writing as much as I normally do. Nothing is wrong, I just won’t be using my computer nearly as much as I won’t come up with any topics worth talking about. I have picked up a book about MySQL and I hope to learn at least the basics about running a MySQL server and how to create and manage a database. It is a shame my college does not offer any courses in database management, but I don’t mind learning about it on my own.

All I will be doing with Spheriosity is updating the comments and working on some of the documentation 🙂


Gradius V — Finally, I beat the dang game!

July 26, 2008

Lately I have had some more time to enjoy the occasional video game. For a few years now I have had an obsession with being able to beat Gradius V. For those who don’t know Gradius V is one of the more popular titles in side scrolling shooters. The game requires you to be able to watch many bullets flying around your ship which being able to take down the necessary enemies. In order to make the game more manageable the hit box for your ship is insanely small. Gradius V could almost be considered a Maniac Shooter, but it falls short in that category because there simply aren’t enough bullets on screen (if you have ever played a true maniac shooter… then you would know).

Even though this game can be insanely frustrating at times it is well worth playing if you can stick with it. When you beat a level it is quite an awesome feeling because of the shear difficulty involved. If you can beat a level without losing a single life, that is even more incredible. I really enjoy challenging my ability to move the ship just tiny bits in order to avoid tight obstacles and swarms of bullets/lasers. It is a true test of one’s ability to have precise control. It is games like this that still bring glory to the standard video game controller. Even though I am a huge fan of the Wii, I can’t see any of the motion sensing features of the Wiimote being used in a Gradius type shooter.

After finishing a game like Gradius V it makes me think there is still a place for the traditional video game controller. I hope to continue playing Gradius V and maybe, some day, I will be able to beat it on a difficulty other than ‘very easy’ (which is anything but… IMO).


Intersection of Spherical Circles (Finally!)

July 23, 2008

For the past two weeks I have been pondering on and off how I would calculate the location of the intersection of two spherical circles. I really want Spheriosity to be as complete as possible and having the intersection tool only work on Spherical lines/line segments seems like a real lame idea. So I have finally devised an algorithm which can correctly determine the intersection points. I will give a brief overview of it here. I will leave out details about different parts of the algorithm which I feel are either trivial or would need their own blog post to explain ;).

So we start with two circles… What are the possibilities for their arrangements in terms of intersections:

  1. The most simple is if the circles are not anywhere near each other and don’t intersect.
  2. One circle could be “inside” the other. In this case the circles obviously do not intersect.

    One circle inside the other

    One circle inside the other

  3. The two circles could intersect only in one spot. This could happen in a number of ways:
    • Case 1:

      First Intersection Case

      First Intersection Case

    • Case 2:

      Intersection Case 2

      Intersection Case 2

  4. Then there is the case where the circles intersect twice. This is perhaps the most common case in terms of what users will see, but since the other cases are easy to construct we must also deal with them.

So that loosely describes my problem and the different scenarios that need to be considered. So… on to the solution! Much of this boils down to finding the intersection of two planes. Each circle on the sphere defines it’s own plane which basically slices the sphere, and the intersection of that plane and the sphere could also be used to form the circle. So the intersection of the planes formed by both circles forms a line which, if it goes through the sphere, can be used to find our points of intersection.

Defining that line of intersection turned out to be the largest problem. It was easy enough to get a vector going in the correct direction but I still needed a point to form the line and perform any sort of calculation on it. Here are the steps I take to find the intersection and make sure one actually exists:

  1. Consider two circles AB and CD (defined by center point and radius point).
  2. Find the normal vector to the planes formed by each circle. We will call these vectors a and c
  3. Take a x c if the magnitude squared of the resulting vector is 0 you can stop now because the planes are parallel, and the circles could not possibly intersect (or are coincident).
  4. Form an axis of rotation, r, between points A and C.
  5. For both circles form lines by rotating their respective center points forward and back by the angle formed between their center points, the center of the sphere, and their radius points. (call these lines l and m.
  6. Find the intersection of lines l and m and call this point I
  7. Find the intersection of the line formed by using point I and the cross product of a and c with the sphere.
  8. Consider the following cases:
    • Case 1: Line does not intersect the sphere: no intersection points
    • Case 2: Line intersects once with the sphere: one intersection point
    • Case 3: Line intersects twice with the sphere: two intersection points

There you have it! I know I left some details out, but writing out every single step would make this a really long post ;).


Python 2.5 and encryption — PyCrypto under Windows

July 20, 2008

[Edit: 8-13-08] – Upon request I have provided a link to an installer for PyCrypto-2.0.1 that is compiled for Python 2.5. You can download it here . I didn’t post it originally because I was not sure how long I could provide a link for. This one should be good for about a year … 🙂 .

Note: The steps listed here will only work on Python 2.5 and above as that is when they added support for allowing MinGW to compile code

I mentioned in a previous post that I was looking for a way to get public/private key encryption in Python and I was having a bugger of a time until I found ezPyCrypto. I don’t know if I have mentioned this before, but I have been an Linux user for about 4 years now. Specifically Gentoo Linux ,and I would never run another OS now that I have been a Linux user. At any rate that’s not the point of this post. Since the script I was developing had to run on Windows I needed to install PyCrypto (it’s a dependency for ezPyCrypto). Well this would be trivial except that PyCrypto has C code that needs to be compiled. There are binary builds on the PyCrypto site for both Python 2.3 and 2.4, but not 2.5 which I am running. I wasn’t going to let this stop me however. The rest of this post will explain what you need to do in order to get PyCrypto working under Windows with Python 2.5. After completing this you will also be able to build your own installer that you could package with your scripts to let people who don’t want to install a C compiler run your code 🙂

The first thing you are going to want to do is install MinGW . MinGW will give you a nice C and C++ compiler for Windows. Just follow the instructions on their site and you will be good to go. After you have MinGW installed you will probably want to add entries to the Windows PATH variable so that when you are in a terminal you will be able to directly access MinGW. [As a note I still run Windows XP so these instructions will be done with that in mind] Doing this is quite simple:

  1. Right Click My Computer –> Properties –> Advanced –> Environment Variables.
  2. You will see User and System variables. If you want all users to use MinGW edit the Path entry in there. Otherwise add a Path variable under User variables.
  3. Go to the Path variable from step 2. Add an entry to the Path that points to MinGW’s bin directory. For me this was “C:\MinGW\bin” but it all depends on where you installed MinGW. Note: Entires in Windows Environment variables are separated by ‘;’
  4. Click ok a bunch of times

While you are changing environment variables you will want to add the main directory for Python to your path. For me this was: “C:\Python25”. This made my final User Path entry: “C:\MinGW\bin;C:\Python25”

Now that you are done changing environment variables you will want to reboot your computer so that Windows gets your changes. Really… you will regret it if you don’t reboot!

Now download PyCrypto and unpackage it. This turned out to be problem enough because not many people keep programs around that open .tar.gz files. Since am a Linux user I just unpacked them in Linux and transferred them to Windows (I run Windows XP under VirtualBox ). However, if that is not an option you can download and install IZarc Archiver. IZarc has been my favorite [Windows] extraction tool for a long time and it supports just about every archive format imaginable.I would recommend extracting the contents of PyCrypto to your desktop. Now is when the first starts!

First, open up a terminal. We will only be using a few commands and you don’t have to be a terminal wiz kid to do this. I normally open a terminal by going: Start –> Run –> “cmd” –> Press Enter

Now that you have a terminal up you want get to the PyCrypto directory. To move between directories in the terminal we use the “cd” command.

cd PATH-GOES-HERE

So, for example here is the command I used to get to where I had PyCrypto:

cd C:\Documents and Settings\Jinto\Desktop\pycrypto-2.0.1

Of course that command will only work for you if your user name is Jinto and you extracted it to the desktop exactly like I did 😉 So modify it to fit your needs. Once you are in the pycrypto directory do a quick:

dir /w

To make sure you see files. Specially make sure setup.py is there. We need to run that setup script and tell it to build the libraries. To do this issue the following command:

python setup.py build -c mingw32

If all goes well it should build without any troubles and all that is left is to install it by issuing the following command:

python setup.py install

You should be all set to use PyCrypto now, or to install ezPyCrypto which uses PyCrypto. As a bonus feature if you are looking to make an executable installer for PyCrypto you can run the following command:

python setup.py bdist_wininst

This will leave you with a .exe file located in the “dist” folder.

I hope this helps everyone get PyCrypto running under Windows. For those of you Linux users who think I am leaving you out check your distro specific respository system they probably already have a package for PyCrypto!


Reading all the Documentation…

July 19, 2008

So apparently when people the time to write something down they wrote it down for a reason. I happened to have missed this simple fact recently when I was working on some Python scripts. I was trying to separate the scripts into different folders based on what they did. Simple enough, but then I ran into the issue of certain scripts needing to go around folder boundries. Of course, Python supports this feature using it’s import system.

This is where my trouble was. I was reading the article about how import works from the standard Python docs. So I read as much as I thought I needed and skipped over, what I thought at the time, to be unimportant. As it turns out the ability to do relative imports using ‘..’ is a feature of only Python 2.5 *sigh*. Alright, so I could have just installed Python 2.5, but I didn’t want to limit this code to running on Python 2.5 for no good reason. So I went back and changed the code to using absolute imports which took around 30 minutes because there were a decent number of files to play with.

So how did I miss this in the documentation? Well I managed to skim over the part which mentions this is only for Python 2.5 (I blame myself for this, btw). I take this as a lesson to pay closer attention when reading documentation. It may have taken me an extra 5 or 10 minutes to read all the documentation about imports and Python… but I will would have saved half an hour of my time if I had taken the extra effort 🙂


The Importance of White Space in Code

July 18, 2008

For almost as long as I have been writing code I have tried my best to improve upon my coding style. This is, of course, an area of much controversy among computer scientists. What makes “good” style? Sometimes its purely a matter of opinion. What I find easy to read might not be so easy for someone else. I suppose this is one reason you might argue that programmers should all try and adhere to similar styles. This way any code would be just as readable to anyone.

Almost every language I have ever used allows you to put in more white space than you need, and anywhere you want to. I find this to be quite a useful feature. Not because I think it allows me to write better algorithms, but because it helps me organize the code and suggest certain things to the reader. To some degree making readable code is very much an art [one that I do not claim to have mastered]. I think adding white space in blocks of code to segment it into different sections is good practice. Lets take the following C code for example. Keep in mind that this could be any language and the same concepts apply:

#include <stdio.h>
#include <stdlib.h>
int main(int argv,char** argc)
{
  int x;
  int y;
  int sum;
  for(x=0;x<10;x++)
  {
    for(y=0;y<10;y++)
    {
      sum=x+y;
      sum=sum+x*x+y*y;
      printf("Random number is %d\n",sum);
      printf("yay for loops\n");
    }
  }
  exit(0);
}

Now, I will be the first to admit that this code is fairly random and doesn’t do anything of extreme usefulness, but that is not the point here. Notice this same program with a little extra white space added:

#include <stdio.h>
#include <stdlib.h>

int main(int argv, char** argc)
{
  int x;
  int y;
  int sum;

  for(x = 0 ; x < 10; x++)
  {
    for(y = 0; y < 10; y++)
    {
      sum = x+y;
      sum = sum + x*x + y*y;

      printf("Random number is %d\n", sum);
      printf("yay for loops\n");
    }
  }

  exit(0);
}

To me, this now really pops out at you, and it is much easier to see the different “areas” of this code. You have the #includes which are separated from main. This lets you know when the real code begins and puts a break between the includes and the actual code. It suggests a change in context! The next place I added white space was between the arguments in the header to main. This allows the variables (argc and argv) to stick out more to the eye.

In the actual function I put a space after I declared all the variables. Again, I use the idea that with the return we will have a change of context going from variable definitions to more concrete code. In each for loop I separated the different parts of the loop constraints to make them a tad easier to read.

Within in the inner for loop I added the most space. First I separated the mathematical statements from the print statements. You may not always want to do this because sometimes you might group a print statement for a specific calculation with the rest of the calculation, it all depends on what you are trying to say. Next I separated the different parts of the second equation to bring out the more
interesting parts. Sometimes I will add parenthesis to really be explicit as to what I am trying to do, but this didn’t see to warrant that.

Finally at the end I separated the exit(0) from the end of the for loops because it is not really part of that block of code.

So there you have it. This is a bit of a trivial example, but I think it shows that a little white space can go a long way in bringing things out to a foreign reader. Remember, the new person who reads your code will be looking for any hints they can get as to its functionality. When you are writing the code you may not need the white space to understand what is going on. However, 6 months later it
could even help you figure out what was originally intended as well as the newcomer to the code base! There’s plenty of other tricks to help make code more readable: descriptive variables, good function names, good functional decomposition, etc. I just wanted to point out one area that I think can be easily overlooked 🙂