Python 2.5 and encryption — PyCrypto under Windows

July 20, 2008

[Edit: 8-13-08] – Upon request I have provided a link to an installer for PyCrypto-2.0.1 that is compiled for Python 2.5. You can download it here . I didn’t post it originally because I was not sure how long I could provide a link for. This one should be good for about a year … 🙂 .

Note: The steps listed here will only work on Python 2.5 and above as that is when they added support for allowing MinGW to compile code

I mentioned in a previous post that I was looking for a way to get public/private key encryption in Python and I was having a bugger of a time until I found ezPyCrypto. I don’t know if I have mentioned this before, but I have been an Linux user for about 4 years now. Specifically Gentoo Linux ,and I would never run another OS now that I have been a Linux user. At any rate that’s not the point of this post. Since the script I was developing had to run on Windows I needed to install PyCrypto (it’s a dependency for ezPyCrypto). Well this would be trivial except that PyCrypto has C code that needs to be compiled. There are binary builds on the PyCrypto site for both Python 2.3 and 2.4, but not 2.5 which I am running. I wasn’t going to let this stop me however. The rest of this post will explain what you need to do in order to get PyCrypto working under Windows with Python 2.5. After completing this you will also be able to build your own installer that you could package with your scripts to let people who don’t want to install a C compiler run your code 🙂

The first thing you are going to want to do is install MinGW . MinGW will give you a nice C and C++ compiler for Windows. Just follow the instructions on their site and you will be good to go. After you have MinGW installed you will probably want to add entries to the Windows PATH variable so that when you are in a terminal you will be able to directly access MinGW. [As a note I still run Windows XP so these instructions will be done with that in mind] Doing this is quite simple:

  1. Right Click My Computer –> Properties –> Advanced –> Environment Variables.
  2. You will see User and System variables. If you want all users to use MinGW edit the Path entry in there. Otherwise add a Path variable under User variables.
  3. Go to the Path variable from step 2. Add an entry to the Path that points to MinGW’s bin directory. For me this was “C:\MinGW\bin” but it all depends on where you installed MinGW. Note: Entires in Windows Environment variables are separated by ‘;’
  4. Click ok a bunch of times

While you are changing environment variables you will want to add the main directory for Python to your path. For me this was: “C:\Python25”. This made my final User Path entry: “C:\MinGW\bin;C:\Python25”

Now that you are done changing environment variables you will want to reboot your computer so that Windows gets your changes. Really… you will regret it if you don’t reboot!

Now download PyCrypto and unpackage it. This turned out to be problem enough because not many people keep programs around that open .tar.gz files. Since am a Linux user I just unpacked them in Linux and transferred them to Windows (I run Windows XP under VirtualBox ). However, if that is not an option you can download and install IZarc Archiver. IZarc has been my favorite [Windows] extraction tool for a long time and it supports just about every archive format imaginable.I would recommend extracting the contents of PyCrypto to your desktop. Now is when the first starts!

First, open up a terminal. We will only be using a few commands and you don’t have to be a terminal wiz kid to do this. I normally open a terminal by going: Start –> Run –> “cmd” –> Press Enter

Now that you have a terminal up you want get to the PyCrypto directory. To move between directories in the terminal we use the “cd” command.

cd PATH-GOES-HERE

So, for example here is the command I used to get to where I had PyCrypto:

cd C:\Documents and Settings\Jinto\Desktop\pycrypto-2.0.1

Of course that command will only work for you if your user name is Jinto and you extracted it to the desktop exactly like I did 😉 So modify it to fit your needs. Once you are in the pycrypto directory do a quick:

dir /w

To make sure you see files. Specially make sure setup.py is there. We need to run that setup script and tell it to build the libraries. To do this issue the following command:

python setup.py build -c mingw32

If all goes well it should build without any troubles and all that is left is to install it by issuing the following command:

python setup.py install

You should be all set to use PyCrypto now, or to install ezPyCrypto which uses PyCrypto. As a bonus feature if you are looking to make an executable installer for PyCrypto you can run the following command:

python setup.py bdist_wininst

This will leave you with a .exe file located in the “dist” folder.

I hope this helps everyone get PyCrypto running under Windows. For those of you Linux users who think I am leaving you out check your distro specific respository system they probably already have a package for PyCrypto!

Advertisements

Java3D, the PickRay, and the PickCylinderRay

July 14, 2008

Today I was struggling to figure out why I could not highlight a line segment in Spheriosity. After much fiddling around I discovered that the problem only occurred when I was using a PickRay to select objects instead of my normal PickCanvas. For those that don’t know a PickCanvas is made to pick objects based on the x and y coordinates of the cursor and a PickRay has to be defined with a starting point and vector indicating its direction. I was a bit disappointed because I thought I had already solved this problem. The general problem here is: ‘How do I take the cursor location and select something in a Java3D BranchGroup?’

My final solution to this problem was to use a PickCylinderRay, which defines an infinite cylinder going in a defined direction from a defined starting point and with a defined radius. Switching to the PickCylinderRay actually solved more than one of my problems. Previously when I was using the PickCanvas I found it very difficult to select objects at times. Especially the line segments (LineStripArrays) . I know that you can set different tolerances to the PickCanvas but I was having a heck of a time getting that happy. The only problem I ran in to with the PickCylinderRay was that if the radius was too large it would pick objects that weren’t even close to the cursor. However, this makes sense 🙂 Here is what the code looks like to define the vector from the camera to the cursor:

public void getCameraToMouseVec(Canvas3D myCanvas, Point clickPos,
                                Point3d cameraPoint, Vector3d dir)
{
     Point3d mousePos = new Point3d();

     //Getting current mouse and camera locations
     myCanvas.getPixelLocationInImagePlate(clickPos.x, clickPos.y, mousePos);
     myCanvas.getCenterEyeInImagePlate(cameraPoint);

     //This block of code converts our image plate coordinates out to virtual world coordinates because
     //that's ultimately what we care about.
     Transform3D motion = new Transform3D();
     myCanvas.getImagePlateToVworld(motion);

     //We do this convertion for both the camera and for the mouse position.
     motion.transform(mousePos);
     motion.transform(cameraPoint);

     //Get the three components of the vector going from the camera location to the eye
     dir.x = (mousePos.x - cameraPoint.x);
     dir.y = (mousePos.y - cameraPoint.y);
     dir.z = (mousePos.z - cameraPoint.z);
}

So after this function completes the variables cameraPoint and dir will be set to the correct values to call .setShapeCylinderRay() of the PickTool class . Here is a sample of the code which takes the output of the previously displayed function and uses it to make a selection:

public void mousePressed(MouseEvent arg0) //Called be Java through MouseListener interface
{
     //In Spheriosity I actually declare these once for the class and reuse them
     Point3d tmpPoint = new Point3d();
     Vector3d tmpVec  = new Vector3d();

     //I assume that 'myMainBranchGroup' is a member of the class that this function is in
     //and is the BranchGrounp we want to be picking from.
     PickTool myPicker = new PickTool(myMainBranchGroup);

     //I also assume that 'myCanvas' is a member of this class
     getCameraToMouseVec(this.myCanvas, arg0.getPoint(), tmpPoint, tmpVec);

     //Use the data gathered from getCameraToMouseVec to make the PickCylinderRay
     myPicker.setShapeCylinderRay(tmpPoint, tmpVec, .1f);

     PickResult result = myPicker.pickClosest();

     ... code to handle what was selected ...
}

There you have it! That is how I solved my little problem with PickRays not working. I’m glad I stumbled upon the original problem because I truly think this solution works much better given the rest of the code base in Spheriosity.

There are a few things to note here. One is that the .1f in the .setShapeCylinderRay() function should be changed depending on your needs (it determines the radius of the cylinder). Lower values mean the cursor has to be closer to the object higher values mean the cursor can be further away. Also the output of the getCameraToMouseVec() function could also be used to define a PickRay if that is truly what you want.

Hopefully this was able to help one or two people trying to select things in Java3D. If not… well at least it serves as a bit of rationale for some of what I have done with Spheriosity 😀


Spheriosity and Efficiency

July 12, 2008

I spent some time today trying to add, what I think, would be a neat feature in Spheriosity. The feature would let you see the line or circle you are about to create when you are trying to make new lines and circles. Currently you just select the end points and then the program draws the line. With this feature, after you select the first point Spheriosity will show you were the line or circle will end up after you place the second point.

I figured this would be pretty simple and that I wouldn’t run into any of the efficiency issues I constantly worry about. Unfortunately I did… This took me completely by surprise because with the point moving feature everything seems to happen quite smoothly! So it was tough for me to believe that now this feature was slow.

I further inspected the code to find that the difference was in how point moving vs. how this feature was implemented. The way the code is currently structured is that all user related activities are fed through a state machine. The different states are like MovePointState , DrawPointState etc.. Each state extends a base class with 6 or 7 (can’t remember exact number) abstract functions. The class which receives all mouse related events then forwards the appropriate data to the functions of the current. Any given state can ignore functions it does not need by simply implementing them as an empty function block:

public void addPoint(Point3f newPt) {}

The function I currently use to get data from the mouse have the following function headers:

/*
 * @param mouseEvent - MouseEvent sent by Java
 * @param worldLoc   - Pointer location mapped to the surface of the sphere
 */
public void newMouseLoc(MouseEvent mouseEvent, Point3f worldLoc);

Java can give you the new location of the mouse in one of two different ways, but now both at once. By implementing the MouseMotionListener interface there are two functions. mouseDragged and mouseMoved. Even though the mouse is being moved while you are dragging in Java choose to implement it such that only one function or the other is called at once. So both mouseMoved and mouseDragged call my function [newMouseLoc]  . In mouseMoved I house the code which determines what lines or points get highlighted depending on the current state. In mouse dragged I simply pass right along the data to newMouseLoc.

So, the MovePointState gets the data from mouseDragged where the only overhead is the calculation I have to do in order to convert the mouse location to the surface of the sphere (quite a large calculation…) Where as in mouseMoved I did the same calculation as in mouseDragged and also did an intersection test with Java3D to see what, if anything, needed to be highlighted. Since part of that intersection test probably involved math that was already done to map the the user click to the sphere… Now… if this calculation was just happening once or twice I’m sure you would never notice. Unfortunately it happens every time the cursor moves. That in itself doesn’t make the the program lag though. Also added to the mix is that to draw each line we really make a bunch of small lines (100 to be exact) because a line on the sphere isn’t the same as the Euclidean version Java3D provides. So now we have that calculation coupled with the code that figures out the end points for the 100 lines and at the very least a for loop with 100 iterations … all each time the cursor moves!

This is why I worry about optimizing Spheriosity. I do my best to keep the code clean, but no one wants to use a slow program.

In terms of a solution to this problem… I moved some of the calculations around so I am doing less math to figure out what to highlight or not, and that seems to solve most of the issues. I still want to optimize it some more as I think I am approaching a dangerous threshold for minimum requirements 🙂 to run the program.


BROWNIES! — The Programmer’s Moment

July 8, 2008

There I was… testing my Python scripts to remotely adminster the radio station computers. I had been coding for a few hours trying to add support for remote admin logins. This required having the server send a public encryption key, letting the client get the potential password from the user, encrypting it with the key, sending it back to the server, having the server decrypt the potential password. Then… I saw it: “Password is:brownies”  shining brightly in the terminal running on the server. Proof of success at long last!

It is that feeling that I call ‘The Programmer’s Moment’ it’s a feeling of overwheling success because you just spent x amount of time on a feature, and you are finally convinced it is going to work like you planned. I am sure other professions experience this same sense of accomplishment, but I experience this most frequently when I am coding. It’s that feeling that makes long nights (or days) staring at a screen worth anything at all. It can motivate you to continue coding and keep you from giving up on a project that seems hopeless.

Here is a little screen shot of that terminal with such simple text that brought some happiness to this crazy coder 🙂

Finally... got the password being sent over the network

Finally... got the password being sent over the network


Python and public/private key encryption

July 5, 2008

Edit (7/20/08): If you are looking to get PyCrypto running under Windows with Python 2.5 I wrote a short guide located here . There are already installers for Python 2.3 and 2.4 on the PyCrypto Homepage

Edit (7/10/08): After noticing that this got some hits on google I rearranged the content so that the first part explains how I got public/private key encryption in python and the second part explains why I wanted encryption in the first place.

I was looking for an easy way to do public/private key encryption in python and I found it!

Enter ezPyCrypto! This package was actually able to generate and export a public key as a string AND import it back in. I should also mention that ezPyCrypto is merely a wrapper for PyCrypto. I have to say ezPyCrypto was quite uh… easy. Here is an example of how you would make a 1024 bit RSA key and export it to a string. It also demonstrates how to import that key and use it to encrypt a message:

from ezPyCrypto import key

myKey = key(1024) #defaults to RSA, 1024 is bits to encrypt with

#Just calling exportKey will export only the public key
publicKey = myKey.exportKey()

#print publicKey  

#Then to load it back in to a new key
myNewKey = key(0)
myNewKey.importKey(publicKey)

#Testing time!
testEnc = myNewKey.encString("cheesecake tastes yummy")

#You should see 'cheesecake tastes yummy'
print myKey.decString(testEnc)

Look at how easy that is! I am quite pleased that I will be able to continue my work, and have some good encryption. The one thing I will say is that ezPyCrypto does not seem like it has seen much development in a while, but oh well. As long as it works when it needs to I will not complain 🙂 Also I only tested this with python-2.5 so your mileage may vary. On to the back story…

So today, among other things, I was looking for some good python libraries to do public/private key encryption. I am trying to write a few scripts that help with the administration of the computers I work on for my college’s internet radio station. I started out writing them all in perl, but now I have most of them written in python. Currently I have written a small backend which starts up the basic services that the radio station requires. At present that consists of: shoutcast server, darkice, a script to monitor the listener counts, and lighttpd. This backend is capable of accepting clients over the network and displaying the basic running status of all these services (IE: running or not). Now, I would like to add the ability to remotely control the services. Hence I would have to have some way of authenticating users who want to log in.

Lets take a step back here. Some of you might be wondering why I don’t just administer everything with ssh. Indeed, this is how I did it for a year, and for someone like me (I loves me a good terminal) this was not a problem. However, I have to face the reality that someone will probably take over the station (at some point or another) who is not at one with bash. This is why I decided to start this project to help ease them into administering the servers, which rarely die anyway. Python and the Tkinter libs were great for this because they would be fully cross platform. Although I hate to admit it, I think the next admin will probably be a Windows user. I think being able to run the server from their dorm will go a long way in helping them accept the job.

Back to the main problem. To authenticate I don’t plan to have anything too breathtaking. The backend will be run as a normal user and I plan to use PAM to authenticate against the user that is running the backend. So I need to be able to send the password over the network! There are plenty of guides to public/private key encryption and a quick google search will provide an explanation better than I will ever be able to give as to how they work. I was able to find a few options. The first is the PyCrypto package. This package seemed quite sufficient but the lack of reasonable documentation for it made me have second thoughts. This isn’t something I want to mess up because we are sending sensitive data across the network. I had a look at the source code but got tired of trying to make sense of it.

Next I found a package called yawPyCrypto (yet another wrapper for python crypto). That seemed quite promising. I was able to work out a simple example where it would generate a 1024bit RSA key pair, and I could encrypt and decrypt a string with it. Only one problem… I could not successfully export and import the string from yawPyCrypto. This meant the server would not be able to the send the client its public key… LAME. Again, I poked around the source code, but I didn’t see anything obvious, and I was not about to reinvent the wheel here.

Then I discovered ezPyCrypto, which as you read from above, solved all my problems 🙂 I should also mention that on my Ubuntu machine I had to play around with the source code in ezPyCrypto to get it to import properly (remove references to variables it couldn’t find). Yeah… sounds stupid, but I was confident that what I was playing with was not related to public/private key encryption which is all I wanted.

I already fear that no matter how much I automate things there will be a problem. That is the way it goes however. I am hoping that I can polish off these scripts soon so I will not have to worry about radio station stuff until the beginning of the school year. I will most likely have more to say about these scripts and maybe a summery of everything I have done with them. That’s all for now though 😀


Spheriosity Alpha 2 Finally Released

July 4, 2008

For at least the next month or so there will be quite a few articles that at least mention Spheriosity. I’m not sure if I’ve mentioned this, but it is my project for the summer. I am trying to make it the best application that I can, given my time frame. I have to say that I have made it further in development than I ever thought I would. I find this quite pleasing and I hope to keep up the pace for the rest of the summer. I have a vacation in there and I imagine that development will slow, or stop, during that time. That is a bit of a shame, but I think it is probably for the best. It will be good for me to take a break and not have to work on ANYTHING. Hopefully I will be able to enjoy that as I sometimes find it rather difficult to truly enjoy free time.

At any rate, this post is about Spheriosity. So… what is new in this version? Basically loads of new features and one giant memory leak. Yes yes, a memory leak in Java. I know, it’s not possible! IT IS. I will talk about that some other time. So here is a rough list of the new features I have added:

  • Parallel Transport
  • Point Rotation
  • Full support for saving files (not supported in applet, because it can’t be)
  • Point moving
  • Small interface improvements
  • Ability to measure triangle area as a % of total spherical area.

There are even more features than I listed, but I would consider those the main and most notable features. I have to say that I was a little reluctant to release Spheriosity in alpha status, but I want it to be out there for people to try and test. I one heard the saying ‘release early and release often’. As long as users are willing to put up with the bugs I think it is better to get their feedback. After all… who is using your software? Though Spheriosity is small and I wouldn’t say it even has a real user base I care immensely about what users think about it. I don’t even know if it would be possible for me to use the program to its fullest potential. As such I want to know what others think about it.

In terms of the documentation I wrote about yesterday. I finished that all up and released that as a rough draft under the GNU FDL. I must admit that I didn’t really want to license the documentation mainly because I find it to be quite a hassle. However it is only a one time cost so I figure it is for the best. I really want to make sure people can use the program 🙂 Funny how I think that right?

I got to do a little shell scripting and made myself a little bash script to collect the files from the repository and build all of the .zip and .tar.gz files that I need to post on SourceForge. That was one of those times that I was soooooo glad I had thought ahead to write a script. This time around all I had to do was make some minor changes and it was so nice to just run a small script and it do 90% of the work for me.

I leave now with a screen shot I made from the new application. To make the screen shot I used the parallel transport and rotate point features that are included in the new version. You can’t tell from just looking at the picture, but if you move point B both ‘eyes’ move the same way. I thought it was a real cool example of what the work I have done is capable of. Granted… it is not a very practical use of the software, but that is another story. Maybe some day I will write about parallel transport and how we (my professor and I) were able to achieve this in Spheriosity 🙂 At any rate here is that screen shot:

Screenshort From Spheriosity Alpha 2

Screenshort From Spheriosity Alpha 2

🙂


My first real experience writing user docs

July 4, 2008

So today at 10:00am I embarked upon my first real attempt at writing user documentation. Spheriosity has been long overdue for some good user documentation. After all, the whole purpose of the application is to help people understand spherical geometry, so if they can’t use it then the program is worthless.

I decided that the best choice for writing the documentation was going to be LaTeX my favorite text processing language. I fell in love with LaTeX when I saw how it could format mathematical equations with ease. I use it almost 100% these days and it makes the most beautiful looking documents! It is suited perfectly for a code oriented person like myself as well because at times it feels like I am programing the document more than anything. Perhaps I will write more about LaTeX some other time.

Also because of LaTeX I realized that I might be able to run latex2html on the document and thus have something I could post on the internet. I have never been a complete fan of the output of latex2html mainly because I think it should have an option to let you view an entire section at once. I’m sure that for some things it is quite nice that it splits up into subsections and then subsubsections if you use them, but for this it doesn’t translate well.

I found that I don’t really mind writing user documentation as much as I thought I would. It’s kind of like commenting code to me. I know a lot of people don’t really enjoy commenting, but it never bothered me much. I feel like it is taking the necessary steps to ensure another human, or possibly myself, will be able to some day figure out what I was thinking. After spending lots of time writing a piece of code it seems worth it to put in that extra effort. I feel the same way about documentation. If it is able to help even a few users work with my application then I consider it well worth the effort. So far I have created 28 pages of documentation. Now, it is not solid text because there are pictures of the program as I try to visually walk the user through certain steps.

I spent a good 10 or 11 hours writing the documentation and I need to take a break from it for sure 🙂 Tomorrow will be a new day and I hope to finish the documentation and place the new alpha build on SourceForge. I don’t know if I could write user documentation all day every day, but it is good to know that I can do it when I need to. I admit that I felt a bit bad not writing any code all day (except a small perl script), but it was for a good cause.

I had a bunch of good ideas about things to write about today so hopefully I will have some material for the next few days. That’s it for today though… I need to get some sleep 🙂