Wednesday, August 26, 2009

A Matrix Sim in JavaScript

Do you ever find yourself working on a large project with a long lead-in time before you have visible results to show? I had been frustrated by this feeling recently. I had been focusing on a personal project which will likely be a long time in the making, so I decided to pick up a short, simple, and quick project to keep things lively and fresh. So our story begins.

During high school, I was wowed by the movie The Matrix. The special effects were mind blowing (for their day) and the story line was something I could really get behind (not so with the sequels, but I digress). I imagine this is how my elders felt when Star Wars was first released.

One of the most memorable visuals in the movie is the code of the matrix zipping by vertically on the screen in a retro green on black. Cypher, makes a statement, which seems to be a favorite among programmers, that he no longer sees the code itself, but what it represents. He has become fluent to the point of transcendence - a geek nirvana experience.

At this point in my life I had written a few moderately complex programs, mostly using Turbo C, and I thought: "Why, I could make my computer look like the matrix." It was one of the more involved programs I had written at the time and I was quite pleased with the result. I brought it in to the computer lab at school, loaded it onto all of the machines, and had the whole lab running it at once. I'm pretty sure that Ben, Yed, and Roman G. remember it. Sadly I doubt the code would run on the DOS emulator that ships with Windows XP and higher. I recall vaguely that Roman ported it to Win32 or something in our high school computer science class. Now that I think about it, I guess you could say that this was my first open source experience.

In thinking of something quick and easy to create with some immediate visual payoff, I decided to recreate my little DOS matrix sim, but this time in the browser. Behold: The matrix in JavaScript.

It turns out that the CSS rules needed to keep columns at a fixed width and height was actually quite complex and took longer than expected to get right. The vertical line height was still elusive in Chrome and Safari, but worked in Firefox and Opera. I don't think that this works yet in Internet Explorer. Though if past traffic is any indication, none of you would have noticed had I not said something :-)

Many thanks to Scudmissile for his mighty CSS kung fu. Oh, and that whirring sound that you hear may be your CPU fan trying to take flight. Apparently rendering a vertical text flow like this using DOM manipulation, CSS, and Math.random may be a bit more processor intensive than my obsolete C code.

Saturday, August 01, 2009

A Test Client for App Engine

I created a simple utility library in Python to help anyone debug their App Engine application. Many of the App Engine apps that I've seen use HTTP, HTML form posts, and the Users API and the easiest way to test these features is to fire up the web browser and click through the web pages generated by the app. However this can be a bit slow and repetative and it is difficult to determine exactly what is being sent over the wire (though this is greatly helped by using wireshark, fiddler, tcpdump, or antoher network packet sniffing tool).

Enter my little App Engine HTTP module. It provides a simple interface for making arbitrary HTTP requests and will print the full request and response to the terminal (though you can turn the noisy printing off if you want). Download, copy the file to you working directory and try it out in your Python interpreter.

For our first demonstration, let's try to visit the Google search page.
import http
client = http.Client()
resp = client.request('GET', '')
You should see your request and the server's response (with the HTML for the Google Search page) in your terminal window. This should work with just about any website out there.

Other HTTP debugging tools can show you the request and response like this, but I find that this kind of simple Python client can be useful in writing end-to-end or integration tests which contact your App Engine app remotely.

Along those lines, one of the things which standard HTTP debugging tools do not provide, is a way to sign in to an App Engine app with a Google Account so that the App Engine Users API can identify the current user. I wrote an extremely simple app which illustrates the Users API, try it out here:

After signing in, the page should simply say, "Hello yourusername (" You'll notice that during the sign in process, you signed in on and were asked to approve access to the app. This kind of interaction works great in a browser, but can be tricky when you are using a command line, browserless, client.

It is possible however, to sign in to an App Engine app without using a browser. You can use the same technique used in appcfg, use ClientLogin and use the authorization token to obtain an app specific cookie which indicates the current user. This simple HTTP library can do this for you and all subsequent requests will use this cookie to tell the App Engine app who the current user is. Try it out by making the request to the simple user app that you visited earlier:
import http
client = http.Client()
resp = client.request('GET',
print resp.body
You should see the following text displayed in the terminal:
Hello, yourusername (
You can use the appengine_login method with your own app, just change the argument to the App ID of the app you want to access.

Along with simplifying access to apps which use Google Accounts, I wanted this library to simplify the process of using another feature used by many web apps: HTML form posts. Now I'm certain you've used HTML forms before, here's a simple example:

The above app uses both the Users API and a simple form. As an alternative to visiting this page in the web browser, you can post your shout-out using the following:
import http
client = http.Client()
client.request('POST', '',
form_data={'who': raw_input('From: '),
'message': raw_input('Message: ')})
If you've even wondered what gets sent across the wire to post on a form like this, look back in your terminal to see the request from your computer and the response from the server (this is of course just the HTTP layer, wireshark will show you traffic on the IP and Ethernet layer as well).

That's really all there is to it. I designed this as just a simple script to use on the command line and I wrote it in less time than it's taken me to write this blog post about it (I borrowed atom.http_core from the gdata-python-client as a foundation). With some tweaks to remove the interactive (getpass and raw_input) calls and replace them with parameters, I could see this module as a utility layer in a larger, more complex, App Engine client application. If you're creating on I'd love to hear about it ;-)

For more information on how the appengine_login method works behind the scenes, see this presentation I gave a few months ago:

Many thanks to Trevor Johns and Nick Johnson for helping me to understand how this ClientLogin-to-cookie exchange works.

I'm sure that App Engine's Java runtime users would appreciate a port of this simple library to Java, if you feel so inclined.