Saturday, May 28, 2011

Go on App Engine Example - Part 1

The App Engine team recently announced support for Go as a runtime for use in apps. Summary up front, the App Engine SDK for the Go runtime is the easiest way I've found yet to get started with Go. As I change my code, it is recompiled in the background when I make a request to my app, so it feels very much like developing in a scripting language.

I've been excited about the Go language for some time now (specifics on why will have to wait for another post) so I was eager to try it out in one of my favorite platforms: App Engine. I wanted to start with something small, so I wrote a simplified version of a web app that I've been itching to write lately, a site for hosting plain text content. Specifically, I want something that preserves whitespace, allows me to line up columns of text, and supports non-English characters (Unicode). Those are the kinds of things I need to share and talk about code. Also there is a great deal more you can do with plain old monospaced text, maybe you'll find this useful as well.

With that objective in mind I give you the Plain Text Machine. This little app lets you enter a small amount of text, somewhere around 2,000 characters, and gives you a link that others can visit to see an HTML reproduction of your writing. I mentioned I wanted to keep this simple, so here's the odd little bit, this app doesn't store your text anywhere. The URL that is generated contains the text, hence the somewhat low limit on message length. It certainly keeps the app simple, the most complex logic is that which converts the text from the URL into HTML.

A request starts by hitting the Init function:
func init() {
http.HandleFunc("/", handle)
http.HandleFunc("/show", show)
The main page, at /, is just static content, we're just interested in the /show handler. It looks like this:
func show(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/html; charset=utf-8")
// Get the message from the URL.
PrintHtml(utf8.NewString(r.FormValue("msg")), w)
The above does two things, sets the content type of our response so that the browser will know it is HTML, and reads the message URL parameter from the request to convert it to HTML.

The PrintHtml method prints out some boilerplate HTML then reads the message one character at a time and converts each character to its HTML-safe equivalent. There's a tiny bit of complexity to make sure that the whitespace is preserved instead of being collapsed as would normally be done with repeated spaces in HTML. Here's the code:
func PrintHtml(text *utf8.String, out http.ResponseWriter) {
spaces := false

fmt.Fprint(out, textHeader, middle)
for i := 0; i < text.RuneCount(); i++ {
currentChar := text.At(i)

if currentChar == 32 && !spaces {
// A first space.
fmt.Fprint(out, " ")
spaces = true
} else {
if currentChar == 32 {
// Space following another space
fmt.Fprint(out, "&nbsp;")
} else if currentChar == 10 {
// Newline
fmt.Fprint(out, "<br>")
} else if currentChar == 9 {
// Tab
fmt.Fprint(out, "&nbsp;&nbsp;&nbsp; ")
} else if currentChar == 38 {
// &
fmt.Fprint(out, "&amp;")
} else if currentChar == 60 {
// <
fmt.Fprint(out, "&lt;")
} else if currentChar == 62 {
// >
fmt.Fprint(out, "&gt;")
} else if currentChar < 31 || currentChar == 128 {
// Skip control characters.
} else if currentChar < 127 {
fmt.Fprintf(out, "%c", currentChar)
} else {
fmt.Fprintf(out, "&#%d;", text.At(i))
spaces = false
fmt.Fprint(out, footer)
The textHeader, middle, and footer variables are string constants containing the wrapper HTML which gives style information.

If you're interested in the full source code for this tiny little app, you can find it in the Plain Text Machine open source project. Hopefully this example provides an easy to understand picture of what Go code for App Engine looks like.

I had quite a bit of fun putting together this app. By keeping it simple I was able to go from idea to done in less time than it took me to write this blog post. As an added bonus, having an app with no persistent storage brings up some interesting philosophical questions. For example, if a message is created but no one stores the link to it, does it still exist?

Monday, May 09, 2011

Setup OAuth2 for Google APIs

Today I'm at I/O Bootcamp and helping out with a walkthrough on how to get starting using Google APIs in a variety of languages.

One of the things I appreciate about the Google APIs is the authorization mechanism which lets me see which applications I've granted access for my data and allows me to revoke access. As an application developer, there are some things that I need to do to identify my application so that Google knows which app is requesting access so that it can show the user more information about my app. The first step, then, in writing an application that uses OAuth2 is registering your app.

You can begin the registration process on the Google API console by creating a project:

Now that you have an application, you'll need to configure it for use with OAuth2 and get the secret tokens that your application will use in its requests. For that create an OAuth2 client ID.

The most vital decision to make during the sign up flow is if your application is a "web application" or an "installed application". If you're a site accessed in a browser and you're able to send the users to a Google web page for authorization and then have the broswer redirect back to your app, then you want web application. For an installed application, the user will still need to authorize your app by visiting a web page, but once authorization is complete, the secret token will be sent to the app either using a redirect to a local running web server or by having the user copy and paste the secret into your application.

For the command line samples I've been playing with I choose installed application.

After creating the client ID you should see information something like this

Client ID:
Client secret: Amzz5Yip2SJPqqq5Jx
Redirect URIs: urn:ietf:wg:oauth:2.0:oob

You'll need to put this information into your application so that it can use the client ID and secret when making requests to get an authorization token from the user. This can be as simple as copying and pasting these strings into your code.

The one other thing that needs to be done before you begin using OAuth2 with one of the Google APIs is to turn on the API for your application. This can be done on the developer console "Services" section.

Let's say that I wanted to access the URL Shortener API. First I would need to enable it for my application.

Then I would need to specify the URL Shortener's API scope when I request authorization from a specific user. The scopes that are requested by my app are turned into a list of APIs that the user must grant access to when they authorize my application.

The scope for an API can be found in the API documentation under authorization.

For an example that brings all of these settings together, see the example:

FLOW = OAuth2WebServerFlow(
credentials = run(FLOW, storage)
http = httplib2.Http()
http = credentials.authorize(http)

Python may not be your bag, but no worries, there are client libraries for the Google APIs in a variety of languages, and even better, there is an API Explorer that lets you try out the underlying protocol without any language specific stuff getting in the way.

For example, here is getting details and stats about a short URL:

And here is creating a new short link:

For all the details on using these APIs, take a look at the documentation. For example here are the URL shortener docs. The common first step for almost all of the Google APIs that access user information is the registration step we started with. For more details on OAuth2 with Google APIs, there is some excellent documentation here.

Tuesday, March 08, 2011

Free Verse #397

Sipping a late,
writing a free verse poem.
Hey look, a haiku!

Sunday, January 23, 2011

Mercurial in Five Commands

Completing any sort of significant programming project can be nearly impossible without version control. For a class project, I recently collaborated with a small group on a large programming assignment. To keep all of our changes in sync while working miles away from each other, we used a centralized version control system.

Since I've been working quite a bit with Mercurial lately, we went with a free private project on bitbucket. It was the first time my teammates had used Mercurial so I gave a crash course that I thought would be helpful for others as well. A distributed version control system has a large number of commands and features, but 90% of the time, you're just dealing with the basics. When working with a team you can get by with just these five Mercurial commands: clone, commit, push, pull, and update.

Setting up the Project

As I mentioned, we were using a hosted repository on bitbucket, so getting a working copy of the repository on our machines started with each of us executing hg clone. It looked a little something like this:
hg clone
This will create a local repository with a copy of each file and you're now free to make changes. You can edit files locally and Mercurial will track the changes for you. If you want to add new files or remove existing files, make sure to use hg add and hg remove. (Bonus commands!). Once you're happy with your changes and you want to gather them up in a logical unit it's time for:

Local Snapshots

To save a set of changes locally, you use hg commit. Be sure to write an informative description of this change set since you and others will want to remember what this set of changes was all about. There are a few arguments to the commit command that come in handy, often I do:
hg ci -u your-username -m 'Reduces codebase entropy. All tests pass!'
For more options on the commit command and any other Mercurial command you can use hg help <command>. You can also see a diff of your current files compared to the most recent snapshot using the hg diff command. (Bonus times two!)

Pinning your changes locally is a fantastic feature of distributed version control systems. As I work I tend to take a local snapshot several times an hour. These revisions just exist in your local copy, so you don't need to worry about clashing with other people's code at this point or making sure that all changes are usable. Sometimes when I decide I've gone down the wrong path I'll take a snapshot of the ill conceived changes before I roll them back, just in case I decide later on that some of the ideas weren't so bad after all.

Once you have something that's ready for others to use, it's time to:


To get your changes into the hands of others, you can push them back up to the central repository. You do this with the hg push command. Since we created our local repository using clone, your local copy knows where to send the changes. Also, if you'd like to do something different, like push your changes to a different location than where we cloned from, you can take a look at more options of the push command using hg help push.

The push command will upload all of your local commits, along with those handy descriptions, for the rest of your team to see.

Now others on your team are pushing up their changes too and it would be great to get their changes so you are all working on the same code. What's that saying, "It's better to give than to...?"


To get the changes others have posted to the repository into your local copy, you use hg pull. Running this command copies the change sets that you haven't received yet to your local repository, but it doesn't edit your files or apply the changes just yet. Since you might want to be selective about what changes you apply, Mercurial splits applying the changes into two steps. First you pull, then you use hg update. When called with no additional arguments like that, hg update applies all of the changes.

I recommend pulling and updating often. In our group, since there were just a few of us, we'd post in a chat room when we pushed changes so others would know to pull. If you push before you pull in other people's changes, you might need to use hg merge. Also, you can see all of the changes which are in your repository with the hg log command. (Triple bonus, hey a hat-trick!)

There you have it!

Want more?

I tried to keep this simple and focused, the bare minimum to work on a project with a small team. Mercurial has more to offer. I haven't touched branches yet, or looking at diffs, or creating your own local repository from scratch, or sharing your changes by running your own server locally. All of these are just single commands! For further reading take a look at Mercurial: The Definitive Guide.

There are a few services which offer hosting of Mercurial repositories. For open source, Project Hosting on Google Code is an option as well as bitbucket which I used for the first time this past week. Any other Mercurial hosting providers that you recommend?