Wednesday, May 15, 2013

Easy Raspberry Pi Setup (Using Windows)

So there I was, new Raspberry Pi in hand. Now to set it up. One small setback, I don't exactly have a monitor, keyboard, and mouse handy. How did I end up with so many laptops?

Turns out setting it up and getting to a command line was a breeze, no monitor (etc.) required. All it took was an Ethernet cable and my router.

Here's what I did.

First, I wrote the operating system disk image to an SD card using the instructions on the Raspberry Pi Quick Start Guide.
  • Downloaded the latest image from raspberrypi.org/downloads
  • Extracted it and found the .img file
  • Installed Win 32 Disk Imager
  • Plugged in the SD card
  • Used the disk imager to write the .img file to the drive for the SD card
Second, I started up the Raspberry Pi to connect to it.

I plugged the Raspberry Pi into the router using that Ethernet cable, then powered it up!

The next step will depend on your router but some routers will show you a list of the devices that are connected to the network from the admin screen. Here's an example


Armed with the IP address, you can ssh into the Raspberry Pi machine. (I used PuTTY.)


There you have it. From the command line you can configure whatever else is needed and you'll never have to scrounge up one of those pesky keyboards.

Thursday, July 19, 2012

waitFor JavaScript

In the spirit of building the simplest thing that could possibly work, I revisit the concept in another of my posts: simple asynchronous JavaScript.

Time and time again I see web developers relying on the order that JavaScript loads in a page to ensure that the site will work. This keeps the mental model simple, "all the stuff above this code will be available here", but it means that people trying to speed up browsers and JavaScript interpreters have their hands tied. Why not instead load and execute as much JavaScript in parallel as possible, like a mutlithreaded application, and wait for your dependencies to load or preconditions to be met before your code runs.

You could do it all in just a few lines:

function waitFor(condition, callback) {
  function waiter(condition, callback) {
    return function() {
      var condMet = false;
      try {
        condMet = condition();
      } catch (e) {}

      if (condMet) {
        callback();
      } else {
        setTimeout(waiter(condition, callback), 5);
      }
    };
  }

  waiter(condition, callback)();
}

Using it looks a bit like this

waitFor(
  // condition
  function() {return window.dependency;},
  // callback
  function() {
    // do things using dependency.
  }
);

For example, here's how you might set this up to check if jQuery has been loaded and that a particular element on the page has been loaded in the DOM.

waitFor(function() {return $('#msgid');}, function() {
  $(document).ready(function(){
    $('#msgid').html('jQuery says "hi"!');
  });
});

Now it doesn't matter what order your script tags are in. Of course, you'll get the best performance if jQuery gets loaded before this waitFor call is made. The wait and retry loop adds some cost. If browsers had an event model available for script loading or conditions then we wouldn't have to poll using setTimeout/Interval. But for existing browsers all the way back to IE 6, this simple helper function will work.

For managing JavaScript dependencies there's also require.js which does quite a bit more (and is larger). I wrote this to be lighter weight and it is more general and flexible than a framework like require.js. Using waitFor, you could set up logic that is triggered if and when certain arbitrary conditions become true.

One little improvement, if you are waiting for lots of things then you would probably want to replace multiple timers with a single timer that will check all conditions. Like this:

function waitFor(condition, callback) {
  waitFor.waitingFor.push([condition, callback]);
  waitFor.check();
}

waitFor.waitingFor = [];

waitFor.check = function() {
  var stillWaitingFor = [];
  for (var i = 0; i < waitFor.waitingFor.length; i++) {
    var condMet = false;
    try {
      condMet = waitFor.waitingFor[i][0]();
    } catch (e) {}
    if (condMet) {
      waitFor.waitingFor[i][1]();
    } else {
      stillWaitingFor.push(waitFor.waitingFor[i]);
    }
  }

  waitFor.waitingFor = stillWaitingFor;
  if (stillWaitingFor.length > 0) {
    setTimeout(waitFor.check, 5);
  }
};

Here's a minified version. I added some newlines for formatting which you can take out.

function w(c,b){w.w.push([c,b]);w.c()}
w.w=[];w.c=function(){for(var s=[],b=0;b<w.w.length;b++)
{var d=!1;try{d=w.w[b][0]()}catch(e){}
if(d)w.w[b][1]();else s.push(w.w[b])}w.w=s;0<s.length
&&setTimeout(w.c,5)};var waitFor=w;

Happy coding!

Friday, May 11, 2012

JavaScript Caching: appendChild vs document.write

I found myself trying to prefetch some data which would appear in an iframe. I wanted to see if requesting the same JSON in the parent page and then in the iframe would cause the second request, made in the iframe, to hit the browser's cache in all cases. It turns out that for Internet Explorer and Web Kit based browsers it depends on how you request the JSON in the parent page.

If you use appendChild in IE the identical JS request will hit the browser cache.
If you use document.write in IE it will not hit the cache.

However, in Chrome and Safari if you appendChild in the parent, you will miss the cache. Using document.write in the parent page causes a cache hit.

Here are the results I gathered.
Cache Hit? both document.write parent document.write
iframe appendChild
parent appendChild
iframe document.write
both appendChild
Internet Explorer miss miss hit hit
Firefox hit hit hit hit
Chrome hit hit miss miss
Safari hit hit miss miss
Opera hit hit hit hit
Android ? ? ? ?
iOS ? ? ? ?

All versions of IE behaved the same, I didn't check different versions of the other browsers.

To test this I wrote a test web application called json-caching-exploration. It's a simple web app that runs on App Engine.

Server and HTML
JavaScript

I haven't run this against mobile browsers yet. If you give it a try please let me know what you find.

It's a shame that these browsers behave differently, especially given the differences in browser behavior between loading JavaScript using document.write and appendChild.

Saturday, May 28, 2011

Go on App Engine Example - Part 1

The App Engine team recently announced support for Go as a runtime for use in apps. Summary up front, the App Engine SDK for the Go runtime is the easiest way I've found yet to get started with Go. As I change my code, it is recompiled in the background when I make a request to my app, so it feels very much like developing in a scripting language.

I've been excited about the Go language for some time now (specifics on why will have to wait for another post) so I was eager to try it out in one of my favorite platforms: App Engine. I wanted to start with something small, so I wrote a simplified version of a web app that I've been itching to write lately, a site for hosting plain text content. Specifically, I want something that preserves whitespace, allows me to line up columns of text, and supports non-English characters (Unicode). Those are the kinds of things I need to share and talk about code. Also there is a great deal more you can do with plain old monospaced text, maybe you'll find this useful as well.

With that objective in mind I give you the Plain Text Machine. This little app lets you enter a small amount of text, somewhere around 2,000 characters, and gives you a link that others can visit to see an HTML reproduction of your writing. I mentioned I wanted to keep this simple, so here's the odd little bit, this app doesn't store your text anywhere. The URL that is generated contains the text, hence the somewhat low limit on message length. It certainly keeps the app simple, the most complex logic is that which converts the text from the URL into HTML.

A request starts by hitting the Init function:
func init() {
http.HandleFunc("/", handle)
http.HandleFunc("/show", show)
}
The main page, at /, is just static content, we're just interested in the /show handler. It looks like this:
func show(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/html; charset=utf-8")
// Get the message from the URL.
PrintHtml(utf8.NewString(r.FormValue("msg")), w)
}
The above does two things, sets the content type of our response so that the browser will know it is HTML, and reads the message URL parameter from the request to convert it to HTML.

The PrintHtml method prints out some boilerplate HTML then reads the message one character at a time and converts each character to its HTML-safe equivalent. There's a tiny bit of complexity to make sure that the whitespace is preserved instead of being collapsed as would normally be done with repeated spaces in HTML. Here's the code:
func PrintHtml(text *utf8.String, out http.ResponseWriter) {
spaces := false

fmt.Fprint(out, textHeader, middle)
for i := 0; i < text.RuneCount(); i++ {
currentChar := text.At(i)

if currentChar == 32 && !spaces {
// A first space.
fmt.Fprint(out, " ")
spaces = true
} else {
if currentChar == 32 {
// Space following another space
fmt.Fprint(out, "&nbsp;")
} else if currentChar == 10 {
// Newline
fmt.Fprint(out, "<br>")
} else if currentChar == 9 {
// Tab
fmt.Fprint(out, "&nbsp;&nbsp;&nbsp; ")
} else if currentChar == 38 {
// &
fmt.Fprint(out, "&amp;")
} else if currentChar == 60 {
// <
fmt.Fprint(out, "&lt;")
} else if currentChar == 62 {
// >
fmt.Fprint(out, "&gt;")
} else if currentChar < 31 || currentChar == 128 {
// Skip control characters.
} else if currentChar < 127 {
fmt.Fprintf(out, "%c", currentChar)
} else {
fmt.Fprintf(out, "&#%d;", text.At(i))
}
spaces = false
}
}
fmt.Fprint(out, footer)
}
The textHeader, middle, and footer variables are string constants containing the wrapper HTML which gives style information.

If you're interested in the full source code for this tiny little app, you can find it in the Plain Text Machine open source project. Hopefully this example provides an easy to understand picture of what Go code for App Engine looks like.

I had quite a bit of fun putting together this app. By keeping it simple I was able to go from idea to done in less time than it took me to write this blog post. As an added bonus, having an app with no persistent storage brings up some interesting philosophical questions. For example, if a message is created but no one stores the link to it, does it still exist?

Monday, May 09, 2011

Setup OAuth2 for Google APIs

Today I'm at I/O Bootcamp and helping out with a walkthrough on how to get starting using Google APIs in a variety of languages.

One of the things I appreciate about the Google APIs is the authorization mechanism which lets me see which applications I've granted access for my data and allows me to revoke access. As an application developer, there are some things that I need to do to identify my application so that Google knows which app is requesting access so that it can show the user more information about my app. The first step, then, in writing an application that uses OAuth2 is registering your app.

You can begin the registration process on the Google API console by creating a project:





Now that you have an application, you'll need to configure it for use with OAuth2 and get the secret tokens that your application will use in its requests. For that create an OAuth2 client ID.



The most vital decision to make during the sign up flow is if your application is a "web application" or an "installed application". If you're a site accessed in a browser and you're able to send the users to a Google web page for authorization and then have the broswer redirect back to your app, then you want web application. For an installed application, the user will still need to authorize your app by visiting a web page, but once authorization is complete, the secret token will be sent to the app either using a redirect to a local running web server or by having the user copy and paste the secret into your application.

For the command line samples I've been playing with I choose installed application.

After creating the client ID you should see information something like this

Client ID:      #######.apps.googleusercontent.com
Client secret: Amzz5Yip2SJPqqq5Jx
Redirect URIs: urn:ietf:wg:oauth:2.0:oob
http://localhost


You'll need to put this information into your application so that it can use the client ID and secret when making requests to get an authorization token from the user. This can be as simple as copying and pasting these strings into your code.

The one other thing that needs to be done before you begin using OAuth2 with one of the Google APIs is to turn on the API for your application. This can be done on the developer console "Services" section.

Let's say that I wanted to access the URL Shortener API. First I would need to enable it for my application.



Then I would need to specify the URL Shortener's API scope when I request authorization from a specific user. The scopes that are requested by my app are turned into a list of APIs that the user must grant access to when they authorize my application.



The scope for an API can be found in the API documentation under authorization.

For an example that brings all of these settings together, see the urlshortener.py example:


FLOW = OAuth2WebServerFlow(
client_id='433807057907.apps.googleusercontent.com',
client_secret='jigtZpMApkRxncxikFpR+SFg',
scope='https://www.googleapis.com/auth/urlshortener',
user_agent='urlshortener-cmdline-sample/1.0')
...
credentials = run(FLOW, storage)
...
http = httplib2.Http()
http = credentials.authorize(http)


Python may not be your bag, but no worries, there are client libraries for the Google APIs in a variety of languages, and even better, there is an API Explorer that lets you try out the underlying protocol without any language specific stuff getting in the way.

For example, here is getting details and stats about a short URL:



And here is creating a new short link:





For all the details on using these APIs, take a look at the documentation. For example here are the URL shortener docs. The common first step for almost all of the Google APIs that access user information is the registration step we started with. For more details on OAuth2 with Google APIs, there is some excellent documentation here.

Tuesday, March 08, 2011

Free Verse #397

Sipping a late,
writing a free verse poem.
Hey look, a haiku!

Sunday, January 23, 2011

Mercurial in Five Commands

Completing any sort of significant programming project can be nearly impossible without version control. For a class project, I recently collaborated with a small group on a large programming assignment. To keep all of our changes in sync while working miles away from each other, we used a centralized version control system.

Since I've been working quite a bit with Mercurial lately, we went with a free private project on bitbucket. It was the first time my teammates had used Mercurial so I gave a crash course that I thought would be helpful for others as well. A distributed version control system has a large number of commands and features, but 90% of the time, you're just dealing with the basics. When working with a team you can get by with just these five Mercurial commands: clone, commit, push, pull, and update.

Setting up the Project


As I mentioned, we were using a hosted repository on bitbucket, so getting a working copy of the repository on our machines started with each of us executing hg clone. It looked a little something like this:
hg clone https://your-username@bitbucket.org/your-username/project
This will create a local repository with a copy of each file and you're now free to make changes. You can edit files locally and Mercurial will track the changes for you. If you want to add new files or remove existing files, make sure to use hg add and hg remove. (Bonus commands!). Once you're happy with your changes and you want to gather them up in a logical unit it's time for:

Local Snapshots


To save a set of changes locally, you use hg commit. Be sure to write an informative description of this change set since you and others will want to remember what this set of changes was all about. There are a few arguments to the commit command that come in handy, often I do:
hg ci -u your-username -m 'Reduces codebase entropy. All tests pass!'
For more options on the commit command and any other Mercurial command you can use hg help <command>. You can also see a diff of your current files compared to the most recent snapshot using the hg diff command. (Bonus times two!)

Pinning your changes locally is a fantastic feature of distributed version control systems. As I work I tend to take a local snapshot several times an hour. These revisions just exist in your local copy, so you don't need to worry about clashing with other people's code at this point or making sure that all changes are usable. Sometimes when I decide I've gone down the wrong path I'll take a snapshot of the ill conceived changes before I roll them back, just in case I decide later on that some of the ideas weren't so bad after all.

Once you have something that's ready for others to use, it's time to:

Share


To get your changes into the hands of others, you can push them back up to the central repository. You do this with the hg push command. Since we created our local repository using clone, your local copy knows where to send the changes. Also, if you'd like to do something different, like push your changes to a different location than where we cloned from, you can take a look at more options of the push command using hg help push.

The push command will upload all of your local commits, along with those handy descriptions, for the rest of your team to see.

Now others on your team are pushing up their changes too and it would be great to get their changes so you are all working on the same code. What's that saying, "It's better to give than to...?"

Receive


To get the changes others have posted to the repository into your local copy, you use hg pull. Running this command copies the change sets that you haven't received yet to your local repository, but it doesn't edit your files or apply the changes just yet. Since you might want to be selective about what changes you apply, Mercurial splits applying the changes into two steps. First you pull, then you use hg update. When called with no additional arguments like that, hg update applies all of the changes.

I recommend pulling and updating often. In our group, since there were just a few of us, we'd post in a chat room when we pushed changes so others would know to pull. If you push before you pull in other people's changes, you might need to use hg merge. Also, you can see all of the changes which are in your repository with the hg log command. (Triple bonus, hey a hat-trick!)

There you have it!

Want more?


I tried to keep this simple and focused, the bare minimum to work on a project with a small team. Mercurial has more to offer. I haven't touched branches yet, or looking at diffs, or creating your own local repository from scratch, or sharing your changes by running your own server locally. All of these are just single commands! For further reading take a look at Mercurial: The Definitive Guide.

There are a few services which offer hosting of Mercurial repositories. For open source, Project Hosting on Google Code is an option as well as bitbucket which I used for the first time this past week. Any other Mercurial hosting providers that you recommend?