This summer, I've been teaching my two oldest kids to program. They're ages sever and four and a half so I'm tailoring our lessons to what I imagine they can grasp in what can be a very abstract field of study. I've been super impressed with what they've learned so far!
The first challenge was thinking how to make programming interesting and compelling. Something too abstract feels like it would be challenging, there needs to be immediate feedback. Type this in, see what changes on the screen. How about we set out to make a game? That was one of the first pursuits that got me into programming in my teenage years.
The second consideration was balancing simplicity and ease of use. When programming, I always felt more confident when I could understand at a pretty low level what was going on. For this reason I wanted to make sure they gained an understanding of the fundamentals. We'd start with a view of a computer as a very simple machine. Even if this means there is some extra coding needed to get the desired effect on the screen. It's better (IMHO) to add abstractions later once they have an underlying understanding than to try to understand a powerful but complicated library that initially gets quick results.
I'm planning to post what we covered each week. We spend 30-60 minutes in a once a week session though some weeks we're taking off. It is summer after all!
Here's what we covered in our first lesson.
1. You can tell a computer what to do!
We opened up the terminal and I told them to type
print "Hello [name]"
Here it is, immediate feedback with the simplest possible "program." To get this working, I added a bash alias print=echo. Seems like print is a more common name across programming languages for "write something to stdout."
This went over pretty well, both kids typed some things.
Time to introduce some programming concepts. I explained that this was calling a function named print. The print function writes out whatever you tell it to write. I hope purist will forgive my painting with a broad brush.
Next I explained that the words that you give to the function to put on the screen are an argument, er, well, a parameter. The looks on their faces when I called these arguments made it pretty clear that they were thinking of what transpires out in the sandbox.
So here we'd covered the main ideas I wanted to teach them in the first lesson. You can tell a computer what to do! The "what to do" is specified using a function and the parameters that you pass to the function to tell it what or how you want things done.
Time for more fun.
Our four and a half year old can read basic words, but I imagined typing and reading might get tedious pretty fast. So next we turned to something just as simple that doesn't require reading from the screen, and not surprisingly was quite a bit of fun: the Mac say program.
If you've never played with this, in the Mac terminal there is command "say" which works like echo except it speaks whatever you give it in a computerized voice. We started with
say "Hello [name]"
Great fun! From there I asked them what parameters we should give the say function next. This filled the rest of our time. By far the biggest hit was say "[Baby's name] are you pooping?"
Yup, teaching kids to program.
Showing posts with label programming. Show all posts
Showing posts with label programming. Show all posts
Wednesday, July 22, 2015
Saturday, May 28, 2011
Go on App Engine Example - Part 1
The App Engine team recently announced support for Go as a runtime for use in apps. Summary up front, the App Engine SDK for the Go runtime is the easiest way I've found yet to get started with Go. As I change my code, it is recompiled in the background when I make a request to my app, so it feels very much like developing in a scripting language.
I've been excited about the Go language for some time now (specifics on why will have to wait for another post) so I was eager to try it out in one of my favorite platforms: App Engine. I wanted to start with something small, so I wrote a simplified version of a web app that I've been itching to write lately, a site for hosting plain text content. Specifically, I want something that preserves whitespace, allows me to line up columns of text, and supports non-English characters (Unicode). Those are the kinds of things I need to share and talk about code. Also there is a great deal more you can do with plain old monospaced text, maybe you'll find this useful as well.
With that objective in mind I give you the Plain Text Machine. This little app lets you enter a small amount of text, somewhere around 2,000 characters, and gives you a link that others can visit to see an HTML reproduction of your writing. I mentioned I wanted to keep this simple, so here's the odd little bit, this app doesn't store your text anywhere. The URL that is generated contains the text, hence the somewhat low limit on message length. It certainly keeps the app simple, the most complex logic is that which converts the text from the URL into HTML.
A request starts by hitting the
The PrintHtml method prints out some boilerplate HTML then reads the message one character at a time and converts each character to its HTML-safe equivalent. There's a tiny bit of complexity to make sure that the whitespace is preserved instead of being collapsed as would normally be done with repeated spaces in HTML. Here's the code:
If you're interested in the full source code for this tiny little app, you can find it in the Plain Text Machine open source project. Hopefully this example provides an easy to understand picture of what Go code for App Engine looks like.
I had quite a bit of fun putting together this app. By keeping it simple I was able to go from idea to done in less time than it took me to write this blog post. As an added bonus, having an app with no persistent storage brings up some interesting philosophical questions. For example, if a message is created but no one stores the link to it, does it still exist?
I've been excited about the Go language for some time now (specifics on why will have to wait for another post) so I was eager to try it out in one of my favorite platforms: App Engine. I wanted to start with something small, so I wrote a simplified version of a web app that I've been itching to write lately, a site for hosting plain text content. Specifically, I want something that preserves whitespace, allows me to line up columns of text, and supports non-English characters (Unicode). Those are the kinds of things I need to share and talk about code. Also there is a great deal more you can do with plain old monospaced text, maybe you'll find this useful as well.
With that objective in mind I give you the Plain Text Machine. This little app lets you enter a small amount of text, somewhere around 2,000 characters, and gives you a link that others can visit to see an HTML reproduction of your writing. I mentioned I wanted to keep this simple, so here's the odd little bit, this app doesn't store your text anywhere. The URL that is generated contains the text, hence the somewhat low limit on message length. It certainly keeps the app simple, the most complex logic is that which converts the text from the URL into HTML.
A request starts by hitting the
Init function:func init() {
http.HandleFunc("/", handle)
http.HandleFunc("/show", show)
}The main page, at /, is just static content, we're just interested in the /show handler. It looks like this:func show(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/html; charset=utf-8")
// Get the message from the URL.
PrintHtml(utf8.NewString(r.FormValue("msg")), w)
}The above does two things, sets the content type of our response so that the browser will know it is HTML, and reads the message URL parameter from the request to convert it to HTML.The PrintHtml method prints out some boilerplate HTML then reads the message one character at a time and converts each character to its HTML-safe equivalent. There's a tiny bit of complexity to make sure that the whitespace is preserved instead of being collapsed as would normally be done with repeated spaces in HTML. Here's the code:
func PrintHtml(text *utf8.String, out http.ResponseWriter) {
spaces := false
fmt.Fprint(out, textHeader, middle)
for i := 0; i < text.RuneCount(); i++ {
currentChar := text.At(i)
if currentChar == 32 && !spaces {
// A first space.
fmt.Fprint(out, " ")
spaces = true
} else {
if currentChar == 32 {
// Space following another space
fmt.Fprint(out, " ")
} else if currentChar == 10 {
// Newline
fmt.Fprint(out, "<br>")
} else if currentChar == 9 {
// Tab
fmt.Fprint(out, " ")
} else if currentChar == 38 {
// &
fmt.Fprint(out, "&")
} else if currentChar == 60 {
// <
fmt.Fprint(out, "<")
} else if currentChar == 62 {
// >
fmt.Fprint(out, ">")
} else if currentChar < 31 || currentChar == 128 {
// Skip control characters.
} else if currentChar < 127 {
fmt.Fprintf(out, "%c", currentChar)
} else {
fmt.Fprintf(out, "&#%d;", text.At(i))
}
spaces = false
}
}
fmt.Fprint(out, footer)
}The textHeader, middle, and footer variables are string constants containing the wrapper HTML which gives style information.If you're interested in the full source code for this tiny little app, you can find it in the Plain Text Machine open source project. Hopefully this example provides an easy to understand picture of what Go code for App Engine looks like.
I had quite a bit of fun putting together this app. By keeping it simple I was able to go from idea to done in less time than it took me to write this blog post. As an added bonus, having an app with no persistent storage brings up some interesting philosophical questions. For example, if a message is created but no one stores the link to it, does it still exist?
Labels:
app engine,
go,
open source,
programming
Monday, May 09, 2011
Setup OAuth2 for Google APIs
Today I'm at I/O Bootcamp and helping out with a walkthrough on how to get starting using Google APIs in a variety of languages.
One of the things I appreciate about the Google APIs is the authorization mechanism which lets me see which applications I've granted access for my data and allows me to revoke access. As an application developer, there are some things that I need to do to identify my application so that Google knows which app is requesting access so that it can show the user more information about my app. The first step, then, in writing an application that uses OAuth2 is registering your app.
You can begin the registration process on the Google API console by creating a project:


Now that you have an application, you'll need to configure it for use with OAuth2 and get the secret tokens that your application will use in its requests. For that create an OAuth2 client ID.

The most vital decision to make during the sign up flow is if your application is a "web application" or an "installed application". If you're a site accessed in a browser and you're able to send the users to a Google web page for authorization and then have the broswer redirect back to your app, then you want web application. For an installed application, the user will still need to authorize your app by visiting a web page, but once authorization is complete, the secret token will be sent to the app either using a redirect to a local running web server or by having the user copy and paste the secret into your application.
For the command line samples I've been playing with I choose installed application.
After creating the client ID you should see information something like this
You'll need to put this information into your application so that it can use the client ID and secret when making requests to get an authorization token from the user. This can be as simple as copying and pasting these strings into your code.
The one other thing that needs to be done before you begin using OAuth2 with one of the Google APIs is to turn on the API for your application. This can be done on the developer console "Services" section.
Let's say that I wanted to access the URL Shortener API. First I would need to enable it for my application.

Then I would need to specify the URL Shortener's API scope when I request authorization from a specific user. The scopes that are requested by my app are turned into a list of APIs that the user must grant access to when they authorize my application.

The scope for an API can be found in the API documentation under authorization.
For an example that brings all of these settings together, see the urlshortener.py example:
Python may not be your bag, but no worries, there are client libraries for the Google APIs in a variety of languages, and even better, there is an API Explorer that lets you try out the underlying protocol without any language specific stuff getting in the way.
For example, here is getting details and stats about a short URL:

And here is creating a new short link:


For all the details on using these APIs, take a look at the documentation. For example here are the URL shortener docs. The common first step for almost all of the Google APIs that access user information is the registration step we started with. For more details on OAuth2 with Google APIs, there is some excellent documentation here.
One of the things I appreciate about the Google APIs is the authorization mechanism which lets me see which applications I've granted access for my data and allows me to revoke access. As an application developer, there are some things that I need to do to identify my application so that Google knows which app is requesting access so that it can show the user more information about my app. The first step, then, in writing an application that uses OAuth2 is registering your app.
You can begin the registration process on the Google API console by creating a project:


Now that you have an application, you'll need to configure it for use with OAuth2 and get the secret tokens that your application will use in its requests. For that create an OAuth2 client ID.

The most vital decision to make during the sign up flow is if your application is a "web application" or an "installed application". If you're a site accessed in a browser and you're able to send the users to a Google web page for authorization and then have the broswer redirect back to your app, then you want web application. For an installed application, the user will still need to authorize your app by visiting a web page, but once authorization is complete, the secret token will be sent to the app either using a redirect to a local running web server or by having the user copy and paste the secret into your application.
For the command line samples I've been playing with I choose installed application.
After creating the client ID you should see information something like this
Client ID: #######.apps.googleusercontent.com
Client secret: Amzz5Yip2SJPqqq5Jx
Redirect URIs: urn:ietf:wg:oauth:2.0:oob
http://localhost
You'll need to put this information into your application so that it can use the client ID and secret when making requests to get an authorization token from the user. This can be as simple as copying and pasting these strings into your code.
The one other thing that needs to be done before you begin using OAuth2 with one of the Google APIs is to turn on the API for your application. This can be done on the developer console "Services" section.
Let's say that I wanted to access the URL Shortener API. First I would need to enable it for my application.

Then I would need to specify the URL Shortener's API scope when I request authorization from a specific user. The scopes that are requested by my app are turned into a list of APIs that the user must grant access to when they authorize my application.

The scope for an API can be found in the API documentation under authorization.
For an example that brings all of these settings together, see the urlshortener.py example:
FLOW = OAuth2WebServerFlow(
client_id='433807057907.apps.googleusercontent.com',
client_secret='jigtZpMApkRxncxikFpR+SFg',
scope='https://www.googleapis.com/auth/urlshortener',
user_agent='urlshortener-cmdline-sample/1.0')
...
credentials = run(FLOW, storage)
...
http = httplib2.Http()
http = credentials.authorize(http)
Python may not be your bag, but no worries, there are client libraries for the Google APIs in a variety of languages, and even better, there is an API Explorer that lets you try out the underlying protocol without any language specific stuff getting in the way.
For example, here is getting details and stats about a short URL:

And here is creating a new short link:


For all the details on using these APIs, take a look at the documentation. For example here are the URL shortener docs. The common first step for almost all of the Google APIs that access user information is the registration step we started with. For more details on OAuth2 with Google APIs, there is some excellent documentation here.
Labels:
google,
oauth,
oauth2,
programming
Sunday, January 23, 2011
Mercurial in Five Commands
Completing any sort of significant programming project can be nearly impossible without version control. For a class project, I recently collaborated with a small group on a large programming assignment. To keep all of our changes in sync while working miles away from each other, we used a centralized version control system.
Since I've been working quite a bit with Mercurial lately, we went with a free private project on bitbucket. It was the first time my teammates had used Mercurial so I gave a crash course that I thought would be helpful for others as well. A distributed version control system has a large number of commands and features, but 90% of the time, you're just dealing with the basics. When working with a team you can get by with just these five Mercurial commands: clone, commit, push, pull, and update.
As I mentioned, we were using a hosted repository on bitbucket, so getting a working copy of the repository on our machines started with each of us executing
To save a set of changes locally, you use
Pinning your changes locally is a fantastic feature of distributed version control systems. As I work I tend to take a local snapshot several times an hour. These revisions just exist in your local copy, so you don't need to worry about clashing with other people's code at this point or making sure that all changes are usable. Sometimes when I decide I've gone down the wrong path I'll take a snapshot of the ill conceived changes before I roll them back, just in case I decide later on that some of the ideas weren't so bad after all.
Once you have something that's ready for others to use, it's time to:
To get your changes into the hands of others, you can push them back up to the central repository. You do this with the
The push command will upload all of your local commits, along with those handy descriptions, for the rest of your team to see.
Now others on your team are pushing up their changes too and it would be great to get their changes so you are all working on the same code. What's that saying, "It's better to give than to...?"
To get the changes others have posted to the repository into your local copy, you use
I recommend pulling and updating often. In our group, since there were just a few of us, we'd post in a chat room when we pushed changes so others would know to pull. If you push before you pull in other people's changes, you might need to use
There you have it!
I tried to keep this simple and focused, the bare minimum to work on a project with a small team. Mercurial has more to offer. I haven't touched branches yet, or looking at diffs, or creating your own local repository from scratch, or sharing your changes by running your own server locally. All of these are just single commands! For further reading take a look at Mercurial: The Definitive Guide.
There are a few services which offer hosting of Mercurial repositories. For open source, Project Hosting on Google Code is an option as well as bitbucket which I used for the first time this past week. Any other Mercurial hosting providers that you recommend?
Since I've been working quite a bit with Mercurial lately, we went with a free private project on bitbucket. It was the first time my teammates had used Mercurial so I gave a crash course that I thought would be helpful for others as well. A distributed version control system has a large number of commands and features, but 90% of the time, you're just dealing with the basics. When working with a team you can get by with just these five Mercurial commands: clone, commit, push, pull, and update.
Setting up the Project
As I mentioned, we were using a hosted repository on bitbucket, so getting a working copy of the repository on our machines started with each of us executing
hg clone. It looked a little something like this:hg clone https://your-username@bitbucket.org/your-username/projectThis will create a local repository with a copy of each file and you're now free to make changes. You can edit files locally and Mercurial will track the changes for you. If you want to add new files or remove existing files, make sure to use
hg add and hg remove. (Bonus commands!). Once you're happy with your changes and you want to gather them up in a logical unit it's time for:Local Snapshots
To save a set of changes locally, you use
hg commit. Be sure to write an informative description of this change set since you and others will want to remember what this set of changes was all about. There are a few arguments to the commit command that come in handy, often I do:hg ci -u your-username -m 'Reduces codebase entropy. All tests pass!'For more options on the commit command and any other Mercurial command you can use
hg help <command>. You can also see a diff of your current files compared to the most recent snapshot using the hg diff command. (Bonus times two!)Pinning your changes locally is a fantastic feature of distributed version control systems. As I work I tend to take a local snapshot several times an hour. These revisions just exist in your local copy, so you don't need to worry about clashing with other people's code at this point or making sure that all changes are usable. Sometimes when I decide I've gone down the wrong path I'll take a snapshot of the ill conceived changes before I roll them back, just in case I decide later on that some of the ideas weren't so bad after all.
Once you have something that's ready for others to use, it's time to:
Share
To get your changes into the hands of others, you can push them back up to the central repository. You do this with the
hg push command. Since we created our local repository using clone, your local copy knows where to send the changes. Also, if you'd like to do something different, like push your changes to a different location than where we cloned from, you can take a look at more options of the push command using hg help push.The push command will upload all of your local commits, along with those handy descriptions, for the rest of your team to see.
Now others on your team are pushing up their changes too and it would be great to get their changes so you are all working on the same code. What's that saying, "It's better to give than to...?"
Receive
To get the changes others have posted to the repository into your local copy, you use
hg pull. Running this command copies the change sets that you haven't received yet to your local repository, but it doesn't edit your files or apply the changes just yet. Since you might want to be selective about what changes you apply, Mercurial splits applying the changes into two steps. First you pull, then you use hg update. When called with no additional arguments like that, hg update applies all of the changes.I recommend pulling and updating often. In our group, since there were just a few of us, we'd post in a chat room when we pushed changes so others would know to pull. If you push before you pull in other people's changes, you might need to use
hg merge. Also, you can see all of the changes which are in your repository with the hg log command. (Triple bonus, hey a hat-trick!)There you have it!
Want more?
I tried to keep this simple and focused, the bare minimum to work on a project with a small team. Mercurial has more to offer. I haven't touched branches yet, or looking at diffs, or creating your own local repository from scratch, or sharing your changes by running your own server locally. All of these are just single commands! For further reading take a look at Mercurial: The Definitive Guide.
There are a few services which offer hosting of Mercurial repositories. For open source, Project Hosting on Google Code is an option as well as bitbucket which I used for the first time this past week. Any other Mercurial hosting providers that you recommend?
Labels:
mercurial,
programming,
version control
Tuesday, June 22, 2010
A Simple Testing Library for C
To prepare for a recent post graduate computer science class, I wrote a small library in C which aids in the creation of lightweight, unit-test-like programs. The code can be found here, and using it looks a bit like this:
#include"asserts.h"The design follows the KISS principle and I think it is a nice fit to the simplicity of C. While there is not much to it, I wrote numerous tests using it over the past couple of months and all of that testing certainly paid off.
int main(void)
{
c7e3_assert(1 == 1, "1 should equal 1");
c7e3_assert(2 == 2, "2 should equal 2");
c7e3_report();
return 0;
}
Friday, June 18, 2010
JavaScript Tricks to Speed up Your Site
One of the techniques which makes the web so powerful is the ability to load code, images, and other resources from all over the Internet. So often though, the process of loading these resources and ensuring that all of the required pieces are in place leads to a slow experience for visitors. With the ability to include so much code from across the web, visiting a site could potentially be like installing a new program when it comes to the amount of stuff that needs to be downloaded.
With this in mind, there are a couple of nifty tricks that can help make your app more responsive and I've written up an example site and testing server that shows some ideas for speeding up the user experience when you need to wait for the DOM to load or for additional JavaScript to be fetched and run. We'll begin with document operations.
Often the JavaScript running on a page manipulates the DOM, using
There is a another way, we could request that resources be loaded in parallel and start executing our code before the page is fully loaded. Chances are, your code doesn't need the complete page to be loaded before it starts running, and running before
Lets say you have a web page, a little HTML which includes five JavaScript files. One may be a library used to do animation, another one for loading the users data. In any case, all of these files need to be loaded and some of them depend on others.
The biggest bottleneck for your users is almost certainly having all of these resources load. Network latency is a killer, and something that is often overlooked during development. To create a simulated network environment which can give a more realistic (or even pessimistic) view of the cost of loading these resources, I wrote a "slow server" which can introduce a delay to the file requested. Here is the code for my testing server (designed to run on App Engine):
If you look at this loading process in a profiler you might see something like this:
Now for our first nifty trick. One way to check to see if the necessary prerequisites are present, is by polling the DOM or the JavaScript environment, to see if conditions are right for the code to run. Here is an example of how this code might be rewritten when using some polling helper functions:
With these changes, we shave several seconds off of the user perceived loading time. Specifically we no longer need to wait for the duplicate load (of the
Now that we've seen a way to work around the need for an onload callback, lets look at another place we can tweak the browser's behavior to make a web page more responsive: dynamic script loading.
The most straightforward way to include new code in your page is to use a script tag, something like:
There are a few parts to this trick. The first is to not put all of script includes in the HTML, you could have JavaScript add new script elements to the page which will cause new code to be loaded as needed. In this way, you could load only the resources that are needed at the moment, perhaps some resources would not end up being requested at all. Including a new script could be done in two ways:
Instead you can perform a check to see if
There is one more trick we can add to this loader. Some browsers will interpret the JavaScript in the order in which the scripts were requested, not the order in which they finished loading. That means that a fast loading script further down the list won't be run until a slower script, which appears above it, is loaded. One way we could defeat this delay, is to break the script includes out of linear execution in the JavaScript. If you use setTimeout to introduce a delay in adding the script include to the page, then the code which sets up the script requests can finish quickly and the browser will get back to the script requests later without the same linear constraints. In our code, we wrap the section of
Through the course of this post, I've written a small library for using these tricks when loading JavaScript dynamically in the page as well as a server for trying it out. These are available here as open source code. There are some improvements that could be made here. Off the top of my head, the checkWaiting function could eventually time out if a condition continues to not be met. Also the loader could do more to check to see if a requested script has already been loaded. Any more ideas?
With this in mind, there are a couple of nifty tricks that can help make your app more responsive and I've written up an example site and testing server that shows some ideas for speeding up the user experience when you need to wait for the DOM to load or for additional JavaScript to be fetched and run. We'll begin with document operations.
Often the JavaScript running on a page manipulates the DOM, using
document.getElementById here and document.createElement there. In order to ensure that all the pieces of the page are in place, web programmers often take advantage of the onload callback. It might be used like this<body onload="runMyCodeNow()">Using this technique ensures that all of the things your code might want to read and write from the page are in place. All images have been downloaded, CSS rules have been applied, the layout is all there. However, all of this comes with a cost, your code doesn't run until every last resource has been fetched and rendered. Even the little footer at the bottom of the page, for example, that your code doesn't care about.
There is a another way, we could request that resources be loaded in parallel and start executing our code before the page is fully loaded. Chances are, your code doesn't need the complete page to be loaded before it starts running, and running before
onload will reduce the delay for your users. Before I dive into how this can be accomplished, lets look at an example which uses the old fashioned way.Lets say you have a web page, a little HTML which includes five JavaScript files. One may be a library used to do animation, another one for loading the users data. In any case, all of these files need to be loaded and some of them depend on others.
The biggest bottleneck for your users is almost certainly having all of these resources load. Network latency is a killer, and something that is often overlooked during development. To create a simulated network environment which can give a more realistic (or even pessimistic) view of the cost of loading these resources, I wrote a "slow server" which can introduce a delay to the file requested. Here is the code for my testing server (designed to run on App Engine):
def FilePath(path):With the above code we can introduce a delay on each individual file. To see this in action with our example, here is some HTML which shows a traditional approach, include script includes and an onload callback when everything has loaded.
"""The requested path into a local file path."""
return os.path.join(os.path.dirname(__file__), 'files', path[1:])
class SleepyRenderer(webapp.RequestHandler):
"""Serves the requested page with a client configured delay.
Delay is given as a URL parameter in hundredths of a second to delay.
For example, 200 means wait 2 seconds before responding.
Example request:
http://localhost:8080/hi.html?delay=300&contenttype=text/html
"""
def get(self):
path = self.request.path
delay = self.request.get('delay')
content_type = self.request.get('contenttype') or 'text/html'
if delay:
time.sleep(int(delay)/100)
http_status = 200
requested_file = None
try:
requested_file = open(FilePath(path))
self.response.out.write(requested_file.read())
requested_file.close()
except IOError:
http_status = 404
self.response.set_status(http_status)
self.response.headers['Content-Type'] = content_type
def main():
application = webapp.WSGIApplication([('/.*', SleepyRenderer)],
debug=True)
util.run_wsgi_app(application)
if __name__ == '__main__':
main()
<html>With the above, the page takes several seconds to load and when the very last script has loaded, the 'output' div gets its contents. In many cases, the code really doesn't need to wait for all resources to load, only the ones that are necessary for the code to run. In this case, since the information is added to the output div, we need the output div to exist in the DOM, but we may not need the entire page to load.
<head>
<script src="/testa.js?delay=500&contenttype=text/javascript"></script>
<script src="/testb.js?delay=400&contenttype=text/javascript"></script>
<script>
function init() {
document.getElementById('output');
output.innerHTML = [
'a is ' + a,
'b is ' + b,
'c is ' + c,
'd is ' + d,
'e is ' + e
].join('<br>');
}
</script>
<script src="/testc.js?delay=300&contenttype=text/javascript"></script>
</head>
<body onload="init()">
<script src="/testd.js?delay=200&contenttype=text/javascript"></script>
<div id="output"></div>
<script src="/teste.js?delay=100&contenttype=text/javascript"></script>
<script src="/testa.js?delay=500&contenttype=text/javascript"></script>
</body>
</html>
If you look at this loading process in a profiler you might see something like this:
Now for our first nifty trick. One way to check to see if the necessary prerequisites are present, is by polling the DOM or the JavaScript environment, to see if conditions are right for the code to run. Here is an example of how this code might be rewritten when using some polling helper functions:<script>The code to track the prerequisites and poll is quite simple:
loader.whenNodePresent('output',
function() {
var output = document.getElementById('output');
loader.whenReady(function() {return window['a'];},
function() {
output.innerHTML += 'a is ' + a + '<br>';
});
loader.whenReady(function() {return window['b'];},
function() {
output.innerHTML += 'b is ' + b + '<br>';
});
loader.whenReady(function() {return window['c'];},
function() {
output.innerHTML += 'c is ' + c + '<br>';
});
loader.whenReady(function() {return window['d'];},
function() {
output.innerHTML += 'd is ' + d + '<br>';
});
loader.whenReady(function() {return window['e'];},
function() {
output.innerHTML += 'e is ' + e + '<br>';
});
})
</script>
loader.waiting = [];In the above we use the
loader.whenReady = function(testFunction, callback) {
if (testFunction()) {
callback();
} else {
loader.waiting.push([testFunction, callback]);
window.setTimeout(loader.checkWaiting, 200);
}
};
loader.checkWaiting = function() {
var oldWaiting = loader.waiting;
var numWaiting = oldWaiting.length;
loader.waiting = [];
for (var i = 0; i < numWaiting; i++) {
if (oldWaiting[i][0]()) {
oldWaiting[i][1]();
} else {
loader.waiting.push(oldWaiting[i]);
}
}
if (loader.waiting.length > 0) {
window.setTimeout(loader.checkWaiting, 200);
}
};
loader.whenNodePresent = function(nodeId, callback) {
loader.whenReady(function () {
return document.getElementById(nodeId);
}, callback);
};
whenReady function which takes a couple of functions, one to return a truthy or a falsey value, and one to call back when the first function evaluates to true. If the condition function isn't true when this first call is made, we check back every so often to see if it is ready.With these changes, we shave several seconds off of the user perceived loading time. Specifically we no longer need to wait for the duplicate load (of the
testa script) at the end of the body. The page also appears to be more responsive because the later script's messages appear just after they load but before the page is complete.Now that we've seen a way to work around the need for an onload callback, lets look at another place we can tweak the browser's behavior to make a web page more responsive: dynamic script loading.
The most straightforward way to include new code in your page is to use a script tag, something like:
<html>When the browser's JavaScript interpreter encounters this script src, it stops whatever it's doing and fetches that resource. It doesn't do any more rendering or executing of code until it's finished. This behavior varies a bit in different browsers and is likely an artifact of an old design in which this kind of single threaded behavior was the only option. Since some sites might depend on this linear behavior to get a script's dependencies all in order, this quirk might be with us for a long time. Most of this time, waiting like this is a really silly idea. How often do the scripts that you include depend on one another?
<head>
<script src="some_great_sites_javascript">
...
There are a few parts to this trick. The first is to not put all of script includes in the HTML, you could have JavaScript add new script elements to the page which will cause new code to be loaded as needed. In this way, you could load only the resources that are needed at the moment, perhaps some resources would not end up being requested at all. Including a new script could be done in two ways:
document.write('<script src="somefile.js"></script>');orvar newScript = document.createElement('script');
newScript.src = 'somefile.js';
document.body.appendChild(newScript);Each of the above is appropriate in different situations. Document write adds HTML directly into the page at the point where the page is being loaded, it should only be used for script tag inclusion if the page is not yet loaded. If that page is loaded, using document.write to add the script tag will wipe out the existing body entirely. I've seen this issue in the wild, if you assume document.write is always safe, you'll be bitten when using it after the page has loaded.Instead you can perform a check to see if
document.body exists, if it does then use document.body.appendChild. If it does not yet exist, use document.write. The code for this loader logic might look something like this:loader.loadScript = function(url) {
if (document.body) {
var newScript = document.createElement('script');
newScript.type = 'text/javascript';
newScript.src = url;
document.body.appendChild(newScript);
} else {
document.write('<scr' + 'ipt type="text\/javascript" src="' +
url + '"><\/scr' + 'ipt>');
}
};Now we can request that new JavaScript code be loaded on the fly and it works when the page has not yet finished loaded as well as after it has. There is one more trick we can add to this loader. Some browsers will interpret the JavaScript in the order in which the scripts were requested, not the order in which they finished loading. That means that a fast loading script further down the list won't be run until a slower script, which appears above it, is loaded. One way we could defeat this delay, is to break the script includes out of linear execution in the JavaScript. If you use setTimeout to introduce a delay in adding the script include to the page, then the code which sets up the script requests can finish quickly and the browser will get back to the script requests later without the same linear constraints. In our code, we wrap the section of
loader.loadScript in a short timeout as follows:loader.loadScript = function(url) {
window.setTimeout(function() {
if (document.body) {
var newScript = document.createElement('script');
newScript.type = 'text/javascript';
newScript.src = url;
document.body.appendChild(newScript);
} else {
document.write('<scr' + 'ipt type="text/javascript" src="' +
url + '"><\/scr' + 'ipt>');
}
}, 1);
};With the above changes in place, our example page from before now loads like this when profiled (note that the messages appear in the order that the scripts were loaded, we don't have to wait for everything before we edit the page):
Through the course of this post, I've written a small library for using these tricks when loading JavaScript dynamically in the page as well as a server for trying it out. These are available here as open source code. There are some improvements that could be made here. Off the top of my head, the checkWaiting function could eventually time out if a condition continues to not be met. Also the loader could do more to check to see if a requested script has already been loaded. Any more ideas?
Labels:
app engine,
JavaScript,
open source,
programming
Thursday, June 25, 2009
Partial Function Invocation
My wife tells me that I often jump into an explanation by starting at the beginning of my train of thought without giving any indication of where I'm going. It would be better if I began with the point I'm trying to make, then explain how I reched my conclusion, How am I doing so far? Oh wait, right... Here is my conclusion:
Allowing a function to be partially invoked, to allow some of the arguments to be specified at different times, can allow for code which is more flexible than by just using objects or pure functions.
I've been thinking about this lately as I refactored the gdata-python-client which is a library which can be used with AtomPub services. I'll spare you all the gory details, and offer a simple example of how partial function invocation might come in handy.
When making a request to a remote server, you might need the following information, just for example: username, password. URL, message body, and content type. So we start out by writing a stateless function to take this information and open a connnecion to the server, format our inputs, transmit our request, and parse the response. We'll call it
Now the question becomes: Did we extract the right pieces of information from the function call into the object? Suppose the code you are writing needs to use a different password for each service you are making a request to, but the content type of the data is always the same. Then it would have made more sense to design our class like this:
To use the above class, you would do:
Allowing a function to be partially invoked, to allow some of the arguments to be specified at different times, can allow for code which is more flexible than by just using objects or pure functions.
I've been thinking about this lately as I refactored the gdata-python-client which is a library which can be used with AtomPub services. I'll spare you all the gory details, and offer a simple example of how partial function invocation might come in handy.
When making a request to a remote server, you might need the following information, just for example: username, password. URL, message body, and content type. So we start out by writing a stateless function to take this information and open a connnecion to the server, format our inputs, transmit our request, and parse the response. We'll call it
post, and using it looks like this:serverResponse = post(url, data, contentType, username,This is all well and good, but suppose the final request, as shown above, is preceeded by a whole series of function calls. Each function would need to dutifully pass along parts of the request. Say for example that the user types in their username and password long before the request is made, so these get passed as parameters to lots of functions which only receive them so they can pass them on. In addition, the username and password are almost always the same from request to request, so the same values are being passed to the post function over and over. In cases like this, we will often use an object to hold common values.
password)
class Requestor {
username
password
method post(url, data, conteentType) {...}
}Now our request will look like:client = new Requestor(username, password)What we've effectively done here is specified some of the information in advance and left other pieces of information to be specified at the last minute. I would argue that this make the code better (cleaner, less chance of human error in listing lots of parameters, perhaps less data on the call stack, etc.).
serverResponse = client.post(url, data, contentType)
Now the question becomes: Did we extract the right pieces of information from the function call into the object? Suppose the code you are writing needs to use a different password for each service you are making a request to, but the content type of the data is always the same. Then it would have made more sense to design our class like this:
class Requestor {
username
contentType
method post(url, data, password) {...}
}Since we've established that not everyone who is using our code has the same usage patterns, lets design for utimate flexibility. Every parameter can be specified either in the object, or in the function call. Also, if the object has a parameter already, we could override it by passing in that parameter when we call the method. This is not too difficult in in Python, so here is a non-pseudocode example:class Requestor(object):If you think this seems a bit excessive, I would agree. I didn't go nearly this far when designing the library that started me thinking about this. There was one request parameter in particular though that does use this pattern. (Five points to the first person to post it in the comments. ;-)
def __init__(self, url=None, data=None,
content_type=None, username=None,
password=None):
self.url = url
self.data = data
self.content_type = content_type
self.username = username
self.password = password
def post(self, url=None, data=None,
content_type=None, username=None,
password=None):
url = url or self.url
data = data or self.data
content_type = content_type or self.content_type
username = username or self.username
password = password or self.password
# Now we have our inputs, code to make
# the request starts here
...
To use the above class, you would do:
requestor = Requestor(username='...')It will also handle our alternate usage where we want to give the password to the post method and set the content type at the object level:
requestor.password = '...'
...
server_response = requestor.post(url, data, content_type)
requestor = Requestor(username='...', content_type='...')We can even override parameters which are set in the object:
...
server_response = requestor.post(url, data, password='...')
requestor = Requestor(password='...', content_type='...')With the above example we end up with a lot of code just to let us specify each parameter in either the object or as a function argument. In fact, this can introduce so cases where the user forgets to specify in either, which is possible because all function arguments are now optional. Wouldn't it be better if we could instead specify some of the function parameters, pass the half-specified function call around, and fill in the ramaining values when we finally invoke. For this illustration, I'm using the following syntax to show a partial invocation,
requestor.username = '...'
...
# Override the content_type, just on this request.
server_response = requestor.post(url, data, content_type='...')
< > around arguments instead of ( ).function post(url, data, contentType, username, password) {...}
started = post<username, password>
...
serverRespense = started(url, data, contentType)Recall our case from earler, what if the contentType is constant but the password is instead more variable:started = post<username, contentType>It turns out I'm not the first person to think of this pattern, not by a long shot. Functional programming often makes use of this pattern, referred to as function currying. I found the following example for Scheme which also shows how easy this is in Haskell. The prototype library for JavaScript includes a bind function which can accomplish the same thing. Here's a paper on the topic in C++: (pdf, Google cache HTML). I also found PEP 309 which was a proposal for this in Python. Perhaps I should have called my Python example above: Function Currying using Classes. If you can think of other examples, I'd love to see them.
...
serverRespense = started(url, data, password)
Labels:
code,
currying,
programming,
python
Sunday, September 07, 2008
A Revived Project
q12 is back!
I'm slowly starting back up again on my note taking wiki application. I created my own version of TiddlyWiki several months ago, but then started working on other projects. I'm planning to rewrite my note taking Ajax application to run on Google App Engine, and as I was getting started I realized that there were a few things missing from the Ajax library that I had written as part of this project.
I had created my own simple unit test framework in JavaScript, and I finally got around to uploading the unit tests for the library to the open source project. I've also been learning about manipulating browser cookies from within JavaScript. Aside: Cookie's in JavaScript are weird! When you say
I've also added a minified version of the q12 library, it weighs in at a mere 10k. Download the library today!
I'm slowly starting back up again on my note taking wiki application. I created my own version of TiddlyWiki several months ago, but then started working on other projects. I'm planning to rewrite my note taking Ajax application to run on Google App Engine, and as I was getting started I realized that there were a few things missing from the Ajax library that I had written as part of this project.
I had created my own simple unit test framework in JavaScript, and I finally got around to uploading the unit tests for the library to the open source project. I've also been learning about manipulating browser cookies from within JavaScript. Aside: Cookie's in JavaScript are weird! When you say
document.cookie = something, reading document.cookie doesn't give you the same thing back (the expiration, domain, and path information are squirreled away somewhere else).I've also added a minified version of the q12 library, it weighs in at a mere 10k. Download the library today!
Labels:
JavaScript,
open source,
programming,
projects,
q12
Tuesday, April 22, 2008
Early vs Late Binding
I've been thinking recently about programming languages (surprised?), specifically about the things that make them different. One of the really nice things about C, is that it compiles into machine code which tends to run lean and mean. By that I mean it is blazing fast and doesn't take up much memory. On the other hand, programming in Python and JavaScript has really been growing on me. There is so much flexibility to create elegant solutions quickly and without rewriting lots of existing code. In fact, I'd say greater ability to reuse existing code is a natural outgrowth of programming language flexibility.
So where does this flexibility come from? One place I tend to notice it most, is in the ability to give an existing function a new body, in other words, you can plug in different behavior in place of the default.
Here's a simple example to illustrate the idea. Let's say that we created a simple checkout register which takes a receipt, adds the sales tax, and spits out the grand total. Here's our code foundation in both Python and JavaScript (these two examples do essentially the same thing):
Python:
Python:
Python:
So what is late binding? The idea is that the computer decides which code should be executed while the program is running. This seems normal in scripting languages, but compiled languages often use this too (I'm looking at you Java and C++). For example, overloaded methods and polymorphism take advantage of late binding. With late binding you can change the meaning of an identifier (for example, change the behavior when you call a specific function) at just about any time.
Now lets take a look at a language which uses early binding. C is a great example. With early binding, the meaning of things like function names are locked in when the code is compiled. There is no dynamic lookup while the program is running to see which code should be executed, instead the address of the desired code is embedded directly into the binary machine code.
Here is how the same calculate-total example might look in C:
Using function pointers, you can store the address of the code that you want to be executed, and change the address while the program is running. We can achieve the same late binding effects that I've illustrated in Python and JavaScript by making some small changes to the C code (marked in bold below). Declare a function pointer named TaxCalculator which will store the address of the desired calculate-tax function, then change CalculateTotal so that it uses the TaxCalculator instead of directly calling a calculate-tax function.
Here's another way to think about this comparison. In high level languages which don't expose pointers, functions, variables, and other identifiers actually act like pointers.
So where does this flexibility come from? One place I tend to notice it most, is in the ability to give an existing function a new body, in other words, you can plug in different behavior in place of the default.
Here's a simple example to illustrate the idea. Let's say that we created a simple checkout register which takes a receipt, adds the sales tax, and spits out the grand total. Here's our code foundation in both Python and JavaScript (these two examples do essentially the same thing):
Python:
def CalculateTax(amount):JavaScript:
return amount * 0.18
class Receipt(object):
def __init__(self, items=None):
self.items = items or []
def CalculateTotal(self):
return sum([item + CalculateTax(item) for item in self.items])
function calculateTax(amount) {
return amount * 0.18;
}
function Receipt(items) {
if (items) {
this.items = items;
} else {
this.items = new Array();
}
}
Receipt.prototype.calculateTotal = function() {
var total = 0;
for (var i = 0; i < this.items.length; i++) {
total += this.items[i] + calculateTax(this.items[i]);
}
return total;
}To use the above code, you might write something like this:Python:
my_order = Receipt([5.50, 10, 7.89])JavaScript:
print my_order.CalculateTotal()
var myOrder = new Receipt([5.50, 10, 7.89]);Now let's say someone asks you to change the tax rate which is used when calculating the total. Here's the catch, you're not allowed to change the existing code. It turns out this is actually really easy. You can define a new function, then make an existing function name point to the new function. Here's an example of how to inject our new code:
alert(myOrder.calculateTotal());
Python:
def CalculateHigherTax(amount):JavaScript:
return amount * 0.25
CalculateTax = CalculateHigherTax
print my_order.CalculateTotal()
function calculateHigherTax(amount) {
return amount * 0.25;
}
calculateTax = calculateHigherTax;
alert(myOrder.calculateTotal());After adding the above code to the foundation we started with, you will notice that the calculate total method now uses calculate-higher-tax instead of the original function, even though you are calling the same method on the same object as before. Congratulations, you have just witnessed late binding in action. So what is late binding? The idea is that the computer decides which code should be executed while the program is running. This seems normal in scripting languages, but compiled languages often use this too (I'm looking at you Java and C++). For example, overloaded methods and polymorphism take advantage of late binding. With late binding you can change the meaning of an identifier (for example, change the behavior when you call a specific function) at just about any time.
Now lets take a look at a language which uses early binding. C is a great example. With early binding, the meaning of things like function names are locked in when the code is compiled. There is no dynamic lookup while the program is running to see which code should be executed, instead the address of the desired code is embedded directly into the binary machine code.
Here is how the same calculate-total example might look in C:
#include<stdio.h>If you try to set CalculateTax to a new function definition, you will get an error at compile time because a function cannot be changed once it is bound. Early binding tends to produce more efficient programs. However, if you want to, you can still use the flexiblity available in late binding in C.
float CalculateTax(float amount) {
return amount * 0.18;
}
typedef struct {
float* items;
int num_items;
} Receipt;
float CalculateTotal(Receipt this_order) {
int i;
float total = 0;
for(i = 0; i < this_order.num_items; i++) {
total += this_order.items[i] + CalculateTax(this_order.items[i]);
}
return total;
}
int main(void) {
Receipt my_order;
float my_items[3] = {5.50, 10, 7.89};
my_order.items = my_items;
my_order.num_items = 3;
printf("%f\n", CalculateTotal(my_order));
}
Using function pointers, you can store the address of the code that you want to be executed, and change the address while the program is running. We can achieve the same late binding effects that I've illustrated in Python and JavaScript by making some small changes to the C code (marked in bold below). Declare a function pointer named TaxCalculator which will store the address of the desired calculate-tax function, then change CalculateTotal so that it uses the TaxCalculator instead of directly calling a calculate-tax function.
#include<stdio.h>There you have it!
float CalculateTax(float amount) {
return amount * 0.18;
}
float CalculateHigherTax(float amount) {
return amount * 0.25;
}
typedef struct {
float* items;
int num_items;
} Receipt;
float (*TaxCalculator)(float) = &CalculateTax;
float CalculateTotal(Receipt this_order) {
int i;
float total = 0;
for(i = 0; i < this_order.num_items; i++) {
total += this_order.items[i] + (*TaxCalculator)(this_order.items[i]);
}
return total;
}
int main(void) {
Receipt my_order;
float my_items[3] = {5.50, 10, 7.89};
my_order.items = my_items;
my_order.num_items = 3;
printf("%f\n", CalculateTotal(my_order));
TaxCalculator = &CalculateHigherTax;
printf("%f\n", CalculateTotal(my_order));
}
Here's another way to think about this comparison. In high level languages which don't expose pointers, functions, variables, and other identifiers actually act like pointers.
Labels:
c,
early binding,
JavaScript,
late binding,
programming,
python
Monday, April 14, 2008
A Musical Interlude... and back to Programming
I've been listening to Daft Punk and Justice quite a bit recently. Apparently I'm on a techno kick again. I've never found any electronic music that I've enjoyed as much as Joy Electric's The White Songbook. The purity of the tones and style has made it one of my all time favorite albums.
This got me thinking about a project which I thought of years ago, started, then abandoned. It was a music synthesizer/sequencer which you would program, by well, programming. I mean that the music would be controlled exclusively through a programming language. This would alter the creative process in several ways. Most music sequencers are graphical and allow you to lay out musical patterns in sequence. Writing a program is extremely non-linear, with classes, functions, and variables being defined in the code long before they are used. In this hypothetical sythesizer language. a composition might look something like this:
In the above example, the first play statement will be executed, then the third play statement (5 bars into the 8 bar intro), then the second solo will play again, probably before the first solo finishes. There are other interesting features in the pseudo-code above, but the fact that these sequences are played out of order was what I really want to highlight.
I've done live coding at several events over the years and I tend to have fun with it. It doesn't always go exactly as planned, but that is the whole idea. Live coding turns programming into a performance piece. I imagine there is a niche group of people who could really get into the live coding music scene.
This got me thinking about a project which I thought of years ago, started, then abandoned. It was a music synthesizer/sequencer which you would program, by well, programming. I mean that the music would be controlled exclusively through a programming language. This would alter the creative process in several ways. Most music sequencers are graphical and allow you to lay out musical patterns in sequence. Writing a program is extremely non-linear, with classes, functions, and variables being defined in the code long before they are used. In this hypothetical sythesizer language. a composition might look something like this:
sequence "intro":
playSample("beat", start=0:32.1, end=0:33.5, beats=[1,3,9,11,15])
playSample("moog", beats=[5,7,11])
shiftPitch(start=A4, end=C4, duration=bars(8))
sequence "solo":
playSample("guitarRiff", start=1:15.3, end=2:09.0)
tempo 150 BPM
play("intro", now)
play("solo", end("intro"))
play("solo" now()+bars(5))
In the above example, the first play statement will be executed, then the third play statement (5 bars into the 8 bar intro), then the second solo will play again, probably before the first solo finishes. There are other interesting features in the pseudo-code above, but the fact that these sequences are played out of order was what I really want to highlight.
I've done live coding at several events over the years and I tend to have fun with it. It doesn't always go exactly as planned, but that is the whole idea. Live coding turns programming into a performance piece. I imagine there is a niche group of people who could really get into the live coding music scene.
Labels:
live coding,
music,
programming,
techno
Tuesday, February 26, 2008
In praise of Haikus
Programmers are no strangers to strict requirements on form and syntax, so working in the poetic medium of the Haiku comes almost naturally.
Programming is fun.
Little virtual widgets.
Poems that do work.
The brevity and compactness of the haiku lends itself well to writing something tighly focused. I find them quite enjoyable to write.
Of course, there are quite a few other poetic structures of note which can offer a fun challenge. The limerick and the sonnet are two of my favorites. Here's a limerick I wrote (beware, obscure programming reference ahead).
There once was a coder named Chuck.
And through all the source code he snuck.
He changed not a line,
it all worked just fine:
he programmed by punching a duck!
A sonnet would be a bit ambitious for this late hour. So unleash your creativity, let's see what you've got.
Programming is fun.
Little virtual widgets.
Poems that do work.
The brevity and compactness of the haiku lends itself well to writing something tighly focused. I find them quite enjoyable to write.
Of course, there are quite a few other poetic structures of note which can offer a fun challenge. The limerick and the sonnet are two of my favorites. Here's a limerick I wrote (beware, obscure programming reference ahead).
There once was a coder named Chuck.
And through all the source code he snuck.
He changed not a line,
it all worked just fine:
he programmed by punching a duck!
A sonnet would be a bit ambitious for this late hour. So unleash your creativity, let's see what you've got.
Labels:
haiku,
limerick,
poetry,
programming
Wednesday, February 13, 2008
My Programming Journey so Far
One of the great things about working in computer science is that you never stop leaning. It seems that many programmer follow a progression from one popular language to the next, and I thought I'd dedicate a post to reminisce about my journey so far. I first learned to program in C. This was at the age of sometime around eleven or thirteen. I was instantly hooked, and since then, I've kept right on learning. I think the path I've taken has been fairly typical. From C, I learned C++ (starting in high school). I learned Java during a summer internship after my junior year of high school. In college, it was more C++, Java, and C (I really learned the ins and outs of C in my networking class) along with some other programming languages.
My favorite two classes in my college computer science curriculum were the ones in which I learned assembly language and designed an arithmetic logic unit and then a simple processor. I finally felt like I understood exactly how computers worked. With assembly language I learned a bit about machine code, how many clock cycles specific operations take, and it felt so good to optimize a bit of code to run blazingly fast. In circuit design I learned where those clock cycles come from, why operations take the time they do, and how those machine language op codes are determined. In all things software, at some point it all comes back to electronics.
College was also a time to get a taste of other, less widely used, but none-the-less important languages. Scheme and Prolog were particularly interesting to me, but I haven't had much occasion to use them very much recently.
Through my career, I've focused primarily on C++, then Java. After that I've had the opportunity to use a large number of languages. I learned Ajax programming using JavaScript, I wrote some PHP, C#, and a rather large amount of Python. Outside of work, I like to explore new concepts and languages and I've taken a look at some other languages too. Some notable examples include Ruby and Common Lisp, but I haven't built anything serious with them yet. Python is a language which I've really grabbed hold of recently and I've been learning quite a bit about it. At least half of the side projects I'm working on in my spare time these days are in Python. There seems to be quite a bit of momentum behind Python, and I'm very interested to see where this all goes.
So there you have it, a small glimpse into my journey thus far. How does it jive or differ from your own?
My favorite two classes in my college computer science curriculum were the ones in which I learned assembly language and designed an arithmetic logic unit and then a simple processor. I finally felt like I understood exactly how computers worked. With assembly language I learned a bit about machine code, how many clock cycles specific operations take, and it felt so good to optimize a bit of code to run blazingly fast. In circuit design I learned where those clock cycles come from, why operations take the time they do, and how those machine language op codes are determined. In all things software, at some point it all comes back to electronics.
College was also a time to get a taste of other, less widely used, but none-the-less important languages. Scheme and Prolog were particularly interesting to me, but I haven't had much occasion to use them very much recently.
Through my career, I've focused primarily on C++, then Java. After that I've had the opportunity to use a large number of languages. I learned Ajax programming using JavaScript, I wrote some PHP, C#, and a rather large amount of Python. Outside of work, I like to explore new concepts and languages and I've taken a look at some other languages too. Some notable examples include Ruby and Common Lisp, but I haven't built anything serious with them yet. Python is a language which I've really grabbed hold of recently and I've been learning quite a bit about it. At least half of the side projects I'm working on in my spare time these days are in Python. There seems to be quite a bit of momentum behind Python, and I'm very interested to see where this all goes.
So there you have it, a small glimpse into my journey thus far. How does it jive or differ from your own?
Tuesday, January 29, 2008
Programming Languages are Languages too
One of the reasons that people keep inventing new programming languages is that humans are good at using language and computers are not. So was we improve computers and add more complexity, programmers endeavor to make using these new features simpler and less painful. As a result, computer languages are moving in a general direction towards more natural human language. There will likely always be differences, but programming languages and human languages are strikingly similar if you understand some of the widely used syntax.
Here's an example. When you see something like this
Here's an example. When you see something like this
z = x * y;it means that you want the computer to "store the value of x times y in the varibale z, then display z on the screen." As you can see, some programming syntax is borrowed from math. This example includes arithmetic and a function. Functions can also be though of as verbs, with variables as the nouns. In object oriented programming, variables can be nouns which are capable of performing actions. If you had a digital carrier pigeon, and you wanted to tell it to carry a letter to your grandmother's house, you might say something like:
print(z);
myPidgeon.payload = myLetter;In human language, there are always multiple ways to say the same thing, and the same applies in programming. The programmer might just as easily design the program to give grandma the letter like this:
myPideon.flyTo(grandma.house);
myLetter.setRecipient(grandma);Now for some fun. What do the following code snippets mean?
myPideon.deliver(myLetter);
if (jack.getWorkPercent() == 100.0 &&
jack.getPlayPercent() == 0.0) {
jack.dullBoyFlag = true;
}Pie aPie = new Pie();
Song aSong = new Song(sixpence);
aSong.sing();
fill(pocket, rye);
aPie.add(new Blackbird()[24]);Mouse mice[3];
for (i in range(3)) {
mice[i] = new BlindMouse();
}
observe(run(mice));party = new Party(jack, jill);
party.setTarget(water);
party.equip(pail);
party.ascend(hill);
Wednesday, January 23, 2008
A spoiled programmer
I've been writing quite a bit of Python code recently and I've become a bit spoiled. It's easy in Python to define new classes on the fly, create new functions, pass them here and there, and return arbitrary collections from a method. C will always have a special place in my heart (I think everyone's first language does), but I often think of ways I could make it a bit easier to do certain things like have functions that return functions or have a function return multiple values.
To explain by way of example, it would be fun to do something like this:
The above is a fairly Pythonic way of doing things, and it seems like it should be possible in C. The first way I thought of is using
If I define a struct for each one of my multi-variable-returning functions, I can create functions which effectively return multiple values instead of just one. Yes, technically I am returning one value, the struct, but you know what I meant :-) Namely, if you look at the program's stack, there is probably no perceptible difference between returning a struct and returning multiple variables.
If I pass in a pointer to the result struct as the first parameter to the function, my program could look like this.
For added effect, note that the anonymous structs don't need to match, you just need to make sure that the shape of the structure is the same so that you don't overwrite data. I could have written
To explain by way of example, it would be fun to do something like this:
/* A function that returns multiple values */
int, char, int myFunction(int a, int b, int c, char d) {...}
...
/* Invoke the function and store the results */
int x, y;
char c;
{x, y, c} = myFunction(5, 6, 7, 'Z');
The above is a fairly Pythonic way of doing things, and it seems like it should be possible in C. The first way I thought of is using
structs. I like to think of a struct as the precursor to a class. It allows the arbitrary grouping of variables into a single collection where they can be referred to by name. (In a couple of earlier posts, I showed how you could use structs to simulate classes in C.)If I define a struct for each one of my multi-variable-returning functions, I can create functions which effectively return multiple values instead of just one. Yes, technically I am returning one value, the struct, but you know what I meant :-) Namely, if you look at the program's stack, there is probably no perceptible difference between returning a struct and returning multiple variables.
/* Create a 2 member struct to hold the return value */This works ok, but I would like to avoid having to create a new struct for each one of my functions. It might be easier if I didn't have to worry about types at all, so the natural choice is to have the function return a type-less void pointer (
struct myFuncReturn {
int first;
char second;
};
/* A function that returns an int and a char */
struct myFuncReturn myFunc(int a, int b, int c, char d) {
struct myFuncReturn to_return;
to_return.first = (a+b)*c;
to_return.second = d;
return to_return;
}
int main() {
struct myFuncReturn pattern;
pattern = myFunc(2, 3, 4, 'Z');
printf("Pattern: %i, %c\n", pattern.first, pattern.second);
}
void*). The calling code would then be responsible for interpreting the function's return struct correctly. If I want to return a new anonymous struct from a function, it might look something like this:void* myFunc(int a, int b, int c, char d);If I use the above, I'll need to allocate memory for the struct and return it's address. This is a bit of a bother as well, because now I need to worry about cleaning up that memory later. Instead of having the function allocate a new structure to return, why not pass in a structure and have the function modify it? The code I would need to write would be more aesthetically pleasing (in my opinion) for both the function definition and the calling code which invokes it, and it might even be more efficient.
If I pass in a pointer to the result struct as the first parameter to the function, my program could look like this.
/* Function definition, the out parameter is the return value */Look ma, no type declarations! Now you might say that writing out the entire struct definition each time is a bit unpleasant, but you could always define a struct and use it instead. I wanted to show that you don't really need to declare a type for each function, which could create a bit of a mess if you start using multi-return functions everywhere. With the above pattern, you could also start to play some interesting games by having functions that actually return different structs in different situations (provided the out pointer's reserved space is large enough for the data you want to send back). If I've lost you by now, I do apologize.
void myFunc(void* out, int a, int b, int c, char d) {
((struct{int first; char second;}*)out)->first = (a+b)*c;
((struct{int first; char second;}*)out)->second = d;
}
int main() {
struct{int first; char second;} pattern;
myFunc(&pattern, 2, 3, 4, 'Z');
printf("Pattern: %i, %c\n", pattern.first, pattern.second);
}
For added effect, note that the anonymous structs don't need to match, you just need to make sure that the shape of the structure is the same so that you don't overwrite data. I could have written
myFunc like this:void myFunc(void* out, int a, int b, int c, char d) {
((struct{int first;}*)out)->first = (a+b)*c;
((struct{int x; char second;}*)out)->second = d;
}Or if you want to go even further, like this:void myFunc(void* out, int a, int b, int c, char d) {
*((int*)out) = (a+b)*c;
((struct{int x; char second;}*)out)->second = d;
}Ah, the joys of programming. It's little games like this that make programming lots of fun. It's like working on a big wide open puzzle that you get to build yourself. No wonder I'm spoiled.
Thursday, November 08, 2007
IE, XMLHttpRequest, and caching
I recently noticed an oft cited irritation with using XMLHttpRequest in Internet Explorer. Turns out, the browser intercepts HTTP GET requests made from within JavaScript and gives your Ajax application a cached response instead of fresh data.
I was running some tests on scorpion_server, using the JavaScript client in the example that I've packaged with the server, and I noticed that I could change the data stored in a resource when using IE6, but I couldn't get the value I had just set. This means my client-server pair is pretty much useless in IE6 (probably 7 too). But I thought of a solution. IE is just performing a test for exact match when checking the cache, so if any part of the URL is different, IE will actually perform the query like I want it to. So I added a timestamp URL parameter to all GET requests made by the client if the browser is IE, and I configured the server to ignore all URL parameters on GET requests. I wasn't using them anyway, perhaps someday I will, but I don't foresee a need because all I want is a super simple remote data store. (Who knows, YAGNI might just be my new mantra.)
Note that IE was the only browser (of those I tested) that exhibited this annoying behavior. Mozilla based browsers (Firefox, SeaMonkey, Flock), Opera, and Safari all behaved correctly and don't require the timestamp URL parameter hack.
I was running some tests on scorpion_server, using the JavaScript client in the example that I've packaged with the server, and I noticed that I could change the data stored in a resource when using IE6, but I couldn't get the value I had just set. This means my client-server pair is pretty much useless in IE6 (probably 7 too). But I thought of a solution. IE is just performing a test for exact match when checking the cache, so if any part of the URL is different, IE will actually perform the query like I want it to. So I added a timestamp URL parameter to all GET requests made by the client if the browser is IE, and I configured the server to ignore all URL parameters on GET requests. I wasn't using them anyway, perhaps someday I will, but I don't foresee a need because all I want is a super simple remote data store. (Who knows, YAGNI might just be my new mantra.)
Note that IE was the only browser (of those I tested) that exhibited this annoying behavior. Mozilla based browsers (Firefox, SeaMonkey, Flock), Opera, and Safari all behaved correctly and don't require the timestamp URL parameter hack.
Labels:
JavaScript,
programming,
scorpion server
Sunday, November 04, 2007
JavaScript client for my Scorpion Server
I've added a JavaScript client to the simple web server project: scorpion server. The client is able to set the username and password and make authenticated GET and POST requests to retrieve and modify resources on the server. Now that this portion is complete, I'm ready to rebuild my Ajax Wiki application to run on top of this simple web server.
Labels:
JavaScript,
programming,
projects,
scorpion server,
web server
Wednesday, October 24, 2007
Varieties in Code Design
One of the interesting things about writing software is the myriad of ways to express what it is that you would like the computer to do. When designing a class, function, or something else, you can make the syntax looks just about any way you like. The real interesting bit, is when multiple people need to work on a large project together. Everyone needs to be able to understand the code, so it's probably a good idea if you agree on code conventions beforehand. All of the below would do equally well, but which is most clear?
x = a + 5;
x = a.plus(5);
x = 5.plus(a); (You could do this in Ruby/)
x = plus(a, 5);
plus(a, 5, x);
a.plus(5, x);
set(x, plus(5, x)); (This looks a bit like Lisp.)
The list goes on and on.
Tune in next time for simple Python web-server fun.
x = a + 5;
x = a.plus(5);
x = 5.plus(a); (You could do this in Ruby/)
x = plus(a, 5);
plus(a, 5, x);
a.plus(5, x);
set(x, plus(5, x)); (This looks a bit like Lisp.)
The list goes on and on.
Tune in next time for simple Python web-server fun.
Labels:
code conventions,
programming,
style
Friday, February 02, 2007
Simulating Classes in C - Part 2
Well the $1 C contest has come to an end after generating lots of interest but no full blown solutions. I'm okay with that, thanks to all of you who expressed interest and started on it. From conversations with several of you over chat, I know that there were some great ideas out there. If you don't mind taking the time, please describe your idea in the comments section. As my friend Matt can attest, I love discussing software, ideas, designs, and just about anything related to creative problem solving. This is probably why I really enjoyed math in high school and college. I looked at each problem as a logic puzzle which could be solved multiple ways. Finding the most elegant solution made me feel like I had just written a poem of supreme beauty.
I wrote my solution a few weeks ago, then started researching how other object oriented schemes are implemented. I found out that my code has several disadvantages and some significant advantages as well. For example, I started to comapre the likelyhood of missing the RAM cache to solutions in C++ and Java. There were a few other intersting comparisons but I won't go into all of the nitty gritty details. Overall, I'm very pleased because I learned a lot. Lets discuss!
/* JSObject functions */
void InvokeObjectMethod(JSObject* object, char* methodName,
JSListOfObjects* inputParams,
JSListOfObjects* outputParams) {
int i;
JSClass* class_of_object;
int method_index = -1;
/* Impossible to invoke a method on a NULL pointer */
if(object == NULL) {
/* ToDo: send a "NULL object" error code in the outputParams. */
return;
}
class_of_object = (JSClass*)(object->type);
if(class_of_object == NULL) {
/* ToDo: send a "NULL class" error code in the outputParams. */
return;
}
/* Find the index of the methodName in the object's class. */
/* This part of the code is O(n) efficient, but this could be
improved if the function names were sorted or if a hash
algorithm was used. Hashing could reduce to O(1). */
/* I could also allow the calling code to specify the desired
method directly by index to avoid requiring a string
lookup for each invocation. */
for(i = 0; i <>method_count; i++) {
if(strcmp(class_of_object->method_names[i], methodName) == 0) {
method_index = i;
break;
}
}
/* If the method name was not found in the list of class methods,
return an error code. */
if(method_index == -1) {
/* ToDo: send a "method not found" error code in the
outputParams. */
return;
}
/* Invoke the method at the index corresponding to the method
name. */
(*(class_of_object->methods[method_index]))(object, inputParams,
outputParams);
}
I wrote my solution a few weeks ago, then started researching how other object oriented schemes are implemented. I found out that my code has several disadvantages and some significant advantages as well. For example, I started to comapre the likelyhood of missing the RAM cache to solutions in C++ and Java. There were a few other intersting comparisons but I won't go into all of the nitty gritty details. Overall, I'm very pleased because I learned a lot. Lets discuss!
Labels:
c,
code,
object oriented,
programming
Wednesday, January 10, 2007
Simulating classes in C
I have to admit, I like to program in C. In some ways, it is simpler than some of the newer, high level languages, and I really enjoy being closer to the machine code. Perhaps I'm a bit obsessive about efficiency. Still, I sometimes long for classes and objects in my C programs, (This is where you tell me that I should use C++ or Objective-C.) so I decided to figure out how I could simulate classes in plain old ANSI C.
It turned out to be quite easy. After I read up on function pointers, I created a struct which contains references to functions (kind of like class methods). Here's a simple example of what I'm talking about:
When I run the above, the program prints "thefunction says 7".
From there, I decided to create a family of structures which would simulate classes, objects, and allow polymorphic function calls.
One way to divide up data and methods in an OO way is to say that the class dictates the methods which can be used with the data in an object instance. So we could say that a class contains a list of functions. Using function pointers, the method called can be changed at runtime, as long as all of the functions have the same stack profile. Have I lost you yet? So what I needed to do was create a generic function prototype which abstracts the parameter list into a common format. While I was at it, I decided to lose the restriction that methods return only one value, so the results of a method call will be stored in a structure which is passed in as a parameter. (The method should probably be called a procedure instead of a function.) Perhaps it would be simpler to show you the code.
I was going to discuss the functions I created to streamline "class" method invocation, but this post is getting a bit long. So I've decided to try something fun. I'm going to ask you, gentle reader, to suggest a design for a function which simplifies invocation of an object method. I will mail $1 (USD) to the person who posts the "best" working solution in the comments below. (I'll be compiling using gcc -ansi. Oh, and US residents only, I don't want to run into any strange rules.). In addition to a dollar, you'll have won bragging rights for the first of my blog challenges. After a week or two, I'll post my solution and we can all compare notes. Happy coding!
It turned out to be quite easy. After I read up on function pointers, I created a struct which contains references to functions (kind of like class methods). Here's a simple example of what I'm talking about:
#include<stdio.h>
typedef struct {
int (*test)(int); /* This is the function pointer */
} functionholder;
/* At runtime, I will point to this function */
int thefunction(int x) {
printf("thefunction says %i\n", x);
return 0;
}
int main() {
functionholder fholder;
fholder.test = &thefunction;
(*fholder.test)(7);
return 0;
}
When I run the above, the program prints "thefunction says 7".
From there, I decided to create a family of structures which would simulate classes, objects, and allow polymorphic function calls.
One way to divide up data and methods in an OO way is to say that the class dictates the methods which can be used with the data in an object instance. So we could say that a class contains a list of functions. Using function pointers, the method called can be changed at runtime, as long as all of the functions have the same stack profile. Have I lost you yet? So what I needed to do was create a generic function prototype which abstracts the parameter list into a common format. While I was at it, I decided to lose the restriction that methods return only one value, so the results of a method call will be stored in a structure which is passed in as a parameter. (The method should probably be called a procedure instead of a function.) Perhaps it would be simpler to show you the code.
typedef struct {
void* type; /* This will point to a JSClass. */
void* data;
} JSObject;
typedef struct {
int object_count; /* number of objects in the array */
JSObject** objects; /* An array of pointers to JSObjects */
} JSListOfObjects;
typedef struct {
char* name;
/* Each JSClass contains a list of pointers to the methods which belong
* to that class. */
int method_count;
/* strings naming the functions (allow method lookup by string) */
char **method_names;
/* An array of pointers to functions */
void (**methods)(JSObject* x, JSListOfObjects* y, JSListOfObjects* z);
} JSClass;
I was going to discuss the functions I created to streamline "class" method invocation, but this post is getting a bit long. So I've decided to try something fun. I'm going to ask you, gentle reader, to suggest a design for a function which simplifies invocation of an object method. I will mail $1 (USD) to the person who posts the "best" working solution in the comments below. (I'll be compiling using gcc -ansi. Oh, and US residents only, I don't want to run into any strange rules.). In addition to a dollar, you'll have won bragging rights for the first of my blog challenges. After a week or two, I'll post my solution and we can all compare notes. Happy coding!
Labels:
c,
code,
object oriented,
programming
Wednesday, November 01, 2006
A Steganography Scheme - Part 2
My text based steganography program is here! I completed the program yesterday, and Vanessa came up with a great name: Steganosaurus. Get in touch with me if you would like me to send you the program and the source code. I'm still considering publishing it on an open source hosting site.
You may recall my previous post about this steganography scheme, and I said I would tell you how it all works. So here goes:
My steganography program needs 4 pieces of information to embed or extract a secret message, it needs a file which will be converted, the base, the shift value, and a filename to which the converted message will be stored. The base and shift need a bit of explanation.
Base: All data on a computer is a number, and a number can be expressed multiple ways. I wrote about this in "A Steganography Scheme - Part 1". So the base tells Steganosaurus how to express the data. Should each number in the source file be converted into a series of values between 0-10, 0-50, 0-200? The choice is yours.
Shift: My program outputs a range of values from the source file, and each of them is between 0 and Base. How does this become readable text? That is the purpose of the shift, it is a value added to each piece of the converted file to make it into a character. So the result of the whole process is the contents of the original file expressed as numbers in the range Shift to Base + Shift. These numbers are converted into Unicode characters (UTF-8) so the end result is a readable file. You can see an example in my wiki entry about Steganosaurus. (It's probably easiest to see an example.) By using different shift values, you can hide the data from your original file in text from any language in the world.
So check it out, give it a shot, and ask me to send Steganosaurus to you.
You may recall my previous post about this steganography scheme, and I said I would tell you how it all works. So here goes:
My steganography program needs 4 pieces of information to embed or extract a secret message, it needs a file which will be converted, the base, the shift value, and a filename to which the converted message will be stored. The base and shift need a bit of explanation.
Base: All data on a computer is a number, and a number can be expressed multiple ways. I wrote about this in "A Steganography Scheme - Part 1". So the base tells Steganosaurus how to express the data. Should each number in the source file be converted into a series of values between 0-10, 0-50, 0-200? The choice is yours.
Shift: My program outputs a range of values from the source file, and each of them is between 0 and Base. How does this become readable text? That is the purpose of the shift, it is a value added to each piece of the converted file to make it into a character. So the result of the whole process is the contents of the original file expressed as numbers in the range Shift to Base + Shift. These numbers are converted into Unicode characters (UTF-8) so the end result is a readable file. You can see an example in my wiki entry about Steganosaurus. (It's probably easiest to see an example.) By using different shift values, you can hide the data from your original file in text from any language in the world.
So check it out, give it a shot, and ask me to send Steganosaurus to you.
Labels:
c,
code,
cryptography,
programming,
steganography
Subscribe to:
Posts (Atom)