Audrey M. Roy

I wish I could post screenshots of the app I'm working on with Fuzzy Rainbow.  But I can't talk too much about it yet.

It's a Python/AppEngine + OpenSocial + JavaScript + jQuery UI app.  All the interaction & graphics are done with pngs and crazy JavaScript tricks because @shazow introduced me to the magic of jQuery & jQuery UI and convinced me that JavaScript could do anything.  Working on Fluffy Puffy Cloud Pets also taught me this to some extent, although I did nothing but draw clouds (and integrate my teammates' code) for that project, and I was skeptical of JS back then.

No canvas, no Flash.  If this turns out to be the wrong approach, I'm not afraid to throw away 2 months of code and rewrite it. 

The app is pretty odd.  Things do funny things.  It's either a game or a tool, depending on the user's perspective.

More pieces are coming down the line.  There will likely be a Pinax-based standalone site centered around the app.  There will also likely be a mobile client (maybe PhoneGap).  And more that I absolutely can't mention.  It's pretty ambitious, but it's just a matter of doing it piece-by-piece and cutting out pieces if needed. 

All the seemingly-scattered hacks & experiments that I've done over the summer are now coming together.  It is much more planned and well-thought-out than it might have appeared.  I am excited that I can say this much & can't wait to reveal more.

Update 9/15:  One of the folks on the GAE team saw this!  First he gave me more apps. Then he sent me another email saying he "just pushed a change live which should fix deployments so that you actually have 250 (instead of the mysterious 100). Please give this a try and let me know!"

This totally changes things.  Wow!

****

I like GAE a lot.  But in the end, it's hard to stick with it for major, time-sensitive projects where you don't have a lot of time to waste on workarounds.  As a developer, you just don't have enough control over your work. 

My biggest troubles:

• You can't pack up and leave appspot.com.  There is the dev_appserver, which you can use to test locally.  But there's no easy, reliable way to run a GAE server clone on your own box.  Appdrop, a project that tried to address this, was last updated in 2008 and seems to have fallen behind the new GAE SDK releases (I could understand - what a pain to keep updating).  Want to run a GAE clone on EC2?  You can't without a lot of work.
• Lots of arbitrary limits.  Sure, they were made with good intentions, to improve performance.  But you're bound to run into one trap or another with no way out.  Max # of files in an app, short timeout for processes, max # of IN queries, etc.
• The local dev server is inadequate for testing.  You need to be able to test your app in an environment identical to your production setup.  I thought I could get away with using one of my 10 appspot apps as a test app, but I was hitting the max 100 deployments/day limit by 4pm and having to wait until midnight for more quota (and the max 250 deployments that they said is untrue, it's really 100).  I had issues with cross-domain requests being treated differently on appspot than on my local server.  When you're working with OpenSocial, you find that different social networks are finnicky about cross-domain requests in different, often surprising ways. 
• (on Python GAE) If an unsupported library isn't pure Python, you can't use it.  Workarounds for this involve using another server, which is irritating.  This is often inevitable - when you're starting a project, you often don't know what libraries you're going to need.
• And finally, you can only have 10 apps.  So it's inadequate for even little hobby or test projects.  As someone who experiments a lot, I usually create (and finish) at least 1-2 mini-projects per week.  I've been reusing my appspot apps, but there are ones that I'd like to leave up and running for others to see and use.

I do want to mention that GAE is a really amazing product that just has a few kinks that need to be worked out.  The GAE team has made great outreach efforts in the developer community and has always been helpful.  And GAE so close to being what I've always wanted.

I'll still use GAE every now and then, and I'm likely to move my OpenSocial app back to GAE in the future.  But GAE has too many roadblocks for rapid development.  I was spending more time on workarounds than on my actual app.

If App Engine were open-source, I could run my own clone of the production server somewhere.  I'd be able to get around all the roadblocks and develop my app using their framework and datastore quickly, in a setup basically identical to production, and then I'd be able to come back and optimize it to meet the appspot limitations if I wanted to have their automatic scaling/load balancing.  I'd also love to be able to peek under the covers for my own knowledge.

Alternately, if App Engine would just let me pay to get around all these issues, I'd get my company's credit card out right now.

Today I tried out Tornado, the Python non-blocking web server optimized for speed & traffic.

[Note: the confusing social media buzz around Tornado might make you think it includes social networking or twittering functionality.  That's incorrect...just setting the record straight here.]

Their chat demo seems pretty snappy, although I didn't see it under large loads or anything stressful.  The long polling stuff is super-cool.  You can start a request and leave it open for awhile, specifying a callback method to go to later.  It's like ajax inside out and backwards.

Testing out their framework...I like their blog demo.  There's a Tornado-App Engine version, which I modified slightly to create a blog to keep track of my canvas experiments (totally unrelated, just killing two birds with one stone)

Check it out: http://labs.fuzzyrainbow.com/graphicsexperiments/

[Note: if you're using Tornado with App Engine, you lose much of the functionality of Tornado that makes it interesting in the first place.  WSGI doesn't support non-blocking requests.  You'll want to use Tornado's own HTTP server.]

Tornado comes with a really simple web framework along the lines of Google App Engine's "webapp" framework.  I already like to use GAE+webapp for really simple projects, like basic API servers, and for projects where a more complex form/template system isn't useful, like very graphical games.  It doesn't have the forms fanciness of Django, but that's good sometimes.

I still have to finish writing the high scores server - the one that all the Fuzzy Rainbow mobile apps will connect to in order to send/receive high score data.  So far it's partly working on GAE+webapp (no auth yet, but I can post and get the scores from a mobile app).  I'm thinking now might be a good time to move it to Tornado. 

A game score tracking server sounds pretty trivial.  But imagine one sending and receiving updates in real-time.  You'd be able to keep tabs on your top players' while they're actually playing.  You could give them live updates on their competitors' scores. 

Not to mention you could run live tournaments and broadcast them publicly.  I didn't anticipate making anything this advanced, but Tornado would make a lot of the implementation easier.

The Google App Engine datastore bulkloader actually does work,
contrary to popular opinion on the issue tracker. It works for both
my local dev server datastore (dev_appserver.py) and my appspot.com
server. I have v1.2.3 of the GAE SDK.
 
If you're on Linux: put your GAE directory someplace nice. Mine is at
/usr/lib/google_appengine. I still don't know if that's the right
place, but it seems nice. Add it to PYTHONPATH, and add PYTHONPATH to
PATH if it's still playing hide-and-seek with you.
 
I had to add my GAE app's directory to PATH to get the bulkloader to
work. There's probably a better way, which I don't know of.
 
Follow their instructions exactly for creating models.py and
whatever_loader.py.
 
Authenticate into localhost using your computer's usual
login/password, which isn't your gmail address/appspot password.
 
Make sure your app.yaml contains this. I put it at the top of mine to
be extra safe. (The "/.*" URL handler messes with the other ones
below it, you know.)
- url: /remote_api
 script: $PYTHON_LIB/google/appengine/ext/remote_api/handler.py
 login: admin
 
I was able to successfully import a .csv full of data into my dev
server and my appspot server.