appengine & virtualenv

21 11 2010

UPDATE: appengine 1.6.1 & uses gaecustomize.py

This article will explain how to setup Google AppEngine (GAE) with virtualenv.

GAE does not provide a “setup.py” to make the SDK “installable”, it is supposed to be used from a folder without being “installed”. GAE actually forbids the use of any python library in the site-packages folder. All included libraries must be in the same folder as your application, this allows GAE to automatically find and upload third-party libraries together with your application code when you upload the code to GAE servers.

So what would be the advantages of using of using virtualenv with GAE? The main reason is to have an environment to run unit-tests and functional tests. It will allow us to use the interactive shell to make operations on DB. And it also enforce you are using the correct python version.

Step 0 – install App Engine SDK

Make sure you use 1.6.1 or later, an important bug was fixed on this release. As described on official docs. Tested with virtualenv 1.6.4.

Step 1 – create and activate a virtualenv

Same as usual…


$ virtualenv --python python2.5 --no-site-packages gae-env
$ source gae-env/bin/activate

Step 2 – add google_appengine path

Add a path configuration file named “gae.pth” to the virtualenv site-packages with the path to google_appengine. This way google_appengine will be in sys.path enabling it to be imported by other modules.

You will need to adjust the content of the file according to where you created your virtualenv and google_appengine location. Mine looks like this:


$ cat gae-env/lib/python2.5/site-packages/gae.pth
../../../../google_appengine

Simple test to make sure your gae.pth is correct:

(gae-env)$ python
>>> from google import appengine

If you did not get any exception you are good to go on.

Step 3 – fix path for third-party libs

The AppEngine SDK comes with a few third-party libraries. They are not in the same path as google’s libraries. If you look at dev_appserver.py you will see a function called fix_sys_path, this function adds the path of the third-party libraries to python’s sys.path. One option would be to add these paths to gae.pth… But I prefer to use the function fix_sys_path so we have less chances of having problems with future releases of the SDK.

Note that this will not look for your config in app.yaml. So you might need to add some extra imports. The example below is using webob version 1.1.1 instead of the default one.

Path configuration files can also execute python code on if the line starts with import. Add a module gaecustomize.py to site-packages:

gae-env/lib/python2.5/site-packages/gaecustomize.py

def fix_sys_path():
    try:
        import sys, os
        from dev_appserver import fix_sys_path, DIR_PATH
        fix_sys_path()
        # must be after fix_sys_path
        # uses non-default version of webob
        webob_path = os.path.join(DIR_PATH, 'lib', 'webob_1_1_1')
        sys.path = [webob_path] + sys.path
    except ImportError:
        pass

And modify gae.pth it calls the above module:

gae-env/lib/python2.5/site-packages/gae.pth

../../../../google_appengine
import gaecustomize; gaecustomize.fix_sys_path()

For some unknown reason gae.pth is being processed twice and on the first time google_appengine is not added to sys.path. Thats why I explicitly call the function fix_sys_path.

Check if it is working fine:


(gae-env)$ python
>>> import yaml

Again. You should not any exceptions on this…

Step 4 – add dev_appserver.py to bin

Not really required but handy.


gae-env/bin $ ln -s ../../google_appengine/dev_appserver.py .

Conclusion

Now you have an isolated environment running AppEngine! But pay attention libraries used your production code should not be installed in your virtualenv, you should do “GAE way” and link them from your application folder. You should install on virtualenv only stuff used on your tests. Check site.py docs for more details on using .pth files.





Slide presentations in reStructuredText -> S5 -> PDF

16 03 2010

Let’s say you want to create a slide presentation and you are not very much into presentation software.

S5 is a good enough HTML based alternative for a slideshow presentation. reStructuredText and rst2s5 can free you from writing HTML yourself…

S5 can generate a "printer-friendly" version of your slides. But I was really missing a way to create a PDF version of my slides to ease its distribution. I finally found a tool that could handle that Prince.

Example (slides.rst):

.. include:: <s5defs.txt>

======================================================================
reStructuredText to PDF
======================================================================

in 2 easy steps

:Author: Eduardo Schettino


(1) rst2s5
=======================

rst => s5

::

  rst2s5 --theme=small-white slides.rst slides.html


(2) prince
=======================

s5 => PDF

::

  prince --media projection -s page.css slides.html -o slides.pdf

Where page.css controls the PDF page size:

@page { size: 1280px 800px }

Caveats

  • Prince is not OpenSource though it provides a free license for non-commercial user.
  • Page footer is displayed only on first page.
  • Some CSS tweaking might be necessary depending on your theme.




json2dom

27 04 2009

I like jQuery but creating DOM elements on the fly with jQuery is pretty boring. I guess I am not the only one. Just take a look at all this plugins. Well I added one more :)

json2dom let you create dom elements on-the-fly using json representation. The problem with other plugins is that they loose the power of jQuery… on json2dom you still can use all jQuery function while representing the data with json.

To create an element just pass a dictionary to the json2dom function where the key is the tag name. the value will be the content of the tag/element. it can be:

  • string -> text content
  • array -> child elements
  • jQeury -> a jQuery object containing a DOM element
  • dictionary -> key: name of a jQuery method. value: parameters to the method

check the plugin page for some examples.





pinyin input/output on ubuntu

31 03 2009

If you are a laowai using ubuntu/debian and want pinyin input and simplified chinese output or pinyin output – with accents ;)  – you should try the package scim-uim. zh-tonepy and zh-pinyin input methods work as I expect.

I guess the package scim-pinyin is for native chinese because i couldnt figure out their input methods :P

Update: If you are for the ü try typing v. I took some time to find it…





designing a javascript widget

4 05 2008

I was playing with the idea on how to create javascript widgets. As an experiment I created a simple picture album viewer.

I have been having some problems with declarative widgets. I guess declarative widgets make it easier to add them to a page but it is bad for customizations and more dynamic behavior.

As an exercise I developed a prototype for a Photo Album Viewer. No magic here, you need to know HTML, JavaScript and CSS.

If you are looking for production-ready javascript widgets to use you wont find it here.

Javascript toolkits are great to solve cross-browser issues and usually give a boost in the productivity. But I will use plain Javascript. Tested on Firefox 2 only.

Photo Album Viewer

I want to design a basic Photo Album Viewer. Just one picture is show as the main content and an series of thumbnails from the album appears on the bottom. The user selects photos from the thumbnails clicking on it. A image from the thumbnail is displayed on the main content.

The whole point of developing a widget is to create something that will provide a richer user experience than a plain old HTML. I will focus on usability but will ignore colors, round borders…

Dealing with Images

Before creating the widget itself lets take a look in some issues of this application.

1) Thumbnails selection

The idea is that all photos will be available on its real size and on reduced size(thumbnail). The thumbnail bar was supposed to be "carousel" where you can go over all pictures in the album and load the thumbnails only when needed. To keep it simple I will load all of them in a scroll bar.

2) loading images dynamically.

Loading all images at once is not an option. Images will be loaded only on request. (though I plan to add some kind of pre-loading later). To load images dynamically, just create a "img" element and set it’s "src" attribute. You can register an event for "onload" to be notified when it is completed loaded.

var imgEle = document.createElement("img");
imgEle.onload = function(){...};
imgEle.setAttribute("class","nodisplay");
imgEle.setAttribute("src",photoURL);
containerNode.appendChild(imgEle);

3) Adjusting picture size

The whole picture must fit inside the display area, and of course the width/height ratio must be preserved. To do this some basic geometric calculation were done.

The real picture dimensions are Picture.width and Picture.height. The Picture.maxWidth/maxHeight are the maximum bounding box for the picture. The picture is adjusted to its biggest size without overflowing the bounding box and the margin is adjusted so the image appears in the center.

Picture.prototype.adjust_size = function(){
    var ratioX = this.width / this.maxWidth;
    var ratioY = this.height / this.maxHeight;
    var used,space;
    if(ratioX > ratioY){
      used = this.height / ratioX;
      space = this.maxHeight - used;
      this.node.style.width = this.maxWidth;
      this.node.style.height = used;
      this.node.style.marginTop = space / 2;
    }
    else{
      used = this.width / ratioY;
      space = this.maxWidth - used;
      this.node.style.width = used;
      this.node.style.height = this.maxHeight;
      this.node.style.marginLeft = space / 2;
    }
};

4) Usability

Images take time to load. So I added a "loading ajax" gif so the users know what is going on. I keep track of the image status (not-loaded/loading/loaded). The images are displayed only when completely loaded. Loaded images are cached.

The widget

Ok. Now we have enough material to think on the widget. The widget is nothing more than putting all parts together.

After creating the widget the DOM is supposed to have this structure.

<div>                             <!-- div containing the whole album -->
  <div class="picview">           <!-- display are for the picture -->
     <img src="ajax.gif" ...>     <!-- loading ajax gif  -->
     <span class="lightbox" ...>  <!-- another indication picture is loading -->
                                  <!-- an image NOT being displayed -->
     <img src="/images/pic1.jpg" class="nodisplay" ...>
                                  <!-- the image being displayed -->
     <img src="/images/pic4.jpg" ...>
                                <!-- more images....>
  </div>

  <div class="thumbbar">            <!-- thumbnails -->
     <a ...><img src="/images/thumb1.jpg"></a>
     <a ...><img src="/images/thumb2.jpg"></a>
                                    <!-- more thumbnails -->
  </div>
</div>

Where "…" indicates that some attributes need to be created dynamically (on client side).

So the million dollar question is: Where should be DOM structure be created? Server, client, both?

The problem is that you can not do everything on the server. You need to attach events to the DOM nodes, keep some runtime info of the widget. Most of the time I see widgets are mix a server and client code, this makes things harder to understand and work. If I could choose server only or client only I would go for server. But between sever & client or just client I go for client only.

So on the HTML page I just add a "onload" function to create the widget:

<script type="text/javascript">
  var ALBUM;
  function start(){
     var picview = document.getElementById("picview");
     ALBUM = new Album(document.getElementById("album"));
  }
</script>

Ok. So now I just need to pass some data. What is the best way to pass data to javascript? No not HTML. JSON of course:

<div id="album" class="data">
    [{thumb:'/media/images/thumb1.jpg',photo:'/media/images/pic1.jpg'},
    {thumb:'/media/images/thumb2.jpg',photo:'/media/images/pic2.jpg'},
    {thumb:'/media/images/thumb3.jpg',photo:'/media/images/pic3.jpg'},
    {thumb:'/media/images/thumb4.jpg',photo:'/media/images/pic4.jpg'},
    {thumb:'/media/images/thumb5.jpg',photo:'/media/images/pic5.jpg'},
    {thumb:'/media/images/thumb6.jpg',photo:'/media/images/pic6.jpg'},
    {thumb:'/media/images/thumb7.jpg',photo:'/media/images/pic7.jpg'}]
</div>

Of course this content would be created dynamically.

So instead of writing some HTML on server than have javascript to parse, modify, and attach events I just pass some data to the javascript widget. The widget is responsible for creating the HTML (easier than parsing and modifying). Another advantage is that you don’t need to add more code to make the widget work programmatically.

Take a look in the full example.








Follow

Get every new post delivered to your Inbox.