Tuesday, May 29, 2007

Demand Led Standards-based Development

I think Peter-Paul Koch's recent article on A List Apart
Evangelizing Outside the Box: Web Standards and Large Companies – ought to make more of the power that clients have in steering the progress of web standards.

The main thrust of the article, that high profile employees of large companies could and should be highly vocal evangelists for web standards in order to add momentum to the movement in general, is totally valid and I certainly wouldn't argue against it. Such evangelism is bound to have an influence on a significant number of developers who are yet to recognise the benefits of web standards.

But the incentives to adopt the good practices of their peers may not be strong enough to enduce most late adopters to put in the work to make the jump to web standards. Why should they bother if they can maintain a viable business using the same methods as they've always used?

However, if their clients demand the use of web standards, these same developers will have to adapt their methods or risk going out of business.

So its down to the larger companies (and everyone else) to educate their clients about the benefits of web standards so that, eventually, there'll be no demand for non-standards based development and anyone who doesn't adapt will become extinct.

ReadSpeaker without nasty page reloads

Readspeaker's a great service that makes it easy for any website owners to provide audio versions of their content on the fly. This is clearly preferable to re-recording content manually every time a couple of words get changed and the quality of the synthesized voice is actually very natural and lifelike.

Every implemetation of ReadSpeaker that I've seen works by providing a link near the text which, when clicked, either forces a page reload so that an audio player can be embedded or - and this is my personal anti favourite - opens a new page containing only the audio player and shrinks the browser window to the size of the player. Take a look at the O'Reilly Radar (click on the 'listen' link) or any number of examples on ReadSpeaker's site to see what I mean.

While in most cases it's probably true that having a badly implemented audio version of a page is better than no audio version, it is possible to implement ReadSpeaker in a nice, elegant and accessible way.

The reason why it's difficult to find any examples is that the ReadSpeaker documentation doesn't make it very clear that it's possible. However, it is possible to retrieve the URL of the generated mp3 file.

This means that it's possible to pass the mp3 url to an audio player that's embedded in the page when it first loads. If this is implemented with a suitable Flash mp3 player, the mp3 won't be downloaded unless the user decides to play the audio.

The way to retrieve the un-encoded URL of the mp3 is to make the following call to ReadSpeaker (if you want the URL to be encoded, set the 'type' parameter to 101):


where CUSTOMER_NAME and CUSTOMER_ID are supplied to you by ReadSpeaker when you open an account.

Your Flash player may be capable of using this URL as it is but if you need to pass the player an actual mp3 URL, you'll need to write a script to retrieve the URL of the mp3 up front.

Here's a Python script that I wrote for use with Plone but it should be easy to adapt to any other platform:

# Python script to retrieve the URL of an MP3 audio file from readspeaker.com.

# Import required functions
from urllib import urlopen, quote

def getReadSpeakerURL(customer_name, customer_id, page_to_read_url):
url_to_open = "http://asp.readspeaker.net/cgi-bin/%srsone?customerid=%s&url=%s&type=100" % (customer_name, customer_id, page_to_read_url)
mp3file = ''

fp = urlopen(url_to_open) # this retrieves a page containing the url of the mp3 file
mp3file = fp.readline() # this retrieves the mp3 url itself

# check for errors
except IOError:
mp3file = ''

# return two versions of the mp3 url, one with the URL encoded and one without
return (mp3file, quote(mp3file))

Then place the following HTML into any pages that you want to be ReadSpeaker-ed (this example is written for Plone so uses Template Attribute Language (TAL) syntax. Sorry if you're not familiar with this but I hope you can still follow it):

<div class="readspeaker" define="customer_name string:[CUSTOMER_NAME]; customer_id string:[CUSTOMER_ID]; rs_url_array python: here.getReadSpeakerURL(customer_name, customer_id, here.absolute_url() + '?hidereadspeaker=1'); rs_url_unquoted python:rs_url_array[0]; rs_url_quoted python:rs_url_array[1];">
<h2>Hear this page read out loud</h2>
<object type="application/x-shockwave-flash" define="flashplayer string:/flash/emff_comments.swf?src=${rs_url_quoted}" attributes="data flashplayer" align="middle" height="28" width="200">
<param value="" name="movie" attributes="value flashplayer">
<p>To hear this page read aloud, get the Flash Player from <a href="www.adobe.com">Adobe's web site</a></p>
<p><a href="" attributes="href rs_url_unquoted">Download</a> - <a href="" attributes="href string:${here/portal_url}/help/index_html#readspeaker-help">Help</a></p>

This way you can have an accessible method of providing audio content without nasty page reloads.

Wednesday, May 23, 2007

Apache rewrite recipe for usable URLs

Here's a little mod_rewrite recipe for Apache that might be useful if you're developing a web app like StripMe which is a very simple service that takes a CSS file and returns a version of the file with all the comments and white space stripped out.

The location of the CSS file to be stripped is passed to the service in the query string of the URL. Normally, this URL would look something like this:
  • http://stripme.org/index.php?cssfile=http://mysite.com/style.css
We wanted to make the service easy to use by letting users pass the URL of their CSS files like this:
  • http://stripme.org/http://mysite.com/style.css
To do this, I ended up using the following rewrites in Apache:
  • RewriteCond %{REQUEST_URI} "http:\/\/"
  • RewriteRule "(.*)http:\/\/(.*)" "$1index.php?http:\/\/$2" [PT]
It's probably not the most elegant solution, but as long as there's a script in your root directory called index.php that knows what parameters to expect, it should work.

Please let me know if it doesn't work or if there's a nicer way of doing it.

Monday, May 21, 2007

One way to fix POSKeyErrors in Zope on Windows

I just spent much longer than should have been necessary trying to fix a Zope problem so I'm going to document it just in case anyone else has the same problem in the future.

The problem started with Zope raising a POSKeyError exception whenever a certain object was accessed in the Zope Management Interface (ZMI). The object could not be removed from Zope using the ZMI in any way.

Investigation revealed that this exception signifies corruption of the Zope Database (ZODB) due to an object that has become browkn in some way.

We have backups (of course), but this worried me because it happened on our development server and we do so much development that even restoring a half a day old backup means losing a significant amount of work. So I looked for a way of fixing the problem without restoring a backup.

Further investigation revealed that this can be achieved using 'zopectl debug' to interact directly with the ZODB in a Python shell and delete the broken object.

The next hurdle was that zopectl doesn't work on Windows and no prizes for guessing what we use.

The solution was to use PloneShell, which does the same thing as zopectl but also works on Windows.

The final solution once I had my interactive PloneShell shell running and attached to the ZODB, was to replace the corrupt object with a new object:
  • container._setOb(idOfBrokenObject, someNewObject)
then delete the broken object:
  • container._delObject(idOfBrokenObject)
and then commit the changes to the ZODB:
  • get_transaction().commit()

Sunday, May 20, 2007

Upscaling Zope with ZEO on Windows

If you're using Plone for production level business applications and you're not yet using Zope Enterprise Objects (ZEO) I'd highly recommend switching over at your earliest opportunity.

If you've got a single server system with multiple CPUs or you want to be able to balance the application load over two or more servers, ZEO is the way to go.

I've recently switched over our Windows-based installation of Zope 2.7 (with Plone 2.0.5). The performance gains are easily worth the effort and I found a couple of resources on the Plone site that made the whole process easy to implement (see links below)

The added bonus is that new servers can now be added easily to the configuration as demand on the server increases making it a great long term solution for a production environment.

Here are the documents I found useful:

Tuesday, May 15, 2007

Setting up Plone on Ubuntu Feisty Fawn

Recently I needed to set up Plone on an Ubuntu server. I wanted to run Plone behind Apache and I needed to be able to remotely administer the server on a Windows machine.

I don't have very much experience with linux so I was expecting all sorts of command line stumbling blocks but I was amazed at how easy everything was to set up, barring a couple of hiccups.

I thought I'd post a log of what I did to get the server working (mostly for my own benefit so I've got a permanent record in case I need to do it again). It's not massively detailed so don't expect a breakdown on how to install and configure things like the secure FTP client WinSCP.

I'm absolutely certain that there are better ways of doing it, and I welcome any comments about how I could have done things differently.

Install Log

On the windows machine
  • Download and install these useful remote administration tools:
    • Putty - For remote login to linux
    • WinSCP - For secure file transfer to and from linux
On the linux machine
  • Install Ubuntu desktop edition (I used a Pentium III, 1GHz CPU with 512MB RAM)
  • Install SSH server - sudo apt-get install openssh-server
  • Install Apache - sudo apt-get install apache2
  • Install Zope (see How to setup Plone from source for some pointers on this) - sudo apt-get install zope2.9 (if you leave off the 2.9 bit it will probably find the package containing the latest version)
  • Make a Zope instance - sudo /usr/lib/zope2.9/bin/mkzopeinstance.py
    • Provide the information requested:
      • Directory: /var/zope/instance001
      • Username: user
      • Password: pass
    • If necessary edit zope.conf to specify port number, etc (it may be necessary to use a terminal to change the owner of zope.conf before you can save the changes to files that you edit)
      • sudo chown user /var/zope/instance001/etc/zope.conf
    • and then back again when you have finished
      • sudo chown zope /var/zope/instance001/etc/zope.conf
  • Run zope - sudo /var/zope/instance001/bin/runzope (or, to run it in debug mode: sudo /var/zope/instance001/bin/runzope -X "debug-mode=on")
    • or:
    • sudo /var/zope/instance001/bin/zopectl start
      • sudo /var/zope/instance001/bin/zopectl stop
  • Install PIL - sudo apt-get install python-imaging
  • Download and install Plone 2.5.2 (you may have to change ownership on the Products folder before you can transfer the Plone files to the Products directory)
Apache setup - See HTTPD - Apache2 Web Server to see how Apache is configured in Ubuntu
  • to create a new virtual host, make a copy of the default site and then - sudo a2ensite site-name
  • To enable a module - sudo a2enmod module-name
  • Enable mod_proxy and mod_rewrite
    • sudo a2enmod proxy
    • sudo a2enmod proxy_http
    • sudo a2enmod rewrite
  • Edit proxy.conf to allow proxying (I could only do this by enabling proxying from the specific client I was accessing the site from but I need to find a better way as this opens up the server to general proxying which is bad)
    • Change 'Proxyrequests Off' to 'Proxyrequests On'
    • Add 'Allow from all' in tag
  • To restart apache:
    • sudo /etc/init.d/apache2 reload
    • or:
    • sudo /etc/init.d/apache2 force-reload
For accessing the ubuntu server using a remote windows environment:
  • On the Ubuntu machine, log in locally to the desktop and open System -> Adminstration -> Login Window
    • In the 'Remote' tab, select 'Same as Local' from the 'Style' drop down list
  • On the Windows machine, Install Cygwin and the X11 components
  • Edit [CYGWIN_INSTALL_DRIVE]:\cygwin\usr\X11R6\bin\startxdmcp.bat setting the IP address of the REMOTE_HOST to the IP address of the Ubuntu server
  • Launch [CYGWIN_INSTALL_DRIVE]:\cygwin\usr\X11R6\bin\startxdmcp.bat
    • Note - By default, this will only allow you to have one session running simultaneously but this can be configured in the Login Window settings in Ubuntu
  • For further help, see: