Mango

Mango loves to cache things. This propensity for saving everything and serving up old versions of things can be annoying, especially while previewing documents locally.

For a while now, it's been possible to clear Mango's "index" cache by visiting /flushcache/. So, over the past few weeks I've got pretty good at the following routine.

1. Make a change to a Markdown document.
2. Visit /flushcache/ in my browser.
3. ⌘← to return to the rendered version of the document.
4. ⌘R to refresh the page and have the change take effect.

While this only takes a second or two each time, similar inconveniences were what drove me from WordPress! The great thing about creating software is that one has the power to ease one's frustrations, which is what I did here.

Click to flush the cache and have the updated content appear

If you add your IP address to INTERNAL_IPS in your project-level settings file, a handy link'll be inserted for you into each page via JavaScript. Clicking on this link will send an Ajax request to /flushcache/; upon receipt of a successful response the page will reload of its own accord. :)

This applies to both local testing and the live site. For local testing you'll need to add '127.0.0.1' to INTERNAL_IPS.

Because the relevant JavaScript snippet is only inserted into the page source for requests from one of the INTERNAL_IPS, only you will see this link.

This is a big deal. Mango has come a long way since the previous release. Changes have been made in many areas, most visible of which is the addition of a default theme (if you're reading this at mango.io you're looking at it).

Navigation (top), and part of a post

Simple search form

Data URIs ensure that the style sheet requires no images

Contact form

Mango themes are just style sheets – they're not coupled with templates the way they are in WordPress.

Changes since 0.3

• It's now possible to enable subscriptions via the SUBSCRIPTIONS setting. With subscriptions enabled, commenters have the option of subscribing to e-mail notifications sent each time a new (non-spam) comment is posted on the thread.

  Since subscriptions require a database, this setting is disabled by default.

• Mango now utilizes Akismet to filter spam comments. Enable this feature by adding AKISMET_API_KEY to your custom settings file.

  This has reduced the number of spam comments that reach my inbox for moderation from dozens per day to just a few per week.

• The default base template now comes equipped with asynchronous Google Analytics code. Activating Google Analytics is now as easy as adding GOOGLE_ANALYTICS_ID to your custom settings file.

• Mango is now BSD licenced, like Django.

• It's now possible to import content from Blogger or WordPress via a couple of handy scripts. Having access to these a couple of months ago would have saved me a great deal of time!

• The cache can now be flushed by visiting /flushcache/. This makes it possible to publish new content and have it appear immediately, rather than waiting for the index cache to expire. While this approach seems like a hack, there's no way for Mango to "know" when the cache should be flushed.

  You may worry that flushing the cache will incur significant overhead. It does not. Mango's caching is two-tiered: each individual document is cached, and the index containing all the cached documents is also cached. Flushing the cache in this way only deletes the index cache, so only new documents and documents which have been modified will be read, converted to Document objects, and cached.

• Mango has search functionality at last. It's rudimentary, but doesn't suck as much as most. Search queries are broken into words and each document containing all the words is returned. Phrases are supported via "double quotes", à la Google.

  Searches are performed against a document's Markdown, so metadata is inspected as well as the document's title and body.

• Internally, Mango now resolves URLs in a much more sensible (and less error-prone) manner. Each Document contains a list of URLs at which it can be retrieved, along with the canonical URL to which its aliases should redirect. Since documents are cached, resolving URLs no longer requires accessing the filesystem, making this process very fast.

• New error handling means that certain common gotchas now result in helpful error messages rather than unhelpful server error pages. This should make getting Mango running for the first time a bit easier.

• The 404 template now has access to Mango's default context variables, which means that it is now styled like any other page on the site. The 500 template is still ugly, but I haven't yet worked out how to pass context variables to the 500 template (if you know the trick please fill me in).

• Mango's cache keys have been changed to ensure that multiple Mango sites can happily share a memcached socket. All of Mango's cache keys are now mango:-prefixed, so Mango should even play nicely with other apps.

• New sanitize filter strips potentially nefarious markup from comments.

Over on MTG apps I'm using Dave Fowler's subdomains middleware to run a Mango blog and another Django app on different subdomains.

Getting this up and running locally was a piece of cake but mirroring the setup on my WebFaction slice proved to be more challenging. The quick fix would have been to create two separate Django projects; one for each app. This would have resulted in unnecessary duplication of shared settings, though, so I was thrilled to find a workable solution in the end.

Here's the project-level urls.py:

from django.conf.urls.defaults import *
from django.core.urlresolvers import resolve

import app1.urls
import app2.urls

def route_request(request):
    urlconf = app2.urls if request.subdomain == 'app2' else app1.urls
    view, args, kwargs = resolve(request.path, urlconf=urlconf)
    return view(request, *args, **kwargs)

urlpatterns = patterns('',
    (r'', lambda request: route_request(request)),

    # associate urlconfs with their respective apps
    (r'', include(app1.urls, app_name='app1')),
    (r'', include(app2.urls, app_name='app2')),
)

It's worth noting that I'm including the urlconfs explicitly even though the requests are all handled by route_request. Django's URL resolver relies on urlconfs and apps being associated in this way. Without this information the url template tag throws horrible NoReverseMatch exceptions.

This setup — without violating DRY — appeared to be getting the job done, but today I noticed that it had broken Django's automatic slash appending. Thankfully, it's possible to add this functionality manually without messing with Mango's source, by taking advantage of the fact that Mango's default URL resolution can be overridden.

# mango/urls/custom.py

from django.conf.urls.defaults import *
from django.core.urlresolvers import reverse
from django.http import HttpResponsePermanentRedirect

from mango.handlers import moderate
from mango.views import archives, search

urlpatterns = patterns('',
    (r'^archive$', lambda request: HttpResponsePermanentRedirect(reverse(archives))),
    (r'^moderate/(approve|delete|spam|close)$', moderate),
    (r'^search$', search),
    (r'^.*[^/]$', lambda request: HttpResponsePermanentRedirect('%s/' % request.path)),
)

The fourth tuple is responsible for appending slashes. The preceding patterns act as guards to ensure that slashes are not appended inappropriately. (Mango is extremely lenient as far as the archives page is concerned – "archive", "archive/", and "archives" will all take you to "archives/".)

A couple of days ago I wrote a Python script which converts a WordPress blog's pages and posts to beautifully-formatted Markdown documents, complete with Mango-friendly metadata.

Blogger is rather popular, as I understand it; my condolences those who've spent more than a few minutes with it. Quite a few of my friends are in this category, so I thought I'd create an import script for Blogger to encourage them to try Mango.

Usage is the same as with the WordPress conversion script, with the exception of the last step:

python blogger.py http://example.blogspot.com/

That's right, no username or password required for this one!

Moving a site from WordPress to Mango just got a whole lot easier. The latest Mango source features a script which connects to an existing WordPress installation using XML-RPC, processes every page and post, and saves each as a Markdown text file in your posts directory. Handy!

Here are the steps to follow in order to give this a go:

1. Install html2text:

   easy_install html2text
   

2. Create a new Django project, if required:

   django-admin.py startproject blog
   

3. Navigate to the new Django project (or an existing Django project):

   cd blog/
   

4. Open the project's settings file:

   open settings.py
   

5. Set your preferred time zone*, then save:

   TIME_ZONE = 'Pacific/Auckland'
   

6. Download the latest Mango source:

   hg clone https://bitbucket.org/davidchambers/mango
   

7. Navigate to Mango's settings directory:

   cd mango/settings/
   

8. Create a file to store customizations to Mango's settings:

   touch custom.py
   

9. Open this file:

   open custom.py
   

10. Tell Mango where to put the documents by adding PATH_TO_POSTS, then save:

    DOCUMENTS_PATH = '~/posts/'
    

11. Navigate to the extras directory:

    cd ../extras/
    

12. Run the import script, using appropriate arguments:

    python wp.py http://example.com/blog/ admin 12345
    

What the script does

The script communicates with the WordPress blog using XML-RPC, retrieving all the pages and posts. It then pulls out the relevant information and formats it as Mango expects. The icing on the cake is the fact that html2text is used to convert body copy to Markdown – WordPress might think it's fine to store content as HTML, but we think it's totally wrong.

The resulting files are named based on their respective slugs, and saved in your posts directory. For example, these are the files that are generated from a new WordPress installation:

# about.text

About
=====

This is an example of a WordPress page, you could edit this to put information
about yourself or your site so readers know where you are coming from. You can
create as many pages like this one or sub-pages as you like and manage all of
your content inside of WordPress.

# hello-world.text

date: 24 June 2010
time: 10:42pm

Hello world!
============

Welcome to **WordPress**. This is your first post. Edit or delete it, then
start blogging!

  1. one

  2. two

  3. three

* A word about TIME_ZONE

Mango always uses UTC when presenting content to the world. The default templates make use of the <time> element to provide hooks for localization of dates and times via JavaScript.

An author, though, should be free to use her local time when adding dates and times to posts. For this reason Mango expects these dates and times to use the TIME_ZONE specified in the project's settings file. Mango handles conversion of these times to UTC for display on the Web.

When importing files from WordPress the timestamps are already in UTC; Mango performs the reverse conversion to ensure that posts dates and times (as they appear in the Markdown files) are localized to the author's time zone.

Conversion scripts for other blogging platforms?

Blogger's probably next on the list, based on my completely unresearched assumption of popularity. If you'd like to try Mango and plan to migrate content from an existing site, let me know what you're moving from and I'll bump your script to the top of the list.

While it is possible to convert posts by hand, having done so myself I'll say that I hope never to need to do so again. I wish I'd written this script two months ago!