daml 0.1.4 Release, A Python Markup Language for the Web

Since my initial post outlining daml as a haml-for-python, a lot has changed. Now it has become something more and plays a new role in a bigger project I am working on. Let’s look at some of the new syntax changes.

CSS Selector Style Attributes

Declaring attributes now follows css selectors appended to a tag hash. For example,

%html
    %body
        #content.main[attr=val][apples=yes] some content

Inlining Tag Hashes

One can now inline tag hashes. The functionality is still being worked on and is broken beyond a single tier. The intent is to handle use cases such as,

%html
    %body
        %ul#nav
            %li %a[href=/] Home
            %li %a[href=/contact] Contact

        #wrapper #container
            #top
            #content

Implicit Python Declaration

Since my last post, all python code had to be preceded with a colon (:). Now, this is only necessary when embedding a function call in plain text (or calling a filter). For Example,

def greet(s):
    return 'Hello, {0}.'.format(s)

nav = ['www', 'www2', 'www3']

%html
    %body
        %p A greeting for you. :greet('Daniel')
        %ul for x in nav:
            %li {x}

The trade off to the above is explicit line breaks for multiline text as similarly done in python,

%html
    %body This is some text \
        that spans multiple \
        lines.

I am still putting alot of thought into this portion of the syntax and it will most likely see some changes in a number of months. Explicit line breaks can become rather confusing when handling complex mixed contents, but again, tools such as daml aren’t particularly suited for writing page content, but rather establishing layout.

New Filters

I added two new filters as a test, these may be removed in future releases (and migrated to my personal set of daml extensions). They are for declaring css and js files. I think the syntax speaks for itself in function,

%html
    %head
        :css /css/
            main.css
            extra.css
            lib/that_lib.css

        :js /js/lib/
            utils.js
            js.js
            jquery.min.js
            etc.js
    %body
        %h1 I do enjoy filters!

I think such filters could be upgraded to support appending additional files, but again, I dont really have a place for such things that aren’t a necessity to a fully featured template engine.

In Summary

This release requires cython >= 0.13 and lxml. I use this regularly on Windows 7 x64 and linux. The 0.1.4 release can be found on pypi, http://pypi.python.org/pypi/dmsl/0.2 and you can follow development at github, https://github.com/dasacc22/dmsl

Advertisements

Python implementation of HAML

UPDATE Dmsl is an active project on github and I am looking for active developers that can submit any bugs they may come across. I have been using it in production environments for nearly one year as of 12/2011. Refer to the repository README for an in depth look at current development, http://github.com/dasacc22/dmsl/blob/master/README

So it has been a busy past few days. During which I have written up an implementation of HAML in python. The closest thing I saw to this before was GRHML, whose site seems non-functional and I had a hard time finding information on its status. So without further ado, let me introduce DAML, my HAML implementation.

:title = 'Hello World!'

%html
    %head
        %title {title}
    %body
        :include('header.html')

        :greeting = lambda x: '%p Hello, {0}'.format(x)

        :greeting('user')
        :greeting('me')
        :greeting('world')

        :items = ['a', 'b', 'c', 'd']

        :def is_last(i):
        :   return len(items) is i+1 and '(class=last)' or ''

        %h2 Loop Method 2 (python list comprehension)
        %ul
            :['%li{0} {1}'.format(is_last(i), x) for i, x in enumerate(items)]

        :include('footer.html')

Ok, now this isn’t all it does but let me talk about a couple things first. For one, this project, as of today, is only 3 or 4 days old. Its hard for me to recall b/c I’ve poured a lot of time into trying to make this run as fast as possible. Today on the other hand has produced a lot of slow downs and hackish code for implementing features like Django code blocks. For you XSL people (which would be me) that means a way to name and call templates from other templates. So anyway, a lot of this code from today (and yesterday) needs some love. With that said, lets go over some differences between HAML and DAML when it comes to marking up HTML.

You’ll notice right away that declaring tags is straight forward and precisely the same. I’ve never actually used HAML so I dont know its exact syntax, but I spent a lot of time perusing the documentation for HAML. Now the actual text processor in DAML is quite fast for building documents. At one point during development, when i was still testing speeds in comparison to HAML, I had results along the lines of 0.21ms processing time for DAML versus 2.4ms processing time for HAML. This was for plain-jane HTML declaration. Lets look at some of that now with DAML.

%html
    %head
        %title Good Stuff

    %body
        #header Some stuff here
            and indentions of plain text
            will all be part of this div

            .span while this child div whose class="span"
                will be embedded within the above div

            while this text is tailed
            and multilined too

        %p and heres some random content too
            %strong that can be played
            however you like.

        %p one thing worth noting is that new tags
            %strong need
            to be on
            %em new lines
            so keep that in mind.

All of this renders fine of course. But I still have a TODO list for handling stuff unrelated to python expression evaluating. Namely comments, escaping, whitespace control. This is pretty much in line with HAML thus far although I often see blank lines in HAML appended with equals-sign and I cant recall what for. Regardless, the next thing to note is the use of variables. Which loosely follows the string.Formatter (and would fully support it if not for speed issues at the moment, but soon hopefully). Lets go over an example of setting and using variables.

:title = 'Hello World!'
%h1 {title}

Simple, eh? Now here’s the deal. There is a sand-boxed python eval going on. All lines starting with :colons are getting added to an evaluation queue that gets compiled and eval’d. Currently you would tag variables just like using string.Formatter and in the future, this will support all the goodies associated with it. Currently though whats going on is its simply accessing the variable declared (versus string.Formatter being setup with the documents namespace and getting called). Never-the-less, this works fine so far. Lets look at some other things we can do with “:”

:l = []
:for x in range(20):
:    l.append(x)

%div
    :['%p {0}'.format(x+1) for x in l]

Notice first, that you can basically freely declare normal python code blocks. You can also declare functions to make use of as shown with the initial intro document. I would say, ideally, the syntax I would want to go for in the majority of cases is something like (for-loop with plain-text block not yet functional)

#body
    :items = ['a', 'b', 'c', 'd']
    :is_last = lambda i: len(items) is i+1 and '(class=last)' or ''
    %ul
        :for i, x in enumerate(items):
            %li:is_last(i) {x}

    #div.class(attr=val, attr2=val2) oh and here's this too

So this is something I am working towards, but you can totally declare :func(*args) inline right now and it works. Notice, going back to the plain-text that you can freely declare %tag#id.class(attr=val) in almost any order, the only except is you can not start a line with (attr=val) nor can you declare (attr=val)(attr=val) though the latter may be added. Part of my TODO list is to have the ability to span attributes across multiple lines.

Ill touch on one last bit here, I implemented something similar to django blocks (at its most basic level). Its a wee bit limited until i implement multiline filter options as part of the preprocessor I *just* started working on not but 4 hours ago. But let me show what the final syntax would look like

:extends('template.daml')

:block header
    %h1 {title} OVERRIDE!

Now the above doesn’t work until i finish a bit on this preprocessor but to show you how that will work, let me show you how the above does currently work

:extends('template.daml')

:block('header\n    %h1 {title} OVERRIDE!')

Now that does work, and as you can see, my preprocessor is basically going to go through for indented portions of text under colon directive calls and push it into the related function thats in the sandboxed globals. This basically means that you can write all sorts of text filters and everything really really easy b/c then back on your end you may have a need to do “blank” with “blah. So..

my_python_code.py
-------------------------
def my_filter(s):
    s = s.splitlines()
    for x in s:
        pass # do blank with blah
    return s

I’ve basically tried to unify a number of features of HAML here so that anything can be easily overridden or customized and extended. Nothing thats just built-in and untouchable. There’s even more to come surely but I am honestly way burnt out on this.

Let me lastly explain just how FAST this works currently. Now before I went in today/yesterday adding in all these crazy bits and pieces, I grabbed the bench suite from genshi that benchmarks all the most popular python templating engines. I wrote my template to do everything to spec as the others were doing and these were my results.

Mako: 0.38ms
DAML: 0.44ms
Cheetah: 0.66ms
Genshi-Text: 0.96ms

and after that it just trailed off into really big numbers with Genshi coming in at 1.5ms?? and django at 2.4ms?? So this thing has the potential to be FAST. Now since the hackery I’ve pulled in the past number of hours I’ve actually almost doubled my processing time and the above test for DAML now runs at 0.85ms but keep in mind i have done no optimization/refactoring/code-cleanup/obvious-bug-fixes-needed to the codebase i just recently committed. So that portion needs some love and I’d love to get the speed back down to what it was just early yesterday morning.

Now, if your interested in the project, pleeease check it out at github, http://github.com/dasacc22/dmsl and play around with it. I’m going to slow my development down to a crawl comparitively as I’m really burnt out. I wont be implementing any new features and I’ll be cleaning up my code base yet again and trying to write beautiful code to everything ive hacked together today/last-nite.

And please feel free to contact me by email or comments or however regarding this project. I would like to get the code cleaned up so that I can collaborate on different things to really bring this project up to par.

Thanks, and enjoy
http://github.com/dasacc22/dmsl

IBM Sliding Puzzle Contest

So someone passed onto me a pdf for an IBM sliding puzzle contest. Basically, it consists of a 3 row by 3 column puzzle with one empty space, you know the ones, and you have to write a piece of software that solves for the answer. The instructional pdf suggests that while its acceptable for your answer to be over 20 moves, it should optimally be about 20 or less and be relatively quick.

At first I had no clue about how to do something like this but found the idea very interesting. Four hours later I had a python script that solves for all possible solutions up to how ever many moves you choose it too. Turns out the shortest answer is 12 moves, according to my script. I haven’t validated the 12 move answer, but i did validate a 14 move answer by hand with success (which was actually a twelve move answer with a repeat move making it 14), and i see little reason for the 12 move answer to be wrong.

Anyway, here it is in all its glory

puzzle = ['0', '4', '2', '5', '8', '3', '1', '7', '6']

answers = []

possible_moves = [
    [lambda p: swap(p, 1, 0), lambda p: swap(p, 3, 0)],
    [lambda p: swap(p, 0, 1), lambda p: swap(p, 2, 1), lambda p: swap(p, 4, 1)],
    [lambda p: swap(p, 1, 2), lambda p: swap(p, 5, 2)],
    [lambda p: swap(p, 0, 3), lambda p: swap(p, 4, 3), lambda p: swap(p, 6, 3)],
    [lambda p: swap(p, 1, 4), lambda p: swap(p, 3, 4), lambda p: swap(p, 5, 4), lambda p: swap(p, 7, 4)],
    [lambda p: swap(p, 2, 5), lambda p: swap(p, 4, 5), lambda p: swap(p, 8, 5)],
    [lambda p: swap(p, 3, 6), lambda p: swap(p, 7, 6)],
    [lambda p: swap(p, 4, 7), lambda p: swap(p, 6, 7), lambda p: swap(p, 8, 7)],
    [lambda p: swap(p, 5, 8), lambda p: swap(p, 7, 8)]
]

def serialize(p):
    return ''.join(p)

def swap(L, m, t):
    if L[t] is '0':
        L[t] = L[m]
        L[m] = '0'
        return serialize(L)
    
def no_dups(S):
    if len(S.split("-")) != len(set(S.split("-"))):
        return False
    else:
        return True

def search(tree, index=0):
    if index < 12:
        for each in tree:
            for i, val in enumerate(list(each.split("-")[-1])):
                if val is '0':
                    generation = []
                    for move in possible_moves[i]:
                        result = each+"-"+move(list(each.split("-")[-1]))
                        if "123456780" in result:
                            answers.append(result)
                        elif no_dups(result):
                            generation.append(result)
                    search(generation, index+1)

search(["-"+serialize(puzzle)])
shortest_answer = sorted(answers)[0]
print "=========="
print "Shortest Answer: " + str(len(shortest_answer.split("-"))-2) + " Moves"
print "++++++++++"
print shortest_answer

Edit: I just thought i would add,
yeah so the thing that was the turning point for solving this thing was how you look at solving the puzzle. Ive always looked at those puzzles as, ok what can i move into the empty space and rotate these pieces around and this and that but thats totally the wrong viewpoint. The way to visualize solving the answer is to look at the empty space as your focus and to move the empty space around, pushing the other numbers around into place. The puzzle suddenly becomes eaiser to solve by hand on your own and thats how the program solves for the answer too, by moving the empty space around, not trying to calculate how to get a particular number to its destination

Google Chrome, Os’, and Web Development

Ok so everyone, their mama, and her pet hamster have written an article about google chrome. Ive also read an interesting article that some-what theorizes on a google os based on chrome in the distant future. Only interesting b/c of what I have been planning to do.

First off, I read about people complaining over initial memory consumption of Google Chrome. I dont know why but I feel a need to state my opinion on the matter. What I care about most is a responsive system and chrome does that, short and simple. If i were to liken chrome to something id say starting an instance of google chrome is like typing

$> ls

at a command prompt. Its freaggin fast. Period. Running chrome nay interferes with most anything else i do. That is using it on a core 2 duo laptop with 2 gigs of ram and a amd64 desktop with 1 gig of ram. Just a quick statement of facts. Anyway, im not really interested in the distant future, what i AM interested in is the possibility of now. Honestly with very few modifications i would use chrome as a full shell replacement, on both linux and windows. Here’s my wish list,

  1. A better file manager
  2. the ability to launch local programs from the address bar
Ok so thats all i have for the most part. Frequently used programs, I could simply bookmark. Launching them from the addy bar would be very similar to what Ive always enjoyed doing on linux using the likes of katapult in the past and gnome-do nowadays (which does alot more interesting stuff). Ubiquity is a mozilla project that has a similarly driven concept of a type-into pop console to accomplish tasks (a little more complex than just launching a program). To accomplish this my local computer would need to be indexed or the standard entries for programs installed would need to be, but id prefer the first so i can search my computer for a file
local:[search-term]
anyone? I think alot of this couldn’t really be accomplished without patching some code (mostly the use of the addy bar).
As for the file manager, the builtin file manager is of the likes of firefox. just a point and click and launch scheme. An actual robust file system could be written as a local webapp. Chrome is like a staging ground for writing a new breed of applications as I see it, I actually just wrote something today to display an index of my movie collection and made a shortcut with chrome, works beautifully. Yes it runs in firefox and anything else for that matter, but would i ever use it in such? likely not b/c when i want to watch a movie, i want to click an icon and i WANT it NOW. I dont have all day to wait for something to start up. Well I do, but its a real buzz-killer.
Ok enough ranting. I am looking at chrome as a means to develop desktop centric applications (one of which is a music app based on the likes of the sndobj library that would allow multiple people to mix at the same time) and I think the best first place to start is to write a file manager and be able to bookmark applications i use and also index my local drive so i can search it. the urls wont be the prettiest with accessing 127. but it will do for now. Then on X startup or on windows startup, chrome is launched instead of gnome or kde or fluxbox or my beloved openbox which has always come to save my day in one way or another, or explorer.exe
Those are my ideas, I will be starting some of them soon and developing in python/cherrypy (unless theres due cause for something else) and then looking into investigating how to create a windows service. If anyones interested in collaborating, feel free to contact me.

Project Eulier and XSLT

Well when i came across Project Euler and saw the first problem, i naturally did what anyone in their right mind would do … solve it using xslt

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
    <xsl:output method="text"/>
    <xsl:variable name="iterations" select="1000"/>
    
    <xsl:template name="sum_multiples">
        <xsl:param name="i">0</xsl:param>
        <xsl:param name="incrementer"></xsl:param>
        <xsl:param name="result">0</xsl:param>
        
        <xsl:choose>
            <xsl:when test="$i < $iterations">
                <xsl:call-template name="sum_multiples">
                    <xsl:with-param name="i" select="$i + $incrementer"/>
                    <xsl:with-param name="result" select="$result + $i"/>
                    <xsl:with-param name="incrementer" select="$incrementer"/>
                </xsl:call-template>
            </xsl:when>
            
            <xsl:otherwise>
                <xsl:value-of select="$result"/>
            </xsl:otherwise>
        </xsl:choose>
    </xsl:template>
    
    <xsl:template match="/">
    <xsl:param name="three">
        <xsl:call-template name="sum_multiples">
            <xsl:with-param name="incrementer">3</xsl:with-param>
        </xsl:call-template>
    </xsl:param>
    <xsl:param name="five">
        <xsl:call-template name="sum_multiples">
            <xsl:with-param name="incrementer">5</xsl:with-param>
        </xsl:call-template>
    </xsl:param>
    <xsl:param name="dups">
        <xsl:call-template name="sum_multiples">
            <xsl:with-param name="incrementer">15</xsl:with-param>
        </xsl:call-template>
    </xsl:param>
    <xsl:value-of select="$three + $five - $dups"/>
    </xsl:template>
</xsl:stylesheet>

the same thing in python could go something like

three = [x for x in range(1000) if x % 3 is 0]
five = [x for x in range(1000) if x % 5 is 0]
print sum(set(three + five))

Edit: Im still learning python, well after posting this and on the ride home i realized i could define my list with one line.

print sum([x for x in range(1000) if x % 3 is 0 or x % 5 is 0])

CherryPy, SndObj, and SVG

Ok so sometime back when i first started tinkering with pysndobj, i was toying with some ideas for a user interface. Primarily i come from a web background and i decided to toy with the idea of a web frontend to a sndobj thread(s) that would write to an output stream that multiple people could connect to to work collaboratively together semi-realtime. Yeah, it sounds a bit amibitious but anyway I decided to take the time to toy around with SVG as well, which ive never worked with, to see what I might come up with.

Ultimately, I got this, cpsndobj.

This is the source code to my simple cherrypy/sndobj/svg demo. Basically I have an svg knob that i made that you can manipulate by clicking and dragging the mouse down on it. Unfortunately with a browser you cannot lock the mouse down in place so it can seem a bit odd if you reach the edge of your screen. But anyways, tar -xzf this bad boy and ./python cpsndobj.py and you will find a local webserver at http://127.0.0.1:8080/. Of course you need to sudo easy_install-2.5 cherrypy if you wanna use it. The rest is javascript(jquery if i remember correctly(been a while since i looked(i think this is why i like python, avoiding all these tags(woot!)))). And I believe I have it set to connect to a jack server so you’ll want to change it to SndRTIO if you want otherwise or start your jack first. After you get it up and running, visit the local address /on to turn on the modulating frequency the visit the root of the site to display the svg knob. Now click and drag it. and youll see it do its thing. Things to note, if your using internet explorer, you’ll need to install the adobe svg viewer (though ive not tested it in IE). Here in firefox land, we apparently like things to run slow so if youd like to experience a smooth svg knob .. experience, then get opera installed and notice the difference.

Uh but yet, i dont use opera past playing with an svg knob and playing flash movies that dont lock up my browser in linux.

One might say, ultimately i basically have a tinker toy thats ok (woot!).

–Edit: I suddenly realized the fallacy of my “i like python” statement above, python still has paranthesis.. duh..