Writing Boring Code

I have bad news. Using Go as a shared library on Android 5.0 is simply not a good idea. In fact, it’s a horrible idea. Android 5.0 introduced a new runtime called ART. This replaces JIT compilation (Just-In-Time compilation).

If you don’t know, a JIT analyzes runtime heuristics and modifies the compiled program to run faster. JIT is one of the reasons you might see javascript outperforming C code on the DNA regex benchmark that’s been floating around for  years.

ART actually precompiles dalvik bytecode to native code and runs that native code in its runtime. As a refresher, when you write java code for android, the java code gets compiled to java bytecode and then gets transpiled to dalvik bytecode to run on the dalvik virtual machine (so no one has to pay oracle any cash).

Tangent, there’s a new experimental compiler for android studio (based on intellij) that compiles java 7.x source directly to dalvik bytecode.

But, the ART runtime is much like the Go runtime, and the two fight over things. For the most part, it works and works well. But, when you begin trying to load native code such as PublisherAdView for ads that load in chromium webview that loads native code, bad things seem to happen.

It is essentially a no-go. For pure Go projects as is the initial target, this is a non-issue. But, for integrating Go into a normal Java app, this is a huge blocker. I’ve given up on using Go for normal apps.

I’ve actually stopped using Scala as well. I like Scala, and programming in a functional language (i.e. Lisp like) has really taught me a few things I appreciate greatly. There just hasn’t been much uptake from the other android developer on our team. He’s much older than me, in his 60’s. He doesn’t have anything bad to say, he just doesn’t pursue it or have much interest.

I really can’t blame him. It’s not just Scala, it’s anything that becomes too involved that’s not well documented. He dropped an email about fixing some SQL entries in a 2 year old word game I wrote but mentioned he wasn’t sure how to build the project.

The issue was two-fold. First, there’s five different versions of the app; google, google pro, amazon, amazon pro, nook. Second, two years ago Google didn’t have tools to help with this kind of stuff. As of maybe two months ago, they finally have stable tools to help with this.

I wrote a Makefile that would build all the apps and collect all the apk files into a single directory. If you’re running OSX or Linux, as most developers on our team are, then it’s trivial to build. If your running Windows, you’re shit-outta-luck.

Another tangent, I found a make.exe command for windows a year back for a browser extension plugin I wrote that worked pretty well, based on the work done by the git-for-windows team.

Still, me giving up scala is actually more like paying respects to Go. One thing I’ve really come to embrace with Go is being boring. Boring works. Boring is readable by people other than me. Boring is quickly buildable by people other than me.

One thing I’ve been considering is the golang/mobile repo added a Dockerfile. It’s really neat, boots up ubuntu, installs build tools, android sdk, go, gradle (used in android studio to build projects), and builds your source code for you. What’s neat is now Docker is available for windows. Even bootstrapping android studio and android sdk with necessary requirements can be a pain. But saying “did you install docker.exe? great just run this file” is really boring and works. This works whether you’re a developer or not.

Hell, even if I’m not writing go code, it works. It works for developers, it works for build servers, whatever.

So I’ve gone back to Java, I’m embracing the boring. I’m also typing “public static void” a lot.

I like Go like I like functional programming. It’s taught me many things and it’s certainly worth review.

Corporate Hackathons: The Fine Line Between Engaging and Exploiting

Corporate Hackathons: The Fine Line Between Engaging and Exploiting

Great write-up from a friend on the Campbell Soup Hackathon.

Git Email Notifications on Push

So I’m doing a private collab and hosting a git repo off my server. Got annoying pretty quick with these occasional emails “hey pushed xyz change that could affect abc for you, make sure to pull the latest”

Enter this script, post-receive-email

To get going with it, I did the following, (line breaks for readability here):

wget -O '/user/local/bin/post-receive-email' \
'http://git.kernel.org/?p=git/git.git;a=blob_plain;f=contrib/hooks/'\
'post-receive-email;h=60cbab65d3f8230be3041a13fac2fd9f9b3018d5;hb=HEAD'

chmod a+x /usr/local/bin/post-receive-email

Next, link to this in your repo with ln -s /usr/local/bin/post-receive-email hooks/post-receive and add something like this to config

[hooks]
mailinglist = "jane@email.com, john@email.com"
envelopesender = no-reply@email.com
emailprefix = "[GIT] "


And that will notify the mailing list of any push that occurs. There’s a number of other options worth exploring by reading the post-receive-email bash script.

Sublime Text 2 and multiple cursors

Two days ago I started using sublime text 2 for projects. I’m coming from a long sprint of VIM usage and I have to say that sublime is pretty awesome. In fact, I’d describe sublime as VIM + awesome. Unfortunately sublime isn’t free and not to be a glory taker, but I hope to produce something on par for free in the future.

One thing that really wow’d me into at least trying sublime was the proposition of multiple cursors. With that being said, this wouldn’t be my first foray into the lust for nonsensical features. But, only after a day of usage I can honestly say multiple cursors in a text editor is just a plain win.

Primarily, I’ve used VIM in the past for all but the most heavily-dependent-on-IDE tasks (namely, java). I love VIM. Just the other day I opened a 6.4GB mysql dump to make some minor changes by hand before passing it on to an AWK script for conversion to postgres (yeay 16GB of system memory), but sublime is, well, simply sublime.

Refactoring tools in an IDE typically consist of being scope aware and allowing you to rename method variables and class members. One thing no refactoring tool can touch though is editing multiple values at the same time. As I worked my way on the very first day through sublime on a python project, I found myself wanting to change a value set on five variables. Instead of a default value of `None`, I wanted it to (effectively) read `kwargs.get(”, None)`, and I thought, “ok! let’s try multiple cursors!”.

I moved into position and slammed `ctrl+d` five times and there they were, five cursors ready to alter the default value of five members.

Afterwards, I reflected on the wonder of how practical multiple cursors really were. “Is this just some cheap skate refactoring tool?”. No. It’s much more powerful. I think that’s one of the many reasons I’m a sublime convert now and hope to see this idea spread through free tools in the future.

Thanks for reading, try sublime, hell buy sublime, and feel the preposterous range for which it defines.

xmonad

During my lunch break yesterday, i ran across a couple comments on slashdot like “maybe emacs just works and thats why ppl are using it” and “tiling window managers are awesome”. So in my usual, screw it – lets see, i fired up emacs (not the first time) which ended pretty quick and I did some quick googling on a decent tiling window manager to try.

Enter xmonad, written in haskell no less (something I was going to learn after I wrap up my C# starter project, a calculator with a lexer so I can insert arbitrary words into a math equation, does proper math too, not just “eval(x)”). Anyway, within 20 minutes, I was sold. A tiling window manager is the best thing since sliced bread. Solves my window management issues I’ve been trying to fix since who knows when, things I’ve looked to for answers from desktop effects like scale, expose, overview of multiple desktops with drag and drop of windows, all that pretty flurry graphical stuff that gives you good feelings but then you rarely use.

Key Points: 

• Focuses on little to no need for the mouse
• The only window decoration is a 1 px border that changes color if it’s focused, you’d be surprised how much faster programs start without decorations
• with no decoration, dragging a window involves alt-click, which is all I do anyway so low barrier to entry for me, but floating a window is rarely needed
• you dont minimize, you tile so no need for decoration buttons
• alt+shift+c closes any window – this could be seen as annoying but everything centers around alt anyway so it becomes natural quick
• Navigating between tiled windows is dead simple, I don’t lose track of what’s opened.
• alt+space swaps to different layout arrangements (3 total by default) which handle any use case I have
• Moving windows between virtual desktops is easy
• Use on a multimonitor setup is where things really shine. If my first monitor is desktop 1 and my second monitor is desktop 4, I can easily swap them by selecting desktop 4 on monitor 1. No need to drag something over or anything like that.
• I can push windows around way to easy

I could probably type in all sorts of blurbs that might not really get the point across. If you’re bothered with window management, give xmonad a try and read up their quick guide to usage. Doesn’t take much time to get use to it.

I’ve honestly been putting off getting another monitor for my setup, i just imagined it being a pain to use but xmonad makes me feel like a master of window management and now I’m eager to get another monitor or two.

AndroidWeb, package for handling concurrent requests with multiple receivers, sqlite caching, and a built-in content provider

Recently, TSG released AndroidWeb, a networking library for android (of which I’m currently maintaining and the original author). This is a great library for handling concurrent networking connections throughout the activity lifecycle, is backed by an sqlite database for caching, and has a ContentProvider allowing share by intent to other apps from the cache.

Check it out, fork it, put in feature requests, and tell me what you think.

https://github.com/dskinner/AndroidWeb

I’ll be doing some follow up posts in the next couple weeks to provide tutorials over specific features as well as providing those as examples in the git source.

Fast Web Development with Damsel for Python

Time feels surmountable when looking forward. In retrospect, I see surmise.

My title to this post feels like a real plug piece, but I guess that’s what happens when you’re looking to get indexed. Recently, I tagged two new versions of dmsl in git. If you’ve never heard of dmsl, please go check out the README on github to see what it’s all about.

https://github.com/dskinner/dmsl

I started off implementing haml in python which gave way to a unique direction for the project. I’ve been using it quite actively in work and am continuously enamored with its simplicity and power. Of course I wrote it and there might be a sense of bias, but this isn’t the first or second or third or fourth attempt I’ve made to write something like this. Each of those projects were short lived in the same way that once they were mostly complete, they just flat out didn’t “have it” for me.

0.3-stable is a wrap up of features and fixes over the course of many months of use and has, in the end, proven reliable. This tag is available on github. There were still some annoyances in the code related to speed and the dependency on lxml though that I badly wanted to fix.

The speed issue was solely in the use of a class inheriting python’s Formatter to handle extensions to string substitutions. These substitutions alone could take three times as much of the time as the parser for the whole document. Granted, on a whole, this was still quite a bit faster then a number of template engines out there, but I felt this could be drastically reduced.

The second issue was the dependency on lxml. Simply put, I wanted that to go.

So recently I addressed these issues and tagged 0.4 in git. I’ve also made this tag available on pypi. If installing from pypi, the only dependency is a build environment and python headers for building the C extensions. If building from github, you will need cython 0.15.1+ installed.

0.4 also has another change with how context variables end up in the templates sandbox. Previously, these items were packed in a kwargs dict available to templates. In 0.4, this is no longer the case and those items are unpacked in the environment for use. Planned though is the ability to revert to the old behaviour as needed for upgrading older installs.

While I feel the changes in 0.4 are drastic, unit tests pass and I’ve upgraded my personal projects successfully to do a little stress testing while also using it in another upcoming public project with success.

So check it out and tell me what you think.

daml 0.1.4 Release, A Python Markup Language for the Web

Since my initial post outlining daml as a haml-for-python, a lot has changed. Now it has become something more and plays a new role in a bigger project I am working on. Let’s look at some of the new syntax changes.

CSS Selector Style Attributes

Declaring attributes now follows css selectors appended to a tag hash. For example,

%html
    %body
        #content.main[attr=val][apples=yes] some content

Inlining Tag Hashes

One can now inline tag hashes. The functionality is still being worked on and is broken beyond a single tier. The intent is to handle use cases such as,

%html
    %body
        %ul#nav
            %li %a[href=/] Home
            %li %a[href=/contact] Contact

        #wrapper #container
            #top
            #content

Implicit Python Declaration

Since my last post, all python code had to be preceded with a colon (:). Now, this is only necessary when embedding a function call in plain text (or calling a filter). For Example,

def greet(s):
    return 'Hello, {0}.'.format(s)

nav = ['www', 'www2', 'www3']

%html
    %body
        %p A greeting for you. :greet('Daniel')
        %ul for x in nav:
            %li {x}

The trade off to the above is explicit line breaks for multiline text as similarly done in python,

%html
    %body This is some text \
        that spans multiple \
        lines.

I am still putting alot of thought into this portion of the syntax and it will most likely see some changes in a number of months. Explicit line breaks can become rather confusing when handling complex mixed contents, but again, tools such as daml aren’t particularly suited for writing page content, but rather establishing layout.

New Filters

I added two new filters as a test, these may be removed in future releases (and migrated to my personal set of daml extensions). They are for declaring css and js files. I think the syntax speaks for itself in function,

%html
    %head
        :css /css/
            main.css
            extra.css
            lib/that_lib.css

        :js /js/lib/
            utils.js
            js.js
            jquery.min.js
            etc.js
    %body
        %h1 I do enjoy filters!

I think such filters could be upgraded to support appending additional files, but again, I dont really have a place for such things that aren’t a necessity to a fully featured template engine.

In Summary

This release requires cython >= 0.13 and lxml. I use this regularly on Windows 7 x64 and linux. The 0.1.4 release can be found on pypi, http://pypi.python.org/pypi/dmsl/0.2 and you can follow development at github, https://github.com/dasacc22/dmsl

Python implementation of HAML

UPDATE Dmsl is an active project on github and I am looking for active developers that can submit any bugs they may come across. I have been using it in production environments for nearly one year as of 12/2011. Refer to the repository README for an in depth look at current development, http://github.com/dasacc22/dmsl/blob/master/README

So it has been a busy past few days. During which I have written up an implementation of HAML in python. The closest thing I saw to this before was GRHML, whose site seems non-functional and I had a hard time finding information on its status. So without further ado, let me introduce DAML, my HAML implementation.

:title = 'Hello World!'

%html
    %head
        %title {title}
    %body
        :include('header.html')

        :greeting = lambda x: '%p Hello, {0}'.format(x)

        :greeting('user')
        :greeting('me')
        :greeting('world')

        :items = ['a', 'b', 'c', 'd']

        :def is_last(i):
        :   return len(items) is i+1 and '(class=last)' or ''

        %h2 Loop Method 2 (python list comprehension)
        %ul
            :['%li{0} {1}'.format(is_last(i), x) for i, x in enumerate(items)]

        :include('footer.html')

Ok, now this isn’t all it does but let me talk about a couple things first. For one, this project, as of today, is only 3 or 4 days old. Its hard for me to recall b/c I’ve poured a lot of time into trying to make this run as fast as possible. Today on the other hand has produced a lot of slow downs and hackish code for implementing features like Django code blocks. For you XSL people (which would be me) that means a way to name and call templates from other templates. So anyway, a lot of this code from today (and yesterday) needs some love. With that said, lets go over some differences between HAML and DAML when it comes to marking up HTML.

You’ll notice right away that declaring tags is straight forward and precisely the same. I’ve never actually used HAML so I dont know its exact syntax, but I spent a lot of time perusing the documentation for HAML. Now the actual text processor in DAML is quite fast for building documents. At one point during development, when i was still testing speeds in comparison to HAML, I had results along the lines of 0.21ms processing time for DAML versus 2.4ms processing time for HAML. This was for plain-jane HTML declaration. Lets look at some of that now with DAML.

%html
    %head
        %title Good Stuff

    %body
        #header Some stuff here
            and indentions of plain text
            will all be part of this div

            .span while this child div whose class="span"
                will be embedded within the above div

            while this text is tailed
            and multilined too

        %p and heres some random content too
            %strong that can be played
            however you like.

        %p one thing worth noting is that new tags
            %strong need
            to be on
            %em new lines
            so keep that in mind.

All of this renders fine of course. But I still have a TODO list for handling stuff unrelated to python expression evaluating. Namely comments, escaping, whitespace control. This is pretty much in line with HAML thus far although I often see blank lines in HAML appended with equals-sign and I cant recall what for. Regardless, the next thing to note is the use of variables. Which loosely follows the string.Formatter (and would fully support it if not for speed issues at the moment, but soon hopefully). Lets go over an example of setting and using variables.

:title = 'Hello World!'
%h1 {title}

Simple, eh? Now here’s the deal. There is a sand-boxed python eval going on. All lines starting with :colons are getting added to an evaluation queue that gets compiled and eval’d. Currently you would tag variables just like using string.Formatter and in the future, this will support all the goodies associated with it. Currently though whats going on is its simply accessing the variable declared (versus string.Formatter being setup with the documents namespace and getting called). Never-the-less, this works fine so far. Lets look at some other things we can do with “:”

:l = []
:for x in range(20):
:    l.append(x)

%div
    :['%p {0}'.format(x+1) for x in l]

Notice first, that you can basically freely declare normal python code blocks. You can also declare functions to make use of as shown with the initial intro document. I would say, ideally, the syntax I would want to go for in the majority of cases is something like (for-loop with plain-text block not yet functional)

#body
    :items = ['a', 'b', 'c', 'd']
    :is_last = lambda i: len(items) is i+1 and '(class=last)' or ''
    %ul
        :for i, x in enumerate(items):
            %li:is_last(i) {x}

    #div.class(attr=val, attr2=val2) oh and here's this too

So this is something I am working towards, but you can totally declare :func(*args) inline right now and it works. Notice, going back to the plain-text that you can freely declare %tag#id.class(attr=val) in almost any order, the only except is you can not start a line with (attr=val) nor can you declare (attr=val)(attr=val) though the latter may be added. Part of my TODO list is to have the ability to span attributes across multiple lines.

Ill touch on one last bit here, I implemented something similar to django blocks (at its most basic level). Its a wee bit limited until i implement multiline filter options as part of the preprocessor I *just* started working on not but 4 hours ago. But let me show what the final syntax would look like

:extends('template.daml')

:block header
    %h1 {title} OVERRIDE!

Now the above doesn’t work until i finish a bit on this preprocessor but to show you how that will work, let me show you how the above does currently work

:extends('template.daml')

:block('header\n    %h1 {title} OVERRIDE!')

Now that does work, and as you can see, my preprocessor is basically going to go through for indented portions of text under colon directive calls and push it into the related function thats in the sandboxed globals. This basically means that you can write all sorts of text filters and everything really really easy b/c then back on your end you may have a need to do “blank” with “blah. So..

my_python_code.py
-------------------------
def my_filter(s):
    s = s.splitlines()
    for x in s:
        pass # do blank with blah
    return s

I’ve basically tried to unify a number of features of HAML here so that anything can be easily overridden or customized and extended. Nothing thats just built-in and untouchable. There’s even more to come surely but I am honestly way burnt out on this.

Let me lastly explain just how FAST this works currently. Now before I went in today/yesterday adding in all these crazy bits and pieces, I grabbed the bench suite from genshi that benchmarks all the most popular python templating engines. I wrote my template to do everything to spec as the others were doing and these were my results.

Mako: 0.38ms
DAML: 0.44ms
Cheetah: 0.66ms
Genshi-Text: 0.96ms

and after that it just trailed off into really big numbers with Genshi coming in at 1.5ms?? and django at 2.4ms?? So this thing has the potential to be FAST. Now since the hackery I’ve pulled in the past number of hours I’ve actually almost doubled my processing time and the above test for DAML now runs at 0.85ms but keep in mind i have done no optimization/refactoring/code-cleanup/obvious-bug-fixes-needed to the codebase i just recently committed. So that portion needs some love and I’d love to get the speed back down to what it was just early yesterday morning.

Now, if your interested in the project, pleeease check it out at github, http://github.com/dasacc22/dmsl and play around with it. I’m going to slow my development down to a crawl comparitively as I’m really burnt out. I wont be implementing any new features and I’ll be cleaning up my code base yet again and trying to write beautiful code to everything ive hacked together today/last-nite.

And please feel free to contact me by email or comments or however regarding this project. I would like to get the code cleaned up so that I can collaborate on different things to really bring this project up to par.

Thanks, and enjoy
http://github.com/dasacc22/dmsl

Line 6 UX2 and Linux

I originally wrote a bad/sad review about the Line 6 UX2 not working under linux as can be seen here:
http://line6.com/community/thread/17663

I was looking to sell it recently when I decided to give it another look over and try out the drivers done for the PodXT devices by someone as can be found here:
http://www.tanzband-scream.at/line6/

The 0.8.1 release failed to compile so i checked out the trunk of the development version
$> svn co https://line6linux.svn.sourceforge.net/svnroot/line6linux
$> cd line6linux/driver/trunk
$> make
$> sudo make install

To my surprise, it worked fairly ok. My line 6 ux2 device lit up and seemed ready to go. One of the first things I noticed was that there was a constant buzzing out of the right channel monitor. This was alleviated by simply plugging my guitar in. Next, I read through the docs to get some general info. I went ahead and fired up alsamixer, increased pcm output and zero’d out the monitor. Note: the docs say to set pcm output to zero as they can be much louder then the guitar monitor. I had no trouble with this when i increased output to 75, volume was normal.

So then I fired up jack and played around with a couple settings. To my surprise I was able to get latency down to 1.5 ms according to ardour (2.87 according to qjackctl)! Either way, this was much lower then I ever managed with my edirol which normally clocked in around 15ms and was littered with xruns that were very audible in the recording. Normally around 22ms I could achieve good sound with no xruns. After a short recording session, I noticed qjackctl had logged numerous xruns but I never heard a thing. Once during the recording, ardour disconnected from jack but this could be totally unrelated.

This is fine news indeed and prompts me to want to hold on to this device. The bad news is I have been unable to get the microphone inputs working. I haven’t had time to look into it fully but hopefully after exploring the line6linux docs some more, I can have some success with getting this working.

So all in all, this is great. What I would really like to see is Line6 spend a few resources on this project or doing something themselves. But that aside, hardware wise (i guess), this little bugger rocks!