Tuesday, April 2, 2013

Thinking in Promises

This morning I had an epiphany: of course some people find promises to be "easy" and "intuitive" at the same time while others find them "hard" and "unwieldy":

Programming in promises requires thinking about type signatures.

Depending on your perspective, this is either a great thing or The Worst Thing EverTM

People who have programmed in statically typed languages before are used to thinking about parameter types and return value types. For these people, it is easier to reason about a function from a URL to a Response object in terms of (URL) => Response rather than (URL, (Error, Response) => void) => void (using jsig notation). The former clearly expresses the expected input and output of the function, while the later only has input which is presumably being evaluated for its side effects.

People who are used to programming in dynamic languages primarily aren't used to having to think in terms of types and signatures, so this additional overhead contributes to their perception of promises as being too hard, too mysterious, or too much to think about. Programmers used to thinking about types are already thinking about this, so this mental overhead is negligible, and the abstraction of promises makes code easier to reason about simply because it let's them reuse the abstractions, patterns, and mental models they're already used to.

My relativistic assertion is that neither approach is correct, just as neither The British colour nor the American color is correct. This assertion is not without controversy, but neither are Promises. I appreciate the clarity that having type signatures gives me in communicating about code to fellow programmers (including my future self).

Finally, I want to clear the air. There is an unfortunate amount of FUD on the side of some promise proponents, whom I've heard say things like "callbacks aren't composable". It is possible to do metaprogramming on callbacks, including things like control flow manipulation (parallelization), map/reduce, decorator pipelining, etc. It's just that this needs to be done either in an ad hoc way or using some purpose-built library like async (or any one of the dozens of other control flow libraries in npm). These are not always clear to reason about. I won't make any claims as to which is easier, but with promises, since they're just values, one can use the same value programming techniques like map and reduce that is typical in synchronous code.

Sunday, February 24, 2013

building F# compiler on Mac OS X 10.8 Mountain Lion

F# is a lovely mixed paradigm functional language targeting the Common Language Runtime, which works great cross-platform on Mono.

I had some trouble getting it installed on my machine, so this documents the exact steps I followed at the end of February, 2013, to get everything up and running. Initially I was following F# 3.0 in the Mac and Mono World. Dave's blog has a bunch of other great content which I recommend you check out.


- Download and run the installer for mono 3.x: http://www.go-mono.com/mono-downloads/download.html

In the console:


# Check installed mono version:
$ mono -V

# I get 3.0.4.

# add mono to pkgconfig path
# in your .bash_profile:
export PKG_CONFIG_PATH=/Library/Frameworks/Mono.framework/Versions/Current/Lib/pkgconfig/


# Git clone fsharpc:
$ git clone git://github.com/fsharp/fsharp.git

# Make sure brew packages are installed and up to date:

$ brew update
$ brew install pkg-config
$ brew install autoconf

# cd to fsharpc directory and comment out mono version check in ./configure:

$ cd ~/dev/fsharpc

# For me, lines 1764 - 1766

# #if ! pkg-config --atleast-version=$MONO_REQUIRED_VERSION mono; then
# # as_fn_error $? "\"You need mono $MONO_REQUIRED_VERSION\"" "$LINENO" 5
# #fi

# Run config script with correct paths:

$ ./autogen.sh --prefix=/Library/Frameworks/Mono.framework/Versions/Current/ --with-gacdir=/Library/Frameworks/Mono.framework/Versions/Current/lib/mono/gac

$ make

$ sudo make install

# verify the install:
$ fsharpc
F# Compiler for F# 3.0 (Open Source Edition)
Freely distributed under the Apache 2.0 Open Source License

error FS0207: No inputs specified


Now you're ready to go.

Sunday, January 13, 2013

call to network citizenship

Aaron wasn't a victim of a cruel Department of Justice, unless we all are. His death is tragic, and this country (both our culture and our society) is terrible at dealing with mental illness. And the grossly disproportionate charges sought by his prosecution, and the prospect of facing more than 30 years in prison for his alleged and victimless crime no doubt caused great stress for a sensitive, idealistic, brilliant 26-year-old.

But we should remember his work, and his message of openness, and take from it a renewed urgency to educate ourselves as citizens of a networked space.

He was an activist, fighting at the vanguard of liberalism and freedom in our bold new networked age. He understood what was at stake in an interconnected world, how easy it is for individuals to be subsumed and left powerless by faceless, distributed, protocol-based power structures.

As network citizens, we must treat ignorance and willful idiocy of networked power structures as no more acceptable than not knowing that there are 50 states, or that the Constitution was not signed in 1776, or that there are three branches of the American federal system.

The network is the dominant power structure of our lifetimes, and it is incumbent upon us to be conversant in this new territory, to understand the stakes, the parties involved, the competing powered and monied interests, and to recognize that we still have legitimate popular sovereignty over these several corporate and special interests. Citizenship is a duty for any individual caught living in an advanced society. Be a citizen.

Requiescat in pace, Aaron Swartz.


Aaron Swartz speaking at a PIPA/SOPA opposition rally in New York.
(Photo (c) selfagency under a Creative Commons BY-SA 2.0 license)

I reproduce here Aaron's Guerrilla Open Access Manifesto:

Information is power. But like all power, there are those who want to keep it for themselves. The world’s entire scientific and cultural heritage, published over centuries in books and journals, is increasingly being digitized and locked up by a handful of private corporations. Want to read the papers featuring the most famous results of the sciences? You’ll need to send enormous amounts to publishers like Reed Elsevier.

There are those struggling to change this. The Open Access Movement has fought valiantly to ensure that scientists do not sign their copyrights away but instead ensure their work is published on the Internet, under terms that allow anyone to access it. But even under the best scenarios, their work will only apply to things published in the future. Everything up until now will have been lost.

That is too high a price to pay. Forcing academics to pay money to read the work of their colleagues? Scanning entire libraries but only allowing the folks at Google to read them? Providing scientific articles to those at elite universities in the First World, but not to children in the Global South? It’s outrageous and unacceptable.
“I agree,” many say, “but what can we do? The companies hold the copyrights, they make enormous amounts of money by charging for access, and it’s perfectly legal — there’s nothing we can do to stop them.” But there is something we can, something that’s already being done: we can fight back.

Those with access to these resources — students, librarians, scientists — you have been given a privilege. You get to feed at this banquet of knowledge while the rest of the world is locked out. But you need not — indeed, morally, you cannot — keep this privilege for yourselves. You have a duty to share it with the world. And you have: trading passwords with colleagues, filling download requests for friends.
Meanwhile, those who have been locked out are not standing idly by. You have been sneaking through holes and climbing over fences, liberating the information locked up by the publishers and sharing them with your friends.

But all of this action goes on in the dark, hidden underground. It’s called stealing or piracy, as if sharing a wealth of knowledge were the moral equivalent of plundering a ship and murdering its crew. But sharing isn’t immoral — it’s a moral imperative. Only those blinded by greed would refuse to let a friend make a copy.
Large corporations, of course, are blinded by greed. The laws under which they operate require it — their shareholders would revolt at anything less. And the politicians they have bought off back them, passing laws giving them the exclusive power to decide who can make copies.

There is no justice in following unjust laws. It’s time to come into the light and, in the grand tradition of civil disobedience, declare our opposition to this private theft of public culture.

We need to take information, wherever it is stored, make our copies and share them with the world. We need to take stuff that's out of copyright and add it to the archive. We need to buy secret databases and put them on the Web. We need to download scientific journals and upload them to file sharing networks. We need to fight for Guerilla Open Access.

With enough of us, around the world, we’ll not just send a strong message opposing the privatization of knowledge — we’ll make it a thing of the past. Will you join us?

Aaron Swartz


July 2008, Eremo, Italy

Sunday, November 4, 2012

Allow me to recapitulate a few of the things I believe and care about

- om. connectedness
- the great opposition to entropy
- work on stuff that matters
- symphony
- excelsior
- be humble yet ambitious
- no ownership, no secrets
- start often
- when in doubt, work harder
- keep moving, moving is living
- you know less the more you go on
- explore
- honor awe
- keep a healthy disregard for the impossible
- be intentional
- play. play nice.
- waste time, money & resources
- break laws, policies, norms and taboos
- hack the world around you
- hack everything. be constructive.
- consume less, create more
- test limits
- challenge metaphors
- build models
- improvise
- travel light
- embrace ambiguity
- avoid pretension
- be better
- be stylish
- be selective & critical
- make truth & beauty
- leave a paper trail
- challenge perspectives
- be incorrigible
- be called a damn fool
- be arbitrary and capricious
- experiment more
- mad science, mad hacking, mad making.
- drink, eat, smoke to excess
- throw stones at cathedrals
- help people in need
- travel abroad. travel off planet.
- document everything
- embellish liberally
- win by your own rules

Tuesday, October 23, 2012

Thoughts on JavaScript from 38,000 feet.




Note: what follows is some stream-of-consciousness writing from the plane on my way back from EmpireJS in New York this past Monday. It's a reflection of some of my current thoughts on JavaScript as a platform.


Let's say I wanted to build a robot to do something really awesome, like make toast or drive in space or fly a rocket crane or something. And I wanted to be able to do that in an expressive, high-level language.
Here's the thing. JavaScript won on the web, because it is everywhere. But with new platforms, do we want to be doing it in JavaScript just because it's familiar?


What, actually, are the good parts about JavaScript? Not just Crockford's favorite language features - I mean in terms of the ecosystem.


Npm is obviously a good part. We've done a great job of finding a good level of granularity to promote incredibly productive code reuse, and at the same time built a community to build thousands of these modules. This is good, and we should keep it. Now, bear with me:


I like programming because, as Fred Brooks so brilliantly put it, it makes me feel like a wizard. You sit at a terminal, type a few arcane and idiosyncratic incantations, and suddenly this machine does things. That's cool. But I want to be able to build things faster. Here's the thing - and I know I'm something of an anomoly here - I don't actually like programming. I like making things. I like being able to see the result, sharing things with people, and doing zany or useful or silly or amazing or trivial things. But i dont like typing things. One of the things I love about code is the ability to change things.


I have very little tolerance for idle bitching. Unfortunately, I also tend to complain a lot. Maybe not complain, but criticize, in the sense of identifying things that could (and should) be better. I'm not always a very happy person, because I see so much badnesss around me. But then I give myself the power to change some of it.


In his talk at EmpireJS, @domenic characterized a lot of what we as JavaScript developers do as Stockholm syndrome - we repeat things to ourselves to convince ourselves that we like and enjoy some of the dumb things in JavaScript which no person in their right mind would actually enjoy. Life is too short, the list of problems to be solved too long.


I submit we should be embracing this prolific explosion of compile-to-JavaScript languages. People are making things which solve their on problems. You dont have to use it. Write in whatever you like. But dont feel obligated to write in things you don't like. Don't tolerate badness. Do you like coffeescript? Maybe, maybe not. Now, do you use modules written in coffeescript in your projects? If not, why not? It better not be for some silly reason. Here's the thing - it shouldn't matter what language the code is written in. JavaScript is our vm. Whatever tools we use to arrive at that JavaScript should be wholly immaterial. We need to stop caring so much about imolementations, and start caring more about interfaces.


I heard a crazy and seditious idea last night - to start publishing tons of C modules to npm. At first that sounded crazy: native modules are evil, etc, etc. But why not? There's an argument to be made that having a common implementation language is better for encouraging community contributions. But most modules, I would contend, are written in whole or in large part by a single author. We shouldd acknowledge and embrace this, and realize that the really interesting things we should care about are tiny piece of functionality - especially domain-specific things. We don't need another damn templating language - we need fun things like math and science modules. It shouldn't matter if someone wants to write in R or Fortran. The biggest platform advantage we can have is fostering the incredible environment of code reuse and productivity, built on top of the right abstractions and interoperability.


Rather than promoting some perverse and conformist orthodoxy, I want to be a crowded bazaar at the intersection of exotic trade routes going to every far flung corner of the internet. I want to encourage mad hacking for its own sake - just to see what we can make. I want us to adopt creative and generative constraints, and discard traditional ones. Color outside the lines. Mostly, I want to look out at the world around me and see everything as mutable, as hackable, and continuously improveable. I want to be able to see the world like Neo from the Matrix - and I want to help others see the world this way, too.

Wednesday, July 4, 2012

Work on stuff that matters



Some people ask me why I'm not doing my own startup. And the answer is I'm sure I will eventually - but not right now. For me, it's not about a big exit, but a big impact. There is a number of things that I could be doing with my time, including scrambling together a team to attack an idea largely for its own sake. But ultimately, I want to be im a place where I can tackle large ideas. That's how I approach my deicions right now: will this course of action position me to work on things that matter and make a difference in people's lives?

Right now, that path is software engineering. But importantly, I want to focus my efforts on the top layer, the membrane between the electronic bits and actual humans. The part that people see, touch, experience. If I do my job right in this area, they won't even notice the interface, but they'll experience joy, accomplish their tasks, or gain new understandings of themselves and the world around them. The technology is cool and shiny, and it's easy to get caught up in that, but I'm serious when I say the technology is only a means to a very human end.

So, the interface is what I want to create. But where do I want to do this? In his book The Passionate Programmer, Chad Fowler compares the career of an engineer to that of a musician. Growth requires practice, but moreover it requires collaboration and feedback with people better than you. Specifically, he advises "always be the worst player in any band you're in".

What's next

In developmental psychology, there's an idea called the zone of proximal development. This is the area of skills and knowledge that is just beyond your present level, and its this knowledge which is most engaging. Since it's similar to what you already know, you have a good foundation for developing the relations between concepts and facts. Since it's new material, it expands your knowledge and skills. The key then is to stay in this zone, to constantly push to take on new challenges, technologies, problems, in a way that builds on your existing experiences.

I feel I'm a pretty competent programmer, and I pick up on new concepts very quickly. Thus, my imperative is to surround myself with committed, passionate software players and creators. I want to be part of a world-class, high-impact, high-performance team. In service of the ultimate goal of working at the problem level - which will require a mix of technical, analytical, social, and business skills - I must seek out the best possible team in which to grow, learn, and contribute.

Finding the right team is tricky.

Great teams can only truly thrive in organizations with certain traits: Trust in team members over processes. Value people as a whole, not as a templated, arbitrarily-tilted job description. Encourage both internal and external communication of a shared vision, even at the expense of giving up some nominal control. Everyone should be empowered - and expected - to articulate the overall vision of the organization and understand how it relates back to their daily work.

The Adam Pisoni, Co-Founder and CTO of Yammer, gave a talk recently where he characterized heavily bureaucratic and process-driven organizations quite well. In these organizations, he says, "you won't do it wrong, you also won't do it better." This kind of aspiration to mediocrity is fundamentally antithetical to my world view. I want to move people forward.

Reach out to me on twitter @leJDen

Thursday, June 21, 2012

On Hypertext

I'm digging up some posts from previous blogs I've written that I think still have value. This is the first in a series. Originally published in 2011.

In reading Vannevar Bush's 1945 piece, As We May Think, in which he is credited for conceptualizing hypertext "associative indexing," I am struck by one crucial difference between the "trails" of his memex and the hyperlinks of the web circa 2011. In the memex, a trail could be created by the reader as he uncovered interesting links. These trails were primarily there for his own recollection, although they could be shared with others by explicitly exporting them. Content could be published with links embedded by the authors, but they could just as easily create their own that made sense in their specific context. Along with associative links between published documents, readers could contribute their own analysis, excerpts, and annotations in situ, so that a trail captured an association of thought more completely.

In today's web, links are the same for every every reader, placed only by the author. Some pages have a mechanism for reader-created links by means of comments, which invariably come at the bottom of the article, and which are frequently nothing more than spam. In any case, these are public, and difficult for the reader to later reference. Methods for annotation and recording thoughts are crude, through the use of blogs, microblogs (like this one), and social media. In this way, individual links can be shared with others, but the concept of creating custom "trails" of associated content is nascent at best, and fully unrealized at worst.

Bush also talks of people who would actively create these links - "trail blazers." In today's web terms, we would call these people curators or mavens - surfacing the best content through their Twitter streams or blogs. However, these sharing mechanisms are still treating content atomicly, and interestingness has a tendency to surface either very timely or very weird content, such that anyone following a stream of links from a popular aggregation source is likely to still feel lost in a sea of lolcats and headline news. Whether this surfacing is done by an individual or a community, such as reddit, it is still subject to a crude, partial approximation of Bush's vision.

The closest approximation to Bush's memex I can think of are wikis. They consist of vast collections of tightly hyperlinked content, due in part to a culture of what I'll call optimistic linking. That is, since they have a controlled domain and others can author content independently, an author can make any given term into a link. If a document exists for that link, then it is automatically referenced. If it does not exist, another author can come along and create that content. Wikis can range in scope from the "sum of all human knowledge" in the case of Wikipedia, to more narrowly focused around a given community or project, for example sites like pbwiki or wikia, which encourage the creation of new, topical wikis.

However, these still fall short of Bush's ideal. They primarily link to internal content. While they can link to external content, they cannot retain context of linking to an arbitrary point within a page, and - more importantly - they can only link one level deep. After a wiki reader clicks through to an external link, she is not longer on the wiki, and she can no longer create a link to an arbitrary associated page.

It is difficult to add annotations or new analysis. While most wikis are set up to allow anyone to contribute by creating and editing pages, they have a culture of preferring "content" rather than "metacontent". Since any edits made by one author are visible to all, there is a certain self-conscious act of "publishing" involved in editing a wiki. Further, an authors edits can be undone or refined by other authors, making a wiki non-permanent and poorly suited for an author's individual note-taking for later recall.

Is there a need for these additional capabilities? I think it depends on our aims in reading and processing the information we find online. If we are merely flitting from one article to the next, possibly stopping to blog or tweet or comment about it along the way as an aside, never to return or further synthesize the information, then our current scheme is sufficient. If, however, we use the web with a more knowledge- and research-centric orientation, then we will soon find it lacking, particularly for personal recall.

This is the aspect of hypertext that Bush most emphasizes, and one which I think is underrealized and potentially quite valuable. He characterizes his hypertext memex machine as "an enlarged intimate supplement to his memory." For simple facts and figures, a generic web search may prove to be immediate enough. "What's the movie with Tom Hanks and the volleyball?" for example. But for more complex intellectual concepts, a less transient, more personal means of assisting research, recall and serendipitous discovery of information is necessary.

Nicolas Carr excoriates hypertext - and computers generally - in his work "The Shallows," claiming that hyperlinks contribute to shorter attention spans and flighty trains of thought. But he's making observations on the current implementations of hypertext. I take issue with his conclusions even on the basis of the current state of the web in 2011, even given the current state of hypertext. But given the addition of certain key conceptual features, ad-hoc hypertext has the promise of greatly increasing our ability to leverage, process, and synthesize the vast swaths of written knowledge. We must invent the tools to enable us to blaze trails across terra semicognita.