I’ve just finished some contract work on the Web App www.VisualStager.com. This is a large scale project that build on top of Express, Mongoose, Backbone, F, and PsuedoClass.

Since Express and Backbone are both lightweight and flexible, it was pretty neat to see them working a part of a well designed large scale production app. My team of four developers had just under two weeks to build out an admin interface for the entire app.

Visual Stager is an interior design app that depends on three.js and clever mathematics to position 3D furniture models inside a 2D photo of a room. We needed to create admin pages for the Account, User, House, Photo, Scene, and Furniture collections. While the experience is still fresh in my mind, I’m going to memoize some interesting takeaways.

Pure Backbone is too lightweight for large scale apps

In a mature framework like Rails or Django, building an admin page is trivial. In our case, we depended on F.ListComponent to render collections of items as tables or lists without boilerplate. F.Component organizes the following into a logical unit of application code:

• A Backbone View
• A Backbone Model
• Handlebars templates
• Public methods
• Sub-components

F Components address one of the main complexity headaches found in backbone apps – There is no Controller class. Additionally, we can subclass F Components using the PsuedoClass inheritance pattern.

No Associations

In ActiveRecord we have “associations”. Imagine a User in our app is a member of many accounts. When we delete the user, we want to automatically remove that user from all accounts it belongs to. This is trivial with ActiveRecord. However, the Mongo/Express stack is not mature enough to have abstracted this behavior. We had to write the association logic manually in our server-side controllers.

Async

This is where Express gets fun. As part of the admin behavior, I found myself writing backend operations that run on large collections of MongoDB documents. The asynchronous style of node.js lends itself perfectly to this type of operation. The async library provides operations that operate on collections, similar to _.each and _.map – the difference between async and underscore is synchronicity. Async iterators may operate on each member of a collection asynchronously.

I’ve been thinking about the internet in terms of Magic. Our phones are portals in our pockets, opening a small window into another world. I like the magic metaphor because it encourages me to think about the internet as a place where fantastic and wonderful things can happen. But can opening up a web browser inspire the same feelings as reading about fictional magic?

Reality Check – the internet isn’t real. Code doesn’t actually DO real things. For me, the excitement I felt when the The Subtle Knife opened a portal into a parallel world was more thrilling than anything I’ve felt on the internet. The Internet can do all sorts of amazing things, but I have come to the conclusion that the price that we pay for the internet is roughly equal to the benefit we get. An equelibrium of sorts. We have a fantastic library of knowledge at our fingertips, but we waste time on Facebook and buzzfeed. We have instant communication all accross the world, but we end up spending more time investing in virtual relationships, and less time with friends in the real world. GPS devices can magically direct us to a place we’ve never been before, but by depending on GPS we are lost without it.

Actually, maybe this isn’t so unlike fictional magic. In most fictitous magical worlds, the magic does enact a kind equelibrium. Think of The Lord of the Rings – magic is a burden that comes with a price. In The Lord of the Rings, and The Golden Compass, both trilogies are concluded with the forfeiture of Magic. Even in Harry Potter, Magic is a force that is equally available to good and evil.

Pushing the metaphor further reveals a score of additional parallels between Magic and Technology. There is a currency that powers magic and a currency that powers technology. After Gandalf’s first encounter with the Balrag he was drained to the point of exaustion. Your smartphone works the same way. We study magic to learn it’s secrets. I’m currently a student at the Hogwarts for Hackers. Our efforts to become more skilled magic users parallel our efforts for personal growth. We aim to harness Magic for pleasure and profit. Like the internet, The Force from Star Wars is a great source that we tap into to manipulate and enhance our lives.

There are Factions that contrast in philosophy, design and implementation. Think of the five “Colors” in Magic: the Gathering. Green is the magic of nature and fecundidty, while red is the magic of Mountains and fire. These factions can interact with varying levels of synergy and incompatibility. Compare these colors to (for example) the functional asycronous style of node.js and the more traditional linear models of Ruby and Python.

There is one important feature of technology that is missing from magic: Layers of Abstraction. In web development we rely on layers of abstraction to build projects so capable and advanced that they may be indistinguishable from magic to and end user. This is the distinction that differntiates magic from technology. Magic is technology that we didn’t invent – just discovered. Once we understand magic, we forfeit magic – and it becomes technology.

There’s one more important parallel between magic and technology. Alladin’s genie couldn’t do it, neither could Dumbledor, and Gandalf may have come closest of all. Magic cannot conquer death. So conclusion I’ve come to is this: Don’t spend your life on technology… or magic. Remeber the equelibrium, and try to be a force that leverages technology for the better.

WebRTC is pretty tricky to work with right now. As of June 2013, support in Chrome and Firefox Nightly is fragmented and incomplete. The best explanation of WebRTC that I have found is the WebRTC Spec Draft from the W3C. This pedantic document is filled with jargon specific to Internet Connectivity Establishment. To elucidate the spec, I put together a list of descriptions for the most fundamental concepts.

Proceed with Caution! Both Chrome and Firefox are still a long way from compliance.

ICE - Protocol for Network Address Translation - Interactive Connectivity Establishment (ICE): A Protocol for Network Address Translator (NAT) Traversal for Offer/Answer Protocols. ICE agents include STUN and TURN servers.

STUN - Session Traversal Utilities for NAT (STUN). Peers connect to a stun server to find their public facing addresses, AND other info on how that peer may be reached fron the internet.

TURN - Traversal Using Relays around NAT (TURN). Like a stun server, but can relay data from one peer to another if the clients are unable to connect directly. Every TURN server is also a STUN server.

NAT - Network Address Translation - When a signal passes through a router, the router may alter the port number – effectively creating a “public facing address”. The public address is effectively mapped to a different address behing the router.

SDP - Session Description Protocol - (Wikipedia) - a format for describing streaming media initialization and connectivity parameters.

An SDP example from someRTCPeerConnection.localDescription.sdp looks like this:

v=0
o=Mozilla-SIPUA-24.0a1 15165 0 IN IP4 0.0.0.0
s=SIP Call
t=0 0
a=ice-ufrag:1f8009b2
a=ice-pwd:04f106d867cbba054729ae220b5c9618
a=fingerprint:sha-256 62:5F:18:CD:23:D3:BD:D9:C9:B4:42:BD:3B:07:2A:6A:EE:B0:06:59:6D:A3:B3:C8:E4:FA:99:B3:37:5E:56:FA
m=audio 41438 RTP/SAVPF 109 0 8 101
c=IN IP4 199.87.82.66
a=rtpmap:109 opus/48000/2
a=ptime:20
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-15
a=sendrecv
a=candidate:0 1 UDP 2113667327 10.0.1.46 65468 typ host
a=candidate:1 1 UDP 1694302207 199.87.82.66 41438 typ srflx raddr 10.0.1.46 rport 65468
a=candidate:0 2 UDP 2113667326 10.0.1.46 57425 typ host
a=candidate:1 2 UDP 1694302206 199.87.82.66 38208 typ srflx raddr 10.0.1.46 rport 57425
m=application 37652 SCTP/DTLS 5000
c=IN IP4 199.87.82.66
a=fmtp:5000 protocol=webrtc-datachannel;streams=16
a=sendrecv
a=candidate:0 1 UDP 2113667327 10.0.1.46 55401 typ host
a=candidate:1 1 UDP 1694302207 199.87.82.66 37652 typ srflx raddr 10.0.1.46 rport 55401
a=candidate:0 2 UDP 2113667326 10.0.1.46 61947 typ host
a=candidate:1 2 UDP 1694302206 199.87.82.66 46592 typ srflx raddr 10.0.1.46 rport 61947

ICE Agent - A peer trying to connect to another peer via ICE. I have also heard this term used to describe an arbitrary node in the Internet Connectivity Establishment Handshake.

The following definitions borrow from the IETF ICE Spec

ICE Peer - From the perspective of one of the agents in a session, its peer is the other agent. Specifically, from the perspective of the offerer, the peer is the answerer. From the perspective of the answerer, the peer is the offerer.

ICE Transport Address - The combination of an IP address, port and transport protocol (such as UDP or TCP) port.

ICE Candidate - A transport address that is a potential point of contact for receipt of media. An agent may have many candidate addresses – each consisting of an IP address and port number. Candidates might include:

• A transport address on a directly attached network interface (a “host” address)
• A translated transport address on the public side of a NAT (a “server reflexive” address)
• A transport address allocated from a TURN server (a “relayed” address)

Candidates have properties – their type (server reflexive, relayed or host), priority, foundation, and base.

ICE Component - A component is a piece of a media stream requiring a single transport address; a media stream may require multiple components, each of which has to work for the media stream as a whole to work. For media streams based on RTP, there are two components per media stream – one for RTP, and one for RTCP.

Helpful WebRTC Resources

Mozilla Hacks blog post mentions some of the discrepancies between chrome and Firefox
Webrtc on Chrome for Beginners
General Description on IETF
WebRTC Spec Draft from the W3C
Peer.js – This WebRTC abstraction library worked really well for me

The Content-Type http request header specifies the content type of the http request payload. The Content-Type header is NOT tied to the content type of the response sent by the server.

Here’s an example using pure JavaScript to make an asynchronous HTTP request from the browser.

var joinRequest = new XMLHttpRequest();  
joinRequest.onload = function() {  
  if (this.status === 200) {  
    console.log('JOINED!', this.response); // this.response is NOT affected by 'Content-Type'
  } 
};

// 'Content-Type' header tells the server what type of data we are sending
joinRequest.setRequestHeader('Content-Type', 'application/json');  
// 'Accept' header tells the server what content types we can accept
joinRequest.setRequestHeader('Accept', 'text/plain'); 
joinRequest.open('POST', '/join', true);  
joinRequest.send(JSON.stringify({name:'Charles'}));  

Don’t confuse Content-Type with dataType, accepts, and Accept.

dataType, and accepts are jquery.ajax options.

• dataType helps jqeury correctly process the server’s response.
• accepts tells jquery to include headers in the request that inform the server what type of data we are interested in.
• Accept is an http request header that informs the server which data types we are interested in. The value for the Accept header must be a semicolon-separated list of MIME types.

I wanted to install Node on a free AWS ec2 instance. As is so often the case with ec2, I had to jump through some hoops. Hopefully my own experiences can save you some time.

As of June, 2013 Node runs perfectly on on Ubuntu 12.X LTS and 13.x. You can $ apt-get install nodejs but this will install node 0.6.0.

If you already installed node 0.6.0, uninstall with

$ sudo apt-get remove nodejs

# If needed clean up symbolic link created when we installed node js
$ hash -r

Add Chris Lea’s PPA to the list of locations that apt-get searches, and get the package.

$ sudo add-apt-repository ppa:chris-lea/node.js  
$ sudo apt-get update  
$ sudo apt-get install build-essential  
$ sudo apt-get install nodejs  
$ sudo apt-get install npm

Now you can npm install, npm update etc

For the record – I also was able to build node from source by following the steps in this excellent blog post on cuppster.com.