Nov 2012

Uniting anonymous crowds through collective collaboration

Can an anonymous crowd of people collaborate to collectively solve a problem? Public events (such as sports games) bring together many people who do not know one another. We wondered whether it is possible to bring a large number of people closer together by tasking them with a collaborative challenge. Each individual contributes part of the solution using a personal mobile device. Players need to take into account the overall progress of the crowd , as shown on a large, shared, public display.
We built and tested Collablocks, a crowd-based, local, collaborative game, during Facebook's CMU hackathon in 2012.

Collablocks is a real-time, browser based 3D building game.

On a large display such as a jumbotron or projector screen, players see a 3D grid of cube-like blocks. A see-through "skeleton" structure represents the goal for the current level. Individuals can join the game by accessing a publicly visible URL from their computers or mobile phones. Players are assigned a unique color and use a simple 2D grid to interact with the game. Each cell of the grid represents a block at the corresponding X,Y position on the shared 3D grid. Blocks can be toggled on or off by the player, creating or removing a block of their color at the top of the corresponding location. Changes to any player grid are instantly reflected on the public display. The goal of the game is to recreate the skeleton structure using player's blocks as closely as possible. Players are responsible for at most one layer of blocks on the public display, but need to take into account blocks created by other users. A shared progress bar visualizes the overall progress of the current level. Having to quickly react to other user's actions becomes the central challenge of the game.

Implementation

We used three.js/WebGL to visualize the public 3D display. The virtual camera continuously rotates and oscillates to reduce occlusion/visibility issues. However, this rotation is constrained to less than 180° in order to facilitate user's association of block location between the public and private displays. Also, blocks are semi-transparent in order to reduce occlusion.
The player's input interface is a responsive HTML5 page that maximizes cross-device compatibility as well as the amount of input space available on mobile devices.
The input and display pages are served by a node.js backend and communicate via WebSockets to minimize latency, an essential feature for fluid gameplay that allows players to visually associate "their" cubes with actions taken. We built on the SocketStream framework using an event-based architecture to keep server and clients in sync. Our backend uses Redis as a fast in-memory store containing session data as well as player's color and input state, allowing them to reconnect without losing access to previously created blocks. When a level is reached, the server generates the next level and broadcasts it to all current players and public display(s), which in turn update their UI. In fast-paced and error-prone environments commonly found at public events, it is important to establish a count of current players to determine a good vertical scale factor for the blocks, enabling the game to adapt to the number of current players. We obtain the count of current players by monitoring the websocket connections on the server side, while employing a periodic "heartbeat" to check on non-websocket based connections.

Results & Outlook

After building it in less than 24 hours, we successfully demoed Collablocks with a crowd of 30-50 users, winning second place at the 2012 CMU Facebook hackathon. One idea for future expansion of the game is to introduce a random assignment of players to different teams in order to create a sense of competition, while displaying current team results side by side on the public display. Furthermore, we see a potential to improve discovery and the onboarding process of the game by advertising ongoing games in the prospective player's vicinity.

Swrm is a platform for honest feedback on the web.

Aug 2012 – ongoing

Swrm is a platform for honest design feedback. We started Swrm after realizing how hard it is to get honest feedback on the web. Currently, online feedback either comes in the form of a 5-star scale with little context, or as part of discussion threads that are typically do not promote a culture of honest critique, making these platforms prone to unconstructive comments and trolling.

Swrm allows designers and creative thinkers to give and get feedback and insights on their work from a large panel of reviewers. Swrm helps creatives to get feedback from designers of all experience levels, improve their work, and learn new skills.

Implementation

We created Swrm as part of Luis von Ahn's Startup Lab class at CMU. I architected the system and together we built a first prototype within a few weeks. Swrm is a single-page, "soft realtime" web app using AngularJS for the frontend, which is bound to our Node.js backend via WebSocket pub/sub. This is facilitated by the SocketStream framework (a WebSocket/socket.io wrapper) built atop Redis' event infrastructure. We use MongoDB for persistent storage. We used HAProxy for reverse-proxying to Node since Nginx does not currently support WebSockets.

Nov 2012

Flox [Flock + Flux] is a particle-based iPad game. On a large, pannable game map, many small particles ("travelers") are generated with one of many randomized start and destination points ("Hubs"). Travelers move slowly towards their destination hubs. Travelers can also move along "routes", in which case they can move a lot faster, much like cars can travel faster on highways than on small roads. The point of the game is to create effective "routes" between common start point/end point combinations, creating a network of routes which help travelers reach their destinations faster. Players receive feedback on the efficiency of routes built and overall create a complex, dynamic, beautiful system in a playful way.

I envision future multiplayer versions to feature both discovery-based and constructive-collaborative gameplay. From an Experience Design standpoint, Flox aims to provide an Optimal Experience: to be intrinsically rewarding and to put the player in a state of Flow. Conceptually, the game is both absorptive and immersive, both entertaining and educational.

The game map is generated ad-hoc from a random seed, allowing future multiplayer (network-based) implementations to quickly reconstruct the map from a single set of coordinates. The game is architected in a way that allows lean network synchronization of the game state across different devices: Hubs are implicitly "encoded" in the random seed; individual particles need not be synced due to their stochastic nature (their overall spawning and targeting behavior is encoded in the hubs), leaving routes and user profiles as the only elements that would need to be synced dynamically.

I built the prototype shown in the video using iOS and Cocos2D in about 5 days, without prior knowledge of iOS. The Cocos2D engine abstracts away peculiarities of OpenGL, which allowed me to quickly reach minimum viable product stage. Some prior experience with C++ and Smalltalk helped smooth the transition to Objective C.

DepthSelect: Leveraging touch pressure to mitigate Pick Ambiguity

Nov 2012

DepthSelect allows users to select occluded objects in layer-based UIs. Layer-based tools such as PowerPoint, Photoshop, or Illustrator make it hard to select objects that are occluded by others, a problem called Pick Ambiguity. Currently used workarounds include selection of all objects with subsequent deselection of unwanted objects, or alternatively, locking all but the layers one is trying to select.

As our submission to the UIST2012 Student Innovation Contest, we used a Synaptics ForcePad (a pressure-sensitive multi-touch input device) to convert touch pressure to "selection depth". By varying touch pressure, objects under the current touch position can be targeted. The selection is finalized by quickly tapping with a second finger. A continuous pressure indicator next to the object list visualizes the current pressure level for the user, allowing the user to more quickly form an accurate mental model of this interaction paradigm. As deeper layers are selected, top layers become increasingly translucent ("onion peeling"), allowing the user to select totally occluded objects.

Implementation-wise, we first wrote a WebSocket wrapper for the C++-based Forcepad SDK, allowing us to accept input events in the browser with low latency. The web frontend is based on paper.js and CSS3. I wrote a basic vector-based layout program; a pressure-based, multi-touch vector drawing tool for quick creation of demo shapes, as well as the 3D selector.

Nov 2009 — Aug 2011

SlingShot is a workflow management system I built for exozet effects.

SlingsShot is used to manage the entire production pipeline and artifacts, including Shots, Plates, Set Data, Tasks, Milestones and Client Feedback. The tool has enabled Exozet's VFX pipeline to scale up to several high-profile productions, for example "Cloud Atlas" and "Abduction".

SlingShot's project statistics provide a compact overview of the entire show, summarizing shot statuses and visualizing task throughput by department.

Personal Dashboards: Artists, Leads and Producers can easy track their work items throughout the show

Shot Overview: one-stop summary of most relevant details

Condensed interfaces for effective management of repetitive, nested tasks

SlingShot's project statistics provide a compact overview of the entire show, summarizing shot statuses and visualizing task throughput by department.

Feb 2009 — ongoing

I co-founded eye.t vision GbR in 2009. Together with my long-term collaborator Martin Büttner, we offer software development and web design services to small businesses. Feel free to browse our company website for more information.