Category Archives: Processing

Slapstream v2.0

The Github repo for this project is located here.

Late last year, I had the opportunity to present an updated version of Slapstream, one of my old projects from my first semester of ITP, at Razorfish’s NY offices as part of Future Interfaces, a one-night technology exhibition run by the NY Media Lab. You can see more details from my older post linked above, but here’s the description from that page:

“Slapstream is a video game where you have to dodge onscreen obstacles by slapping yourself in the face.

It uses a Kinect to track the position and velocity of your hands and their distance from your face. Once it detects a slap, your onscreen character will move in the direction that your face reels. A more powerful slap to the face will result in greater movement onscreen.

The idea came out of my interest in making a novel game interface, and also sort of as an experiment to see how much inconvenience people will put up with for the sake of fun or competition.”

Videogames traditionally never emphasized the means of control as a way to add to the total experience of gaming. At first, controllers were designed to be purely functional. As they evolved, they became more ergonomic, so that the player notices them less and less as they become increasingly immersed in the game’s world. However, more recently there has been a reaction in the opposite direction with innovations such as the Wii and Kinect, where the mode of interaction becomes physical and interfaces with the onscreen world to become part of the experience of play. I designed Slapstream as a different take on this new mode of interaction in order to explore the entertainment value that comes from doing something unexpected and shocking to play an otherwise simplistic game.

I had left the project in an extremely prototypical state, and it was fun to revisit an old project now that I have a little more knowledge about programming. I worked on refining the slapping mechanic, overhauling the interface, outfitting the game with retro-style graphics, and adding a high-score system to encourage competition. You can see the video I made from exhibition footage above.

Update 7/2015: Slapstream has since been shown at Tribeca Film Festival’s Interactive Playground and at NYC Resistor’s annual interactive art show.

NYC Food Crawl – ITP Winter Show 2013

This post assumes knowledge of the concept behind this project. To view the project proposal, click here.
The Processing code that I wrote is posted on Github. Click here to view.


NYC Food Crawl is a physical data representation that uses diorama sets with live cockroaches, along with an accompanying screen-based visualization, to represent the frequency of vermin violations in New York City area restaurants. I took the restaurant food inspection results dataset from NYC OpenData and brought the main database and the violation codes database into Google Refine and Excel for cleaning. After cleaning this data and isolating the vermin-specific violation codes, I brought both of these datasets into Processing. By doing so, I was able to calculate how many restaurants were in each inspection grade level, how many of those restaurants had had a vermin-related violation recorded, and how many of those violations had occurred within the past year or sooner.

I then used these figures to build the physical part of the project: restaurant dioramas, one for each grade, with a representative amount of live cockroaches inserted into each one to reflect the data. I decided to keep the representation simple: the number of cockroaches in each grade’s diorama would represent the percentage of restaurants in that grade that had had a vermin violation. Since my order of cockroaches (linked if you’re curious – most people at the show were) came in mixed sizes, I decided to make cockroach size indicative of recency: the ratio of larger cockroaches to total cockroaches in each diorama is the ratio of recent violations to total violations. More simply, the more detectable the presence of the large cockroaches were in each box, the higher percentage of recent violations there were for that grade level.

A concern I had from the perspective of the viewer was that, while memorable and attention-getting, the cockroach dioramas provide a very shallow representation of the data. Also, viewers might receive a skewed representation of the data depending, as the cockroaches like to hide out of sight and may not be all visible at once. To address this, I built out an interactive graphical visualization in Processing that both emulates the physical display for each grade level and provides additional statistics. I also included statistics for the data sorted by borough instead of inspection grade level, in case viewers wanted to explore the data further. You can see a video of this below:


Here are some pictures of my setup for the show, with both the dioramas and screen visualization:




New York Times API Search Tool



Code for this project can be seen on GitHub.

The assignment for the week was to use the New York Times API in order to chart a cultural shift over time. I initially chose to track the terms “payphone” vs “cellphone” to try to see the moment in time in which cellphones made payphones obsolete.

For how simple this result is, I had a lot of trouble wrapping my head around the coding process for some reason. One big sticking point was the need to combine the date ranges for both sets of data into one complete range that would encompass both. The solution was to take one of the search terms’ date ranges, add it to an ArrayList, comb the other term’s range for dates that aren’t duplicates, use a little Java (the Collections class) to sort the combined results, and then re-export that out to a String array.

At one point, I was getting a complete graphing for “payphone” but an incomplete graphing for “cellphone” (it would only draw the results for 2oo6 and 2007, even though some debug efforts were showing that the data was still there for other years). After trying many many things and wasting lots of time to no avail, I eventually realized that the culprit was using the “==” operator instead of “.equals()” for a String array. I ALWAYS forget about this and my lazy side wishes that Processing would just let you get away with the former option. What I don’t understand is why the first result set still graphed correctly and why only the second was affected – I was using the same display code for both.

Eventually, I got it working and took the screenshot above. I found the results kind of boring, so then I set about making the program modular so that you can define whatever two strings you want at the beginning of the code and the graph scale and date range will adjust itself accordingly. When I accomplished that, I took some screenshots of some more visually interesting results:



Strange how “cassette” briefly resurfaces in 1998. I wonder if some alternate use for the term came up?



This one shows you a president sandwich. I thought it looked neat.

World’s Hotels – Manipulating A Large Dataset

First, I’ll start with the video (HD recommended):

And the Git repo with the code:

This visualization takes a random sampling of one-tenth of all hotels in the world and displays them in a map view by default. The points on the map are color-coded by the hotel’s star rating, with 0/unrated being the darkest orange and 5 stars being the brightest yellow. If you press S, the visualization changes to star ranking mode, in which it divides the random 10% set into proportionally-sized squares for each ranking. By pressing M, you can return to map view.
Development Process:

The first challenge with this project was wrangling the huge dataset into some sort of usable shape. I downloaded and went through some tutorials for openRefine, and then loaded the set it. After poking around, I noticed that some of the most complete categories included the hotel name, the star rating (after combing through and setting all unrated hotels to 0), the latitude and longitude, and the country name. In order to try to reduce the amount of data for Processing to parse, I cleaned for those categories and deleted the data I didn’t plan on using (later I ended up taking out the hotel name field, which further cut the file size by about 50%).

Once loaded into Processing, I set out trying to plot the data into a map view. I looked around for map libraries, but the most prominent one only worked with Processing 1.5 or lower. Since I was using Table classes to organize my data (2.0+), I couldn’t use it. After some experimenting, I found that mapping the latitude and longitude coordinates to the edges of the screen makes for a perfectly usable (and rather neat-looking) map. I color-coded them by star ranking by mapping their rating to green in an RGB color with red set to 255. That way, the 0-stars are red, and the hotels get more yellow the higher they’re ranked.

After that, I set about animating the map. It was around then that framerate became a major issue (the screen only refreshed once every 1-3 seconds or so). By putting in a random-selection filter, I found that limiting the data to 8% of the total produced a usable framerate, which was disappointing. (Later, I added a motion blur which let me get away with a slightly chunkier framerate, letting me bump it to 10% – it could probably go to 15 but it would start to really show at that point.)

The final step was to add the star ranking view. In order to make the squares proportionally sized to each other, I had Processing sort/count the number of hotels in each category in the setup() loop. I noticed unrated was the largest, so I mapped all the other squares against that. I also ended up square-rooting each category, so that the size difference of the squares would be exaggerated.





Slapstream is a video game where you have to dodge onscreen obstacles by slapping yourself in the face.

It uses a Kinect to track the position and velocity of your hands and their distance from your face. Once it detects a slap, your onscreen character will move in the direction that your face reels. A more powerful slap to the face will result in greater movement onscreen.

The idea came out of my interest in making a novel game interface, and also sort of as an experiment to see how much inconvenience people will put up with for the sake of fun or competition.

To see my initial project proposal, click here.


Current code:

The code is available on Github here. There is also some incremental update history here, but I lost tracking of some of my later development updates because my saving practices broke my Github directories. I’ll try to restore them to working order before I continue work on the program.

Current state and future plans:

When I demoed Slapstream, it was still very much a prototype – the hero and obstacles were represented by circles, debug information was visible all over the screen, and many features I wanted to implement were missing.  However, I was pleased with the actual action of slapping – it was fairly responsive, and the movement of the character onscreen did map pretty well to the direction and speed of the user’s slaps.

I’d like to implement a scoring system, make obstacles fall more quickly over time, add 2-player capability, and replace the circles with less abstract graphics.

I got started learning the process of coding with the Kinect from the book Making Things See by Greg Borenstein.

The parallax motion of the stars was just very slightly modified from the motion seen in William Smith’s sketch on OpenProcessing.

Special thanks to Dan Shiffman, Mark Kleback, Ben Smith, and Genevieve Hoffman for coding help.