And here’s a selection of work that showcases my broad skill set in user experience, interaction design, content development, visual design and programming!
02
Arctic mOcean
Designing and programming interactive spaces
An interactive game independently conceptualized and programmed on processing for Cirque du Soleil as part of my IxD master’s program.
Interaction Design & Programming
Section overview
Project overview
“arctic mOcean” is an audience-driven interactive game that challenges the traditional notion of a static spectator within the theatrical performance.
The concept aims engage the audience by allowing them to participate actively in the show’s plot progression through an interactive collaborative game. The experience not only gives the audience the ability to “occupy” the stage via the game, but it also creates a simple audience-driven variable that can be used to define the plot evolution: the audience is divided into two teams, each representing one of two sides within the plot (protagonist vs. antagonist). Individuals in each group must work collaboratively to manipulate waves to move icebergs towards the ‘goal’. The team that manages to get more icebergs to the goal wins.
Role throughout project
This project was carried out independently.
Research - ideation - prototyping - interaction design
Programming - technical spec design and setup
Media production - motion design - visual design - audio production
The tools used were
Processing
OpenCV
Box2D
Minim
Competitor / state-of-the-art analysis
Desk research
Storyboarding
Prototyping
Storytelling
●
The brief
“arctic mOcean” was produced following a brief proposed by Cirque du Soleil for a winter-themed performance arena. The prescribed objective was to create an immersive experience designed to encourage audience engagement using ‘ice’ and/or its composing characteristics at the heart of the concept.
The client was
Cirque du Soleil
The client needed
A way to engage audiences during a show
A winter-themed concept that involved ice
The users were
Audience members
The users needed
A memorable and entertaining show.
A sense of connection and engagement with show and fellow attendees.
●
The research
In developing the “arctic mOcean” experience a variety of team-based dynamics were referenced - from modern digital gaming to traditional team sports and even to collaborative interactions in animals.
Video games and interactive storytelling
Player participation in defining storytelling progression and outcomes is a common aspect of most role-playing video games like Zelda and Final Fantasy.
Team sports and collaboration
The experience also draws from real-life sports that use group effort within a team playing setting, i.e. within ice hockey, where players pass the puck between one another in a collaborative attempt to get it to the goal.
The wave
Team sports are particularly interesting because of their ability to foster competitive spirit and sentiments of allegiance within fan cultures and audiences. I was particularly interested in capturing unified/coordinated crowd responses to the game - ie. chanting or ’the wave’.
Orcas and wave washing
Inspiration is also drawn from a naturally occurring scenario in Antarctic marine nature where groups of Orca (killer whales) work together in a highly coordinated attempt to produce targeted waves in order to prey on seals. This precisely orchestrated team effort called “wave washing”, destabilises the seals as they seek refuge on floating icebergs. This collaborative group manipulation of waves is the source of inspiration for the gaming ‘tool’ that allows the crowd to interact with the game.
Floating icebergs
In keeping with the brief’s guideline to use ‘ice’, the “arctic mOcean” game dynamics take from the motion of icebergs within ocean waves. “arctic mOcean” also reiterates this concept in its name by playing on the words “ocean” and “motion”.
●
The prototype: development
Rough sketches
Before addressing the best or most appropriate tools to be used for realising the prototype, the game’s desired interaction was formed through preliminary sketches. An idea was selected based on the notion of needing a way for users to work together to reach one unified goal and as a result, the wave was chosen as the ideal example for such a scenario.
Tools and frameworks
Based on prior knowledge as well as some early research, some initial speculation on the possible tools were selected. This included a variety of physics depiction/calculation examples that involved vector interpolation, visualised ripples through pixel manipulation, and sine-produced variations to simulate waves.
Finally, a dedicated physics library was chosen for the twofold benefit of having built-in functions for ease of implementation and more robust performance handling. Two choices of closely related physics libraries were considered (box2D and fisica libraries), but finally box2D was selected for its manipulability and wealth of available documentation online.
On the video end, openCV was also chosen for its performant handling of video, in particular it offered advanced processes such as finding contours and applying grayscale and threshold to moving images. The first version of the prototype was created using only one (internal) video input, and was as a result a one player game. This was an ideal setup to start testing and optimising the physics reactions of the game-world in relation to real-world video input. However, subsequent versions introduced two external logitech C615 webcams for a two-team player version of the game. The benefit of using the external logitech cameras is that unlike the internal macBook camera, camera functions such as brightness, contrast and exposure can be manually set using an application such as WebcamSettings. Moreover, the logitech C615 webcams offer out-of-the-box compatibility with macOS.
●
The prototype: implementation
Image projection
The final prototype used a projection of the “arctic mOcean” game that was programmed and run on Processing. The projection itself was projected onto the floor, using an angled mirror to transform a horizontal projection into a vertical one (from a wall projection to a floor projection). The projection was displayed on a large custom-sized paper surface on the floor.
Lighting
The first step to run the experience requires illuminating two spotlights placed behind each of the teams. Each spotlight is then directed towards the players to create a contrasted silhouette against the white backdrop behind them. This specific lighting arrangement was chosen to accommodate to restrictions on spotlight quantity, placement and small room dimensions (venue lighting would be better equipped for a more accurate lighting scenario).
Image capture
After the lighting is set up, two webcams, one dedicated to each team, capture the image of the players and send the live feed to a computer via 2x 5m USB booster cables (one for each webcam). The image fed to the laptop is then used as input for the processing program running the “arctic mOcean” game.
The program
I used the Processing application to program the experience. I used the program’s built-in Capture library to access the two video inputs, and pass the content frame by frame to the openCV library for image processing. This transforms the image to grayscale and applies a threshold filter to create a clear-cut black and white silhouette image. Using the openCV library, the ‘contour’ of the image was then extracted. This contour was used to create and update an array of points in the box2D physics world. This array of box2D points would then be able to allow the players to manipulate all the other items in the box2D world (the waves and the iceberg). The box2D physics engine then handles the interpretation of the motion and collisions, and all resulting physics interactions are displayed in the processing program. The processing image is updated in realtime and sent from the computer via miniDisplay output to the projectors VGA input for the game participants’ display.
Sound effects
The processing program also processes a soundtrack and sound-effects using the minim integrated sound library. The sound-effects are launched by the program at appropriate times. The sound is output to the speakers via usb.
●
The challenges
Some difficulties and obstacles encountered were
Coming up with a simple and intuitive concept that fit the brief.
Working entirely independently throughout ideation and prototyping process.
Working with a processing library I wasn’t previously familiar with.
Solving technical and compatibility issue.
Working within a very limited time constraint (1 week) to come up with a concept that fits the brief and putting together a working prototype that would be ready by the exhibition date.
Let’s focus on one
Coming up with a simple and intuitive concept that fit the brief.
Because I wanted to focus on a game-based dynamic as a way to create audience engagement, I initially referenced some popular RPGs. I tried to focus on adventure and journey, but those kinds of games tended to be robust and complex when it came to characters and storytelling. This was especially problematic to considering the short time frame available to complete the brief. I think such constraints (time and scope) ultimately proved very useful in honing in on a simple and straightforward concept.
●
The takeaways and results
The successes were
The various technical difficulties were overcome and the project ran successfully throughout the whole exhibition.
The project was completed and exhibited on time despite the short deadline
Participants during testing and the exhibition clearly enjoyed and were drawn by the experience.
The experience encouraged group participation and collaboration.
Some next steps and thoughts on how to improve the project
Because of the limited time constraint on this project, I was unable to iron out all kinks in the game’s object physics. The game worked well enough for the purposes of the exhibition, but it was not completely seamless at times. If I had more time I would likely start by working on that. Then I would go on to flesh out the game more to explore more levels and challenges that could foster further audience engagement and collaboration.
