@heyhexx – work in progress show
 @heyhexx is an interactive social media responsive robot puppet theatre installation. This installation acts as an intermediary tool between the digital and physical world, and aims to make the abstract concept of emotions, in the context of social media interaction, easily understandable by translating it into puppet theatre in the physical space. The hope is that by interacting with Hexx, people will be able to grasp the emotions embedded in their messages sent on social media.
 Interaction System
For the interaction system, we have built automated software to stream incoming tweets that mention the twitter handle @heyhexx, and we then analyse these tweets for their emotion using IBM Watson’s Natural Language Understanding. The emotion then gets sent to the physical installation, and the software records a video in real time of the short theatre piece, and sends the video back through twitter.
We are currently developing a system in which we can analyse the personality of the twitter timelines of people interacting with Hexx, in order for each person to get their custom personality of Hexx.
There are 3 sections combined into this project.
- 1: Input
- 2: Hexx
-
3: Output
1 Input
Currently, the system that we focus on is everyone can interact with Hexx by sending him a text message through Twitter. Then we extract the emotion from text to emotion for sending forward to Hexx and also his surrounding world.
We currently using service from IBM Watson Natural Language Understanding which gives the result as a percentage of 5 emotion, joy sad fear anger and disgust.
For the future development, we plan to classify more emotions based on Robert Plutcnik’s emotion wheel out of 5 primary emotions from IBM Watson by using machine learning. And also expand input to physical interaction by grabbing the result from multi-sensors for example pressure sensor, GSR sensor.
2 Hexx
Hexx now developed 5 behaviours based on emotion analysis.
[vimeo 269231262 w=640 h=360]
3 Output
Puppetry, theatre, mechanics, and paper craft are the main elements of the concept. The set is comprised of dynamic origami structures that can move and be animated to display different emotive behaviours through the use of various mechanical techniques, such as motors and pulleys.
Theatre :
Hexx is maneuvered around the circular stage by a collaborative industrial robot arm. Circular shape provides first a proper shape for the robot arm to move Hexx around the city within its reach and second, easy accessibility for people to observe Hexx’s world from around the table.
The locations in the Hexx’s world mimic from the places our world, there are Hexx’s house, his workplace, a park and scary shop. For work in progress show, we made one-third of the whole set combines 2 sets, a park and the city.
Origami and paper craft mechanic :
We made dynamic origami and other paper craft technique to test on the movement to see the possibility of adding to the scene. Type of the paper, thickness and colour are the aspects of the quality of paper that we take into the consideration through the process.
[vimeo 277950150 w=280 h=158]Â Â [vimeo 277950291 w=280 h=158]
For further development of the project, we would like to explore going beyond one to one interactions, and try to use @heyhexx as a tool for adding commentary to conversations happening on social media. We feel that this may bring about more possibilities for the use of @heyhexx, such as visualizing the emotions in the context of political debates.
Sisyphus
Sisyphus is a robotic art installation that features two robots engaged in an endless cyclic mechanism. A team of small robots keep building brick arches while a giant robot pushes them down. The robo...
Submit a Comment