Thesis plan

Aesthetic, Purpose

My intention is to illustrate what and how we feel, rather in our subconsciousness or either in consciousness. I want to explore the correlation between our physical and mental actions, using the metaphor of cells building up a living organism. Our physiological action and reaction of imaginations and imagery, which have been submerged under our consciousness will be revealed through the surrealistic imagery through my project.

Description

My work will be a screen projection of 3D virtual imagery illustrating the visitor’s mental state. The system I am building is an interactive database, which will generate unique images according to the data collected from the sensors. Eventually, a collective of generated organisms, formed and interacting with the visitor’s physiological state, will be shown.

Technology

I will be using various biofeedback devices such as a brainwave sensor (ex. Neurosky’s mindset), pulse sensor, etc., to detect the visitor’s physical and mental states. I will be using python (or c++) to connect and convert the data from the sensors to visual imagery. A screen projection with real time generative drawings will be filling up a wall.

Courses

Graphic Programming, Interactive Installation, Physical Computing, Interactive Media, Advanced Interactive Media, Fundamental for computer Graphics, Digital Art in Context, Video Editing, Audio for Digital Media

Flow Chart

Installation Plan

Time Line

Artists

Artists that I came to know more, who explores the intersection btw technology, nature.

Ken Rinaldo
http://www.kenrinaldo.com/

Olafur Eliasson
http://www.olafureliasson.net/

Brian Knep
http://www.blep.com/


Thecla Schiphorst
http://www.sfu.ca/~tschipho/

Thesis Plan

Introduction
We replicate nature through code, by finding out the algorithmic structure. Nature is a huge system. Everything is connected to each other, and some animals such as ants can communicate and transmit their physiological state or environmental conditions through pheromones. Can humans also do this through our emotions?

Aesthetic, Purpose
My intention is to visualize the action of emotions and activate (amplify) them. Your feelings that have been submerged will be brought out through surrealistic imagery. The concept of visualizing and replicating nature by Form can be traced back to the Platonic definition for Form. Awe (homage) towards the structural beauty of nature is also what drove me into this project.

Description
My work will be a screen Projection of 3D Virtual landscape and creatures generated by the visitor’s mind. The system I am building is an interactive imagery data bank, which will generate a unique image according to the data collected by the sensors. It will first start up with presenting several random sounds or images. When a visitor’s reaction is strong for a certain image, either negative or positive, then that imagery will lead the visitor to another stage. Eventually, collective landscape of generated creatures growing up, interacting with the visitor’s physiological state, will be shown.

Technology
I will be using various biofeedback devices such as a brainwave sensor (ex. Neurosky’s mindset), galvanic skin response, pulse sensor, etc., to detect the visitor’s physical and mental states. I will be using python (or c++) to generate and call up a 3D imagery in real time. The physiological state, such as pulse, and brainwave will be giving the energy for the virtual creature to grow up. A screen projection of 3 angles will be filling up the whole room. I will use the algorithmic design of George Stiny, while I am drawing a live screen projection.

Experience
You walk into the room and you are tired. You wear the sensors. You see several different creatures coming out. You do not care. Suddenly, you are immersed into the image. The creatures start to grow and fluctuate, following your heartbeat. Suddenly you are surprised by an unexpected sound. The whole environment changes and the creatures change their state. You get bored. Everything slows down. You take off the sensors and leave. You see how the creatures are accumulated, forming a universe.

Thesis Research Presentation

I was nervous of my thesis presentation.

It was because I did not have a concrete idea of what I will be doing, and also I did not know what was possible, and I was just not confident enough.

I said what I get interested and I like.
1. Aggregation within lives of being a way of expansion, and eventually the object itself becoming and forming as part of the environment.
2. Emotion as a cellular activity, and as a electronic transmission
3. Biological growth and Metamorphosis

But I did not provide a proper image for this.


Theoretically, I said I want to get sensors to detect people's emotion and generate a surrealistic imagery corresponding to that.
Questions aroused. Is it really possible, how diverse and sensitive is it. I actually did not know well. I realized I have to research more.


Also, Oguz gave me this idea to put an object to trigger people's emotion better. Linda said that I have to think of a real walk through of what the feeling would be within interacting with my work.

I really just think I have to know more and experiment more. Cuz the less you know, the more you get scared of. I am pretty amazed at how I can really detect people's emotion! It is not that far, and a lot of people are interested in it already!

Ways to sense emotions

I was afraid of which kind of software or hardware to use to capture emotion. I thought it would be expensive or hard to get. But actually not! The artist that I saw in eyebeam used Neurosky, and when I checked online, it was $200, which is payable. I think I should also get that.

But the problem is that it only give an information of stressfulness and relaxation, which is not a diverse range of emotion.

When I searched online, there were several other ways to capture emotion, such as speech recognition, ECG, which is detecting heart beat and stuffs like that, and also Galvanic Skin response. It also seemed like to capture emotion, more than one kind of sensors were needed.

There are also DIY websites of how to make cheap sensors, but I am quite afraid to make it. What if I make a mistake and shock the users! What if it looks scary, not user friendly! I should see and try more. There are forums and communities, which helps people to go and do something. I also want to know how for humans to communicate with plants.

Brainwaves reacting with images

Brainwaves, which is also known as EEG, gives you the information of relaxation or stress.

I went to eyebeam open studio and saw this artist Scott Kildall using brainwaves as to capture what people feel from images from mass media. I also participated in the survey and it was quite fun and amazing to see the results. They(the surveyors who are watching at my brainwave) actually knew of how I feel towards the image, even before I realize or feel how I feel. I think I did not feel, but unconsciously feeling something, and they captured it precisely. I was very happy to see a guy's masculine naked body, and also a cute puppy. I was stressed to see a gun, a patient in a hospital, and a wedding cake.


I was thinking of doing my thesis project, using people's emotion, and Oguz gave me an idea to put some objects to trigger people's emotion, and actually I think was a fresh idea, and this project also makes me a broader view.

plants in urban life

This might be irrelevant with my thesis project, but I enjoy seeing plants submersed(?) within urban life. I also want to plant something in my window.




I went to the open studio in eyebeam, and was pretty amazed to see this planting system designed appropriate for farming in urban life. It uses hydroponic planting.







The musee du quai branly in Paris, which was designed by architect Jean Nouvel is awesome. The whole facade of a building is covered with plants like moss.






When I went to siggraph 2009, New Orleans, the project "Biological Instrumentation" by
Nina Tommasi was interesting, in terms of using real life within media art.

Video

Soliloquy from jean chu on Vimeo.

This is the video work that I have done for the video editing class.
I shoot the moon jellies from the NY Aquarium. I like the way how they look like cells.
I used blurring, to make them depict the mental stage between consciousness and unconsciousness.

What I am not sure is how to treat the texts. Should I make them merge more with the image? Should I make them unreadable?

Thinking about the interface, the experience, what is the limit?

As I keep thinking, I realize that my idea is becoming a mixture and a cluster of too many things so that it gets not clear. I should free my limit or boundary, but should make a frame of the way how my ideas would flow through.
I need to cut off stuffs now, and concentrate more on one strong theme. I want everything to be simple and coherent. I do not want to put too much of settings. I want to make my interface as minimal as I can.

Whatever I do, I will be drawing generative images that can be considered alive. What I am not sure is how much interactive I will make them to be, and what will the interaction or the interface , input be. Also, what I should think is what and how to display. It does not necessarily have to be interacting with human, it does not necessarily have to be showing the interaction in real time. I can even utilize real alive amoeba or the data of the nature as the data of interaction. I can just show the result of the diary afterward. It can be only just a screen projection or a set of monitors. This setting makes me more free.

I am interested in replicating the nature visually within coding and randomness. Also, I am interested in putting metaphors within my work, such as communication of the visitor's emotion, because somehow I want the interaction to be meaningful. I also want to be able to configure shapes and build up my algorithm of drawing shapes. I think I can make them to become all 1, such as make the emotions feed in to the virtual world and make creatures grow by eating them. But I am not clear yet of how to make it more interesting and fun, and artistic. I think I should not make what things happen to direct or apparent. Maybe filter it to be abstract, yet beautiful. I want the images to be ephemeral and organic, yet structured. I should draw some images to think of the scenes of the metamorphoses.

This might sound stupid but what I am not clear enough and would like to think more is how do I define something to be "alive". It should know what it is doing, what is going on outside, and it should be doing something responding to the current stage of circumstance. Also, it should have the possibility to die, which means it should somehow have random reactions that might screw up everything. It is because death makes existence and life more precious.

Draft of plan for my thesis project

Introduction
We can replicate nature through randomness by coding. I am trying to replicate nature as a way that is similar to how ants and primitive creatures talk with pheromone. How can I do this when my subject is the human? This is the focus of my work.

Aesthetic, Purpose
My intention is to find a way to enable visualization and communication of diverse perceptions of objects between other people in a setting that is similar to a play set. The concept of visualizing and replicating the nature by Form can be traced back to Platonic definition for Form. Awe (homage) towards the structural beauty of the nature is also what drove me into this project.

Description
Within the play set I mentioned above, I will present the experience of a different way to sense or perceive the world, and translate them into forms. People will interact with that setting, and the sensors that are triggered by the visitors as they interact with the objects. According to the various reactions there will simultaneously be a line projection with a corresponding sound which will appear on a screen, reflecting the reaction as read by the system I am building.

Technology
I will be using various sensors such as touch sensor, galvanic skin response, a sound sensor, and a proximity sensor to detect the visitor’s physical and mental states. I will be using vibration motors, speakers, an LCD monitor table or a screen projection on a table to make the reaction happen. I will use the algorithmic design of George Stiny, while I am drawing a live screen projection.

Radio Head music video “house of cards”

This is a music video which has been done with collaboration with a new media artist Aaron Koblin, using the new media technology. I like this piece because it visualizes the sound really well, as well as showing the singers. It is abstract, yet shows a lot of what is there in the real world. Using the new technology of 3D scanning, it presents a new way of video taking. This piece experiments in the new technologies, but still follows the structure of the usual music video. The camera goes back and forth of the closed up scene of the face of the singer, to the whole group of people and the scenery of a village.

http://code.google.com/creative/radiohead/
http://www.aaronkoblin.com/work/rh/index.html

Never loose yourdream.

I realized that if I loose my dream, I get lost of what I want to do and should be doing with this digital media.

Nowadays, everybody is sharing their codes, and it is very natural to use libraries developed by others. The result is that your work can easily become similar with others.

I am a person who is easily amazed and do works driven by technolgy. But now is the time that I need a shift of paradigm, because technology changes so fast, and then I really want to find my own style or aesthetic of work.

I want a more interesting idea or concept. That is what I think is most important in digital artworks. Technologies, or visuals are easy to modify or change. The core is always what drives you here, what is unique in your mind.

I want to establish my own manifesto.

Researching of Artists, Aesthetics, Techonolgies

I have been researching artists who have been doing works that bridge text, emotion, bodies, and computer graphic images within a certain algorithm.



Camille Utterback
http://www.camilleutterback.com/

Camille Utterback's work is an attempt to bridge the conceptual and the corporeal. She said “How we use our bodies to create abstract symbolic systems, and how these systems (language for example) have reverberations on our physical self is a matter of great concern to me.”

Central to her work “Text Rain” is the tension between the abstract realm of ideas and the corporeality in which we live and interact with these ideas. The falling letters are not random, but form lines of a poem about bodies and language. 'Reading' the phrases in the Text Rain installation becomes a physical as well as a cerebral endeavor.

This is a very old, classic project, but I still think it is effective and strong, and I like it really much as an interactive poetry.


Diane Gromala
http://www.lcc.gatech.edu/~gromala/art.htm
http://www.heelstone.com/meridian/gromala/gromalab.html
http://transliteracies.english.ucsb.edu/post/research-project/research-clearinghouse-individual/research-reports/biomorphic-type%E2%84%A2-2

BioMorphic Typography is Diane Gromala’s term for a family of fonts that continually morph in real-time response to a user’s changing physical states, as measured by a biofeedback device. Part of a larger initiative, Design for the Senses, the goal is to develop approaches to experiential design that focus on the senses and “the history of the body.” The writer is hooked up to a biofeedback device, which measures her heart rate, respiration, and galvanic skin response. As she writes, these continuous streams of data affect the visual character of the typeface.

In traditional caligraphs, the writer's emotion and physical state was able to get captured, and the readers were able to read it. But through digital typing, there is no way to capture the writer's emotion or characteristic. Through this biofeedback technology, the writer, reader, and the action of writing is able to be reconnected.

I have been searching on diverse devices, that I can detect human emotions.
Biofeedback Devices
http://en.wikipedia.org/wiki/Galvanic_skin_response


Galvanic skin response (GSR), also known as electrodermal response (EDR), psychogalvanic reflex (PGR), or skin conductance response (SCR), is a method of measuring the electrical resistance of the skin. GSR measurement is one component of polygraph devices and is used in scientific research of emotional arousal. You can easily think of a lie detector.


Brain Wave Sensors
http://www.neurosky.com
Brain Wave Sensors detect different wavelengths of the brainwaves and it can indicate different emotional states, like a focused awareness, a meditative state, or drowsiness. It is used also on psychological experiments of human emotions. A game using brain wave sensor as a device is actually already on developement by neurosky's MindSet.



Breath Detection
“Back to the mouth” is a game system activated by breath and mouth odor. The user eats/drinks something to control the user’s mouth odor. Then it is possible to kill monsters on a screen by blowing/sucking a blowgun-type device.

It was fun for me to think as a good example to connect the virtual monster and the real physical state of my body.

http://www.youtube.com/watch?v=fwYriNtuhW0&feature=PlayList&p=2C6C0F8A0D22160E&index=6




Kinetic Memory
http://www.topobo.com/
(Construction Kit with Kinetic Memory)
Topobo is a toy that people can sculpt a behavior through a kinetic memory within the toy. I was amazed by its constructive assembly system within kinetic memory.

This might be irrelevant to my work, but I am just inspired by this technology of being able to sculpt motions.



I have also been researching on how to derive a computational image which can be deconstructed or constructed under a certain structure and formula as a life pattern. This is a brief history of computation on forms.


•1950–cellular automata (Ulam, von Neuman)–generative grammars (Chomsky)
•1960–pattern grammars (Fu)
•1970–shape grammars (Stiny, Gips)
•1980–artificial life (Langton), self-organizing systems



Shape Grammers
http://radicalart.info/AlgorithmicArt/grammar/ShapeGrammar/index.html
http://ocw.mit.edu/OcwWeb/Architecture/4-520Fall-2005/LectureNotes/
http://www.shapegrammar.org/

The Shape Grammar formalism, developed in the 1970's by George Stiny and James Gips, was inspired by the rewrite grammars of Chomskyan formal language theory.

It is about special algorithms, about applying rules to generate figures. The earliest Shape Grammars worked with line segments, but extensions were developed that deal with color and with three-dimensional shape. It has been inspiring forms of architecture.

Yet, I still more time to absorb his idea, and apply it into my own way to build up a structure of unique emotional icons and characteristics.



Fid.Gen
http://postspectacular.com/code/start#fid.gen
Toxi's works are always inspiring and fabulous. Toxi's fiducials and his forever sculpture uses algorithm to generate a random yet never repeating figures.

It was interesting for me because I have never thought that a computer generated figure can be an identical from each other, and work as an ID marker. I came to think to generate characters of artificial lives which are random, but has each individual characteristic on it.