We had some problem at first with the receiver side and we didn’t find an effective to debug it. It was pretty clear that the transmitter is working fine since while I was using just xbee explorer board as receiver I can directly see the data coming from the serial channel. However I cannot get the code to work when I plugged the xbee back to arduino circuit.

I ended up using a processing sketch to print out serial data from arduino since I cannot see it directly by printing out data to the serial terminal in arduino code. So lesson learned:

1. the resister used to augment sensor output value to fit xbee’s 3.3v requirement has to be carefully handled, otherwise you do not get to use full range of the sensor output and it makes it harder to figure out the range to look for in the receiver code,

2. a processing sketch is necessary for debugging. it gives a lot of confidence while you can see what is in the data stream, at least for me.

simple remote xbee sensor

with feedback

My thesis project is a software program that creates generative drawings on its own, one drawing per day, and updates its own drawing blog.

The program was built on the reverse of most generative computer graphics artwork or simulation approaches that creates realistic graphics; instead of creating complex visual graphics based on intensive computational power, it is trying to simulate human drawing in a very low-level approach of recreating emotion and gestures.

Rationale

I choose not to build another program to appreciate visual complexity because it comes down to the question of the value of the work of art, which in my opinion has a lot to do with mastery of skills and efforts put in it. So for me people appreciate complex visuals in computer graphics because those visuals are extremely hard to achieve by manual effort. However it is not always true for computers and that is why I am not super excited about computer generated complex abstracts. The real effort would be assigning emotion to fully automated process, and achieve humanity in the drawings produced.

In short, I am building this program in a way that is extremely hard for the computers to do, giving up the significance that could be easily achieved (compared to human effort) by a computer program. I would like to show the slow and painful process of this computer generated drawing and its evolution towards its own aesthetics.

There are infinite number of ways to build a program that draws on itself, and I am building it to mimic my own way of drawing/doodling. From the way the brush moves on the canvas, the shape of the brush strokes, to the composition preferences and training materials feed to the program for inspirations, all implementation would be following my own subjective choice. However the machine is running all by itself without constant intervention even by myself. Ideally it could live as part of my ghost in a different shell. The program could be an extension of the body, virtually. Would it be an even closer way to communicate directly through my ghost?

Goals

The goal of this project is to create new genre of computer graphic that expresses humanity by mimicking the doodling experience and generating drawing based on emotional factors and gestures.

It is not the goal of the project to produce tradition painting through simulation of painting tools and media. Drawings produced by the program should be visually recognizable as a computer generated artwork, while the content of the drawing can be subjective, emotional and less “visually mathematical”.

Technically, I would like to show the process of how a computer program would make sense of visual inputs and produce creative result out of it. And ideally it should also be able to publish its evolution from creating basic shapes and curves to being able to produce complex compositions (but not necessarily always doing so).

Audience and Location

The project is expecting a wide audience, essentially anyone that has prior experience of using a computer and interested in computer generated graphics. It would also probably draw attention of people that has particular interest in drawings.

This project itself is a very personal expression, meaning that I decide to design the algorithm independently without too much concerns about potential audience. While I am building the project not for particular target audience, I do expect different responses from different groups of people, and there are several that I am more interested in: a) generative graphics artists, b) illustrators, c) artificial intelligence scholars or engineers.

In terms of presentation, I would like the piece to be accessible online including the whole archive of the drawings it created. It is meant to be experienced alone than in a group. While it’s most preferred to be viewed through a web browser or a downloadable application running on one’s own computer, the project could possible still be suitable for public spaces like galleries or museums.

For now it seems that a website would provide a most complete experience for this project given its non-linear nature and technical difficulty to replicate its infrastructure. If I would have enough time to release the application itself to the public or make it easily deployable to other people’s website, and the copies of this project would be able to evolve independently, the final presentation of this decentralized artificial intelligence would have to be on the internet, or at least using internet to aggregate the productions/artworks from all copies.

Core Features and Functionality

The core of the project is a software that generates drawings. Every time the program launches, it creates a drawing and terminates itself. The drawing takes inspiration from the program’s external visual memories mixed with some “live” input that adds emotional factors to it. The drawing process is monitored and supervised by the composition and aesthetics assessment algorithms. The program determines the scale of the drawing and the moment to finish drawing.

There will be a few peripheral components to assist the core drawing module.

a) Visual Memory Storage, an important basis for building up the aesthetics preference of the program, as well as its raw material for inspiration. The memory is prioritized based on an arbitrary “impression” algorithm that puts the program’s most favorite visuals on the top.

b) Interface for Visual Memory allows the program to accept multiple source of external graphical inputs. The source could be a streaming video feed from a live webcam on justin.tv, or a favorite image feed from my ffffound.com image bookmarking account. The sources could be used as either positive or negative training material to build up the program’s aesthetic preference. The interface should be easily configured to maintain the input feeds which the program is listening to, and whether or not they are used as training materials. Ideally it could be easily accessible on the web.

c) Aesthetic Preference database, is a set of rules that the program collectively builds up over time through the process of learning, drawing and self assessment. The database is referred to during the creation of the drawing, and will be updated through training, drawing, and feedback on the published drawings.

d) Coordination Scripts that launches the program and publishes its creation to a website. Since the program itself is designed to be modular for future portability, some coordination scripts are required to help keep the whole thing running. They would help monitoring the visual feeds and compiling any updates to the visual memory, launch the drawing module at a preconfigured time, publish
the drawing created in this iteration and collect the feedback on the drawing to reflect into the preference database.

e) A website, technically a blog, that showcases the daily creation of the program.

People will be able to browse through the drawings the program created over time. They will probably not be able to search, since there will not be enough metadata associated with the drawing for filtering. If there has to be a filter or category, it could be based on timeline, emotion, or arbitrary themes (if the program evolves to name its creation).

For individual drawings, people will be able to:

a) watch a playback of the drawing process;
b) see the inspiration of the drawing, or the pictures that the program referred to in the creation;
c) rate the drawing, the result might or might not be used as training material;
d) (optional) order a print of the drawing;

This project is not intended to be instantly interactive because I would like to show the slow and painful process of creating a drawing, not to draw people’s attention to the coolness of computer generative graphics. I would like each of the drawing the program made would take a while for people to digest. I would like people to notice and appreciate the drawings themselves as work of art without knowing that they are created by a computer. This is already a very challenging task both in AI and fine art.

The software program shares my attitude on computer generative graphic art. While it is capable of creating complex generative visuals, it chooses to (as I choose to make it ) take the hard way and to apply its own (as my own) preference of visual beauty to its creation. It is a very personal critique on current computer generative art.

The core drawing module will either be written in Processing or openFrameworks, depending on my research on the portability of each option. I would like the program to be able to executed on a command line tool that does not require user intervention in the drawing process, so that it would be easily automated after it is deployed to a linux server.

For visual memory storage, couchDB seems a competitive choice for my data structure given its document oriented concept and flexibility in data types. MySql will be a safe but second choice since I’m more familiar with it.

Housekeeping scripts will be a compilation of a set of tools written in combination of shell scripts, python and php. It is most likely that I will build the website in php or just hack around the wordpress CMS framework.

As a software program, it does not have a user interface. It should not even require an input. It could be simply described as an application that generates one drawing at a time.

The showcase website, as the final presentation of the program, does have a user interface. I have in mind a blog site with clean minimalist design, highlighting latest drawing created by the program. And it should be easy enough to navigate through the archive of drawings to see how the algorithm evolves.

Success Measures / Future Plans

The goal of this project is to achieve machine intelligence in artworks, meaning that it should essentially create drawing that is visually or emotionally appealing to its viewers. Although the concept of machine intelligence evolution made public is as important, I would like to make the project beyond just being conceptual. The diversity of the drawings that it creates and the fact whether or not people appreciate those drawings will be also very important and valuable information for me.

For thesis, of course I would like to build the whole project to its greatest completion. However given the time constraints and technical challenges, I could still sort out some of the components to be lower prioritized than others.

The core drawing module is first priority, but it does not mean that the evolving algorithm needs to be perfect. The most important thing is the self driven composition process.

Secondly important is the infrastructure that automates the program and publish the drawings. Even if the algorithm is not perfect, it still makes a lot of sense if the whole evolution procedure is online and accessible to all.

Next will be the external interface of the program to the outside world, which makes it easier to feed training materials, to supervise the training process and to apply the feedbacks.

For future I will be most interested in making it portable enough so that there could be multiple instances of this program evolving individually and people could observe how different they are becoming over time.

 
Collect training materials

 
Supervise training process

Training personal drawing style

A public online gallery showcases all creations

Get back to the drawing machine that learns from human drawing, I finished the brush stroke and drawing API encapsulation last semester. From now on until the end of the semester, I would like to finish the genetic algorithm for evolving machine aesthetics and the publishing module, so that I can at least setup a solid basis for this project. Even if I cannot finish all features that I planned, I will still be able to have a running platform to improve the underlying intelligence on.

Doorbell for insistent visitors from My Journey on Vimeo.

Switch Code

Buzzer Code

In my project I would like to envision in a near future how would the re-definition and the relation of private and public evolves, in either of the following ways or their combinations:

a) a narrative animation that describes the imagined situation and how the transition is going to happen;
b) a tool that exposes one’s digital identity trace;
c) a data visualization focusing on realtime data exposure;
d) an installation that explores the new social norms under this imagined circumstances;
e) a collection of objects/installations that illustrates different aspects of possible social changes;

Related Projects:

Personas
Project website: http://personas.media.mit.edu/

Personas was created by Aaron Zinman, with help from Alex Dragulescu, Yannick Assogba and Judith Donath, as a component of the Metropath(ologies) exhibit in 2009. “It uses sophisticated natural language processing and the Internet to create a data portrait of one’s aggregated online identity. In short, Personas shows you how the Internet sees you.”

Personas visualizes the process of a machine making sense of your public data on the internet. It is not meant to be utilitarian, but more to be a critique on data mining process, which is in most cases happening behind the scene.

There are a few things that I like a lot about this project.

Firstly it is taking only public data on the internet as input. Data collection wise I would also (have to) take a very similar approach as data input for my project. Even I am predicting a future in which almost all digital information would be public and transparent (not necessarily to human beings, at some point we should be taking machines into account too), that would be a most efficient way of gathering information and it well illustrates the blurred borderline between public and private information online.

Secondly it is not aiming to be a useful tool but to expose a completely automated process. This could easily be misinterpreted given its presentation online. I also find it hard to build something that 1) looks useful/practical and 2) runs in a browser to be properly interpreted in its intended context. I got it wrong at first glance too, not a lot of people will actually read the long background description. Given that it was designed originally for a museum exhibition, it is probably facing a completely different situation while being shown separately online. If an online interaction piece is the direction I would be pursuing, I should be very careful with the presentation, and try to make it looks less utilitarian.

Also worth noting is the clean elegant interface of the project. The very limited input and interaction might have a positive effect on driving people to find out more about the underlying idea.

The personas project inspires me in a sense of the possibility of tracing online identity, and it provides an interesting presentation of showing a process but not bothering the user by too much technical details.

Master Plan
Project website: http://masterplanthemovie.com/

Master Plan is a short animation directed and produced by Ozan Halici and Jürgen Mayer in 2007. The script is based on the book “What Barry Says” by Simon Robson and “The Google Story” by David A. Wise.

Master Plan is a clear alert of the seldom noticed online privacy issue back at time, and sent a very strong message through the narrative. This project is inspiring as it was the first beautifully crafted info-graphic video project I had ever seen. Not only the presentation is great, the topic is also very close to the issue that I am interested in.

While Master Plan is mostly summarizing what already happened and proposed a very open question that it did not intend to answer, I would like to predict the future that tries to answer a few questions that have been on my mind for a while. What information we considered private but is actually widely accessible thus being pseudo private? What information is pseudo public because of misinterpretation and loss of context? How will machine make sense of the mass amount of data we have been accumulating once they get smart enough to decode them? Are we human beings going to be still in control of those data by then?

If I could get a fairly clear picture through my research in next couple weeks, and if I would be able to put together a narrative that is strong enough to support a visual conversation, I would like to borrow the form of storytelling from Master Plan to be at least part of my project to tell the story.

We Feel Fine
Project website: http://wefeelfine.org

We Feel Fine is created by Jonathan Harris and Sep Kamvar in May 2006.

As far as I know, We Feel Fine remains a lot of people’s favorite ever since its launch. It is a best example of how utilizing publicly available but usually invisible information can create a powerful art project. I am not exactly enthusiastic about the technical details behind the scene, but the idea of combining their subject interest together with the user profile information of the source makes a huge change to the nature of the project. For We Feel Fine it enables the features like filtering and clustering, but for me it means that it made possible to monitor any specific person/profile and connect that to the statements they made. Any data can become sensitive while accumulating through time, it makes a big difference to people emotionally. Any web services that holds user data for a long period of time should all realize the strong attachment of these data to their originators, no matter how boring it is.

There are so much I can take from this project, not only the visuals and interactions, but also the important emotional component that an art project should feature. The method they use for data collection will also work for my case if I would like to create an interactive tool for visualizing aggregated statistics, unlike Personas’ personal perspective, I will probably need to build some infrastructure like We Feel Fine does, to support this kind of data mining.

We Feel Fine does not trace down to personal details, but they have all the data available to do so. This could also be an extra point that I would like o address in my project, is the easiness of data replication. Once the information is out there, there is no way back. You can take back one of your blog posts that you regret that you posted from your site, but there is no way to take it back from all kinds of crawlers all over the internet. People are not allowed to make mistakes anymore, and not a lot of people are aware of this. That changes the rule of sharing information and you probably would like to think twice before posting anything intimate to the internet. This is not to say people should be paranoid about data collection agents/devices/softwares, it is part of the mind change that I foresee in a near future, and I would like to explicitly explain in my project.

Ten Thousand Cents
Project website: http://www.tenthousandcents.com/

Ten Thousand Cents is created by Aaron Koblin in collaboration with Takashi Kawashima in 2008. The total labor cost to create the bill, the artwork being created, and the reproductions available for purchase are all $100. The work is presented as an interactive/video piece with all 10,000 parts being drawn simultaneously. The project explores the circumstances we live in, a new and uncharted combination of digital labor markets, “crowdsourcing,” “virtual economies,” and digital reproduction.

Bicycle Built For 2000
Project website: http://www.bicyclebuiltfortwothousand.com/

Bicycle Build For 2000 is created by Aaron Koblin and Daniel Massey in 2009. It is comprised of 2,088 voice recordings collected via Amazon’s Mechanical Turk web service. Workers were prompted to listen to a short sound clip, then record themselves imitating what they heard.

I am putting these two projects in the same category not just because they are from the same author but they both feature crowd-sourcing to create the final product, and the workers were not even told what exactly they are working on. There are two things that I am especially interested in, involving a large group of people, and this unconsciousness in the creation process.

If I could find a proper form of involving people to help revealing the ideas of privacy redefinition, meaning that I can find a minimized task that help me get a clue of what is on people’s mind, distributing these tasks through mechanical turk seems a great idea, and this unconsciousness will also help reinforce the idea of people creating a future without having the big picture in mind.

Track Me Not
Project website: http://mrl.nyu.edu/~dhowe/trackmenot/

TrackMeNot is a lightweight browser extension that helps protect web searchers from surveillance and data-profiling by search engines, created by Daniel C. Howe, Helen Nissenbaum in August 2006. It does so not by means of concealment or encryption (i.e. covering one’s tracks), but instead, paradoxically, by the opposite strategy: noise and obfuscation. With TrackMeNot, actual web searches, lost in a cloud of false leads, are essentially hidden in plain view.

This is a utility tool, that actually deals with the identity issue directly. It relates to my subject of interest because it is providing an answer with an effort of not changing social norms and presumptions and trying to solve a specific identity exposure case. I am not against the attempt but I see this approach as a passive way to look at our issues. My biggest problem is not with large corporates collecting data, I am more worried about not a lot of people care or even know about it.

Resources

Boyd, D. (2007, May 22). Social network sites: public, private, or what?. Retrieved from http://kt.flexiblelearning.net.au/tkt2007/edition-13/social-network-sites-public-private-or-what/

Boyd, D. (2010, January 16). Facebook’s move ain’t about changes in privacy norms. Retrieved from http://www.zephoria.org/thoughts/archives/2010/01/16/facebooks_move.html

Zimmer, M. (2010, January 12). Zuckerberg’s remarks aren’t surprising, nor new, nor true. Retrieved from http://michaelzimmer.org/2010/01/12/zuckerbergs-remarks-arent-surprising-nor-new-nor-true/

Kirkpatrick, M. (2010, January 18). The Facebook privacy debate: what you need to know. Retrieved from http://www.readwriteweb.com/archives/facebook_privacy_explanation_debate.php

Hill, D. (2008, Feburary 11). The City as a platform. Retrieved from http://www.cityofsound.com/blog/2008/02/the-street-as-p.html

Kerr, Ian, Steeves, Valerie, & Lucock, Carole. (2009).Lessons from the identity trail. Oxford University Press, USA.

Work Plan

I have not yet got to a stage where I can work on a detailed work plan. I have several milestone points that I would like to keep. Unfortunately it is not making a lot of sense to me to go more detailed than that at this point.

If I have to switch my thesis idea, I would like to turn in another research paper and work plan by next week (Feb 17) together with new assignments.

Week 5 (Feb 17)

Finish background research, at least get to one narrowed-down but still exciting idea to work on. This is prerequisite to all other components of this project, and I have already spent a great amount of time on research. If I will not be able to nail down anything specific to work on, I would like to keep the research open but switch my thesis project to a practical idea that will fit in the timeline.

At the same time, keep working on background research of another project which I already had a firm idea and a clear vision with, so that

Week 8 (March 10)

Build a bare-bone prototype that complete the structure of the project, leaving details for polishing or working out. If it is an animation, I should have script, storyboard and graphical elements settled. If I will be working on an installation, I should have the algorithm ready, and a bare-bone software program running.

THE STORY

Kalpa is the creature of stillness. There is a large amount of them existing but rarely seen. Their bodies are too small to be noticed by human eyes but they have an extremely strong sense of any kind of movement around them.

They gather and reproduce only in a still environment. A slightest change can stop them from reproducing and kill them very quickly. Therefore, even though they reproduce surprisingly fast, the ever changing nature of the world makes them ephemeral and become extinct.

Fortunately, except for having a still environment there is another way to keep kalpa alive.

The first person who discovered this was Sri Aurobindo. Aurobindo was born in a noble family in India. He went to London for education at seven. Being always passionate about politics, he dropped out of school when he was 20, went back to India to join national independence movement. By late 18th century when the revolution was suppressed by the government, downhearted Aurobindo ended his 17-year political life and started seclusion. During the seclusion, he read a lot ancient Indian, Syrian and European manuscripts and became interested in the Absolute Stillness Theory in alchemistry.

During an austerity practicing at the age of 58, he claimed that he felt the mysterious creature that had written records only in ancient manuscripts. He then named it “Kalpa” and implied in his book On Bhagavad Gita that the ancient Hindu prophets achieved immortality through concentration and having kalpa gather around their bodies. In the following 80 years, Aurobindo practiced four times. The last one lasted for 40 years without him eating anything. Just when others believed he was dead, he walked out his chamber and peacefully passed away in the sun.

When Aurobindo’s book spread to Japan, it was treated as The Criterion of Immortality by local monks. Combined with Tao Theurgy of ancient China, they created a new sensational approach. The devoted ones must do the following steps. The first step is to become empty in three years. They should exercise everyday to burn out all the body fat first and then eat only nuts and seeds to minimize the movements of their stomachs and to slow down the metabolism. The second step takes another three years. They eat only tree barks and grass roots and drink only a thimbleful of water. They meditate instead of exercise to become sweeny in order to maximize the reduction of metabolism.

Before the last step, they need to drink a kind of tea that is specially made of sumach extracts to extrude body water and so to minimize fluid movements inside their bodies. Then they go into a small basement to practice austerities. According to master Yentao, this brutal and suicidal approach can help the gathering of kalpa around their bodies so for them to achieve immortality. It was banned by the government in late 19th century.

Sociologist David Philips from University of California San Diego (UCSD) was the first one who associated kalpa with time. In his 1970 doctoral thesis paper Death as a Form of Social Activism he pointed out that “to defer death by controlling body movement is actually a common phenomenon although the amount of time earned is limited…”. The cases he collected suggested that in a lot of areas, dying elderly people were able to defer their deaths until after major festivals merely by reducing the amount of food they eat, body movements and using a lot of will powers.

Inspired by Aurobindo, Philips proposed his new theory about time in the Annual North America Psychology Conference (ANAPC) in 1991. He believed that not only physical stillness but also concentration can increase the reproduction speed of kalpa and what is more important is that at the same time the gathering of kalpa can also help reinforce the stillness, meaning when they reach a certain amount, they could actually stop time. But unfortunately there was no evidence to prove the existence of kalpa so the theory was called ridiculous, and Philips was considered a lunatic. Philips was not able to get any funding and had to stop the research on kalpa. In 2005, he spent his last days in a nursing home in Connecticut and did not leave behind any empirical record on kalpa.

A couple of years later, almost when everyone has forgotten about this academic farce, two young art students claimed that they managed to capture kalpa. They made a wood box, in which they placed a specially made clock. They brought the box into a quiet room, stared at it for 10 minutes and then quickly sealed it with plaster. So they have a living sample of kalpa trapped in the box. Nobody would believe it was so easy to capture this legendary creature until after they stare at the box for a while, they were all surprised to see the time slow down. This specimen was later brought to NASA and became the first cornerstone in the making of time machines.

Kalpa is my collaborative project with Li Li and was shown at ITP Winter Show 2009. More information at http://j.mp/kalpa-project

I moved on from the brush stroke and started to work on a set of drawing functions that incorporate the stroke.

Currently coverage:

• moveTo,
• lineTo,
• rect,
• circle

the moveTo function is necessary since the dynamics of the brush motion would depend on the last stroke, both position and speed.

currently i’m working on the curve function, it’s quite a challenge to get the brush movement natural enough and calculation is also dependent on the stroke length and brush size too. once this is done I think all basic drawing functions are covered for now and I will move onto the composition stage.

this is a simple test:

stroke update from Liangjie Xia on Vimeo.

The machine draws with dripping ink, which is loaded into the cup and slowed down by the sponge.

The machine draws while the movable paper holder is pushed, leaving passive trails on the paper.