lithium flowers bloom » final

Automating The Brush Stroke

L.J — Fri, 20 Nov 2009 18:40:40 +0000

Here’s a rough diagram of the drawing machine architecture that I have in mind. I’m building the components from bottom to top.

I moved on from the brush stroke and started to work on a set of drawing functions that incorporate the stroke.

Currently coverage:

moveTo,
lineTo,
rect,
circle

the moveTo function is necessary since the dynamics of the brush motion would depend on the last stroke, both position and speed.

currently i’m working on the curve function, it’s quite a challenge to get the brush movement natural enough and calculation is also dependent on the stroke length and brush size too. once this is done I think all basic drawing functions are covered for now and I will move onto the composition stage.

this is a simple test:

stroke update from Liangjie Xia on Vimeo.

Humon beings update

L.J — Sat, 02 May 2009 07:55:59 +0000

Li and I presented the humons on Wednesday and Thursday in both Redial and Nature of Code class. Although we’ve been playing around for weeks and a lot of people already saw it, it’s still so joyful to see a lot people actually play with it at the same time.

Here’s the video from the nature of code class presentation.

A short demo of the basic motions we have so far: basic motion, jelly-ish body shape, gravity, collision, multiple ground tracks. The physics are implemented with the great toxi verletphysics library.

Basic interactions are all accessible from the phone, which allow a lot of people to interact at the same time without introducing more human-computer interfaces. In this demo video: basic movements (horizontal moving), jumping, message recording.

Another key feature of the humon project is to allow people to record and broadcast voice messages to the whole community. The instant playback will probably be replaced by a queueing system in short future, so that the audience may be able to focus on certain messages.

Some more screenshots from early processing sketches and different scenes:

The development weblog is now live too, we’ll keep updating there more often.

Check it out: humon doings

I stayed up late to check if we got into the spring show, but it’s still not out yet. Time to sleep!

US Intervention 1798 – 2009

L.J — Mon, 27 Apr 2009 18:35:17 +0000

“The United States military has been intervening in other countries for a long time. In 1898, it seized the Philippines, Cuba, and Puerto Rico from Spain, and in 1917-18 became embroiled in World War I in Europe. In the first half of the 20th century it repeatedly sent Marines to ‘protectorates’ such as Nicaragua, Honduras, Panama, Haiti, and the Dominican Republic. All these interventions directly served corporate interests, and many resulted in massive losses of civilians, rebels, and soldiers.” (via A BRIEFING ON THE HISTORY OF U.S. MILITARY INTERVENTIONS, By Zoltán Grossman, October 2001)

People might be familiar with recent US military interventions in Middle East, or those in Asia in 1970s, however in the past 200 years’ history, overseas interventions actually happened a lot more frequent than people usually think and played an important role to this country’s development.

This visualization is not intended to create a narrative piece of intervention history, it’s more created as a tool to go through the mass amount of history events and discover the correlations among those events in a certain period of time, or within a certain region.

DOWNLOAD

Download app for mac:

http://leejayxia.com/processing/us_intervention_mono_present.zip

For 1024 x 768:

http://leejayxia.com/processing/us_intervention_mono_present_1024.zip

I’m still working on the source data since world wars are currently excluded. Download links will be updated when I finish.

Storytelling stickers update

L.J — Thu, 09 Apr 2009 22:28:27 +0000

Wrapping up what’s happened to the storytelling stickers!

THE READER

The RFID reader is working! It took a while though. I did some search and ended up taking arduino codes from http://blog.formatlos.de/2008/12/08/arduino-id-12/ featuring the AF_SoftSerial library. It’s very well written and comprehensive. The scripts added a kicking off step and a disconnection detection as well. Great work! A simple processing sketch was made to display the identifier code.

The tag needs to be really close to the reader to be recognized, which is fine in my project. But since I probably needs to get reading from both ends of a miniature trumpet/horn, that might not be good enough. I’m trying to build some antenna for the reader and see if it could extend the accessibility.

THE TAG

The tags arrived too. I ordered 300 raw EM4001 RFID tags (meaning that they come with exposed coil antennae) and unfortunately a lot of them broke. A lot of soldering is ahead! But I got enough to start anyway, like a hundred.

THE SOUND

I got the wave sound shield from Adafruit and finished assembling while procrastinating. But I haven’t tried to use them because it doesn’t record to the card and I decided to fall back to use a computer in this phrase.

So now it works like this. Whenever a card/tag/sticker is seen/read, according to the mode an audio clip associated to the tag ID would be recorded or played, by the computer. All sound files are stored on the computer instead of any hardware/memory card as I previously planned. The processing sketch was adapted to keep track of a known card and pick the corresponding sound file to play/record.

[Videos to be uploaded soon]

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281

import ddf.minim.signals.*;
import ddf.minim.*;
import ddf.minim.analysis.*;
import ddf.minim.effects.*;
import processing.serial.*;

Serial myPort;
boolean connected = false;

String activeTag = "Waiting...";
PFont f = createFont("Helvetica", 50);

Hashtable cards;
Minim minim;
AudioInput in;
AudioRecorder recorder;
AudioPlayer player;

String previousCard = "";
boolean playMode = false;
boolean autoPilot = true;

void setup()
{
println("connecting...");
myPort = new Serial(this, Serial.list()[0], 9600);
myPort.clear();
cards = new Hashtable();

loadCards();

minim = new Minim(this);
in = minim.getLineIn(Minim.STEREO, 2048);

size(800,600);
background(0);
fill(255);
noStroke();
textAlign(CENTER);
textFont(f);
text("WAITING",width / 2, height / 2);
}

void draw()
{
background(0);
if (!connected)
{
myPort.write('C');
}
else
{
text(activeTag,width / 2, height / 2);

if (playMode)
{
text("PLAYING",width / 2, height / 2 + 90);
}
else
{
text("RECORDING",width / 2, height / 2 + 90);
}
if (recorder != null)
{
stroke(255);
if (playMode)
{
if (player != null)
{
for(int i = 0; i < player.left.size()-1; i++)
{
line(i, 50 + player.left.get(i)*50, i+1, 50 + player.left.get(i+1)*50);
line(i, 150 + player.right.get(i)*50, i+1, 150 + player.right.get(i+1)*50);
}
}
}
else
{
for(int i = 0; i < in.bufferSize() - 1; i++)
{
line(i, 50 + in.left.get(i)*50, i+1, 50 + in.left.get(i+1)*50);
line(i, 150 + in.right.get(i)*50, i+1, 150 + in.right.get(i+1)*50);
}
}
}
}
}

void closePlayer()
{
if (player != null && player.isPlaying())
{
try
{
player.close();
}
catch (Exception e)
{

}
}
}

void closeRecorder()
{
if (recorder != null/* && recorder.isRecording()*/)
{
try
{
recorder.endRecord();
recorder.save();
}
catch(Exception e)
{

}
}
}

void loadCards()
{
cards.clear();
String []lines = loadStrings("index.txt");
if (lines == null)
{
return;
}

for (int i = 0; i < lines.length ; i++)
{
String card = trim(lines[i]);
cards.put(card, true);
}
}

void dumpCards()
{
PrintWriter writer = createWriter("data/index.txt");

for (Enumeration e = cards.keys() ; e.hasMoreElements() ;)
{
String card = e.nextElement().toString();
writer.println(card);
}

writer.flush();
writer.close();
}

void serialEvent(Serial port) {
String myString = port.readStringUntil(' ');
if (myString != null)
{
myString = trim(myString);
if (myString.equals("connected"))
{
connected = true;
println("connected~!");
}
else
{
if (myString.equals("0000000000"))
{
activeTag = "Disconnected";

if (playMode && player != null)
{
println(previousCard + ": Disconnect from playing");
closePlayer();
}
else
{
println(previousCard + ": Disconnect from recording");
closeRecorder();
}
previousCard = "";

}
else
{
activeTag = myString;

if (!cards.containsKey(activeTag))
{
cards.put(activeTag,true);
dumpCards();
if (autoPilot)
{
closePlayer();
playMode = false; //auto switch to record
println("Autopilot: switch to record mode");
}
}
else //found
{
if (autoPilot)
{
playMode = true; //auto switch to play
closeRecorder();
println("Autopilot: switch to play mode");
}
}

if (playMode)
{
if (cards.containsKey(activeTag))
{
println(activeTag + ": Start playing new file, disconnect from previous " + previousCard);

closePlayer();

println("loading " + activeTag + " to play");
player = minim.loadFile("data/" + activeTag + ".wav", 2048);
player.play();
player.loop();
}
else
{
println(activeTag + ": Not recorded yet");
}
}
else //record mode
{
if (activeTag != previousCard)
{
println(activeTag + ": Start playing new file, disconnect from previous " + previousCard);
closePlayer();
closeRecorder();
}

recorder = minim.createRecorder(in, "data/" + activeTag + ".wav", true);
recorder.beginRecord();
}

previousCard = activeTag;
}
}
}
}

void stop()
{
in.close();

closePlayer();
closeRecorder();

minim.stop();
super.stop();
}

void keyPressed()
{
if (key == ' ')
{
playMode = !playMode;

if (playMode)
{
println("play mode");
}
else
{
println("record mode");
}
}

if (key == 'a')
{
autoPilot = !autoPilot;

if (autoPilot)
{
println("Autopilot: ON");
}
else
{
println("Autopilot: OFF");
}
}
}

THE NAME

Any thoughts on a name for the toy?

More experiments with globe

L.J — Thu, 09 Apr 2009 20:51:11 +0000

Some sketches to play with 3D globe and its motion. I have to made this to see if it makes sense at all if I plot the timeline data long the radius of the globe. Next step is to create better visual elements and put in text captions.

The graph reads from the center to the outside as from oldest to latest events. The size of the cubes indicates the length of the intervention event.

Download the sketch executable for Mac OS X.

Face Blind (Revisited)

L.J — Sun, 22 Feb 2009 01:00:04 +0000

Face Blind was my final project for Visualizing the Five Senses last semester. However for some reason I never had a chance to document it, and as a kicking off of my “blog catching up” week I would like to start with this project.

Just as a quick recap of the project, Face Blind is a visualization of prosopagnosia symptom and calls for the awareness of this perception disorder. On Wikipedia’s prosopagnosia page, the brief intro is given as follows,

Prosopagnosia (sometimes known as face blindness) is a disorder of face perception where the ability to recognize faces is impaired, while the ability to recognize other objects may be relatively intact. The term usually refers to a condition following acute brain damage, but recent evidence suggests that a congenital form of the disorder may exist. The specific brain area usually associated with prosopagnosia is the fusiform gyrus.[1]

Few successful therapies have so far been developed for affected people, although individuals often learn to use ‘piecemeal’ or ‘feature by feature’ recognition strategies. This may involve secondary clues such as clothing, hair color, body shape, and voice. Because the face seems to function as an important identifying feature in memory, it can also be difficult for people with this condition to keep track of information about people, and socialize normally with others.

The difficulty of face-blind people could hardly be explained and understood by people around because we are taking face recognition so for granted and seldom think of how faces are specifically optimized in our memory system. The analogy of the stones were used by Cecilia Burman on her website on prosopagnosia and I found it a relatively effective approach of making this explanation short. I decided to create a set of cloud masks for every student in the ITP community to illustrate this idea in a similar yet different way.

The clouds are generated by several different features on peoples faces and are overlaying on their faces to obscure them The optimized mechanism for face-recognition in our brain would not be activated since most of the facial features are turned off by the noises (clouds). However since the clouds are generated from an abstraction of facial features so ideally I could still create a unique cloud mask for each person. In this way, we could probably experience the difficulty of face blindness and, however, could also experience the “piecemeal” or “feature by feature” recognition strategy by observing the shape, texture and other features of the clouds.

Cloud Generation

My first difficulty in the process is creating the cloud texture programmatically. I first thought of using stock photos of clouds textures and trimming them for my use. After some research I decided to create my clouds by Perlin Noise, which seems to be a classic and efficient way of generating the texture. Since I’m not only looking for the randomness of the clouds, I would also like to gain control of the generated result through a set of configuration parameters (which would be the output of my face recognition/analysis tool).

The first iteration and the later revision of the cloud generator looked like this. The clouds used in the final visualization were composition of two layers of cloud highlights and another two layers of shadows, plus a adjusting filter to make sure the generated cloud cover at least 50% of the face area.

Facial Recognition

Since I’m not building a sophisticated robot with computer vision that watch your door or build you an iron armor, my goal for the facial recognition program was to just extract certain features from the face, which in this case includes aspect ratio of the face shape, positions of the eyes, nose and mouth, and whether the person is wearing glasses or not.

In fact it worked pretty well. Click on the thumbnails below to view larger pictures.

Rectangle on the top left indicates aspect radio of face shape;
Red stripe – position of eyes, brown of wearing glasses;
Lower stripes – position of nose/mouth, not really accurate from time to time;

Composition

After the preparation of the previous two steps, I tried to develop a showcase for the visualization. The performance problem emerged as I was putting everything together. The realtime rendering of the cloud for each individual took about 3-4 seconds in average and created an unacceptable latency in the viewing experience. Due to the short of time, I pre-rendered all graphics beforehand. I cannot argue this is the best way to present the final product. It might be better to cache the whole cloud space to optimize calculating time than pre-rendering. Anyway this is what I presented at the final.

See if you could recognize anyone. Click on the thumbnail for higher resolution. All headshot images courtesy of Social Genius. If you’re interested in getting your “cloud” just drop me a message at leejayxia [at] gmail.com.

Final Presentation: Face Blind

L.J — Thu, 18 Dec 2008 08:24:37 +0000

On Tuesday we had Paola Antonelli, Natalie Jeremijenko, Allyson Torrisi as guests in the Visualizing the Five Senses class final presentation. I showed my final work on the Face Blind project.

Face blind

L.J — Sun, 07 Dec 2008 07:31:34 +0000

I chose the topic of face blindness of my final project for “visualizing the five senses” because I was simply stunned while I bump into the website of prosopagnosia. It was almost at once I decided to create something under this topic and also I think it’s highly related to this class.

Prosopagnosia, also known as P.A. or face-blindness, is a symptom with which people lose the ability of identifying people by their faces. In my understanding the optimization module in the brain taking care of the facial recognition is not functioning, so that people can no longer associate the visually seen faces with the memory of people. Recognizing people in the social life becomes as hard as looking for one specific stone among thousands of them.

My proposed project is to recreate this experience for normal people by obscuring faces on the screen. Meanwhile the faces will be replaced by the texture of cloud. The idea is to create a mapping function between human faces and cloud shapes/patterns. Ideally I would be able to find a consistent conversion so that two similar faces should generate similar clouds.

some early sketches:

special thanks to “social genius”, images courtesy of ITP community

Update on Physical Computing Final

L.J — Sun, 07 Dec 2008 07:05:19 +0000

I’ve been working with Liesje on the pcomp final and we had a lot of fun playing with magnets. Now it comes to a crucial question what are we gonna do to put some real pcomp into this project. Finally today we came to narrow down our ideas and focused on a still exciting solution! We still want to keep the organic look and feel of this piece and we’re adding a little bit servo-driven motion just to strengthen the interaction and exaggerate the motion of the magnets.

some 3 days ago pictures.

Wrapping-up the five senses: poster and proposal

L.J — Sun, 23 Nov 2008 04:44:10 +0000

The semester is coming to the end and we were creating a collage of all previous class works… and it turns out to be no-so-systematic as in one poster.

Here is the archive for all posts regarding the class this semester.

coin and collage:

poster and proposal (more images):

I’ll keep working on the odor database and try to find out how our sense of smell are manipulated and cheated by the artificial tastes. Ironically it is a reflection on the “mango cavier” effect presented by professor Ken Kirshchenbaum on last week’s class.

images for reference slash inspiration: cartography / mutant food and taste / visual cue of the sensor / representation of the food and taste / visual cue of the sense