lithium flowers bloom » Processing

Communication Setup for Blinded

L.J — Sat, 15 May 2010 20:52:00 +0000

Li and I worked on a group of robotic/kinetic creatures to create an ironic and self-balanced networked system called Blinded.

A very detailed documentation of the construction iterations are up on Li’s website, so I’ll just write a bit about the communication between the creatures.

For the first iteration we had only one creature fully assembled so I built a quick “single character communication protocol” that allows the creature to be remotely controlled from a computer. I have two pre-paired series one xbees which are perfect for this basic cable replacement purpose.

The arduino on the creature side is expecting specific “single character” functions from the serial channel via xbee, available instructions are:

1. L – turn left
2. R – turn right
3. F – step forward
4. anything else – stop

At the same time, for monitoring purpose the arduino is reporting if the creature is on its target by comparing its left and right sensors and their historic readings, the reported status is written into the serial channel simply with notation of 1 (on-target) and 0 (target lost).

Because we had only one creature ready back then, the pursuing status is not entirely making sense, so the computer mainly served as a remote controller. The controller setup is pretty simple though, it can be just an xbee explorer board and any serial terminal program can be used to send commands to the remote creature. Following video shows one of our field tests in the park.

After more creatures were added to the group, the situation became much more complicated and I have to put up a processing sketch to coordinate input from multiple creatures. I did not use the xbee library for processing although it did look pretty promising in the examples and taking care a lot of the heavy liftings. But I dig into the source code a little bit and got a feeling that it might be a pain to use with series 2 xbees. So I just wrote some very simple parsing functions to take apart I/O packets and make sure I have access to any digital or analog pins without too much hack.

For physical setup, I switched all remote xbees to series 2 and put their firmware to router-at mode. The base station that stays with the computer is in coordinator-api mode. I used 4 pins of the remote xbees, 2 for digital output (remote commands), 2 for analog input (transferring sensor values to the remote computer/coordinator). Each remote xbee will still need to handle status reporting and command receiving, and the arduino on the creature remains as local intelligence that parses the sensor information and actually controls the motion.

Commands for Xbee configuration:

ATD04
ATD14
ATD22
ATD32

During runtime, the computer is able to set the remote xbee digital pin state through processing sketch. Code snippet for setting

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55

void setRemotePin(byte []address, String command, int value)
{
if (address.length != 8)
{
println("invalid address");
return;
}
if (command.length() != 2)
{
println("invalid command");
return;
}
serialSelector.port.write(0x7E); // start byte
serialSelector.port.write(0x0); // high part of length (always zero)
serialSelector.port.write(0x10); // low part of length (the number of bytes that follow, not including checksum)
serialSelector.port.write(0x17); // 0x17 is a remote AT command
serialSelector.port.write(0x0); // frame id set to zero for no reply
// ID of recipient, or use 0xFFFF for broadcast

for (int i = 0;i < 8; i++)
{
serialSelector.port.write(address[i]);
}

/* broadcast */
/*
serialSelector.port.write(00);
serialSelector.port.write(00);
serialSelector.port.write(00);
serialSelector.port.write(00);
serialSelector.port.write(00);
serialSelector.port.write(00);
serialSelector.port.write(0xFF); // 0xFF for broadcast
serialSelector.port.write(0xFF); // 0xFF for broadcast
*/
// 16 bit of recipient or 0xFFFE if unknown
serialSelector.port.write(0xFF);
serialSelector.port.write(0xFE);
serialSelector.port.write(0x02); // 0x02 to apply changes immediately on remote
// command name in ASCII characters
serialSelector.port.write(command.charAt(0));
serialSelector.port.write(command.charAt(1));
// command data in as many bytes as needed
serialSelector.port.write(value);
// checksum is all bytes after length bytes
int sum = 0x17 + 0xFF + 0xFE + 0x02 + int(command.charAt(0)) + int(command.charAt(1)) + value;

for (int i = 0;i < 8; i++)
{
sum += address[i];
}

serialSelector.port.write( 0xFF - ( sum & 0xFF) ); // calculate the proper checksum
//delay(10); // safety pause to avoid overwhelming the serial port (if this function is not implemented properly)
}

Sensor Network: Data Logging and Visualization

L.J — Mon, 01 Mar 2010 07:50:18 +0000

I’m working with Sebastian and Michael as “visualization” team for the sensor network assignment for this week’s Sociable Objects Workshop. It’s very challenging for the whole class to work on one single project, but so far it went on pretty well. Tasks are split and assigned and we also find ways to work with dependencies and such constraints.

Since we are just focusing on the visualization part, meaning that we take on only after all the data are aggregated at the coordinator xbee, our main effort will be data logging, mining and visualization. So the first thing we built is a pseudo data generator that feeds data in the same format as the data will be from the coordinator. We talked to the base station team and agreed that we will be getting raw API I/O RX packet and we will take care of the parsing and data storage.

I put together some simple arduino code to feed pseudo data, they could be easily replaced by a real coordinator and ideally we will not have to change the processing code on the other side which is listening to the serial channel.

#define ADDRESS_COUNT 9
int addresses[9] = {
0x0001,
0x0002,
0x0003,
0x0004,
0x0005,
0x0006,
0x0007,
0x0008,
0x0009,
};

void setup()
{
Serial.begin(9600);
randomSeed(analogRead(0));
}

void loop()
{
for (int i = 0;i < ADDRESS_COUNT;i++)
{
if (random(1000) > 990)
{
if (random(1000) > 500)
{
sendPseudoPackage(addresses[i], 1);
}
else
{
sendPseudoPackage(addresses[i], 0);
}
}

}
delay(50);
}

void sendPseudoPackage(int address, int value) { // pass either a 0x4 or and 0x5 to turn the pin on or off
Serial.print(0x7E, BYTE); // start byte
Serial.print(0x0, BYTE); // high part of length (always zero)
Serial.print(0x17, BYTE); // low part of length (the number of bytes that follow, not including checksum)
Serial.print(0x92, BYTE); // 0x92 is I/O RX Packet

//05-12: 64-bit address
Serial.print(0x00,BYTE);
Serial.print(0x13,BYTE);

Serial.print(0xA2,BYTE);
Serial.print(0x00,BYTE);

Serial.print(0x40,BYTE);
Serial.print(0x33,BYTE);

int low = address & 0xFF;
int high = (address - low) >> 2;

Serial.print(high, BYTE);
Serial.print(low, BYTE);

//13-14: 16 bit of recipient or 0xFFFE if unknown
Serial.print(0xFF, BYTE);
Serial.print(0xFE, BYTE);

//15: Receive Option
Serial.print(0x01, BYTE);

//16: Num Samples
Serial.print(0x01, BYTE);

//17-18 Digital Channel Mask
Serial.print(0x00, BYTE);
Serial.print(0x01, BYTE); //only Digital I/O 0

//19: Analog Channel Mask
Serial.print(0x00, BYTE);

//20-21: Digital Samples
Serial.print(0x00, BYTE);
Serial.print(value, BYTE);

//no analog samples

// checksum is all bytes after length bytes
long sum = 0xFF; //fake
Serial.print( 0xFF - ( sum & 0xFF) , BYTE ); // calculate the proper checksum
delay(10); // safety pause to avoid overwhelming the serial port (if this function is not implemented properly)
}

Then I wrote a basic processing sketch which filters the data and stores only changes to the databases. The sketch itself servers as some sort of realtime visualization on the input, and we get a feeling of how often data are changing and potentially detects sensor failures if one sensor is not active for a very long time.

The processing sketch is sending data to a website that handles data logging. The website also provides query api that returns sensor history in json format, and functions like filtering by certain address. The site itself is lightweight and fairly easy to add new functionality.

The plan is to meet tomorrow and start building a web interface upon the api for the final visualization. Hopefully we could also get to verify if the pseudo data is identical to the real format, and maybe we can get a test run with the real setup.

~~Stay tuned!~~

UPDATE: A list of available API functions:

url: http://itp.nyu.edu/~lx243/sensornet/index.php/api/

all (meaning that the url will look like http://itp.nyu.edu/~lx243/sensornet/index.php/api/all)

returning all records in json format

find/:address

returning all records from address specified
address should be a string of 64bit address, e.g. 0013A20040330001

find_today/:address

find_by_day/:address/:day

day should be in the format of yyyymmdd, e.g. 20100301

status/:address

returning latest records from specified address

status_at/:address/:time

returning latest records by the time specified
time should be in the format of yyyymmddHHMMSS, e.g. 20100301140000

changes/:address

returning all changes at specified address

changes_today/:address

returning changes happened today at specified address

changes_by_day/:address/:day

returning changes happened today on specified date at specified address
day should be in the format of yyyymmdd, e.g. 20100301

stats/:address

returning open time, close time, changes, and total active time of the sensor

stats_today/:address

returning above stats from specified address

stats_by_day/:address/:day

returning above stats from specified address on specified day

UPDATE 2: Visualization

It took a while for Sebastian and I to make processing.js to talk to the API and get real time update. We used jQuery as a bridge to retrieve the data and feed it on the page to processing.js. But eventually all is fine:

Humon beings update

L.J — Sat, 02 May 2009 07:55:59 +0000

Li and I presented the humons on Wednesday and Thursday in both Redial and Nature of Code class. Although we’ve been playing around for weeks and a lot of people already saw it, it’s still so joyful to see a lot people actually play with it at the same time.

Here’s the video from the nature of code class presentation.

A short demo of the basic motions we have so far: basic motion, jelly-ish body shape, gravity, collision, multiple ground tracks. The physics are implemented with the great toxi verletphysics library.

Basic interactions are all accessible from the phone, which allow a lot of people to interact at the same time without introducing more human-computer interfaces. In this demo video: basic movements (horizontal moving), jumping, message recording.

Another key feature of the humon project is to allow people to record and broadcast voice messages to the whole community. The instant playback will probably be replaced by a queueing system in short future, so that the audience may be able to focus on certain messages.

Some more screenshots from early processing sketches and different scenes:

The development weblog is now live too, we’ll keep updating there more often.

Check it out: humon doings

I stayed up late to check if we got into the spring show, but it’s still not out yet. Time to sleep!

US Intervention 1798 – 2009

L.J — Mon, 27 Apr 2009 18:35:17 +0000

“The United States military has been intervening in other countries for a long time. In 1898, it seized the Philippines, Cuba, and Puerto Rico from Spain, and in 1917-18 became embroiled in World War I in Europe. In the first half of the 20th century it repeatedly sent Marines to ‘protectorates’ such as Nicaragua, Honduras, Panama, Haiti, and the Dominican Republic. All these interventions directly served corporate interests, and many resulted in massive losses of civilians, rebels, and soldiers.” (via A BRIEFING ON THE HISTORY OF U.S. MILITARY INTERVENTIONS, By Zoltán Grossman, October 2001)

People might be familiar with recent US military interventions in Middle East, or those in Asia in 1970s, however in the past 200 years’ history, overseas interventions actually happened a lot more frequent than people usually think and played an important role to this country’s development.

This visualization is not intended to create a narrative piece of intervention history, it’s more created as a tool to go through the mass amount of history events and discover the correlations among those events in a certain period of time, or within a certain region.

DOWNLOAD

Download app for mac:

http://leejayxia.com/processing/us_intervention_mono_present.zip

For 1024 x 768:

http://leejayxia.com/processing/us_intervention_mono_present_1024.zip

I’m still working on the source data since world wars are currently excluded. Download links will be updated when I finish.

I wish I know more trigs

L.J — Tue, 14 Apr 2009 06:46:10 +0000

Did too much soldering today. I’ll just post some screenshots as update for the U.S. intervention history visualization. Since last the most effort regarding this project was spent on figuring out the tech details on building interaction into the spinning globe. I wish I know more about trigs!

Several tips to put down for myself:

- move textFont() out of the loop;
- do not create font on the fly if using huge font size;
- screenX and screenY are your friends (while creating mouse over interaction in 3D) (Thanks Dan!);
- rotation is headache, rotation in 3D is huge headache:

I had some rotation and translation params of a point generated by geo data. It’s easy to get params in those forms when you’re dealing with a sphere. Such point can easily be drawn with a series of rotate() and translate(), and a box() or sphere(), however I want to calculate the actual coordinates of those points in the 3D space to place attractors / draw lines / whatever.

I wish I paid enough attention while I learned trigs in high school, but I ended up looking into processing core source code and did something like this:

1
2
3
4
5
6
7
8
9
10
11

PVector rotation = getRotation(); // the rotation params that i do have
float offsetZ = 500; // some arbitrary number

PMatrix3D m = new PMatrix3D();
m.rotateY(rotation.y);
m.rotateX(rotation.x);

//more 3D transformation here

m.translate(0,0, offsetZ); // just do whatever I would do in normal draw loop
PVector l = new PVector(m.m03,m.m13,m.m23);

It’s basically just reusing the PMatrix3D class which is used internally in 3D transformation in processing, but I’m so happy not having to figure out the math myself.

More experiments:

Storytelling stickers update

L.J — Thu, 09 Apr 2009 22:28:27 +0000

Wrapping up what’s happened to the storytelling stickers!

THE READER

The RFID reader is working! It took a while though. I did some search and ended up taking arduino codes from http://blog.formatlos.de/2008/12/08/arduino-id-12/ featuring the AF_SoftSerial library. It’s very well written and comprehensive. The scripts added a kicking off step and a disconnection detection as well. Great work! A simple processing sketch was made to display the identifier code.

The tag needs to be really close to the reader to be recognized, which is fine in my project. But since I probably needs to get reading from both ends of a miniature trumpet/horn, that might not be good enough. I’m trying to build some antenna for the reader and see if it could extend the accessibility.

THE TAG

The tags arrived too. I ordered 300 raw EM4001 RFID tags (meaning that they come with exposed coil antennae) and unfortunately a lot of them broke. A lot of soldering is ahead! But I got enough to start anyway, like a hundred.

THE SOUND

I got the wave sound shield from Adafruit and finished assembling while procrastinating. But I haven’t tried to use them because it doesn’t record to the card and I decided to fall back to use a computer in this phrase.

So now it works like this. Whenever a card/tag/sticker is seen/read, according to the mode an audio clip associated to the tag ID would be recorded or played, by the computer. All sound files are stored on the computer instead of any hardware/memory card as I previously planned. The processing sketch was adapted to keep track of a known card and pick the corresponding sound file to play/record.

[Videos to be uploaded soon]

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281

import ddf.minim.signals.*;
import ddf.minim.*;
import ddf.minim.analysis.*;
import ddf.minim.effects.*;
import processing.serial.*;

Serial myPort;
boolean connected = false;

String activeTag = "Waiting...";
PFont f = createFont("Helvetica", 50);

Hashtable cards;
Minim minim;
AudioInput in;
AudioRecorder recorder;
AudioPlayer player;

String previousCard = "";
boolean playMode = false;
boolean autoPilot = true;

void setup()
{
println("connecting...");
myPort = new Serial(this, Serial.list()[0], 9600);
myPort.clear();
cards = new Hashtable();

loadCards();

minim = new Minim(this);
in = minim.getLineIn(Minim.STEREO, 2048);

size(800,600);
background(0);
fill(255);
noStroke();
textAlign(CENTER);
textFont(f);
text("WAITING",width / 2, height / 2);
}

void draw()
{
background(0);
if (!connected)
{
myPort.write('C');
}
else
{
text(activeTag,width / 2, height / 2);

if (playMode)
{
text("PLAYING",width / 2, height / 2 + 90);
}
else
{
text("RECORDING",width / 2, height / 2 + 90);
}
if (recorder != null)
{
stroke(255);
if (playMode)
{
if (player != null)
{
for(int i = 0; i < player.left.size()-1; i++)
{
line(i, 50 + player.left.get(i)*50, i+1, 50 + player.left.get(i+1)*50);
line(i, 150 + player.right.get(i)*50, i+1, 150 + player.right.get(i+1)*50);
}
}
}
else
{
for(int i = 0; i < in.bufferSize() - 1; i++)
{
line(i, 50 + in.left.get(i)*50, i+1, 50 + in.left.get(i+1)*50);
line(i, 150 + in.right.get(i)*50, i+1, 150 + in.right.get(i+1)*50);
}
}
}
}
}

void closePlayer()
{
if (player != null && player.isPlaying())
{
try
{
player.close();
}
catch (Exception e)
{

}
}
}

void closeRecorder()
{
if (recorder != null/* && recorder.isRecording()*/)
{
try
{
recorder.endRecord();
recorder.save();
}
catch(Exception e)
{

}
}
}

void loadCards()
{
cards.clear();
String []lines = loadStrings("index.txt");
if (lines == null)
{
return;
}

for (int i = 0; i < lines.length ; i++)
{
String card = trim(lines[i]);
cards.put(card, true);
}
}

void dumpCards()
{
PrintWriter writer = createWriter("data/index.txt");

for (Enumeration e = cards.keys() ; e.hasMoreElements() ;)
{
String card = e.nextElement().toString();
writer.println(card);
}

writer.flush();
writer.close();
}

void serialEvent(Serial port) {
String myString = port.readStringUntil(' ');
if (myString != null)
{
myString = trim(myString);
if (myString.equals("connected"))
{
connected = true;
println("connected~!");
}
else
{
if (myString.equals("0000000000"))
{
activeTag = "Disconnected";

if (playMode && player != null)
{
println(previousCard + ": Disconnect from playing");
closePlayer();
}
else
{
println(previousCard + ": Disconnect from recording");
closeRecorder();
}
previousCard = "";

}
else
{
activeTag = myString;

if (!cards.containsKey(activeTag))
{
cards.put(activeTag,true);
dumpCards();
if (autoPilot)
{
closePlayer();
playMode = false; //auto switch to record
println("Autopilot: switch to record mode");
}
}
else //found
{
if (autoPilot)
{
playMode = true; //auto switch to play
closeRecorder();
println("Autopilot: switch to play mode");
}
}

if (playMode)
{
if (cards.containsKey(activeTag))
{
println(activeTag + ": Start playing new file, disconnect from previous " + previousCard);

closePlayer();

println("loading " + activeTag + " to play");
player = minim.loadFile("data/" + activeTag + ".wav", 2048);
player.play();
player.loop();
}
else
{
println(activeTag + ": Not recorded yet");
}
}
else //record mode
{
if (activeTag != previousCard)
{
println(activeTag + ": Start playing new file, disconnect from previous " + previousCard);
closePlayer();
closeRecorder();
}

recorder = minim.createRecorder(in, "data/" + activeTag + ".wav", true);
recorder.beginRecord();
}

previousCard = activeTag;
}
}
}
}

void stop()
{
in.close();

closePlayer();
closeRecorder();

minim.stop();
super.stop();
}

void keyPressed()
{
if (key == ' ')
{
playMode = !playMode;

if (playMode)
{
println("play mode");
}
else
{
println("record mode");
}
}

if (key == 'a')
{
autoPilot = !autoPilot;

if (autoPilot)
{
println("Autopilot: ON");
}
else
{
println("Autopilot: OFF");
}
}
}

THE NAME

Any thoughts on a name for the toy?

TicTacToe Game with Asterisk

L.J — Thu, 09 Apr 2009 21:08:42 +0000

and Processing…

Thanks to Dan’s example codes on the threading multiple callers and socket communication, it’s not hard to route the AGI scripts to processing. The most effort was actually spent on the player scheduling and game developing (written from scratch). Kinda tired of these white on black UIs but it’s still the most handy design for quick prototype.

In the video the game was played between a cellphone and a softphone on Gizmo.

More experiments with globe

L.J — Thu, 09 Apr 2009 20:51:11 +0000

Some sketches to play with 3D globe and its motion. I have to made this to see if it makes sense at all if I plot the timeline data long the radius of the globe. Next step is to create better visual elements and put in text captions.

The graph reads from the center to the outside as from oldest to latest events. The size of the cubes indicates the length of the intervention event.

Download the sketch executable for Mac OS X.

update on mainstreaming info

L.J — Mon, 30 Mar 2009 13:48:21 +0000

I broadened my mainstreaming info project to look into all US military interventions in the past 200 years instead of only nation building attempts for several reasons.

After putting all nation building facts together and categorizing them by countries, it’s still hard for me to reveal some significant patterns among all these attempts for me to use as an axis to organize all the verbal materials;

I made the sketch below trying to plot the history events onto the timeline. It turns out that first it would take much longer time for me to verify all the facts and second I don’t feel confident enough to simply “tag” the history events with my shallow understandings for now;

Sketch: military intervention shown on top of the timeline bar, main history events displayed along the side of the bar. (blue – independence, red – controlled by another country, orange – dictatorship)

This timeline graph is not enough to carry the huge amount of information due to the capacity of information it’s trying to carry. It’s not easy to arrive on a simple conclusion from this sea of history data. A visualization is always an illusion, however I do not wish to over simplify the history in this case. I would have to put more texts and labels on the screen. (This is like a common struggle while I’m designing a visualization. It’s becoming more like a data mining toolset other than a complete presentation of an idea.)
So the user still have to read through the texts to get more info on a certain event, which is not a bad thing. They could combine the external intervention facts together with country key events to get to their own conclusions, or not. From there I started to think if it would be better to just show the big picture of all military interventions and use nation building as an addition.

The intervention list I’m using was found here: http://www.globalpolicy.org/empire/history/interventions.htm. History commons has another well organized list of intervention history too. I wrote a quick gantt chart sketch in processing to see how the data look like, and was surprised to find China and Nicaragua to be the two countries with most US intervention in history.

Next step would be playing back the events in a 3D space to better illustrate the geo location of all these data, and I would like to create more organic visual forms for the history events. Some experiments starting with basic geometry forms:

Visualizing wolfram patterns with monmonmonsters

L.J — Mon, 30 Mar 2009 05:59:37 +0000

Interested in creating a simple form of “stand alone complex”, I decided to use wolfram’s one-dimension cellular automata as the underneath engine for the monsters. The color, dancing pattern, dancing amplitude are driven by different sets of CA rules.

Play monmon “CA” monster here: http://leejayxia.com/processing/monmonmonster_wolfram/

or on openprocessing: http://openprocessing.org/visuals/?visualID=1326