lithium flowers bloom / the weblog

content-aware media retargeting

22-Oct-09 | 885 views

seam carving for content-aware image resizing

Seam Carving for Content-Aware Image Resizing

Effective resizing of images should not only use geometric constraints, but consider the image content as well. A seam is an optimal 8-connected path of pixels on a single image from top to bottom, or left to right, where optimality is defined by an image energy function. By repeatedly carving out or inserting seams in one direction one can change the aspect ratio of an image. By applying these operators in both directions we can retarget the image to a new size. The selection and order of seams protect the content of the image, as defined by the energy function. Seam carving can also be used for image content enhancement and object removal. We support various visual saliency measures for defining the energy of an image, and can also include user input to guide the process. By storing the order of seams in an image we create multi-size images, that are able to continuously change in real time to fit a given size.

This technology/algorithm is presented at SIGGRAPH 2007.

project page: http://www.faculty.idc.ac.il/arik/SCWeb/imret/index.html
youtube demo: http://www.youtube.com/watch?v=vIFCV2spKtg (algorithm explained at 1’12)

online demo: http://swieskowski.net/carve/
implementation by aviary: http://aviary.com/blog/woodpecker

improved seam carving for video retargeting

This one is pushing the algorithm to a new level by applying to realtime videos as well as improving the seam carving algorithm.

http://www.youtube.com/watch?v=AJtE8afwJEg

DRAGON: in-scene video navigation
http://hci.rwth-aachen.de/download/DRAGON/1057-karrer.mov

Direct manipulation interaction technique for frame-accurate navigation in video scenes.
This technique benefits tasks such as professional and amateur video editing, review of sports footage, and forensic analysis of video scenes.

By directly dragging objects in the scene along their movement trajectory, DRAGON enables users to quickly and precisely navigate to a specific point in the video timeline where an object of interest is in a desired location. Examples include the specific frame where a sprinter crosses the finish line, or where a car passes a traffic light.

cross posted: http://itp.nyu.edu/RepresentingEarth/?p=519

Filed in ITP, Representing Earth | Tagged Fall 2009, ITP, seam carving | Comments (0) »

Extended Study

14-Oct-09 | 498 views

I found this ref photo on ffffound for extended drawing study

Filed in Drawing Machine | Tagged drawing, Fall 2009, ITP | Comments (0) »

Gestures and Form

14-Oct-09 | 515 views

Gesture Studies on the street:

Form Study:

Filed in Drawing Machine | Tagged drawing, Fall 2009, form, gesture, ITP | Comments (0) »

Digitized Memories

08-Oct-09 | 1,342 views

Virtual reality is apparently not just about the vision, it’s always about using all possible help of the technology to trigger a similar (but altered) experience that is already stored in the human brain through all senses, including very subtle emotions. It’s a magic of bringing the memory to the present. It is not too hard to imagine that virtual reality or telepresence technologies be applied to a more emotional and psychic topic: to bring the deceased back.

We’re creating much more personal information than ever on the internet nowadays and what is going to happen to these data after we die? How much is it possible to restore partially or even a small piece of presence, that is visually, audibly or emotionally connected to ourselves?

One conceptual approach was “Cemetery 2.0” by Elliot Malkin. It works by keeping an active satellite Internet connection that keeps the gravestone in constant communication with the Web services.Visitors to the physical memorial can view related memorials on the device display, while visitors of any of the online memorials will recognize that their browsing is associated directly with the actual burial site. The Cemetery 2.0 device is connected to Hyman Victor’s GEDCOM file. The file includes The Kaddish (the Jewish prayer for the dead) to neutralize an inadvertent posthumous baptism.

95f748fc54eee1936564fdc3212229b2

Digital Remains is an art project by Michele Gauler, which is more closely connected to the topic of virtual presence. It assumes a world in which our data is stored on the network creating digital archives of generations of people. Personal access keys are used to remotely log on to the digital remains of a person and receive their data on our own digital devices. Based on data tags and meta data, search algorithms dig through a deceased person’s data, presenting us with content that is most likely relevant to us. For instance, a photograph from a holiday we spent with the person 10 years ago or the person’s favourite piece of music which they typically listened to while writing e-mails.

35a3ffe5275c29a247e4e86743a91a2b

One interesting is the presentation of the concept: an over-head projection of hand and arm movements, which are in synch with the movements on the monitor screen. Placing one of the access keys on the table, starts the access key spinning and triggers the relevant scenario. The presentation itself is stunning however a little confusing too, it certainly conveys some emotional connection to the deceased friend.

9bcda9cdccf143134029710a6834ba47 ee2646d548e05a322af39519f26e70e8

There used to be demonstration videos online but I cannot find them anymore. An audio brief is still available on MOMA’s website.

A similar concept could also be seen in “Family Digital Shrine“. By using tea cups with trackable tags (as identity of viewer) it’s able to reveal different content for the views. It’s a great way of storytelling, and of encouraging storytelling, but it’s also a shortcoming in terms of usability. A youtube video is available online. I’m wondering how they collect their data and categorized them.

The most ambitious one might be the mission eternity. A person’s data is packed into a data package called ARCANUM CAPSULE, which is securely stored in a p2p network. Spec of the capsule content: pdf. The presentation mode is called BRIDGE, and they’re planning a mobile graveyard (called SARCOPHAGUS) which is a standard 20 foot cargo container equipped with an immersive LED screen of 17’000 pixels that cover the walls, ceiling and floor on which the visitors can walk.

fe24465882a4bbba4c522a036cda95f4

I would like to finish with a purely emotional piece which has nothing to do with any specific human being however dramatically arouse my emotions, and it’s stunningly beautiful (and also bizarre too). Sometimes we feel more with eyes closed.

Funktionide by Stefan Ulrich

http://vimeo.com/5509560/

One last interesting fact: all above projects are from 2006.

cross posted: http://itp.nyu.edu/RepresentingEarth/?p=256

Filed in Representing Earth | Tagged afterlife, Fall 2009, ITP, memory, presentation | Comments (0) »

image bookmarking websites as reorganization for image metadata

24-Sep-09 | 5,970 views

Whereas social bookmarking websites are serving the purpose of personal stash of online assets for people, they are also performing extremely effective reorganization on the massive but scattered information on the web. Taking just image bookmarking tools/websites for instance, they are generating much more semantic metadata for all accessible images on the web, and are essentially “curating” more meaningful and better-quality image contents out of billions and billions.

Examining currently popular image bookmarking websites reveals the possibility of creating semantic image library on top of the whole image inventory of the internet. Here we’re not talking about image hosting services, although they also provide FOLKSONOMIC tools that help themselves evolve, but bookmarking services are clearly bringing in something more interesting.

Services specifically targeting visual images emerge due to the human nature of pursuing beauty. This fact does affect the categories of images being collected. We see most image bookmarking sites are really design/photography oriented even though it might not necessarily be their initial intention. Not a lot of people are bookmarking parties photos anyway, they have facebook already. It also shows that we do have different tools for collecting different aspects of information out of an image.

Besides those traditional folksonomic tools that most image hosting services are also providing, I think the most interesting features from these bookmarking sites are smart recommendation and the possibility to track cross posting (or reposting, or whatever).

Smart recommendation features algorithm that calculates image similarities. It could be tag based but in fact ffffound is implementing this just based on the image collection of a certain user and all users that find the same image favorable. It’s very similar to what Pandora and Last.fm is doing, I’m not very sure if there’s any image fingerprint stuff happening under the hood. Discussion?

Reposting tracking and duplication check are essentially technical tools that bookmarking sites come up with to release server burdens. However they could provide very interesting data on how images get referred to on the internet and could be an effective search helper.

FFFFOUND!

http://ffffound.com/

collect: bookmarklet, IE extension
redistribute: screensaver, iphone view
discover: no search, no tags, similarity comparing, smart recommendation based on personal taste, follow/follower system
misc: no way to find how many photos are logged, report inappropriate/copyright issue

VISUALIZE.US

http://vi.sualize.us/

collect: on-site url ref, bookmarklet, firefox extension

redistribute: unofficial facebook application, add-this style sharing
discover: social tagging, tag search, watchlist(fan system), does seem to be able to find similar images
misc: safe filter, duplication detection

74,446 bookmarks

WE HEART IT

http://weheartit.com/

collect: bookmarklet only
redistribute: ?
discover: social tagging, tag-only search, similar entries (tag based), contacts system
misc: unsafe filter

31795 * 20 = 635900 bookmarks

imgfave

http://imgfave.com/

collect: bookmarklet only
redistribute: facebook integration (sharing), friendfeed integration, blog sidebar widget
discover: no search, no tag, collection is a tag-like concept
misc: nudity alert on bookmarklet, related images seem irrelevant

tumblr

http://www.tumblr.com/

collect: on-site, bookmarklet, facebook app, iphone app, api
redistribute: embed widget, api
discover: tagging, search, repost, follower system, trending page, realtime popularity radar(http://www.tumblr.com/popular)
misc: tracks reposting, multi media types (videos everything), no similarity match

Filed in ITP, Representing Earth | Tagged billions and billions of photos, Fall 2009, image bookmarking, research, social bookmarking | Comments (2) »

line and contour

21-Sep-09 | 515 views

1. Place one (large) drop of ink on a piece of paper;

2. Tilt the paper to push the ink towards the other corner of the paper but keep the ink from the edge of the paper;

Contour Studies

Filed in Drawing Machine, ITP | Tagged contour, drawing, Fall 2009, ITP, line, Week1 | Comments (0) »

Humon and Intervention visualization are in ITP Spring Show 2009

10-May-09 | 500 views

Humon and Visualization of US Military Interventions are both in the show!

You should come if you’re in New York at this time. Only one day left!

Check it out on Monday, May 11, 2009 -9pm!

http://itp.nyu.edu/show

Filed in ITP | Tagged ITP, Spring 2009, Spring Show | Comments (0) »

Humon beings update

02-May-09 | 957 views

Li and I presented the humons on Wednesday and Thursday in both Redial and Nature of Code class. Although we’ve been playing around for weeks and a lot of people already saw it, it’s still so joyful to see a lot people actually play with it at the same time.

Here’s the video from the nature of code class presentation.

A short demo of the basic motions we have so far: basic motion, jelly-ish body shape, gravity, collision, multiple ground tracks. The physics are implemented with the great toxi verletphysics library.

Basic interactions are all accessible from the phone, which allow a lot of people to interact at the same time without introducing more human-computer interfaces. In this demo video: basic movements (horizontal moving), jumping, message recording.

Another key feature of the humon project is to allow people to record and broadcast voice messages to the whole community. The instant playback will probably be replaced by a queueing system in short future, so that the audience may be able to focus on certain messages.

Some more screenshots from early processing sketches and different scenes:

The development weblog is now live too, we’ll keep updating there more often.

Check it out: humon doings

I stayed up late to check if we got into the spring show, but it’s still not out yet. Time to sleep!

Filed in Redial:Interactive Telephony, The Nature of Code | Tagged final, Interactive, ITP, monster, presentation, Processing, Spring 2009 | Comments (1) »

US Intervention 1798 – 2009

27-Apr-09 | 1,011 views

“The United States military has been intervening in other countries for a long time. In 1898, it seized the Philippines, Cuba, and Puerto Rico from Spain, and in 1917-18 became embroiled in World War I in Europe. In the first half of the 20th century it repeatedly sent Marines to ‘protectorates’ such as Nicaragua, Honduras, Panama, Haiti, and the Dominican Republic. All these interventions directly served corporate interests, and many resulted in massive losses of civilians, rebels, and soldiers.” (via A BRIEFING ON THE HISTORY OF U.S. MILITARY INTERVENTIONS, By Zoltán Grossman, October 2001)

People might be familiar with recent US military interventions in Middle East, or those in Asia in 1970s, however in the past 200 years’ history, overseas interventions actually happened a lot more frequent than people usually think and played an important role to this country’s development.

This visualization is not intended to create a narrative piece of intervention history, it’s more created as a tool to go through the mass amount of history events and discover the correlations among those events in a certain period of time, or within a certain region.

DOWNLOAD

Download app for mac:

http://leejayxia.com/processing/us_intervention_mono_present.zip

For 1024 x 768:

http://leejayxia.com/processing/us_intervention_mono_present_1024.zip

I’m still working on the source data since world wars are currently excluded. Download links will be updated when I finish.

Filed in Mainstreaming Info | Tagged final, ITP, Military Intervention, Processing, Spring 2009, Visualization | Comments (1) »

I wish I know more trigs

14-Apr-09 | 473 views

Did too much soldering today. I’ll just post some screenshots as update for the U.S. intervention history visualization. Since last the most effort regarding this project was spent on figuring out the tech details on building interaction into the spinning globe. I wish I know more about trigs!

Several tips to put down for myself:

- move textFont() out of the loop;
- do not create font on the fly if using huge font size;
- screenX and screenY are your friends (while creating mouse over interaction in 3D) (Thanks Dan!);
- rotation is headache, rotation in 3D is huge headache:

I had some rotation and translation params of a point generated by geo data. It’s easy to get params in those forms when you’re dealing with a sphere. Such point can easily be drawn with a series of rotate() and translate(), and a box() or sphere(), however I want to calculate the actual coordinates of those points in the 3D space to place attractors / draw lines / whatever.

I wish I paid enough attention while I learned trigs in high school, but I ended up looking into processing core source code and did something like this:

1
2
3
4
5
6
7
8
9
10
11

PVector rotation = getRotation(); // the rotation params that i do have
float offsetZ = 500; // some arbitrary number

PMatrix3D m = new PMatrix3D();
m.rotateY(rotation.y);
m.rotateX(rotation.x);

//more 3D transformation here

m.translate(0,0, offsetZ); // just do whatever I would do in normal draw loop
PVector l = new PVector(m.m03,m.m13,m.m23);

It’s basically just reusing the PMatrix3D class which is used internally in 3D transformation in processing, but I’m so happy not having to figure out the math myself.

More experiments: