Designing an environment to create mobile music instrument

Mobile phones are little multi-media computers. They have rich sensor capabilities and a diverse set of rather high quality actuators. If we consider mobile phones primary computational platforms, how can we build an environment that supports full flexibility, yet supports all the strengths yet allows us to dodge the weaknesses of the mobile device.
There are many difficult question regarding space, performance, architecture. In this talk we'll take a casual walk through the various aspects of an environment called UrMus. Along the way we will explore UI design on mobile devices, multi-rate and flexible-rate multi-media processing, universal on-the-fly mapping and much more. We'll discuss how we learned from the UI design of computer games and how we try very hard to avoid keyboard input and replace it with multi-touch interaction paradigms.