IDEA FLOW
Winter 2003

PDF | Table of Contents | Go to Bottom of this Page

SENSING AND RESPONDING: Mani Chandy's Biologically Inspired Approach to Crisis Management

rganisms receive streams of information from their senses, yet they manage to avoid information overload and system breakdown by instantaneously aggregating information to identify threats and opportunities, and responding appropriately. Organisms are "sense-and-respond" systems that detect and respond to important events in their environments. Abstracting this relationship of organisms to their environment and turning it into a computer-based information system was the challenge that Mani Chandy, Simon Ramo Professor of Computer Science, set for himself almost a decade ago.

What he came ups with could be thought of as the inverse of a traditional database: rather than repeatedly accessing well-defined static data structures with different queries, imagine data that constantly change, have no well-defined structure, but questions about the data that are more or less constant. You can liken this to an organism's day-to-day stance to its environment—organisms get continuous streams of data, with varying structures and formats from their senses, but the conditions that define threats and opportunities change slowly. Occasionally there are "events"—sudden, dramatic, sometimes catastrophic happenings that require immediate response. Organisms that respond to non-events waste energy. Organisms that don't detect threats and opportunities in the environment don't survive.

In 1998, Chandy took an academic leave from Caltech to start iSpheres, a company devoted to creating information systems for crises management. In the aftermath of the TWA 800 disaster, Chandy was inspired to develop a system so that all kinds of information (weather reports, emails, engineering data, police and fire department responses, and so on) could be accessible and useful for rescue workers and later, investigators, responding to the crises. This idea was the seed that led to iSphere's eventual products: decision-making systems for financial and trading institutions, manufacturing concerns, and corporations requiring a "sense-and-response" approach to tremendous amounts of unpredictable data in all sorts of formats and configurations. The software had not only to duplicate the "sense-and-response" functions that humans bring to decision-making, but also to amplify human capabilities, in both the amount and types of data sensed, and the time it takes to respond.

Chandy began working on this biologically inspired, inverse database problem in 1992/93, but in 1998 had the key breakthrough of "sense and response" as the organizing principle behind his conceptions. As given his nature, Chandy put all his results on his web site, including downloadable software. One day, Caltech's Office of Technology Transfer called him and suggested he take everything down, as they were getting inquires from companies who wanted to license the ideas! This phone call led to the germination of iSpheres.

iSpheres faced several challenges. Sense-and-respond systems have to manage large volumes of heterogeneous data ranging from stock ticks at 10,000 per second to news stories and email. These different streams of data have to be integrated with information in databases and business-intelligence warehouses to detect threats and opportunities such as arbitrage. Then the system must respond appropriately to events, and appropriate response often requires orchestration of services within and outside the organization. The fundamental problem is to build sense-and-respond platforms, layered on top of information technologies such as databases and application servers, which can be configured to deal with different applications. Though the applications are usually complex, the interfaces for configuring the platform are relatively simple: after a few weeks of training, clients can configure the software themselves, for their particular applications.

An exciting part of starting a company, Chandy explained, was talking to users in different industries ranging from energy and stock trading to supply chain management, and solving their concrete problems directly. Creating a product that affects people's lives was exhilarating and demanding because the product is used in mission-critical applications. In the academy, the "metrics for success are fuzzier," the problems solved are removed from concrete requirements in different industries, the pressure to make systems work 24/7 is less intense, and sometimes solutions are appreciated only by other academics. It was also a revelation to pitch ideas to investors—the process of getting your ideas funded in the venture-capital marketplace was quite different than getting your ideas funded by granting agencies, and more satisfying in many ways.

Chandy learned how industry works, how essential effective leadership is, and that a company is not simply a technology. The culture of the company is crucial, and he strove to make iSpheres a nurturing environment for employees, an environment where they could grow and evolve. However, the discipline of the marketplace was the constant and final arbiter of what the company could (or could not) do. This fiscal discipline was brutal, especially concerning hiring and firing, and this was perhaps the most difficult part of starting a company.

he original idea of crisis-management systems that would assist rescue workers and investigators evolved at the company into decision-making systems for financial institutions and manufacturing concerns (the investors realized that the core product had to be revenue generating). So, Chandy is back in his Caltech lab figuring out ways to improve the software for crisis-management applications. He feels that for some (but not all) problem spaces, giving away systems freely is more effective than building commercial products. He's come full circle, but in the process created and disseminated ideas far and wide. Other application areas he is working on include systems that would assimilate the vast amounts of heterogeneous biological information available to detect significant opportunities, as well as systems that would allow one to study new types of financial markets (see article on Information Science and Technology, page 18). He is eager to become engaged again with the deeper mathematical issues of the work. If he were younger, he hinted that perhaps working in startups periodically, or even on a fulltime basis, would be a tempting direction; but now, with his start-up company out of its infancy and on its way to childhood, he is very satisfied to delve back into the challenging theoretical concerns at Caltech. ENG