In my early days as a graduate student, I went on a snorkeling trip off the coast of the Bahamas. I'd actually never swum in the ocean before, so it was a bit terrifying. What I remember the most is, as I put my head in the water and I was trying really hard to breathe through the snorkel, this huge group of striped yellow and black fish came straight at me...and I just froze. And then, as if it had suddenly changed its mind, came towards me and then swerved to the right and went right around me. It was absolutely mesmerizing. Maybe many of you have had this experience. Of course, there's the color and the beauty of it, but there was also just the sheer oneness of it, as if it wasn't hundreds of fish but a single entity with a single collective mind that was making decisions. When I look back, I think that experience really ended up determining what I've worked on for most of my career.
I'm a computer scientist, and the field that I work in is artificial intelligence. And a key theme in AI is being able to understand intelligence by creating our own computational systems that display intelligence the way we see it in nature. Now, most popular views of AI, of course, come from science fiction and the movies, and I'm personally a big Star Wars fan. But that tends to be a very human-centric view of intelligence. When you think of a fish school, or when I think of a flock of starlings, that feels like a really different kind of intelligence. For starters, any one fish is just so tiny compared to the sheer size of the collective, so it seems that any one individual would have a really limited and myopic view of what's going on, and intelligence isn't really about the individual but somehow a property of the group itself.
Secondly, and the thing that I still find most remarkable, is that we know that there are no leaders supervising this fish school. Instead, this incredible collective mind behavior is emerging purely from the interactions of one fish and another. Somehow, there are these interactions or rules of engagement between neighboring fish that make it all work out.
So the question for AI then becomes, what are those rules of engagement that lead to this kind of intelligence, and of course, can we create our own?
And that's the primary thing that I work on with my team in my lab. We work on it through theory, looking at abstract rule systems and thinking about the mathematics behind it. We also do it through biology, working closely with experimentalists. But mostly, we do it through robotics, where we try to create our own collective systems that can do the kinds of things that we see in nature, or at least try to.
One of our first robotic quests along this line was to create our very own colony of a thousand robots. So very simple robots, but they could be programmed to exhibit collective intelligence, and that's what we were able to do. So this is what a single robot looks like. It's quite small, about the size of a quarter, and you can program how it moves, but it can also wirelessly communicate with other robots, and it can measure distances from them. And so now we can start to program exactly an interaction, a rule of engagement between neighbors. And once we have this system, we can start to program many different kinds of rules of engagement that you would see in nature.
So for example, spontaneous synchronization, how audiences are clapping and suddenly start all clapping together, the fireflies flashing together. We can program rules for pattern formation, how cells in a tissue determine what role they're going to take on and set the patterns of our bodies. We can program rules for migration, and in this way, we're really learning from nature's rules.
But we can also take it a step further. We can actually take these rules that we've learned from nature and combine them and create entirely new collective behaviors of our very own.
So for example, imagine that you had two different kinds of rules. So your first rule is a motion rule where a moving robot can move around other stationary robots. And your second rule is a pattern rule where a robot takes on a color based on its two nearest neighbors. So if I start with a blob of robots in a little pattern seed, it turns out that these two rules are sufficient for the group to be able to self-assemble a simple line pattern. And if I have more complicated pattern rules, and I design error correction rules, we can actually create really, really complicated self assemblies, and here's what that looks like.
So here, you're going to see a thousand robots that are working together to self-assemble the letter K. The K is on its side. And the important thing is that no one is in charge. So any single robot is only talking to a small number of robots nearby it, and it's using its motion rule to move around the half-built structure just looking for a place to fit in based on its pattern rules. And even though no robot is doing anything perfectly, the rules are such that we can get the collective to do its goal robustly together. And the illusion becomes almost so perfect, you know—you just start to not even notice that they're individual robots at all, and it becomes a single entity, kind of like the school of fish.
So these are robots and rules in two dimensions, but we can also think about robots and rules in three dimensions. So what if we could create robots that could build together? And here, we can take inspiration from social insects. So if you think about mound-building termites or you think about army ants, they create incredible, complex nest structures out of mud and even out of their own bodies. And like the system I showed you before, these insects actually also have pattern rules that help them determine what to build, but the pattern can be made out of other insects, or it could be made out of mud. And we can use that same idea to create rules for robots.
So here, you're going to see some simulated robots. So the simulated robot has a motion rule, which is how it traverses through the structure, looking for a place to fit in, and it has pattern rules where it looks at groups of blocks to decide whether to place a block. And with the right motion rules and the right pattern rules, we can actually get the robots to build whatever we want. And of course, everybody wants their own tower.
So once we have these rules, we can start to create the robot bodies that go with these rules. So here, you see a robot that can climb over blocks, but it can also lift and move these blocks and it can start to edit the very structure that it's on. But with these rules, this is really only one kind of robot body that you could imagine. You could imagine many different kinds of robot bodies. So if you think about robots that maybe could move sandbags and could help build levees, or we could think of robots that built out of soft materials and worked together to shore up a collapsed building—so just the same kind of rules in different kinds of bodies. Or if, like my group, you are completely obsessed with army ants, then maybe one day we can make robots that can climb over literally anything including other members of their tribe, and self-assemble things out of their own bodies. Once you understand the rules, just many different kinds of robot visions become possible.
And coming back to the snorkeling trip, we actually understand a great deal about the rules that fish schools use. So if we can invent the bodies to go with that, then maybe there is a future where I and my group will get to snorkel with a fish school of our own creation.
Each of these systems that I showed you brings us closer to having the mathematical and the conceptual tools to create our own versions of collective power, and this can enable many different kinds of future applications, whether you think about robots that build flood barriers or you think about robotic bee colonies that could pollinate crops or underwater schools of robots that monitor coral reefs, or if we reach for the stars and we thinking about programming constellations of satellites. In each of these systems, being able to understand how to design the rules of engagement and being able to create good collective behavior becomes a key to realizing these visions.
So, so far I've talked about rules for insects and for fish and for robots, but what about the rules that apply to our own human collective? And the last thought that I'd like to leave you with is that science is of course itself an incredible manifestation of collective intelligence, but unlike the beautiful fish schools that I study, I feel we still have a much longer evolutionary path to walk. So in addition to working on improving the science of robot collectives, I also work on creating robots and thinking about rules that will improve our own scientific collective. There's this saying that I love: who does science determines what science gets done. Imagine a society where we had rules of engagement where every child grew up believing that they could stand here and be a technologist of the future, or where every adult believed that they had the ability not just to understand but to change how science and technology impacts their everyday lives. What would that society look like? I believe that we can do that. I believe that we can choose our rules, and we engineer not just robots but we can engineer our own human collective, and if we do and when we do, it will be beautiful. Thank you.
