Hi! We're Robert Katzschmann and Joseph DelPreto, researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) who work in the field of soft robotics.
We just published a paper in Science Robotics on “SoFi” (pronounced Sophie), a soft robotic fish we created that can swim alongside real fish in the ocean. We’re hoping a platform like SoFi could enable nature filmmakers to better document marine life up-close, monitor ocean pollution, and someday even inspect underwater infrastructure like oil rigs and pipelines.
We’ll be online today at 2pm EST to answer your questions. Feel free to ask us about SoFi, our other projects (including a soft gripper for robot hands, a dynamic soft manipulator arm, 3d printed soft robots, and robots corrected by brain signals), our academic backgrounds, and anything else you want to know about.
Note: Access to full Science Robotics paper via our lab website, search for and click on link called "Paper: Official full text version formatted/edited by Science Robotics".
Requisite disclaimer: we are by no means speaking for MIT or CSAIL in any official capacity!
Is this kind of swim movement more energy efficient than a normal propeller?
we did not study this within our work, but there is the potential for this swim movement of a compliant fish body to be more efficient, see for example this video and study: Dead fish swimming upstream D.N. Beal, F. S. Hover, M.S. Triantafyllou, J.C. Liao, and G.V. Lauder. "Passive propulsion in vortex wakes." Journal of Fluid Mechanics 549 (2006): 385-402. Link to paper ~Robert
What happens when one breaks? Does it just sink to the bottom and become trash, does it decompose, what happens to a bigger fish that eats it, can it digest it, how does it not harm the other fishes insides?
fortunately, non of the prototypes broke yet. In case one breaks, we pick it up, repair it and deploy it again. The current prototype is build using silicone elastomer, 3D printed ABS, couple of electronic parts. These things won't decompose within any reasonable time frame. There are research efforts on making the soft bodies out of degradable rubber, but we will have to wait for those materials to become available. ~ Robert
Awesome! Thank you for your time.
When I first saw this [awesome] robot, I had a couple of questions. Mind you, I haven't read in-depth about the equipment yet.
1) Do you see this robot as being able to contribute on an ecological level? In other words, using it to somehow replenish the ocean in different ways.
2) How deep do you think you could get this hush puppy to go in ideal conditions?
3) How many prototypes were developed prior to the master bot?
Happy to answer! to 1) We see the robot as a means to learn more about the ocean by studying marine life, but also by monitoring the ocean using sensors incl. temperature, water samples, video footage. 2) Our current version of the hush puppy SoFi was tested to 18 m in depth, but changing some of the materials we used, we could see it conceivably go much deeper. 3) We have been looking at this since 2012. This is the 4th generation of our robotic fish, with a few prototypes per generation as we were tweaking the functionalities. ~Robert
What powersource does the fish use and how long can it swim at full speed?
The fish uses a 35 watt-hour LiPo battery, and was able to swim for about 40 minutes of operation in the ocean on a single charge (we tested it during 6 dives over the course of 3 days). This was at a range of swimming speeds, so it might be a little less if constantly at full speed, but hopefully that gives a general estimate. ~Joseph
Hi guys, Automatics and Robotics PhD student here. I'd like to ask you about 3 things:
1) What would you say is the hottest topic in AI right now?
2) Do you maybe use ANFIS in your labratory? I am considering making control system for a wearable rehabilitation device based on ANFIS classifiers and would appreciate any insights on what to avoid. :)
3) How does your daily routine as MIT's employees look like?
1) I am especially excited about using AI to give our robotic fish more autonomy, such as using using its onboard footage in real time to track and follow real fish or to localize itself within a coral reef and identify structures within the reef. Using AI for autonomous driving or for material handling are two very hot areas.
2) Joseph and I have not used ANFIS yet for our projects.
3) Speaking of the lab that we are part of: We have a lot of independence in allocating our time and financial resources for projects, all driven and motivated by moving our research projects forward, gaining insights, and seeing results. We build lots of robots, iterate on their design, and test them. We also have a fair share of paper writing to do. We have weekly meetings with our whole group (20 people), and individual meetings with our advisor. Some days are 9 to 6, others can go much longer, so we have to manage our time wisely. ~Robert
How deep will y'all be able to swim with it and how long does it last under water?
The current version was designed to be compatible with typical human divers, so we've tested it down to about 18 m. We could use some different materials and increase the structural integrity to go deeper though, which could be especially useful if we make the robot more autonomous in the future (for example by using the camera to automatically track fish or explore reefs). It currently lasts about 40 minutes on a single battery charge (using a 35 watt-hour battery). ~Joseph
This AMA is being permanently archived by The Winnower, a publishing platform that offers traditional scholarly publishing tools to traditional and non-traditional scholarly outputs—because scholarly communication doesn’t just happen in journals.
To cite this AMA please use: https://doi.org/10.15200/winn.152215.55063
You can learn more and start contributing at authorea.com
Thank you for archiving!
I saw in your conclusions you mention:
The next steps are to use SoFi as an instrument to 1) study the behavior of marine life over long periods of time without human interference with the scene, 2) study if SoFi can be used to influence the behavior of marine life, and 3) create robotic swarms.
But other than point 3, I was left wanting to know more. What major challenges do you see for long term operation (i.e. how are you going to address biofouling, are you working on a long-term autonomy controller, etc.) . Maybe my definition of "long term" is different than yours, but for me it means at least being deployed for a month+.
And in general, what kind of environmental approval headaches did you have when trying to get the go-ahead to put this robot in the water?
Some major challenges for a long-term deployment would probably include long-term autonomy of the robot, power supply, self-localization, communication, and also biofouling.
We could imagine increasing the autonomy by using existing on-board sensors such as the camera to track and follow live fish or to independently navigate underwater habitats. This would also reduce the needed input from a human controller, making them more of a higher-level mission planner (for example, choosing to follow a fish or deciding on areas to explore).
Regarding power, it would be interesting to consider approaches such as adding solar panels on top of SoFi and having the robot periodically surface to recharge its batteries.
The use of prerecorded maps can be helpful for SoFi to perform self-localization within the mapped environment.
Since the humans would then be farther from the robot or likely above water, we could conceivably use radio signals to communicate with a floating buoy or similar device that translates radio-signal-based air commands to acoustic underwater commands; this would decrease the necessary acoustic transmit power and help increase deployability for long-term missions. While the communication is currently unidirectional, the current robot hardware supports transmission as well so we could possibly transmit localization pulses from the fish. Also, the use of alternative means of underwater communication, such as optical or electrical, are methods we will investigate.
Biofouling will definitely be an issue for long-term deployments, but depending on the length of a mission, this can be mitigated by regular maintenance/cleaning of SoFi. In this regard the biodegradability of the materials used in SoFi will be part of future research efforts as well.
Environmental approval depends on where you deploy the device. In terms of types of materials used, our robotic fish is quite similar to underwater camera and SCUBA gear.
~Joseph & Robert
How long until we see this as the hottest pool toy of the summer?
Probably when it replaces "Ridiculous Inflatable Swan-Thing" on ebay: https://www.ebay.com/itm/David-Shrigley-Ridiculous-Inflatable-Swan-Thing-Collectable-Blow-Up-Pool-Toy-/273060933968
How long until we see this as the hottest pool toy of the summer?
On the notion of when SoFi is available for others to use: We hope that within a year, we can support biologists with their studies using SoFi as a platform. It would probably be 2-3 years from now until SoFi is commercially available for anyone to buy - this also depends on if and when investors will fund the commercialization...
i have a couple of questions
what are the implications for human prosthetics, would you guys say you have made a breakthrough for lifelike robotics?
i read that you guys want to use it to monitor fish populations, I think that it's really cool. how do you expect to transmit the data collected by the fish? assuming that they are very spread out, that could be a hard task. also are you planning to use AI to analyze the videos you guys collect?
Thanks for bringing up these great points! It's not clear that this particular fish would be directly applicable to prosthetic devices, but it does help further the field of soft robotics more broadly. Soft devices could conceivably help enhance prostheses by making them more compliant (perhaps aiding grasping) or lifelike, although achieving a sense of embodiment by the owner of such a prosthetic device would probably take significant design effort regarding form and control. More generally though, soft wearable devices are also a promising area for human augmentation - for example, check out some of the work by Conor Walsh at Harvard: https://biodesign.seas.harvard.edu/soft-exosuits.
Regarding scaling the system to populations of fish and multiple robots, there are a few directions we could consider. For example, it could be fashioned as a distributed system (with pairs of robots exchanging data when they are nearby) or as a centralized system (with each robot periodically sending data to a central computer). We could continue to use acoustic communication underwater, but could also combine it with radio-based methods above water; one option could be using buoys as stations that convert underwater acoustic messages to above-water radio messages and vice versa. Depending on the data rates and distances needed underwater, we could also consider other underwater methods such as optical communication (which often has higher data rates but is susceptible to ambient light and scattering).
Whatever communication method is chosen, the robot would be made more autonomous for such a mission so it could log data onboard and operate mostly independently with a human providing high-level mission objectives. Towards this end, we would likely use AI to increase the robot's autonomy as you suggest; for example, we could analyze its onboard camera footage in real time to track and follow real fish or to localize itself within a coral reef and identify underwater structures. We could also use AI offline on previously recorded footage, automatically extracting useful segments and key frames (our lab has worked on a concept called coresets that can be applied to video segmentation - for example, see http://groups.csail.mit.edu/drl/wiki/index.php?title=Project_iDiary or https://people.csail.mit.edu/rosman/papers/icra_17_medical.pdf).
~Joseph & Robert
What are the long term goals of your project?
What level would you say we are to autonomous robotics ?
How do you see you project spreading innovation in the medical sector?
Thanks for your time and innovation
Answer to 1): The future steps are to use SoFi as an instrument to 1) study the behavior of marine life over long periods of time without human interference with the scene, 2) study if SoFi can be used to influence the behavior of marine life, and 3) create robotic swarms.
Mid-term goals along those lines are: We will be working on several improvements on SoFi, including enabling its onboard camera to automatically track objects like real fish so that it can follow them. In addition, we plan to build multiple “SoFis” to see how additional robot fish will impact the behavior of schools of fish and increase the possibilities for detailed oceanic observation. We also plan to increase the speed of the fish by improving its pump system and tweaking the design of its body and tail.
Long-term goals: We've so far been focusing on testing the robot's swimming capabilities and how well it can navigate complex environments while observing marine life. Moving forward, we'd like to see SoFi used for more in-depth studies of ocean environments and fish behavior. We're excited by the possibility of creating swarms of soft robot fish to monitor larger areas and to observe or influence more schools of fish. Using SoFi to create this type of underwater observatory could help further our understanding of the mysteries our oceans contain. We could also imagine using these robots to monitor pollution throughout underwater habitats, adding additional sensors to SoFi and have solar panels that allow SoFi to recharge while surfacing.
Answer to 2) For the fields of flying, driving, walking locomotion and underwater locomotion, all of these four fields are at different levels of autonomy. Most advanced is probably the realm of autonomous driving with many companies tuning and working on getting the corner cases right to increase safety and minimize failures. Varying lighting conditions and the unpredictability of humans in traffic pose a lot of challenges, but I think we are soon there for companies to release first publicly available systems. For flying, with its restrictions in what part of airspace to use and with other goals in mind (compared to driving), there are focused products that can do a lot of autonomy within a specific task (e.g. follow and film a person). Package delivery over long distances will take a bit longer, more uncertainties arise when providing services for partially unmapped areas with temporary changing arrangements. For walking locomotion, the challenges are in the design and the control of the system, soft robotics or at least added compliance to the mechanisms can help to make the walkers more robust when walking or recovering from falls. Boston dynamics has shown some amazing quadrupeds, I would not be surprised if one of those will be released as a product soon. For underwater, just the deployment of the system and the system cost in general is quite inhibiting. Systems like our robotic fish can help to lower the cost for creating swarms, our robot costs a couple hundred dollars to make. There are also a couple of new startups starting to create lower cost drones for less than $10k. But going deeper in the ocean and dealing with the difficulties of communication for use in localization as well as poor vision limit the autonomy in this field.
to 3) Soft robotics has already started show some new solutions to surgery. Soft robots have the potential to be inherently safer compared to rigid robots. For example work like this is useful to assist in minimally invasive surgery.
- t3_87hthf_comments.json 84 KB
This article and its reviews are distributed under the terms of the Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and redistribution in any medium, provided that the original author and source are credited.