Ag robotics companies have historically faced a frustrating trade-off: build a machine optimized for one task or crop, or add more flexibility, which increases cost and complexity.
California-based Bonsai Robotics is on a mission to change that equation with a vision-based autonomy stack that it claims can turn 2D camera images into a scaled 3D understanding of the farm environment.
This, says cofounder and CEO Tyler Niday, enables the same underlying system to operate across orchards, vineyards, berries and other specialty crops without developers having to rewrite thousands of lines of code for every new application.
While it continues to retrofit existing equipment, it is also building its own machines following its acquisition of Farm-ng, including larger, heavy duty versions of its Amiga platform for spraying, hauling and lifting.
The goal is not simply to replace labor, but to drive down the overall cost of farm operations by combining autonomy with lower-cost, more flexible machinery, says Niday.
AgFunderNews (AFN) caught up with Niday (TN) at the inaugural Ruggedize ag robotics conference at Reservoir Farms in Salinas to discuss how AI is changing the economics of ag robotics, Bonsai’s push into multiple specialty crops, and why previously intractable challenges such as robotic harvesting may finally be solvable.
AFN: Give me the 60-second pitch…
TN: Bonsai Robotics is a full-stack robotics company for the rugged world, meaning we build an operating system to control and monitor and manage your farm, the autonomy stack, which can go on existing tractors or new form factors, and then full stack, meaning we build robots too.
So behind me, you’re seeing some of our new products, and when we build these robots, we’re not just capturing the labor savings from autonomy. We’re additionally reducing the capex, the cost of the machine, and the operating expenses. The machine behind me can burn about three gallons of diesel per day instead of 30. So lots of stacked up savings with autonomous applications.
AFN: You recently talked about moving systems from 2D images to 3D scaled worlds on any crop at any time with no code required. What does that mean in practice?
TN: I’ve been building ag autonomous systems for a long time, and historically in robotics, you write hundreds of thousands of lines of “if-then” statements, more or less, to get to a solution for one crop. If there’s a plug on a plow, you have to write code around that.
Bonsai Robotics has a learned model, so we’ve had a ton of data, and we’re able to more or less build this foundation model that has multiple different 3D understandings, whether it’s an elevation map, a voxel map of the trees [a 3D spatial representation of the trees/environment], semantic occupancy [is the space occupied by a tree, the ground, a person, a machine etc].
So this one model can really go to all these different crops very quickly just from a 2D camera. So it’s a 2D camera to this scaled 3D world, really enabling us to go across the whole spectrum of specialty crops, which is really unique versus just focusing on one.
AFN: So what have you been training it on?
TN: We’ve deployed 400 units and more data is a huge piece of our puzzle. We we have about a million acres of data collected across specialty crops, and we’ve also trained an Nvidia world model [an AI model designed to build an internal representation of how a physical environment looks and behaves].
We use [Nvidia’s] Cosmos [family of world foundation models for physical AI]. It’s an on-road self-driving car model, and we put all our ag data on it, so we have really good data. So if we need to simulate a new environment or a new crop or rain or an elephant walking through the screen, we are able to really advance and scale all of that as well. But our foundation model is all built internally with a full automated self-labeling pipeline.
AFN: So you’ve got this a 2D image, and then you use AI to extrapolate from that this 3D representation: where the ground is, how far away things are and so on?
TN: Correct. So like normally in robotics applications, you need depth. So historically, firms have used lidar, where you shoot a laser out and measure the distance, the time of flight it takes to hit something, or stereo vision where you’re matching pixels. Instead of doing those algorithms, we have a learned model, the machine learning, the transformer model is learning the depth based on historic data.
A great example is normally if you have dust in robotics and you’re using a lidar, you’re blind. But because we’re a learned model, our camera system is inferring through that and guessing what’s next, guessing what’s actually after. You have a lot more contextual understanding because it’s not a rule set to get to an answer. It’s learned by a transformer.
AFN: You started doing a lot of work in orchards with almonds. How have you broadened your scope with the acquisition of Farm-ng?
TN: When I think about autonomy in the rugged world, the outback of Australia has probably some of the most adverse conditions. There’s no GPS connection inside these 30-year-old orchards. There’s dust. There’s sand hills. It’s really challenging to navigate, and you can’t use traditional robotics technology.
So Bonsai Robotics started there to really pioneer our vision stack in models, and now we’ve scaled outward because we’ve been able to create this model that works in some of these adverse conditions. This one model can go into strawberries, vineyards, apples, table grapes, citrus fruit… the whole array of specialty. Because specialty has the largest labor requirements rather than corn, cotton, and soy. But each each market’s a niche, so if we can do all of it, it’s really the enabler, and that’s what our foundation model enables.
AFN: And in some of those other crops, are you supplying your own Amiga machines or retrofitting existing farm equipment?
TN: Both. So the split from a revenue perspective is about half and half right now. Horsepower is always required, so contractors are always going to be needed. But there are specific applications where an autonomy first platform design like the Amiga Max behind us, that’s a strawberry sprayer, can really do more than just labor replacement and drop the cost significantly on a per hour operational basis too.
So we’re doing both, and we’ve really pioneered these new bigger Amiga platforms to go after bigger farming applications.
AFN: So we’re here at Reservoir Farms. What is the value that it can bring to a company at your stage of development?
TN: So Reservoir is one of the most spectacular things I’ve been a part of in ag. I’ve been in ag my whole career. I’ve been through some acquisitions, but when I started Bonsai, it took me seven months to find an orchard to run in. I was shaking trees, right? People do not want me touching their trees and breaking them and knocking them over.
So I didn’t have a lot of connections in strawberries or leafy greens or all of these [crops grown at Reservoir Farms], and it was like when our machine was ready, we had testing capabilities. At the snap of a finger, we had literally all the decision-making growers from this valley here in a few days to watch this demo.
It was the same with Napa [Valley]. I don’t have a lot of connections in the grape industry, so [via Reservoir] the same thing happens. It’s been such an enabler for us from a testing perspective, but also [in providing] connections to the community to know what to build so we build a product that is usable.
AFN: So, where are you commercially today? Â
TN: Bonsai Robotics has sold now over 400 units in total. About 75 of them are OEM retrofit units, and the rest are Amigas. And with the new product line, the Amiga Maxes, it’s the hybrid electric platform where we’ve actually sold out of this year. So we’re going to be racing to just build the units that we’ve already deployed. So it’s a pretty exciting time for us. We’ve gotten a lot of demand on the new products, but also scaled our existing operations on the OEM side, in the big tree crops.
AFN: How important is flexibility in making the economics of these machines add up?
TN: I think it’s everything. You’ll struggle with autonomy subscriptions on just a tractor, right? The cost of autonomy is generally expensive. So when Bonsai Robotics did that [autonomy] platform for almond orchards the OMC [Orchard Machinery Corporation] AR-500 is an orchard shuttle truck [for collecting and transporting almonds], and it cost somewhere around $200,000. But it was historically parked [for most of the year]. That’s a power unit, that’s a lot of horsepower [to be idle for most of the time], right?
Turning it into a tractor [that can do multiple tasks] is a huge enabler, and that’s a big aspect of why people want to buy it. You need to go past just a labor replacement and find other ways to drop the cost of these machines and services.
AFN: Danny from Reservoir was talking earlier about how ag robotics is becoming more investable…
TN: Ag investment cycles are different, and there’s been a lot of learnings over the course of the years that are making it more investable. The technology has increased at a rate where we can actually solve problems.
Five years ago, I would have told you harvest is not a solvable problem. Now, I think it is with end-to-end models and just some of the things you can do. So I definitely thinking it’s more approachable. I think the boom in physical AI is huge. Sure, we don’t pay people as much as an excavator operator, but the demand is going to go through all these physical industries. It’s the ultimate problem to solve. And from my perspective, I think agriculture is the most important one, so I’m excited for it.
*B-roll courtesy of Bonsai Robotics
Further reading:
🎥 ‘The whole system is now more investable’: Reservoir makes the case for ag robotics
Castoro Cellars deploys Saga Robotics’ UV-C bots across 600 organic acres
