It’s one thing to build a robot that can harvest produce. It’s quite another to build one that can do so faster than humans and in an economical manner, says Brandon Contino, co-founder and CEO of ag robotics startup Four Growers.
The Pennsylvania-based company initially sought to relieve labor challenges in the greenhouse. What’s since come out of that is an autonomous harvesting machine that uses a robotic arm calibrated with four stereo cameras to precisely detect and harvest produce, starting with tomatoes. (Cucumbers and peppers are on the near-term horizon.)
Four Growers just released the second generation of its machine, which also includes heatmaps and forecasting for yields as well as patented packing cart technology. “We’ve been able to prove that we can harvest fast and now we’re at the stage where we’re showing the daily operations and integration into the customer workflow,” says Contino.
The company currently has systems running in locations in the Netherlands and Canada, including one at Syngenta’s TomatoVision facility in the Netherlands.
Below, Contino (BC) tells the company’s origin story, why it chose greenhouses over vertical farms, and the myriad technical challenges involved in building robotics for agricultural environments.
AFN: What led you to agriculture robotics?
BC: I got really interested in neural prosthetics: controlling robot arms with the brain.
I went to the University of Pittsburgh because they had some really amazing prosthetic work. I ended up writing my own grants to get funded to do a low-cost Internet of Things, water quality sensors. As I got into water quality, then I learned about water scarcity and how agriculture is the number one user of our freshwater resources.
I wanted to figure out how to reduce the demand for fresh water. My cofounder [CTO Dan Chi] and I started the hydroponics club. He was working with leafy greens and thought vertical hydroponic was an amazing way to grow. We were actually going to build a vertical farm in Pittsburgh.
Then we went to Canada and saw the economics of vertical farming. We realized [vertical farming] wasn’t something, at least in the near term, that would really be able to have wide-scale impact. The only way we can see the business model working is if you have really high prices for your produce, which could work for some segments, but it’s not going to have large impact in the near term.
So we took a step back and looked at Dutch-style glasshouses and were blown away by how they operate and how they grow [produce]. They had all the same kind of sustainability benefits of vertical farms, except they had a smaller carbon footprint because they didn’t need all the indoor lighting. And they actually had a viable business model that’s been proven for 50+ years.
So we started cold-calling all these different greenhouse owners and operators, asking what were their biggest limitations.
Everybody always said it was labor. They had acres to build more greenhouse but they couldn’t find enough people.
Diving deeper into the problem, we realized the biggest problem was around harvesting. With robotic harvesting, it was things like identifying fruit with the computer vision, knowing what [the machine] can take, what not to hit, then doing the motion planning so you’re not damaging the crop.
Between my background in the software side, computer vision, motion planning and Dan’s background in mechanical design, we had this skillset to at least take a stab at building something for greenhouses.
AFN: What have been a couple challenges along the way?
BC: There’s the technical challenge of trying to pick tomatoes in this super dense environment. I think a lot of times what you’ll see is [a machine] pick a tomato once; it’s not the hardest thing to do at once. But it just took 25 seconds to pick that one tomato and you look at a person who’s doing it a second. That’s going to be a hard economic business model.
When you pick tomatoes, they’re not all facing you. You have to reach around, they’re at different angles. There’s a whole bunch of tech challenges we had to solve to make that happen.
AFN: What are some of those technical challenges?
BC: Dealing with organic matter being makes the vision aspect a little bit tougher. You’re also in an outdoor environment in terms of lighting. For the machines, it’s getting them to see a tomato — a nice bright red thing against a green background — but also getting them to see everything else. Just knowing the tomato isn’t enough. What are things the machine can hit or damage?
If we damage a stem on week two and we break the plant, we cost that grower 30 weeks of yield. Once you have the data, it’s about being able to successfully perform motion planning and in an economical manner: you can’t take 20 seconds to figure out a path to then pick one tomato. You have to be able to execute on it quickly and get everything calibrated correctly without using extremely expensive sensors. Then the last piece of course is then picking tomatoes. We actually use a vacuum-based approach to ensure that as we pick the tomato we don’t damage it.
There’s also the integration of logistics. What we’ve learned from all the time we’ve spent in the greenhouse with our customers is that although harvesting is the hardest part of the problem, if we didn’t solve the packing and logistics of the harvested tomatoes, our solution had limited value to them. Because of this, in addition to the harvesting functionality that we’ve developed, we’ve also had to develop a secondary system that handles the management of all the harvested fruit – we call this the packing cart. This allows the system to easily integrate into their existing workflow and reduces the amount of human interventions that are required with the system.
AFN: What crops don’t necessarily make sense right now to use a robotic harvester for?
BC: The ones that are really challenging at the moment when we think about our future expansion are in the outdoor market. Apples are predominantly trellis [which enables easier picking] or at least moving very much in that direction. I actually visited a farm where lemons might be trellis in the future, but I think any of the really bushy crops are the toughest right now.
AFN: What about non-technical challenges?
BC: One that a lot of agricultural companies deal with is that we want to not just build something because we think it’s cool, but build something that’s valuable to the grower.
Our whole origins story started from the growers telling us what they needed and really working with them from day one. And so for us, it’s being in the field with them every day, running the robot everyday getting that close connection. For us, that’s not always easy because outside of California, greenhouse growers are a bit more spread out. A lot of them are in Canada. So being able to have a team based in Pittsburgh and still having a really good connection and really a deep understanding of the farmer [is important]. In the beginning, that meant living in Airbnbs for months at a time in Canada.
AFN: How has it been operating in a challenging environment for indoor ag?
BC: Indoor tends to lump the vertical and greenhouse together in the same category, which I don’t think anyone really had a problem with when vertical had a lot of hype. But now that vertical no longer has the hype, I think there’s probably going to be a little bit more of a distinction.
It’s sad to see that the vertical farms are shutting down. But going back to the origin story when Dan and I were looking at becoming vertical farmers, I can’t say I’m totally surprised. At the end of the day, you’re selling lettuce, but you’re taking on all these R&D costs.
I do think there will be people that make it, and I think there are some interesting companies that are still out there. But I think a lot of the vertical farming aspect is just fundamentally just challenging because of the lighting costs and the amount of energy it requires. [But] I think there’s some really cool applications around nurseries, and transplanting.
Greenhouses, on the other hand, have already been proven, so they’re not hit as hard. It almost seems that greenhouses are getting more momentum now that vertical has kind of lost it. And we’re seeing a lot of private equity moving to greenhouses.
AFN: Does Four Growers’ future involve more than just greenhouses?
BC: We don’t really think of ourselves as like a “greenhouse tomato harvesting robot company.” We’re really building an automated AI platform that also performs analytics as well.
We actually have already tested — with minimal effort — swapping out the grip on the robot arm. We had a new gripper and we retrained it on cucumbers, then peppers. We’re looking at potentially broccoli, cauliflower, all kinds of high-value crops, we can apply the core technology to.
On the flip-side, there’s the analytics piece. We’re looking at each individual plant two to three times a week, providing feedback on the yield heat maps and all the other data that we collect, helping growers with forecasts, helping them increase their yields. And just some of the initial data we’ve gotten out from our systems that are running every day in the Netherlands and Canada, I think I think there’s a lot of opportunity to really help these growers get another layer of understanding of their crops.
