The $16.8 million dollars has been granted to University of Auckland’s department of electrical and computer engineering, over five years. It will allow the university, along with six institutional research partners across New Zealand, encompassing more than 30 researchers and 10 PhD-students to develop a robotic automation system to upskill workers, as well as to undertake manual tasks.
Professor Bruce MacDonald, who is the leader of the Centre for Automation and Robotic Engineering Science (CARES) says the funding came after lengthy talks with members of the horticultural and viticultural industries.
“We wanted to know what their key priorities were,” he says, “and labour was the number one issue. (To solve that) is a three-step process. First we learn from expert operators in the vineyard, then help the labour that is available, and then eventually we automate it.”
The research programme plans to utilise the developing fields of artificial intelligence and augmented reality to help train workers in the field.
“We are planning to gain the knowledge from the people who know what they are doing and then put that into the vineyard or orchard, so the people who are doing the job can be trained as they are doing it. And as a third step it is ultimately necessary to analyse, how such knowledge transfer should be used to automate some of the work.”
All such technology-based support and automation involves people as staff, their supervisors and decision makers in grower’s organisations. Therefore, the project involves sociologists and anthropologists that have a clear mandate to learn and understand people’s interests and engagement in such unavoidable changes of their work environment. But it is also critical to find out, how to improve the way how people can be involved in developing this technological future, especially in improving the data literacy and readiness for such technology-based changes in the industries.
Partnering with Plant and Food, Lincoln Agritech and the Universities of Waikato, Canterbury and Otago will provide the research team with a wide range of resources.
Armin Werner from Lincoln Agritech says technology can help remove what is known as the fingerprint of individual staff.
“At the end of key areas such as pruning in the vineyard or apples, or even hand thinning, we see the highly varying results – the finger print of different people,” he says. “This is where the grower comes and says, we know we could be better, we could utilise better yields for higher profits if our performance in the vineyard or orchard was more uniform. But this is not always possible as it comes back to people. It is not always possible for them to be consistent, like a machine that does it the same way every time.”
With knowledge from experts in their field, it will allow the researchers to provide workers with instant feedback on their decisions.
“At the end this model can help the less experienced to make a better decision on the vine,” Werner says. “We want to use augmented reality, pointing with glasses or some other device, so the pruner gets the information that yes, it is okay to cut that cane, but it may have negative effects the next year. It can then suggest something else. We want people to make decisions, not become robots, so it is long term skill development through teaching.”
Professor MacDonald says the research will be working closely with the New Zealand Winegrowers Research Centre, based in Blenheim.
“For us it is a key link. It’s a way for us to understand what the industry needs are and they are a good intermediary between us and the industry.”
Yield estimation project update
Armin Werner from Lincoln Agritech is also the leader of the research programme into developing an automated yield estimation tool for vineyards.
The aim of the project is to develop sensor based equipment that can be attached to any form of platform, and moved between rows to measure the number of grape bunches and their size or mass. Part of the software in the equipment being developed will then be able to predict how those bunches and berry sizes will relate to a final yield at harvest.
The research programme which is into the third of its five years, has had some “serious development finalised” Werner says.
“At the moment we are improving our methods on how to see grape bunches behind leaves with microwaves, and combine this with machine vision, a camera base.
“We then combine these two sensors with an artificial intelligence approach, so we are learning by understanding what the patterns are between these two sensors. This will give us good accuracy at finding optically occluded grape bunches.”
Werner says they will go into the next season with a tested approach, and “hopefully that will allow us to show how accurately we can count and size grape bunches.”
The research is funded by MBIE as well as NZW and conducted by experts from Lincoln Agritech, Plant and Food Research, Canterbury University, Lincoln University and CSIRO. The development of a prediction model has relied on a great deal of data from growers from Marlborough over the past two years, he says, but more is needed.
“We want data that shows the relation between their counts they did in January and the yield they harvested for that block, bay or vine. It would be great if more vineyards could provide that.”
Adding to the link between the wine industry and Lincoln Agritech, a Memorandum of Understanding (MoU) was signed between the latter and the NZWRC, in November.