Skip to main content
Intended for healthcare professionals
Open access
Research article
First published online November 30, 2023

A Prototype Pick and Place Solution for Harvesting White Button Mushrooms Using a Collaborative Robot

Ava Recchia, BASc https://orcid.org/0009-0001-0672-7126, Dora Strelkova, BASc, […], Jill Urbanic, PhD [email protected], Eunsik Kim, PhD, Alvee Anwar, BSc, and Aditya Subramani Murugan, MIE+3View all authors and affiliations
All Articles
https://doi.org/10.1089/rorep.2023.0016

Abstract

Mushroom harvesting is a labor-intensive process. Implementing intelligent automation can improve the work environment for harvesters, reducing work-related musculoskeletal disorders caused by repetitive movements and awkward postures. This research prototypes a robotic harvesting solution for white button mushrooms using a systematic approach. The fungi growth cycle, bruising characteristics, and picking motion dynamics were all considered. To establish a bruising threshold, compression loads were incrementally applied to mushrooms until visible damage occurred. A force measurement glove was used to collect data from harvesters at a mushroom farm to determine the average force exerted on a mushroom during picking. Computer vision-based motion analysis was performed to define picking dynamics. Several compliant grippers were designed, simulated in Autodesk Inventor Nastran, and realized via 3D printing. Prototypes underwent durability testing using iterative cycle counts of 100 as well as moisture absorption testing to study performance in high humidity. A collaborative robot with specialty end of arm tooling was explored to harvest mature mushrooms from a selected area, validating the automation strategy. This system repeatedly located, picked, and placed mushrooms without damage. Recommendations for future work include a decision-making algorithm for mushroom grading preharvest and refinement of gripper tips.

Introduction

Agricultural automation challenges and benefits

Implementing automation in agriculture comes with its own set of challenges and uncertainties due to the complex nature of the environment. As classified by Bac et al.1; the objects (i.e., the food product, leaves, stems) within a crop, the crop itself, and the growth environment are the three main sources of variation. Factors of object variations include size, shape, color, and texture. Factors of crop variations include indeterminate growth and location patterns. The growth environment variation factors include lighting, temperature, and high humidity. These complexities can limit the accuracy of vision systems to detect produce,2 the ability for end-effectors to manipulate produce without damage,3 and the image processing capabilities of AI.4 This explains why manual labor is still commonly used for harvesting.
However, this approach presents the risk of developing work-related musculoskeletal disorders (WMSDs) such as hand-wrist tendonitis and carpal tunnel syndrome due to repetitive motions and awkward postures.5 WMSDs can potentially increase production expenses due to the high insurance or medical costs for treatment of injured workers.6 Automation can be introduced to eliminate these strenuous harvesting activities and improve the efficiency of the work environment. This research explores the harvesting process for white button mushrooms (Agaricus bisporus) and how it can be automated. A prototype robotic pick and place solution for white button mushrooms is presented.

White button mushroom growth cycle and conditions

In a typical white button mushroom farm, growth occurs on substrate beds in Dutch shelving systems (Fig. 1a, c). For a local farm, the mushroom beds are 1.5 m (5 ft) wide, and there is approximately a 35 mm gap between the trays. The working distance is assumed to be 25 cm, as the bedding material is 2–3 cm higher than the tray top. The mushroom heights need to also be considered.
FIG. 1. (a) The mushroom beds, showing the height constraints, (b) the dense mushroom packing, (c) five layers, (d) a schematic sketch of the key dimensions.
Mushrooms are characterized as round, soft, delicate objects (Fig. 1b) with a growth cycle of ∼1 month, depending on the farm. The environment is controlled, and the relative humidity (88%–95%), temperature range, and CO2 levels are closely monitored.7 The first 2 weeks of the growth cycle are typically dedicated to the irrigation and pinning processes. Pinning is the process by which mushroom fruit bodies begin to form, and noticeable outgrowth occurs.8 The beds and floor of the growing rooms are watered intermittently until mushroom pins first appear.
Harvesting occurs during the following 2 weeks of the growth cycle once the pins have developed into dense clusters of mature mushrooms (Fig. 4). Mushrooms are harvested in cycles called flushes, where each flush consists of picking for 3-to-5 days followed by a 1-to-2-day period of no picking.8 Farms may operate using two or three flush cycles.8 On the first day and flush of harvesting, large mushrooms in clusters are normally picked to open the growing space. This is called thinning. Each subsequent day of picking involves selecting mushrooms that meet the demand requirements of the farm.

Literature Review

The harvesting process and related research

Mushroom harvesting activities can be divided into two broad categories, such as picking and postprocessing (i.e., stem slicing, sorting, and transport). These in turn can be manual or automated.

Manual harvesting

Mushrooms intended for the fresh market are typically harvested by hand, to ensure the highest quality. Manual harvesting ensures minimal bruising, protects from discoloration, and enhances the shelf-life.9 However, manual harvesting is labor-intensive which entails high costs, sometimes greater than 40% of the total cost of production.10 Moreover, the situation is exacerbated by absenteeism of workers during seasonal changes.11 In 2019, Canadian and American mushroom farms demonstrated a job vacancy rate of ∼20%.12,13 In addition, manual harvesting is time-consuming which has a negative impact on the efficiency of the process. For instance, mushrooms can be mechanically harvested at greater speeds than those harvested by hand.14

Automated harvesting

The growing characteristics of mushrooms present several challenges for harvesting automation, including maneuvering around clusters, avoiding bruising or damage, and operating within the space constraints of the shelving units. Despite this, automated solutions have been developed for mushroom harvesting as an alternative to the manual approach. This is summarized in Table 1.15–17
Table 1. Mushroom Harvesting Automation Review Matrix, and the Legend
AuthorEnd effectorTransport systemErgonomic analysisEconomic analysis
Reed et al.15
Kuchinskiy16
Yang et al.17
Huang et al.18
Rowley19
This work
SymbolDescription
Vacuum cup
Gripper
Included
Some aspects included
Not included
Many current solutions implement suction cup end-effectors, which limit the cost-effectiveness and feasibility of the tooling in the harvesting environment. For example, in the study conducted by Huang et al.,18 a robot end-effector which used a vacuum cup was designed to pick mushrooms from customized tubs. The vacuum cup was connected to a moveable platform which would allow it to travel along the bed to pick up mushrooms. The contact forces as well as the twisting and bending motion characteristics related to harvesting are required to be known for an effective automation solution. To date, only one study included aspects of an ergonomic analysis by collecting force and picking data based on human movement in a lab setting.18
A separate study also considered an economic analysis of the vacuum cup design,19 summarizing the attributing costs to this automation strategy and it's financial adoptability. Although the vacuum method may be effective for a controlled lab setting, it would be challenging to implement within a commercial growth environment. There are tight spatial constraints (Fig. 1d), and as these systems require a constant supply of dry, compressed air, they can be less efficient than other actuation methods, resulting in greater energy consumption and operating costs.
Therefore, the goal of this research is to clearly understand the problem domain and develop a prototype pick-and-place solution for harvesting white button mushrooms. Applied forces and motion dynamics are studied from an ergonomic point of view to replicate the real-world picking dynamics. The designed gripper employs a novel approach, featuring a compliant mechanism with an interlocking feature that achieves both flexibility and designed rigidity, emulating the observed harvesting activities. The controlled compliance allows for gripping without requiring force sensor feedback, reducing the system complexity. The prototype solution was tested in a lab environment with mushrooms that vary in size, location, and orientation. The experimental activities highlighted the merits of the approach taken and provided a foundation for subsequent automation research activities.

Methodology

The methodologies employed in this research are shown in Figure 2. For mushrooms intended for the fresh market, the cap sizes, weights, and bruising characteristics need to be established. The forces and motions used by harvesters need to be quantified, and the postprocessing requirements defined. This data and information establish the baseline for the end of arm tooling, the physical prototypes, and the simulation and validation activities. A collaborative robot (cobot) is utilized as a foundation for pick-and-place activities. Cobots are a multidisciplinary tool that are quick to program, fast to setup, and safe to use.20 Although they are typically employed in a manufacturing setting, they have great potential in agricultural automation, and are ideal for prototyping and testing new harvesting systems. The OMRON TM5-700 cobot was used throughout this project to validate gripper designs and test mushroom-picking strategies once the force and bruising thresholds were defined.
FIG. 2. Constraint analysis, simulation, and validation methodologies employed for prototyping an automated mushroom harvesting solution.

Constraint analysis

Mushroom classification

Physical characteristics of mature mushrooms can vary greatly. Therefore, a classification scheme that considered both mushroom cap size and physical appearance was created. Fungi size is defined as either small or large, based on the cap width, the cap height, and the mushroom's mass (Table 2 and Fig. 3).
FIG. 3. (a) Examples of grade 1 and (b) grade 2 mushrooms.
Table 2. Average Size and Mass Measurements Based on 10 Small and 10 Large Mushrooms
SizeCap width (mm)Cap height (mm)Mass (g)
MeanStandard deviationMeanStandard deviationMeanStandard deviation
Small27.351.9418.061.486.701.10
Large60.333.6034.952.0540.254.27
Physical appearance is generalized into two grades:
1.
Grade 1: white, clean, and round, and (Fig. 3a).
2.
Grade 2: dirty, bruised, or damaged, “feathering” on the surface (Fig. 3b), misshapen.
Fresh mushrooms sold in grocery stores are typically of Grade 1 with a firm and smooth appearance. Grade 2 mushrooms are habitually used for processed foods.

Bruising threshold via compression testing

Mechanical handling can cause mushroom bruising and reduced quality,21 thus a test was developed to establish a bruising force threshold. Mushrooms were acquired from a commercial farm and tested the same day to avoid discoloration due to aging.22 To measure bruising forces, a force gauge (Yunrus HP-500) with a load range of 0–500 N and accuracy of ±5% was used.
The bruising threshold test procedure is shown in Figure 4. Twenty trials were conducted, using 10 small and 10 large-sized mushrooms to determine the bruising threshold. To define a lower threshold, a 5 N base force was applied to a mushroom. After the force was applied, the mushroom was visually checked for bruises. If no bruises were found, a new mushroom was selected to avoid fatigue loading. On this new mushroom, the 5 N force was increased by 1 N, with a total of 6 N now applied to the mushroom. Each time a new mushroom was tested, the total force applied was incremented by 1 (i.e., 6 + 1 = 7 N applied for next iteration). This process was repeated until bruising was observed, and once observed, the total force was recorded. Based on this data, the average bruising force for the small and large mushrooms were determined. This force data was later used to determine a force–distance relationship that would be leveraged for the cobot end-effector picking motions.
FIG. 4. Flowchart of “Bruising force” measurement process.

Picking force threshold in an industrial setting

In this study, the picking test was conducted with two expert harvesters at a commercial farm (the protocol was approved by the University of Windsor's Research Ethics board [REB# 22-205]). Harvesters wore a force glove and picked mushrooms with a normal picking posture and speed (Fig. 5a). Subsequent motion-related data collection activities utilized specialized pickers. Therefore, the collected data are reflective of the real-world harvesting environment.
FIG. 5. (a) Force measuring glove used for measuring the mushroom picking force (b) key points detected on the harvester's hand in the start frame and (c) end frame.
The force glove contained an Adafruit Metro M423 board and Force Sensitive Resistors (FSR) sensors24 with a range of 0–100 N. Weights from 20 g to 1000 g were placed on the FSR sensors, logging respective mass and resistance values. Curve fitting was performed on the data to derive a function to calibrate sensor force readings. As harvesters primarily used an index finger and thumb when picking, two FSR sensors were attached to these digits on the glove. The board was programmed using CircuitPython25 to sample data at a frequency of 100 Hz and communicate serially via a laptop.

Manual motion dynamics

Based on the experimental observations, the mushroom picking motion was defined as a combination of twisting and bending/pulling. As the force glove could interfere with a harvester's workflow, computer vision was used to monitor picking dynamics. Motion data for five specialized pickers were captured via video for three trials each (15 trials). Video frames from when the harvesters started and finished picking were studied. Using a hand tracking framework known as Mediapipe,26 21 landmarks were extracted from the harvester's hands in these frames (Fig. 5b, c).
To calculate the twist rotation, two vectors BA⃗andBC⃗ were defined from the thumb and index fingertip coordinates [Eq. (1)]. The angle and rotation axis between vectors BA⃗andBC⃗ were also calculated [Eqs. (2) and (3)]. The “SciPy”27 library was used to convert the rotation from axis-angle representation to Euler form using yaw to describe the twisting motion, and pitch and roll for the bending motions:
v1=BA⃗=xb−xa,yb−ya,zb−zav2=BC⃗=xb−xc,yb−yc,zb−zc.
(1)
angleofrotation=arccosv^1⋅v^2.
(2)
axisofrotation=v1×v2 ^,
(3)
where
xb,yb,zb=worldcoordinatesofthumbfrominitialframe.
xa,ya,za=worldcoordinatesofindexfingertipfrominitialframe.
xc,yc,zc=worldcoordinatesofindexfingertipfromfinalframe.
v^1=normalizedv1.
v^2=normalizedv2.

End-of-arm-tool design

An OMROM TM5-700 cobot with Robotiq's “Hand-E” Adaptive Gripper was used for the prototype harvesting system.28 Specialty gripper arms to accommodate varying mushroom cap diameters and interchangeable compliant contact point tips are designed and built (Fig. 6a). This approach can be readily parameterized to accommodate other robotics solutions.
FIG. 6. (a) Model of 3D printed gripper assembly featuring arm (gray) with quick-change keyway for mounting interchangeable tips (teal) and (b) real 3D printed gripper tip and examples of (c) zig zag fill and (d) concentric fill toolpaths.
Compliant mechanisms are strong and flexible devices capable of transmitting precise forces through elastic body deformation and shape memory.29 Leveraging elastomers and the flexibility of Fused Filament Fabrication (FFF) additive manufacturing (AM) build strategies allow designers to explore novel gripper structures. Five gripper geometries with interlocking features for strategic rigidity were explored, where the tips were compliant in compression and rigid for twisting and bending motions. Interior and exterior slots, along with varying wall thicknesses were tested as seen in Table 7. All designs included curved contact surfaces to allow the tip to conform to the mushroom caps.
The gripper tips were designed using Solidworks and Mastercam and printed using a flexible 85A durometer thermoplastic polyurethane (TPU).30 Zigzag and concentric toolpaths were studied to quantify their effect on compliance for the final gripper tip design (Fig. 6b, c). All 3D printed features were fabricated on the Prusa i3 MK3S+ FFF machine.

Simulation and validation

Finite element analysis and toolpath voids in 3D prints

A calibrated finite element analysis model was made in Autodesk Nastran for optimizing subsequent tip designs and estimating their deflection characteristics. A symmetrical model of each gripper tip was used to simplify the computation. Boundary conditions were applied, including a fixed constraint on the bottom surface, slider constraints on the symmetric cut surfaces, and the appropriate top surface compressive loads (Fig. 8a).

Experimental testing

Deflection characteristics for the five gripper tip designs with differing infill strategies (zigzag and concentric) were examined (Fig. 6b and Table 7). Displacements for the 7 and 10 N applied loads were recorded as these loads will not introduce bruising (Bruising Threshold section). Select gripper tip designs were soaked in water for 3 days to observe the influence of moisture on gripping performance as the farm environment is very humid. Durability tests were employed to grip and release a stationary piece of bar stock (38 mm) for 100, 200, and 300 cycles for the baseline and moisture-exposed samples to determine the influence of moisture and fatigue on the gripper tips.

Results

Constraint analysis results

Bruising threshold

The force measurement data in Table 3 indicate that the average force exerted on a mushroom cap without bruising is 18.50 and 11.33 N for small and large mushrooms, respectively. This demonstrates that total force applied during picking can be dependent on the cap size.
Table 3. Force Acting on the Mushrooms Without Causing Damage (10 Small, 10 Large)
SizeForce (N) without damageTravel (mm)
AverageStandard deviationAverageStandard deviation
Small18.500.923.650.67
Large11.330.525.731.13
The results in Table 4 show that for large mushrooms, a force of 12 N would damage more than 60% of the mushrooms and a force of 13 N would damage 100% of the mushrooms. For small mushrooms, a force of 19 N would damage 50% of the mushrooms and a force of 21 N would damage 100% of the mushrooms. Thus, small-sized mushrooms have a greater resistance to force compared to larger ones. It is therefore recommended to apply no more than 17 N for small mushrooms and 11 N for large mushrooms as the maximum picking force. Figure 7 shows examples of damage to the mushrooms as indicated by the red circles. It was observed that employing a travel distance <3 mm could also be utilized (Table 3).
FIG. 7. Examples of damage to the mushrooms as indicated by the red circles.
Table 4. Percentage of Mushrooms Bruised (10 Small, 10 Large)
Force (N)1112131415161718192021
Cumulative mushrooms bruised (%)Large066.7100        
Small000000012.55087.5100

Picking force threshold

Table 5 summarizes the mean and standard deviations of the picking force data from this farm study and that of Huang et al.,18 which used lab personnel. In this study, the expert harvester data were collected using the force measurement glove for 1 minute (Fig. 5b, c). Four useable data points were collected from the first harvester and three from the second. Results from these seven measurements showed a higher thumb peak force and lower index finger peak force from the expert harvesters. In Huang et al.18 the force exerted via the thumb was greater than the force exerted via the index finger. However, as presented in Table 5, similar mean trends are observed. The standard deviations are significantly different. The variation in worker skills may have contributed to potential discrepancies for the standard deviations.
Table 5. Comparison of Force Exerted on Mushrooms Collected via Force Measurement Glove (Data from Seven Mushrooms)
 Thumb peak force (N)Index peak force (N)Total force (N)
μσμσμσ
Farm study3.811.033.320.997.131.43
Previous study by Huang et al.182.401.304.203.306.603.57
When the harvester twisted the mushrooms, the force exerted via the index finger and the thumb was similar, but when pulling the mushrooms out from the bed, the harvester exerted a greater force via the thumb. The harvesters employed much less force than the bruising threshold of the mushrooms. Based on this data, forces no greater than 7–10 N were used for the gripper simulation and deflection testing. The controlled compliance for the gripper tips is to be designed for 7 N force with 2 mm travel distance for the compressive motion.

Manual motion dynamics

The mean and median twisting angles were calculated for the X-axis (pitch), Y-axis (roll) and Z-axis (yaw) directions, as shown in Table 6. The data showed a positive skewness in the distribution, suggesting that the median value of the twisting angle would be a better measure than the mean value. Therefore, the median values were used for the picking motion.
Table 6. Twisting Angles During Mushroom Picking (Data from 15 Trials)
 Euler angles (degrees)
X-axis (pitch)Y-axis (roll)Z-axis (yaw)
Mean4.4415.7010.59
Median2.408.878.54
Standard deviation4.2413.249.13

End-of-arm-tool design results

The gripper geometries in Table 7 were deflection tested, and the results are shown in Figure 8b for different forces and infill build strategies. The grippers were rigid for twisting and bending motions once the V–M faces locked. With the solid infill, the gripper tips with exterior expansion slots demonstrated greater deflection compared to those with interior compression slots. However, the opposite was observed with the 25% infill.
The simulations provided smaller deflection values compared to the experimental results. The gripper tips demonstrated more compression during testing than via the simulation (Fig. 8c). This can be attributed to the presence of internal voids in the 3D printed details despite their solid infill. Therefore, a printing build strategy optimization needs to be considered if the FFF method is to be employed in practice.
FIG. 8. (a) FEA model for gripper tip simulation (b) experimental force-deflection results for TPU gripper tips at solid and 25% infill and (c) comparison of simulation and experimental results. FEA, finite element analysis; TPU, thermoplastic polyurethane.
As shown in Figure 8c and Table 7, gripper E was the most flexible, whereas grippers A and D showed the greatest rigidity. Thus, features from grippers B and C were used to develop the final design, keeping the exterior expansion slots for selective flex, and including a more-defined V–M face for surface contact. One inner compression slot was added for controlled compliance of the gripper such that the V–M face repeatedly locked. Testing this gripper tip built with 100% infill indicated that this solution is flexible in compression until 7 N force was applied, and rigid for twisting. The final gripper assembly weighed roughly 10 g and was used for all data collection tests presented in Figure 9.
Table 7. Gripper Tip Designs, Final Gripper Geometry, and Table Legend
Gripper tip styleAnnotation
A (thick wall)
B (thick wall)
C (thin wall)
D (thick wall)
E (thin wall)
FIG. 9. Experimental results using final gripper geometry (a) postdurability testing force-deflection data (b) soaked versus dry gripper tips force-deflection data (c) toolpath dependent force-deflection data.
No visible wear or damage was observed on the gripper tips after the durability testing. However, the post force-deflection tests demonstrated a slight performance loss. At 100 and 200 cycles, the deflection decreased ∼15%, and at 300 cycles, it decreased approximately by 30%. Figure 9a demonstrates these results. This indicates an ∼15% deflection decrease every 100 cycles for the TPU built grippers. A replacement strategy would need to be considered for this material/build option.
TPU materials are stable, even at high temperatures (up to 100°C, depending on their hardness) when in contact with the test oils ASTM 1, IRM-902, and IRM-903, yet are still susceptible to hydrolysis.31 The 85A TPU material has a 0.22% moisture absorption rate within 24 hours.30 A 3-day period was chosen to observe whether an extended period of constant moisture would affect the material and the performance results. There was no significant difference between the durability results for the gripper tips soaked in water versus the dry tips (Fig. 9b).
A simple main effects analysis showed that toolpath did have a statistically significant effect on gripper tip deflection (p = 1.15E-05). It is concluded that tip performance can thus be sensitive to the fabrication process, emphasizing the importance of toolpath optimization. The two-way analysis of variance (ANOVA) demonstrated that the individual effects of moisture did not have a statistically significant effect on deflection (p = 0.479). There was also no statistically significant interaction between the effects of toolpath and moisture (p = 0.268). It can be concluded that the moist environment of a mushroom farm will not affect the performance of TPU gripper tips. Figure 9c demonstrates these results.

Final prototype

The OMRON TM5-700, a six degree of freedom collaborative robot, was used for the harvesting prototype. The custom gripper was made of polylactic acid and TPU, which are light-weight materials (∼10 g). As the target mushroom mass is light (Table 2), there was no concern for exceeding the maximum payload. The OMRON system utilizes TM Flow Software. Although the cobot had a built-in camera module (2D), this only provided mushroom location. The Pickit L 3D vision system was explored to provide angle and depth information required to pick mushrooms that were growing in various orientations. This was mounted above the Hand-E end-effector, providing a bird's-eye, top-down view of the test fixture.
Mushrooms were randomly placed on the test fixture to replicate select growth orientations for a proof of concept. Nails coated in hot glue were used to prop and secure mushrooms at varying angles and depths as if they were fixed in compost. The cobot was programmed to map motion trajectories based on detected mushrooms using the vision system and to compress the gripper tips 2 mm while picking. This translated to an applied force less than the determined bruising threshold (Table 5). The complete test system is featured in Figure 10.
FIG. 10. (a) System architecture and (b) cobot testing system.

Discussion

The implemented solution was able to pick randomly placed single mushrooms from the test fixture without bruising or damage. The gripper system is compact and can be readily parametrized for similar automation solutions. The proof-of-concept system designed for this research highlighted several practical issues. A different vision system is required due to the significant space constraints (35 cm height, Fig. 1d). Using AM to realize designs proved advantageous with respect to time and budget constraints for the end-effector designs. The costs to build the gripper assembly is summarized in Table 8.
Table 8. Cost Analysis of 3D Printed Details for Realization of the Mushroom Gripper
Printed detail, solid infillMaterial1 kg Spool cost (in CAD, no tax)Material required (in grams)Cost of material used (in CAD)Print time (in hours)Cost of power consumption (in CAD)aTotal cost of material and power (CDN $)
ArmPLA21.997.630.171.430.0190.19
TipTPU 85A85.003.000.260.350.00460.26
a
A 13.0 ¢/kWh rate and 100 W average machine use is assumed for cost of power consumption per print.
CAD, Canadian dollar; CDN, Canadian; PLA, polylactic acid; TPU, thermoplastic polyurethane.
The gripper arms were fabricated out of PLA. Although suitable for prototyping, PLA has poor layer adhesion over time and would be prone to fractures and breaking in a collision.32 Thus, a plastic with higher impact strength or even a metal AM system should be considered to fabricate the arm components.33 Stereolithography or digital light processing should also be explored for fabricating the grippers tips. This could achieve consistent flex in printed details regardless of tip geometry or build orientation due to the isotropic nature of resin printed details. However, the material costs and build time for these new strategies need to be rigorously assessed, including the additional cleaning and sintering/curing post processes.
For a compliant mechanism design, voids can be used strategically to improve part flexibility. Grippers built using a concentric toolpath demonstrated greater force deflection than that built with the zigzag toolpath (Fig. 9c); therefore, this could be a design for manufacturing consideration.
Although the prototype system was successful in a lab setting, the geometry of the OMRON TM5-700 cobot and the vision system creates challenges for direct implementation in a mushroom farm. One benefit of a cobot is the element of safe operation around human workers; therefore, a cobot system is recommended to be used in future research, as the optimal harvesting solution could be a combination of manual labor and automation.
Harvesting mushroom clusters needs to be addressed, and this is future work. Decision-making algorithms need to be developed to determine the picking order and yes/no criteria for mushrooms within mushroom clusters. A modified test bed will be developed to allow for physical testing, and modifications to the gripping tips designs will be implemented if required. Once a foundation for harvesting mushroom clusters is developed, a compact vision system will be selected, and testing with uneven lighting will be performed.
For additional future research in mushroom harvesting, the growing environment needs to be reversed engineered in real time to determine the location, size, shape, and orientation of the mushrooms before harvest. Decision-making algorithms need to be developed to grade mushrooms preharvest and develop optimal motion trajectories. Classifying compression characteristics of different parts of the mushroom (i.e., stem, cap, gills, flesh, etc.) would be useful in developing a more accurate bruising threshold for feedback control with the cobot. Postharvest operations, such as structured packaging strategies (Fig. 10a) should also be explored, further leveraging the vision system and decision-making algorithms.

Conclusion and Future Works

Many agricultural harvesting activities are still performed manually as it is challenging to automate, but cobots and AM allow designs to create unique automation solutions. However, it is necessary to understand the environment and growth conditions of the produce in question. Quantifying the baseline motions and forces is necessary before developing a solution.
This research focused on automating picking and placing of white button mushrooms. A data collection glove, machine vision, and force–distance gauge were used to gather the essential design information. Using a cobot, compliant mechanisms, and FFF AM, novel gripper solutions were designed, built, and tested. This showcased the potential of these strategies for addressing the unique variability and complexities associated with agricultural harvesting.
Gripper tip designs and build processes are to be refined with additional durability and chemical resistivity tests performed on selected materials. The gripper tip materials must not only be compliant but also be resistant to the chemicals and biocides used in the farming process. Therefore, the resistance of the proposed tip materials to common chemicals used in industry needs to be explored to ensure no unexpected behaviors emerge during usage. This will determine the gripper tip lifecycle and allow for optimal operating procedures to be developed.
As the agri-automation domain is rapidly growing, it is important to have an established roadmap to guide the implementation of new technology. Given any harvesting environment, growth and produce characteristics can be defined. A custom end-effector can then be designed using preestablished constants. AM is a cost-effective strategy that enables gripper designs to be built, tested, and validated with prompt turnaround.

Abbreviations Used

AM
additive manufacturing
FFF
fused filament fabrication
FSR
force sensitive resistors
PLA
polylactic acid
TPU
thermoplastic polyurethane
WMSD
work-related musculoskeletal disorder

Acknowledgments

The authors of this article would like to acknowledge the St. Clair college team for their robotics expertise and the availability of their laboratory and automation equipment. An acknowledgment is also extended to our industry partner, Ron from Sinopa who established a connection with a local mushroom farm. This facility was visited to observe the mushroom growth environment, the harvesting process, and to collect data for the basis of this research.

Author Disclosure Statement

No competing financial interests exist.

Funding Information

No funding was received for this article.

References

1. Bac CW, van Henten EJ, Hemming J, et al. Harvesting robots for high-value crops: State-of-the-art review and challenges ahead. J Field Robot, 2014; 31(6):888–911;
Crossref
Google Scholar
2. Gongal A, Amatya S, Karkee M, et al. Sensors and systems for fruit detection and localization: A review. Comput Electron Agric, 2015; 116:8–19;
Crossref
Google Scholar
3. Zhang B, Xie Y, Zhou J, et al. State-of-the-art robotic grippers, grasping and control strategies, as well as their applications in agricultural robots: A review. Comput Electron Agric, 2020; 177:105694;
Crossref
Web of Science
Google Scholar
4. Fountas S, Malounas I, Athanasakos L, et al. AI-assisted vision for agricultural robots. AgriEngineering, 2022; 4(3):674–694;
Crossref
Google Scholar
5. Meyers JM, Miles JA, Faucett J, et al. Ergonomics in agriculture: Workplace priority setting in the nursery industry. Am Ind Hyg Assoc J, 1997; 58(2):121–126;
Crossref
Google Scholar
6. Kirkhorn SR, Earle-Richardson G, Banks RJ. Ergonomic risks and musculoskeletal disorders in production agriculture: Recommendations for effective research to practice. J Agromedicine, 2010; 15(3):281–299;
Crossref
Google Scholar
7. Meigs Beyer D. Basic Procedures for Agaricus Mushroom Growing. 2017. Available from: https://extension.psu.edu/basic-procedures-for-agaricus-mushroom-growing [Last accessed: January21, 2023].
Google Scholar
8. Meigs Beyer D. Six Steps to Mushroom Farming. 2022. Available from: https://extension.psu.edu/six-steps-to-mushroom-farming [Last accessed: January21, 2023].
Google Scholar
9. Van Griensven LJLD. The cultivation of mushrooms: Its present status and future developments. Outlook Agric, 1987; 16(3):131–135;
Crossref
Google Scholar
10. Zied DC, Pardo-Giménez A, (eds). Edible and Medicinal Mushrooms: Technology and Applications. John Wiley & Sons: Chichester, United Kingdom; Hoboken, NJ; 2017.
Google Scholar
11. Navas E, Fernández R, Sepúlveda D, et al. Soft grippers for automatic crop harvesting: A review. Sensors, 2021; 21,8):2689;
Crossref
Google Scholar
12. Mushrooms Canada. Mushrooms Canada's Election 2021. 2021. Available from: https://www.mushrooms.ca/wp-content/uploads/2021/09/2021-09-01-RELEASE-LabourCrisisRecommendations-MushroomsCanada.pdf [Last accessed: December10, 2022].
Google Scholar
13. Pennsylvania State University. An Integrated Approach to Address the Labor Shortage on Mushroom Farms through Smart Agriculture | Penn State College of IST. 2021. Available from: https://ist.psu.edu/research/projects/mushroom-labor [Last accessed: March21, 2023].
Google Scholar
14. Azoyan A. Feasibility Analysis of an Automated Mushroom Harvesting System. Masters Thesis. University of Georgia: Athens, Georgia; 2004.
Google Scholar
15. Reed JN, Miles SJ, Butler J, et al. AE—Automation and emerging technologies. J Agric Eng Res, 2001; 78(1):15–23;
Crossref
Google Scholar
16. Kuchinskiy NA. Development of an Autonomous Robotic Mushroom Harvester. Masters Thesis. Western University: Ontario, Canada; 2016.
Google Scholar
17. Yang S, Ji J, Cai H, et al. Modeling and force analysis of a harvesting robot for button mushrooms. IEEE Access, 2022; 10:78519–78526;
Crossref
Google Scholar
18. Huang M, He L, Choi D, et al. Picking dynamic analysis for robotic harvesting of Agaricus bisporus mushrooms. Comput Electron Agric, 2021; 185:106145;
Crossref
Google Scholar
19. Rowley JH. Developing Flexible Automation for Mushroom Harvesting (Agaricus bisporus). PhD Thesis. The University of Warwick: England, United Kingdom; 2009.
Google Scholar
20. Djuric AM, Urbanic RJ, Rickli JL. A framework for Collaborative Robot (CoBot) integration in advanced manufacturing systems. SAE Int J Mater Manuf, 2016; 9(2):457–464;
Crossref
Google Scholar
21. Weijn A, Tomassen MMM, Bastiaan-Net S, et al. A new method to apply and quantify bruising sensitivity of button mushrooms. LWT Food Sci Technol, 2012; 47(2):308–314;
Crossref
Google Scholar
22. Eastwood D, Burton K. Mushrooms—A matter of choice and spoiling oneself. Microbiol Today, 2002; 29:18–23.
Google Scholar
23. Adafruit. Adafruit Metro M4 Feat. Microchip ATSAMD51. 2019. Available from: https://www.adafruit.com/product/3382 [Last accessed: October24, 2022].
Google Scholar
24. Adafruit. Force Sensitive Resistor (FSR). 2012. Available from: https://learn.adafruit.com/force-sensitive-resistor-fsr/overview [Last accessed: October24, 2022].
Google Scholar
25. CircuitPython. The Easiest Way to Program Microcontrollers. n.d. Available from: https://circuitpython.org/ [Last accessed: October24, 2022].
Google Scholar
26. Lugaresi C, Tang J, Nash H, et al. Mediapipe: A Framework for Perceiving and Processing Reality. In: Third Workshop on Computer Vision for AR/VR at IEEE Computer Vision and Pattern Recognition (CVPR): Long Beach, CA; 2019.
Google Scholar
27. SciPy. SciPy API Development Guide—SciPy v1.9.2 Manual. n.d. Available from: https://docs.scipy.org/doc/scipy-1.9.2/dev/api-dev/api-dev-toc.html [Last accessed: October24, 2022].
Google Scholar
28. Robotiq. Robotiq Hand-E for OMRON-TM Series Robots Instruction Manual. Robotiq Inc.: Lévis, Québec, Canada; 2021.
Google Scholar
29. Strelkova D, Urbanic RJ. Design and Fabrication of Novel Compliant Mechanisms and Origami Structures for Specialty Grippers. In: Towards Sustainable Customization: Bridging Smart Products and Manufacturing Systems. (Andersen A-L, Andersen R, Brunoe TD, et al. eds.) Lecture Notes in Mechanical Engineering. Springer International Publishing: Cham; 2022; pp. 55–62.
Crossref
Google Scholar
30. NinjaTek. NinjaFlex-TDS.Pdf. 2016. Available from: https://ninjatek.com/wp-content/uploads/NinjaFlex-TDS.pdf [Last accessed: October24, 2022].
Google Scholar
31. Covestro. Chemical Properties: Technology by Covestro. n.d. Available from: https://solutions.covestro.com/en/highlights/articles/theme/product-technology/chemical-properties-tpu [Last accessed: October25, 2022].
Google Scholar
32. UltiMaker Cura. Ultimaker PLA Material: Highly Versatile, Easy to Print. n.d. Available from: https://ultimaker.com/materials/pla [Last accessed: October24, 2022].
Google Scholar
33. Markforged. Markforged Metal 3D Printer: The Metal X 3D Printing System. n.d. Available from: https://markforged.com/3d-printers/metal-x [Last accessed: October24, 2022].
Google Scholar
Cite this article as: Recchia A, Strelkova D, Urbanic J, Kim E, Anwar A, Murugan AS (2023) A prototype pick and place solution for harvesting white button mushrooms using a collaborative robot. Robotics Reports 1:1, 67–81, DOI: 10.1089/rorep.2023.0016.

Also from Sage