A Concept of a Compact and Inexpensive Device for Controlling Weeds with Laser Beams

agronomy

Article A Concept of a Compact and Inexpensive Device for Controlling Weeds with Laser Beams

Ildar Rakhmatulin 1 and Christian Andreasen 2,*

1 Department of Power Plants Networks and Systems, South Ural State University, 454080 Chelyabinsk City, Russia; [email protected] 2 Department of Plant and Environmental Sciences, Faculty of Science, University of Copenhagen, Højbakkegaard Allé 13, DK 2630 Taastrup, Denmark * Correspondence: [email protected]; Tel.: +45-3534-3353

Received: 1 October 2020; Accepted: 19 October 2020; Published: 21 October 2020

Abstract: A prototype of a relatively cheap laser-based weeding device was developed and tested on couch grass (Elytrigia repens (L.) Desv. ex Nevski) mixed with tomatoes. Three types of laser were used (0.3 W, 1 W, and 5 W). A neural network was trained to identify the weed plants, and a laser guidance system estimated the coordinates of the weed. An algorithm was developed to estimate the energy necessary to harm the weed plants. We also developed a decision model for the weed control device. The energy required to damage a plant depended on the diameter of the plant which was related to plant length. The 1 W laser was not sufficient to eliminate all weed plants and required too long exposure time. The 5 W laser was more efficient but also harmed the crop if the laser beam became split into two during the weeding process. There were several challenges with the device, which needs to be improved upon. In particular, the time of exposure needs to be reduced significantly. Still, the research showed that it is possible to develop a concept for laser weeding using relatively cheap equipment, which can work in complicated situations where weeds and crop are mixed.

Keywords: laser-weeding; laser-based weed control; non-chemical weed control; thermal weed control

1. Introduction Weeds are one of the most significant yield-reducing factors for crop production worldwide [1]. Herbicides have been widely used with great success since 1950s, but today, herbicide-resistant weeds are becoming an increasing problem in agriculture [2]. The extensive use of herbicides has resulted in increasing public concerns, and this has led to further restrictions for herbicide use in Europe and elsewhere [3,4], resulting in the banding of many herbicides due to risk of unwanted side-effects [5]. The possibilities of developing new effective herbicides, which can meet the environmental and safety requirements of today, seemed to be exhausted, as no new mode of action of herbicides has been developed since the 1980s [5]. This situation, together with the increasing public interest in organic food [6], calls for integrated weed management approaches to reduce weed problems in the future. Today, soil tillage and crop rotation seem to be the main alternative to herbicides [7,8]. However, soil tillage also has disadvantaged as it increases the risk of soil erosion and leaching of plant nutrients, dries out soils with limited water content, and harms beneficial soil organisms like earthworms. Flame weeding has been used in organic agricultural production but requires a large amount of gas [9] and may not be considered environmentally sustainable in the long term due to the CO2 derived production. Therefore, there is a need for developing new techniques supplementing or replacing present weed control methods [10,11].

Agronomy 2020, 10, 1616; doi:10.3390/agronomy10101616 www.mdpi.com/journal/agronomy Agronomy 2020, 10, 1616 2 of 18

The fast development in laser technology seems to open up new opportunities for weed control based on electricity. Laser beams can deliver high-density energy on selected spots, which warm up the plant tissue and may result in the death of the plant [12]. Heisel et al. [13] used a laser beam to cut weed stems for weed control. Treatments were carried out on greenhouse-grown pot plants at three different growth stages and from two heights. Plant dry matter was measured two to five weeks after the treatment. The relationship between dry weight and laser energy was analyzed using a non-linear dose–response regression model. They found that regrowth of weeds appeared when dicotyledonous plant stems were cut above the meristems, indicating that it is essential to cut close to the soil surface to obtain a significant effect. Marx et al. [14] proposed a weed damage model for laser irradiation, but the recognition system did only recognize one type of weeds under ideal laboratory conditions. Mathiassen et al. [15] investigated the effect of laser treatment directed toward the apical meristems of selected weed species at the cotyledon stage. Experiments were carried out under controlled conditions using pot-grown weeds. Two lasers and two spot sizes were tested, and different energy doses were applied by varying the exposure time. The biological efficacy was examined on three different weed species (Stellaria media (L.) Vill., Tripleurospermum inodorum Sch. Bip., and Brassica napus L.). The experiments showed that laser treatment of the apical meristems caused significant growth reduction, and in some cases, killed the weed species. The biological efficacy of the laser control method was related to wavelength, exposure time, laser spot size, and laser power. The effectiveness also varied between the weed species. Xiong et al. [16] developed a prototype robot equipped with machine vision and a gimbal- mounted laser pointers. The robot consisted of a mobile platform modified from a small commercial quad bike, a camera to detect the crop and weeds on the cotyledons stage, and two steerable gimbals controlling the laser pointers. The robot was able to identify weeds in the indoor environments and carrying lasers to irradiate the weeds. It controlled the platform in real-time realizing continuous weeding. As several technologies have to be combined to develop a successful laser-weeding system, it may end up being costly. This paper aims to present a concept for a compact and inexpensive device for controlling weeds with laser beams. In contrast to previous research with laser weeding, we studied the effect of different laser types on relatively tall E. repens plants.

2. Materials and Methods

2.1. Weed Detection A Raspberry Pi 3 Model B+ computer (Raspberry Pi Foundation, Cambridge, UK) with a Quad-core Broadcom BCM2837B0, Cortex-A53 64-bit SoC @ 1.4 GHz processor and 1GiB LPDDR2 SDRAM memory was applied for weed detection which is a relatively cheap and compact device. Python 3.6, vision library OpenCV 3.4.1 (The University of Birmingham, Birmingham, UK) was used as the programming language. However, the use of deep neural networks is almost impossible for quick recognition of plants due to the limited amount of RAM (14 GB) and the low processor speed (1.5 GHz) of the Raspberry Pi 3 Model B+. Therefore, the Viola-Jones algorithm [17] combined with SqueezeNet [18] was used for weed detection. The Viola–Jones algorithm is an object-recognition framework that allows the detection of image features in real-time. It was originally developed for fast multi-view face detection using Haar feature-based cascade [17]. SqueezeNet is a deep neural network for computer vision. SqueezeNet was created as a small neural network with few parameters that could easily ﬁt into the computer memory and could easily be transmitted over the computer network. We used a SqueezeNet model of 1 1 and 3 3 convolutional × × kernels, with a data storage capacity of 5 MB. However, even in this case, the result of processing an image for the presence of the desired object was approximately one second, which is too slow for a device for weed detection under normal ﬁeld management conditions. Agronomy 2020, 10, 1616 3 of 18 Agronomy 2020, 10, x FOR PEER REVIEW 3 of 18

The Viola–Jones algorithmalgorithm isis aa machinemachine learning-basedlearning-based approachapproach inin whichwhich aa cascadecascade functionfunction is trainedtrained fromfrom manymany positivepositive andand negativenegative images.images. The motive of images was selected visually placing thethe weedweed stalkstalk inin thethe centercenter andand includingincluding otherother weedweed partsparts inin thethe image.image. It isis thenthen usedused toto detectdetect objects inin otherother imagesimages inin realreal time.time. Eight hundredhundred positive examples of weedsweeds andand 12001200 negativenegative examples were used to train Haar cascade and SqueezeNet. The image size was 250 800 pixels (width examples were used to train Haar cascade and SqueezeNet. The image size was× 250 × 800 pixels height). Eight hundred of the negative examples were images of tomato plants, and 400 images were ×(width × height). Eight hundred of the negative examples were images of tomato plants, and 400 imagesimages ofwere other images plants. of Someother plants. positive Some and negativepositive and examples negative are examples shown in Figureare shown1. in Figure 1.

(a)

(b)

Figure 1.1. Images used to train Haar cascades. ( a)) Examples Examples of of images images with with E. repens (positive image). ((bb)) ExamplesExamples ofof imagesimages withwith otherother plantsplants (negative(negative images).images).

ImageImage preprocessing waswas complicatedcomplicated byby thethe factfact thatthat thethe weedsweeds andand crop werewere inin aa similarsimilar huehue range,range, and,and, atat thethe samesame time,time, closeclose toto eacheach other.other. Therefore, we had to use 2000 images to train Haar cascades.cascades. OpenCV library functions were used for feature extraction (Figure2 2).). However,However, theythey diddid not result inin aa clearclear separationseparation of crops and weeds,weeds, and, consequently, wewe decideddecided usingusing thethe imagesimages obtained fromfrom thethe camera without any transformations.transformations. Haar cascades were trained on plants located verticallyvertically inin anan angleangle ofof 9090°◦ fromfrom thethe surface.surface. To assessassess thethe accuracyaccuracy of of the the weed weed recognition, recognition, we we used used a confusiona confusion matrix. matrix. A A confusion confusion matrix matrix is ais tablea table or or chart chart that that shows shows the the prediction prediction accuracy accuracy of of a a classiﬁer classifier forfor twotwo oror moremore classes (Figure 33).). TheThe classiﬁer’sclassifier’s predictions areare onon thethe X-axis,X-axis, andand thethe resultresult (accuracy)(accuracy) isis onon thethe Y-axisY-axis (see (see Results). Results). We testedtested ourour modelmodel forfor thethe weedweed recognitionrecognition onon 224224 randomrandom imagesimages withwith weeds.weeds. TheThe imagesimages werewere taken from the the image image collection collection prepared prepared for for checking checking the the model. model. We We used used Python Python 3.6 3.6and and the thestandard standard library library for forvisualization visualization (seaborn (seaborn and and matplotlib) matplotlib) for for displaying displaying the the confusion matrixmatrix ((https://www.python.org/downloahttps://www.python.org/downloadsds/release/python-360/)./release/python-360/).

Agronomy 2020, 10, 1616 4 of 18 Agronomy 2020,, 10,, xx FORFOR PEERPEER REVIEWREVIEW 4 of 18

Figure 2. 2. ExamplesExamples of feature of feature extraction: extraction: (a)) receivedreceived (a) received imageimage withwith image weedweed with andandweed tomato,tomato, and ((b)) tomato, imageimage thresholding(thresholdingb) image thresholdingbyby cv2.thresholdcv2.threshold by (img,127,155,cv2.THRESH_BINARY),(img,127,155,cv2.THRESH_BINARY), cv2.threshold (img,127,155,cv2.THRESH_BINARY), ((c)) imageimage thresholdingthresholding (c) image byby cv2.threshold(img,127,155,cvthresholding by cv2.threshold(img,127,155,cv2.THRESH_BINARY_INV),2.THRESH_BINARY_INV), (d)) imageimage thresholdingthresholding (d) imagebyby cv2.threshold(img,127,155,cv2.THRESH_TOZERO),thresholding by cv2.threshold(img,127,155,cv2.THRESH_TOZERO), (e)) imageimage (e)thresholdingthresholding image thresholding byby cv2.threshold(img,127,155,cvby cv2.threshold(img,127,155,cv2.THRESH_TOZERO_INV),2.THRESH_TOZERO_INV), (f)) morphologicalmorphological (f) morphological transformationstransformations transformations byby cv2.morphologyEx(img,by cv2.morphologyEx(img, cv2.MORPH_BLACKHAT, cv2.MORPH_BLACKHAT, kernel1), (g)) morphologicalmorphological kernel1), (g transformations)transformations morphological byby cv2.morphologyEx(img,transformations by cv2.morphologyEx(img, cv2.MORPH_OPEN, cv2.MORPH_OPEN,kernel2), (h)) morphological kernel2), transformations (h) morphological by cv2.morphologyEx(img,transformations by cv2.morphologyEx(img, cv2.MORPH_CLOSE, cv2.MORPH_CLOSE,kernel2), ((jj)) morphologicalmorphological kernel2), transformationstransformations (j) morphological byby cv2.morphologyEx(img,transformations by cv2.morphologyEx(img, cv2.MORPH_GRADIENT, cv2.MORPH_GRADIENT, kernel2), (k)) kernel2), floodflood (k)fillfill flood fillbyby cv2.floodFill(im_floodfill,by cv2.floodFill(im_floodfill, mask, mask, (0, 0), (0, 255), 0), ((ll 255),)) segmentationsegmentation (l) segmentation byby cv2.kmeans(pixel_values),cv2.kmeans(pixel_values), by cv2.kmeans(pixel_values), ((ll)) none,none, criteria,(l) none, criteria,10. cv2.KMEANS_RANDOM_CENTERS), 10. cv2.KMEANS_RANDOM_CENTERS), (m)) (distancedistancem) distance transformtransform transform byby cv2.distanceTransform(opcv2.distanceTransform(opening,cv2.DIST_L2,5),ening,cv2.DIST_L2,5), ( n()n color)) colorcolor detection detectiondetection by cv2.inRange(hsv, byby cv2.inRange(hsv,cv2.inRange(hsv, h_min, h_max),h_min,h_min, h_max),(p) edge ( detectionp)) edgeedge detectiondetection cv2.Canny(img,100,200). cv2.Canny(img,100,200).cv2.Canny(img,100,200).

Figure 3. Illustration of information necessary to construct a confusion matrix. Figure 3. IllustrationIllustration ofof informationinformation necessarnecessary to construct a confusion matrix. 2.2. Method to Hit the Target 2.2. Method to Hit the Target We applied OpenCV stereo vision functions to determine the distance to the objects, which made it We applied OpenCV stereo vision functions to determine the distance to the objects, which made possibleWe applied to dispense OpenCV with thestereo use vision of distance functions measurement to determine sensors the distance (infrared to sensors, the objects, sound which locators made or it possible to dispense with the use of distance measurement sensors (infrared sensors, sound locators itlaser possible radars) to dispense [19–22]. We with chose the use a camera of distance that was measurement compatibility sensors with (infrared the Raspberry sensors, Pi–CSI sound connector, locators or laser radars) [19–22]. We chose a camera that was compatibility with the Raspberry Pi–CSI orhad laser a low radars) price and[19–22]. a reasonable We chose image a camera quality. that The was Raspberry compatibility Pi and Sony with IMX219 the Raspberry Exmor with Pi–CSI the connector, had a low price and a reasonable image quality. The Raspberry Pi and Sony IMX219 Exmor connector,following specifications had a low price were and used a reasonable (Figure4 ):image Pi camera quality. with The Sensor Raspberry type: Pi Sony and IMX219 Sony IMX219 Color CMOSExmor with the following specifications were used (Figure 4): Pi camera with Sensor type: Sony IMX219 with8MP; the Sensor following size: 3.674 specifications2.760 mm were (1/ 4used inch (Figure format); 4): number Pi camera of pixels: with Sensor 3280 type:2464 (activeSony IMX219 pixels) Color CMOS 8MP; Sensor× size: 3.674 × 2.760 mm (1/4 inch format); number of× pixels: 3280 × 2464 Color3296 CMOS2512 (total 8MP; pixels); Sensor pixel size: size:3.674 1.12 × 2.7601.12 mmµm; (1/4 lens: inch M12 format); customizable, number telephotoof pixels: fisheye3280 × 2464 lens; (active× pixels) 3296 × 2512 (total pixels); pixel× size: 1.12 × 1.12 μm; lens: M12 customizable, telephoto (activeviewing pixels) angle: 3296 horizontal × 2512 (total FOV pixels); 70 degrees; pixel video: size: 1.12 1280 × 1.12720 μm; combined lens: M12 and customizable, cropped up telephoto to 60 fps; fisheye lens; viewing angle: horizontal FOV 70 degrees; video:× 1280 × 720 combined and cropped up fisheye1080P cropped lens; viewing to 30 fps; angle: 1640 horizontal1232 full FOV FOV 70 binding degrees; mode, video: up 1280 to 30 × fps;720 3280combined2464 and full cropped FOV, 0.1 up to to 60 fps; 1080P cropped to 30× fps; 1640 × 1232 full FOV binding mode, up to 30× fps; 3280 × 2464 full to15 60 fps; fps; board 1080P size: cropped 36 36 to mm;30 fps; IR 1640 sensitivity: × 1232 full built-in FOV 650binding nm IR mode, cut filter, up to not 30 sensitivefps; 3280 to× 2464 IR light; full FOV, 0.1 to 15 fps; board× size: 36 × 36 mm; IR sensitivity: built-in 650 nm IR cut filter, not sensitive to FOV,focal Length:0.1 to 15 3.15fps; mm;board aperture size: 36 (F):× 36 2.35. mm; IR sensitivity: built-in 650 nm IR cut filter, not sensitive to IR light; focal Length: 3.15 mm; aperture (F): 2.35.

Agronomy 2020, 10, 1616 5 of 18 Agronomy 2020, 10, x FOR PEER REVIEW 5 of 18

FigureFigure 4. 4. TheThe positions positions of of the the cameras cameras to to determine determine the the distance distance to to the the object: object: dd isis a a value value called called disparity/offset,disparity/oﬀset, TT isis the the distance distance between between the the cameras, cameras, ZZ isis the the distance distance to to the the object, object, and and ff isis the the focal focal lengthlength camera camera ratio. ratio.

AA depth depth map map was was built built from from a a stereo stereo pair. pair. For For each each point point in in one one image, image, a a search search was was performed performed forfor its its paired paired point point in in another another image, image, and and by by a a pair pair of of corresponding corresponding points. points. We We triangulated triangulated and and determineddetermined the the coordinates coordinates of of their their prototype prototype in in the three-dimensional space. Knowing Knowing the the 3D 3D coordinatescoordinates of of the the pre-image, pre-image, the the depth depth was was calc calculatedulated as as the the distance distance to to the the camera camera plane. plane. TheThe paired paired point point was was looked looked for for on on the the epi-polar epi-polar line. line. Accordingly, Accordingly, to to simplify simplify the the search, search, images images werewere aligned aligned so so that that all all the the epi-polar epi-polar lines lines are are parallel parallel to to the the sides sides of of the the image image (usually (usually horizontal). horizontal). Moreover,Moreover, the the images images are are aligned aligned so sothat that for fora point a point with with coordinates coordinates (x0, (x0,y0) the y0) corresponding the corresponding epi- polarepi-polar line lineis given is given by the by theequation equation x = xx0;= x0;then, then, for foreach each point, point, the the corresponding corresponding paired paired point point must must bebe searched searched for for in in the the same same line line in in the the image image from from the the second second camera. camera. This This process process of of aligning aligning images images isis called called rectification. rectification. Usually, Usually, rectification rectification is is performed performed by by remapping remapping the the image image and and is is combined combined withwith getting getting rid rid of of distortions, distortions, but but this this was was all all do donene by by the the OpenCV OpenCV library. library. We We only only had had to to calculate calculate thethe distance distance between between the the cameras. cameras. ForFor simplicity simplicity and and increased increased accuracy, accuracy, special special stereo stereo cameras cameras can can be be used—for used—for example, example, the the IMX219-83IMX219-83 8MP 8MP 3D 3D Stereo Stereo Camera Camera Module, Module, which which ha hass a a fixed fixed and and highly highly accurate accurate position position between between thethe cameras. cameras. The The distance distance between between the the camera camerass was was calculated calculated by by the the following following formula: formula:

(T(T-d)/(Z-f)=T/Z,d)/(Z f ) = T /Z,(1) (1) − − where T is the distance between the cameras, d is a value called disparity/offset, Z is the distance to thewhere target,T is and the distancef is the focal between length the camera cameras, ratiod is (Figure a value 4). called disparity/oﬀset, Z is the distance to the target,After and thef is machine the focal vision length had camera confirmed ratio (Figurethe presen4). ce of a weed, a laser guidance system (Figure 5) convertedAfter the the machine two-dimensional vision had coordinates conﬁrmed the of presencethe target of into a weed, three-dimensional a laser guidance coordinates. system (Figure 5) AgronomyconvertedThe 2020 laser the, 10 guidance, two-dimensional x FOR PEER system REVIEW cons coordinates isted of ofa galvanometer the target into (speed: three-dimensional 20 kpps) (Unbranded/Generic, coordinates. 6 of 18 style DMX Stage Ligh type: DHR814494, protocol for laser control: ILDA DB25) with two motors and two mirrors guiding the laser beam (Figure 5). This galvanometer is relative cheap (ca. $ 62). The device has two mirrors; one placed horizontally, and one placed vertically. Both mirrors were controlled by Raspberry PI that controlled the motor drivers. Drivers change the angle of the mirror. The following expression describes the control of the positions of the galvanometer mirrors:

𝑑𝑖∗ 𝑛𝑖+1∗ (𝐶𝑖+1− 𝑃𝑖) 𝑃𝑖+1 =𝑃𝑖 + (2) 𝑑𝑖+1 ∗ 𝑛𝑖+1 i is the time step, d is a value called disparity/offset, C is the three-dimensional coordinates of a point on the mirrors, n is the normal vectors of the units of the mirrors, and P is the target point on each mirror. It was necessary to calibrate(a) the mirrors to get them(b )to work correctly by using the potentiometerFigureFigure 5. GalvanometerinGalvanometer the laser board operating operating before principle. principle. applying (a) Photo( aFormula) Photo of a galvanometer;of(2). a galvanometer; (b) the principle(b) the principle of operation. of operation.1 and 2 are 1 motors,and 2 are 3 ismotors, the laser 3 is beam, the laser and beam, 4 are mirrorsand 4 are for mirrors laser reﬂection. for laser reflection.

2.3. TheThe Laser laser System guidance system consisted of a galvanometer (speed: 20 kpps) (Unbranded/Generic, style DMX Stage Ligh type: DHR814494, protocol for laser control: ILDA DB25) with two motors Three lasers were tested: a 0.3 W laser (LBX Series, OXXIUS company, Lannion, France) with a and two mirrors guiding the laser beam (Figure5). This galvanometer is relative cheap (ca. $ 62). wavelength of 405 nm, a 1 W laser (Blue Violet, TTL/PWM Focus Golden, Tgleiser, China) with a wavelength of 635 nm, and a 5 W laser (Jilin, MBL-W-457, Tgleiser, China) with a wavelength of 450 nm.

2.4. Decision Model for the Weed Control System Figure 6 shows the decision model for the developed device. The Viola–Jones algorithm was used in the decision model.

Figure 6. A decision model for the weed control device.

Agronomy 2020, 10, x FOR PEER REVIEW 6 of 18 Agronomy 2020, 10, 1616 6 of 18

The device has two mirrors; one placed horizontally, and one placed vertically. Both mirrors were controlled by Raspberry PI that controlled the motor drivers. Drivers change the angle of the mirror. The following expression describes the control of the positions of the galvanometer mirrors:

di ni+1 (Ci+1 Pi) Pi+1 = Pi + ∗ ∗ − (2) d + n + i 1 ∗ i 1 i is the time step, d is a value called disparity/oﬀset, C is the three-dimensional coordinates of a point on (a) (b) the mirrors, n is the normal vectors of the units of the mirrors, and P is the target point on each mirror. It was necessaryFigure 5. Galvanometer to calibrate the operating mirrors principle. to get them (a) toPhoto work of correctly a galvanometer; by using (b the) the potentiometer principle of in the laseroperation. board before 1 and 2 applying are motors, Formula 3 is the (2).laser beam, and 4 are mirrors for laser reflection.

2.3.2.3. The The Laser Laser System System ThreeThree lasers lasers were were tested: tested: a a 0.3 0.3 W W laser laser (LBX (LBX Series, Series, OXXIUSOXXIUS company,company, Lannion,Lannion, France)France) withwith a a wavelength of 405 nm, a 1 WW laserlaser (Blue(Blue Violet,Violet, TTLTTL/PWM/PWM FocusFocus Golden,Golden, Tgleiser, China)China) with a a wavelengthwavelength of 635 nm, and and a a 5 5 W W laser laser (Jilin, (Jilin, MBL-W-457, MBL-W-457, Tgleiser, Tgleiser, China) China) with with a awavelength wavelength of of450 450nm. nm.

2.4.2.4. Decision Decision Model Model for for the the Weed Weed Control Control System System FigureFigure6 shows 6 shows the decisionthe decision model model for the for developed the developed device. device. The Viola–Jones The Viola–Jones algorithm algorithm was used was inused the decision in the decision model. model.

. Figure 6. A decision model for the weed control device. Figure 6. A decision model for the weed control device.

Agronomy 2020, 10, 1616 7 of 18 Agronomy 2020, 10, x FOR PEER REVIEW 7 of 18 Agronomy 2020, 10, x FOR PEER REVIEW 7 of 18 2.5. Experimental Setup 2.5.2.5. Experimental Experimental Setup Setup Tomato plants were used as crops and couch grass (E. repens) as weeds. The plants were TomatoTomato plantsplants werewere usedused asas cropscrops and couch grass (E. repensrepens)) asas weeds.weeds. TheThe plantsplants werewere established in pots (10 cm Ø) with sphagnum and regularly watered ensuring that the plants had establishedestablished in in potspots (10(10 cmcm Ø)Ø) withwith sphagnumsphagnum and regularly watered ensuringensuring thatthat thethe plantsplants had had suﬃcient water during the experiments. Tomatoes were established from seeds and E. repens from sufficientsufficient water water during during thethe experiments.experiments. Tomatoes were established fromfrom seedsseeds andand E.E. repens repens from from rhizomes. No fertilizer was added to the pots during the experiment. rhizomes.rhizomes. No No fertilizer fertilizer was was addedadded toto thethe potspots during the experiment. The device thermally damaged the stem of E. repens. After the stems of the weeds were hit, TheThe device device thermally thermally damageddamaged thethe stemstem of E. repens. AfterAfter thethe stemsstems ofof thethe weedsweeds were were hit, hit, the the the plants either completely fell to the ground or bent. Figure7 shows the prototype of the weed control plantsplants either either completely completely fellfell toto thethe groundground or bent. Figure 7 shows thethe protprototypeotype ofof the the weed weed control control device. The laser machine was able to move horizontally to mimic natural conditions. The experimental device.device. TheThe laserlaser machinemachine waswas ableable toto move horizontally toto mimicmimic naturalnatural conditions.conditions. TheThe setup with E. repens between two tomato plants is shown in Figure8. experimentalexperimental setup setup with with E.E. repensrepens betweenbetween twotwo tomato plants isis shownshown inin FigureFigure 8. 8.

(a(a)) ((bb))

FigureFigure 7. 7. TheThe The prototype prototypeprototype of of thethe weedweed control control device. ( a)) 1. 1. Pointing Pointing toto to thethe the cameras;cameras; cameras; 2. 2. 2. galvanometer; galvanometer; galvanometer; 3. 3. 3. RaspberryRaspberry computer;computer; 4.4. laser laser rangeﬁnder;rangefinder;rangefinder; 5.5. laser; 6. power supply; 7. 7. motormotor drivers, drivers, 8. 8. electronic electronic signalsignal processing processing board, board, 9.9. Arduino ArduinoArduino board board to control the position ofof thethe laser laser setup. setup. ((b (b)) )A, A, B, B, and and C C showshow the the location location ofof thethe crops.crops.

Figure 8. The experimental setup with E. repens between tomato plants. 1. Elytrigia repens plant; 2. the Figure 8. The experimental setup with E. repens between tomato plants. 1. Elytrigia repens plant; 2. the Figurelaser installation; 8. The experimental 3. the device setup was allowing with E. repens the laserbetween to move tomato horizontally. plants. 1. Elytrigia repens plant; laser2. the installation; laser installation; 3. the 3device. the device was allowing was allowing the laser the to laser move to movehorizontally. horizontally. Figure 9 shows the process of determining the diameter of the stem of the weed. Figure9 9 shows shows thethe processprocess ofof determiningdetermining thethe diameter diameter of of the the stem stem of of the the weed. weed.

Agronomy 2020, 10, 1616 8 of 18

AgronomyAgronomy 2020 2020, ,10 10, ,x x FOR FOR PEER PEER REVIEW REVIEW 88 of of 18 18

(a(a) ) (b(b) )

FigureFigureFigure 9.9. 9. Determination Determination of ofof the thethe diameter diameter of of the the stem stem stem of of of the the the weed. weed. weed. ( a(a) ()aIdentification Identification) Identiﬁcation of of the ofthe theweed; weed; weed; ( b(b) ) (calculationbcalculation) calculation of of the ofthe the length length length of of the of the the weed. weed. weed.

ItItIt waswas was not not not possible possible possible to to to determine determine determine the the the diameter diameter diameter of of theof the the stem stem stem from from from the the the Haar Haar Haar cascade cascade cascade as theas as the the diameter diameter diameter of theofof the weedthe weedweed was determinedwaswas determineddetermined together togethertogether with the withwith leaves thethe because leleavesaves because webecause used we imageswe usedused of images wholeimages plants. ofof wholewhole Therefore, plants.plants. HaarTherefore,Therefore, cascades Haar Haar perceived cascades cascades the perceived perceived straw and the the leaves straw straw asand and one le leaves feature.aves as as one one The feature. feature. length The ofThe the length length weed of of was the the calculatedweed weed was was withcalculatedcalculated an error with with of an 10% an error error as the of of 10% program10% as as the the knew program program the coordinatesknew knew the the coordinates coordinates of the bounding of of the the bounding rectanglebounding forrectangle rectangle a detected for for a a object.detecteddetected Knowing object. object. Knowing theKnowing length the ofthe thelength length weed, of of wethe the couldweed, weed, calculate we we could could its calculate calculate diameter. its its Figure diameter. diameter. 10 shows Figure Figure the 10 10 path shows shows of thethethe laserpath path beamof of the the tolaser laser the beam weedbeam to plant.to the the weed weed plant. plant.

Figure 10. The experimental set up. 1. Focusing the laser beam; 2. mirrors from the galvanometer; 3. FigureFigure 10. 10.The The experimental experimental set set up. up 1.. 1. Focusing Focusing the the laser laser beam; beam; 2. 2. mirrors mirrors from from the the galvanometer; galvanometer; 3. 3. thethethe laser;laser; laser; 4.4. 4. laserlaser laser beam;beam; beam; 5.5. 5. galvanometer;galvanometer; galvanometer; 6.6. 6. schematic schematic image image of of thethe the laserlaser laser beam;beam; beam; 7.7. 7. cameras.cameras. cameras.

2.6.2.6.2.6. ExperimentsExperiments Experiments 2.6.1. The Relationship between Plant Lengths of E. repens and Plant Diameters 2.6.1.2.6.1. The The Relationship Relationship between between Plant Plant Lengths Lengths of of E. E. repens repens and and Plant Plant Diameters Diameters By using a caliper (Neiko 01407A; accuracy 0.02 mm), we manually measured the length and ByBy using using a a caliper caliper (Neiko (Neiko 01407A; 01407A; accuracy accuracy 0.02 0.02 mm), mm), we we manually manually measured measured the the length length and and diameter of 75 E. repens plants for 80 days and described the relationship mathematically. diameterdiameter of of 75 75 E. E. repens repens plants plants for for 80 80 days days and and described described the the relationship relationship mathematically. mathematically. 2.6.2. The Relationship between Required Energy to cut E. repens and Plant Diameters 2.6.2.2.6.2. The The Relationship Relationship between between Required Required Energy Energy to to cut cut E. E. repens repens and and Plant Plant Diameters Diameters We used the 75 plants with diﬀerent diameter to study the necessary power to cut E. repens. WeWe used used the the 75 75 plants plants with with different different diamet diameterer to to study study the the necessary necessary power power to to cut cut E. E. repens repens. .We We We used a spot diameter of 0.55 mm for the laser beam. usedused a a spot spot diameter diameter of of 0.55 0.55 mm mm for for the the laser laser beam. beam.

Agronomy 2020, 10, 1616 9 of 18

2.6.3. The Relationship between Required Energy and the Diameter of the Laser Spot We estimated the mathematical relationship between energy consumption in Joule and the diameter (mm) of the spot of the laser beam for the 5 W laser. One second of exposure corresponded to 5 J.

2.6.4. The Relationship between Spot Diameter of the Laser Beam and the Eﬀect of the Exposure Dose-response experiments were conducted with two diﬀerent lasers (1 W laser, 435 nm; 5 W laser, 450 nm) and with varying sizes of the focus spots of the laser beams on E. repens plants. We chose cheap lasers with the most common wavelength for the laser type. The internal spot diameters were 0.30 mm, 0.55 mm, and 0.80 mm ( 0.05 mm). Twelve dosages ± × 5 E. repens plants were used for each laser type. A dosage corresponded to the time (s) the plant was exposed to the laser beam. The plants were harvested 24 days after exposure, and the fresh weights were measured.

2.6.5. The Eﬀect of the Three Lasers for Cutting Young Shoots of E. repens

1 Nine pots with three E. repens plants pot− (approximately 100 mm tall and 35 days old) were used for each treatment. The three lasers irradiated plants from a distance of 15 20 cm in the three − pots. We placed the pots with weeds randomly between the tomatoes (Figure8). The tomato plants had an age of 90 days and approximately 10 internodes. Each laser was ﬁred approximately 200 times.

2.7. Data Analysis and Statistic Linear regression was done using the GLM package in the open-source program R version 4.0.01 (The R Foundation for Statistical Computing, Vienna, Austria, http://R-project.org) to test whether plant length and diameter were linear related. Variance homogeneity was assessed by visual inspection of residual plots. The signiﬁcance level was set to 0.05. Statistical analysis of dose-response experiments was estimated using the add-on R package drc [22]. The eﬀect of the irradiation on the fresh weight of weed plants was described by a four-parameter log-logistic model where the endpoint y depends on the dose x,

D C y = C + − , (3) 1 + exp b[log(x) log(ED )] { − 50 } 1 where y is the fresh weight of the plant (g pot− ), and x is the amount of energy applied (Joule). The parameter C denotes the lower limit of the curve. The parameter ED50 is the eﬀective dose (energy in Joule) required to reduce the biomass by 50% between D and C. The parameter b is proportional to the slope at dose x equal to the parameter ED50. The relationship between required energy and the diameter of laser spots was estimated using an exponential function using the lm R package.

3. Results

3.1. Detection of E. repens The detection accuracy of E. repens depends on the number of datasets (images used in the Haar cascades) and how well the datasets reﬂect the real conditions. When using images collected under real conditions, we succeed with a detection accuracy of up to 88%. There were several reasons for failures: 1. The plants were not correctly detected 2. The machine vision did not detect E. repens. 3. The weed was detected, but its shapes were incorrectly determined. 4. Crop plant parts were deﬁned as a weed. We tested our model for weed recognition on 224 images with weeds; the visualization of the matrix is shown in the Figure 11. Agronomy 2020, 10, 1616 10 of 18 Agronomy 2020, 10, x FOR PEER REVIEW 10 of 18

FigureFigure 11. 11. AA confusion confusion matrix matrix showing showing the the prediction prediction accuracy accuracy of of a aclassifier classiﬁer for for two two or or more more classes. classes. TheThe classifier’s classiﬁer’s prediction predictionss are are on on the the X-axis X-axis and and the the result result (accu (accuracy)racy) is is on on the the Y-axis. Y-axis. TP TP is is a a true true positivepositive decision; decision; we we predicted we foundfound aa weedweed and and it it was was a a weed. weed. FP FP is is a falsea false positive positive decision; decision; we wepredicted predicted it was it was a weed, a weed, but but it was it was not not a weed. a weed. FN FN is ais false a false negative negative decision; decision; we we predicted predicted it was it was not nottomato tomato and and it was it was not not true. true. TN TN is a is true a true negative negative decision; decision; we predicted we predicted that thisthatis this not is a not tomato a tomato and it andwas it true. was true.

TheThe tracking tracking accuracy accuracy was was calculated calculated to to 80% 80% based based on on the the information information from from the the confusion confusion matrix matrix (Figure(Figure 11) 11) using using the the following following formula: formula:

((𝑇𝑃TP+𝑇𝑁+ TN)) 𝑥x 100100 𝐴Accuracy𝑐𝑐𝑢𝑟𝑎𝑐𝑦 == (𝑃(P ++𝑁N)) where TP is a true positive decision; we predicted we found a weed and it was a weed. TN is a true where TP is a true positive decision; we predicted we found a weed and it was a weed. TN is a true negative decision; we predicted that this is not a tomato and it was true. P is the number of true negative decision; we predicted that this is not a tomato and it was true. P is the number of true results (95 + 84 = 179) and N is the number of false results (17 + 28 = 45). The false alarm factor: FPR = results (95 + 84 = 179) and N is the number of false results (17 + 28 = 45). The false alarm factor: , where FP is a false positive decision (we predicted it was a weed, but it was not a weed) was FP 100 FPR = ×N , where FP is a false positive decision (we predicted it was a weed, but it was not a weed) estimatedwas estimated to 37%. to 37%.The probability The probability of detection of detection = TP x= 100/P,TP 100 where/P, where TP is TPa trueis a positive true positive decision decision (we × predicted(we predicted we found we found a weed a weed and it and was it a was weed) a weed) was estimated was estimated to be to77%. be 77%.

3.2.3.2. The The Relationship Relationship between between the the Plant Plant Le Lengthsngths of of E. E. repens repens and and Plant Plant Diameter Diameter TheThe relationship relationship between between plant plant lengths lengths and and plant plant diameters diameters of of 75 75 E.E. repens repens plantsplants was was described described mathematically:mathematically: y = 0.0030.003xx ++ 1.5251.525 (4) (4) 2 wherewhere yy isis the the plant plant diameter diameter (mm), (mm), and and xx isis the the length length of of the the plant plant in in mm mm (R (R2 == 0.91)0.91) (Figure (Figure 12). 12).

Agronomy 2020, 10, x FOR PEER REVIEW 11 of 18 Agronomy 2020, 10, 1616 11 of 18 Agronomy 2020, 10, x FOR PEER REVIEW 11 of 18

Figure 12. The relationship between plant diameters (mm) of E. repens and plant lengths (mm). Figure 12. The relationship between plant diameters (mm) of E. repens and plant lengths (mm). Figure 12. The relationship between plant diameters (mm) of E. repens and plant lengths (mm). 3.3.3.3. The Relationship between Required EnergyEnergy toto CutCut E.E. repensrepens andand StrawStraw DiametersDiameters 3.3. The Relationship between Required Energy to Cut E. repens and Straw Diameters FigureFigure 1313 showsshows thethe relationshiprelationship between the required energy to cutcut E.E. repensrepens andand thethe plantplant diameterdiameterFigure for 13 the shows 55 WW laser.laser. the relationship Figure 13 and between Table1 1 theshow show required that that the the energy dose-response dose-response to cut E. Model repensModel (1) and(1) ﬁtted fittedthe plant the the datadatadiameter well.well. for the 5 W laser. Figure 13 and Table 1 show that the dose-response Model (1) fitted the data well.

FigureFigure 13.13. TheThe relationshiprelationship between energy consumptionconsumption (Joule) for cutting plants of E. repensrepens withwith variousvariousFigure shoot13.shoot The diametersdiameters relationship (mm)(mm) between (5(5 WW laser).laser). energy consumption (Joule) for cutting plants of E. repens with Tablevarious 1. Estimatedshoot diameters parameters (mm) (5 and W standardlaser). errors (SE) of the dose-response models (model 3) for Table 1. Estimated parameters and standard errors (SE) of the dose-response models (model 3) for experiments with a 5 W laser showing the relationship between energy consumption (Joule) for experiments with a 5 W laser showing the relationship between energy consumption (Joule) for cuttingTable 1. plants EstimatedE. repens parameterswith various and shootstandard diameters errors (SE) (mm) of corresponding the dose-response to the models curve (model in Figure 3) 13for. cuttingexperiments plants withE. repens a 5 Wwith laser various showing shoot the diameters relationship (mm) between corresponding energy toconsumption the curve in (Joule)Figure13. for The parameter ED50 is the eﬀective dose (energy in Joule) required reducing the biomass by 50% Thecutting parameter plants E.ED repens50 is the with effective various dose shoot (energy diameters in Joule) (mm) requiredcorresponding reducing to thethe curve biomass in Figure13. by 50% between D and C. The parameter b is proportional to the slope at dose x equal to the parameter ED50. betweenThe parameter D and C.ED The50 is parameter the effective b is proportionaldose (energy toin the Joule) slope required at dose xreducing equal to thethe parameterbiomass by ED 50%50.

betweenLaser Type D andLaser C. RelativeType The parameter Relative Slope Slope b is Lower proportional Lower Limit Limit (Joule) to (Joule the )slope Upper Upperat doseLimit Limit x (Joule equal (Joule)) toED the50 (mm parameter)ED 50 (mm) ED50. b C D Laser Type b Relative Slope Lower CLimit (Joule) Upper Limit D(Joule) ED50 (mm) 5 W (450 nm) 11.61 (1.17) 1.19 (1.17) 40.20 (0.43) 1.38 (0.01) 5 W (450 nm) 11.61 (1.17)b 1.19 (1.17) C 40.20 D (0.43) 1.38 (0.01) 5 W (450 nm) 11.61 (1.17) 1.19 (1.17) 40.20 (0.43) 1.38 (0.01) 3.4. The Relationship between Required Energy and Laser Spot Diameter

3.4. The Relationship relationship between between Requi energyred Energy consumption and Laser (Joule) Spot Diameter and the spot diameter of the laser beam (5 W laser)The relationship is shown in between Figure 14. energy consumption (Joule) and the spot diameter of the laser beam (5 W laser) is shown in Figure 14.

Agronomy 2020, 10, 1616 12 of 18

3.4. The Relationship between Required Energy and Laser Spot Diameter

AgronomyThe 2020 relationship, 10, x FOR PEER between REVIEW energy consumption (Joule) and the spot diameter of the laser12 beam of 18 Agronomy(5 W laser) 2020 is, 10 shown, x FOR inPEER Figure REVIEW 14. 12 of 18

Figure 14. The relationship between the energy consumption and the diameter of the laser spot (mm). FigureThe larger 14. TheThe the relationshiplaser relationship spot diameter between between has the the to energy energy be to damage consumptio consumption a shootn and and of E.the the repens diameter diameter, the moreof of the the energy laser laser spot spot is needed. (mm). (mm). The larger the laser spot diameter has to be to damage a shoot shoot of of E.E. repens repens,, the the more more energy energy is is needed. needed. 3.5. The Relationship between Spot Diameter of the Laser Beam and the Effect of the Exposure 3.5. The Relationship between Spot Diameter of the Laser Beam and the Eﬀect of the Exposure 3.5. The Relationship between Spot Diameter of the Laser Beam and the Effect of the Exposure Figure 15 shows the dose-response curves with the two laser types (1 W laser, 5 W laser) where Figure 15 shows the dose-response curves with the two laser types (1 W laser, 5 W laser) where threeFigure intervals 15 shows of spot the sizes dose-response have been tested.curves Thewith smaller the two the laser spot types of the (1 Wlaser laser, beam 5 W was, laser) the where better three intervals of spot sizes have been tested. The smaller the spot of the laser beam was, the better threethe weed intervals control of spotand utilizationsizes have ofbeen energy. tested. The The pa rameterssmaller the of spotthe dose-response of the laser beam model was, are the shown better in the weed control and utilization of energy. The parameters of the dose-response model are shown in theTable weed 1. The control models and fittedutilization the data of energy. very well The (small parameters SE-values) of the (Figure dose-response 13, Table model 2). are shown in Table1. The models ﬁtted the data very well (small SE-values) (Figure 13, Table2). Table 1. The models fitted the data very well (small SE-values) (Figure 13, Table 2).

Figure 15. Dose-response curves. (a) E. repens plants were exposed to a 1 W laser (435 nm) beams and (b) to a 5 W laser (450 nm) beams. Three diameters (+/ 0.05 mm) of the internal spot of the laser beams Figure 15. Dose-response curves. (a) E. repens plants −were exposed to a 1 W laser (435 nm) beams and were used: 0.30 mm; 0.55 mm; 0.80 mm. The plants were harvest 40 days after laser exposure. Figure(b) to a 15. 5 W Dose-response laser (450 nm) curves. beams. ( aThree) E. repens diameters plants (+/ were− 0.05 exposed mm) of to the a 1internal W laser spot (435 of nm) the beamslaser beams and # (bwere) to aused: 5 W laser ■ 0.30 (450 mm; nm) ○ 0.55beams. mm; Three □ 0.80 diameters mm. The (+/ plants− 0.05 were mm) harvesof the internalt 40 days spot after of laserthe laser exposure. beams were used: ■ 0.30 mm; ○ 0.55 mm; □ 0.80 mm. The plants were harvest 40 days after laser exposure. Table 2. Estimated parameter and standard errors (SE) of the dose-response models (model 3) for Tableexperiments 2. Estimated with two parameter laser types and and standard three intervals errors (SE) of internal of the dose-response laser spot diameter models corresponding (model 3) for to experimentsthe curves in with Figure two 15. laser The types parameter and three ED 50intervalsis the effective of internal dose laser (energy spot indiameter Joule) required corresponding to reduce to thethe curves biomass in byFigure 50% 15.between The parameter D and C. EDThe50 parameter is the effective b is proportional dose (energy to in the Joule) slope required at dose tox equalreduce to thethe biomass parameter by ED50%50 .between D and C. The parameter b is proportional to the slope at dose x equal to

the parameter ED50. Laser Type Laser Spot Diameter Relative Slope Lower Limit (g pot−1) Upper limit (g pot−1) ED50 (Joule)

Laser Type Laser Spot Diameter Relativeb Slope Lower LimitC (g pot−1) Upper limitD (g pot−1) ED50 (Joule ) 1 W (430 nm) 0.30 mm 5.0b (0.4) 24.3C (1.3) 92.0 D (1.2) 51.0 (0.9) 11 W W (430 (430 nm) nm) 0.30 0.55 mm mm 5.04.4 (0.4) (0.5) 24.334.6 (1.3) (1.6) 92.092.4 (1.2) (1.3) 51.0 53.1 (0.9) (1.3) 11 W W (430 (430 nm) nm) 0.55 0.80 mm mm 4.43.2 (0.5) (0.4) 34.639.4 (1.6) (2.7) 92.493.6 (1.3) (1.2) 53.1 60.6 (1.3) (2.3) 1 W (430 nm) 0.80 mm 3.2 (0.4) 39.4 (2.7) 93.6 (1.2) 60.6 (2.3)

Agronomy 2020, 10, 1616 13 of 18

Table 2. Estimated parameter and standard errors (SE) of the dose-response models (model 3) for experiments with two laser types and three intervals of internal laser spot diameter corresponding to

the curves in Figure 15. The parameter ED50 is the eﬀective dose (energy in Joule) required to reduce the biomass by 50% between D and C. The parameter b is proportional to the slope at dose x equal to

the parameter ED50.

Laser Spot Lower Limit Upper limit ED50 Laser Type Relative Slope 1 1 Diameter (g pot− ) (g pot− ) (Joule) b C D 1 W (430 nm) 0.30 mm 5.0 (0.4) 24.3 (1.3) 92.0 (1.2) 51.0 (0.9) 1 W (430 nm) 0.55 mm 4.4 (0.5) 34.6 (1.6) 92.4 (1.3) 53.1 (1.3) Agronomy 2020, 10, x FOR PEER REVIEW 13 of 18 1 W (430 nm) 0.80 mm 3.2 (0.4) 39.4 (2.7) 93.6 (1.2) 60.6 (2.3) 5 W (450 nm) 0.30 mm 3.9 (0.4) 20.3 (2.0) 98.6 (2.0) 46.4 (1.2) 5 W (450 nm) 0.30 mm 3.9 (0.4) 20.3 (2.0) 98.6 (2.0) 46.4 (1.2) 5 W (450 nm) 0.55 mm 6.2 (0.7) 32.7 (1.2) 94.6 (1.2) 53.0 (0.9) 5 W (450 nm) 0.55 mm 6.2 (0.7) 32.7 (1.2) 94.6 (1.2) 53.0 (0.9) 5 W (450 nm) 0.80 mm 3.3 (0.4) 35.4 (2.7) 92.7 (1.1) 62.9 (2.1) 5 W (450 nm) 0.80 mm 3.3 (0.4) 35.4 (2.7) 92.7 (1.1) 62.9 (2.1)

Table2 2 gives gives an an overview overview of of the the experimental experimental series series with with the the three three lasers. lasers. TheThe mostmost successfulsuccessful was the 5 W laser.

3.6. The Effect Effect of the Three Lasers for Cutting Young ShootsShoots ofof E.E. repensrepens Table3 3 shows shows howhow e efficientfficient thethe systemsystem was.was. The 0.3 laser needed much more time to cut the shoots than the two other lasers, but it also harmed the tomatotomato plantsplants less.less. The 5 W laser was able to cut mostmostE. E. repens repensshoots shoots within within 6 s.6 Thes. The weed weed detection detection system system found found about about 86% of86% the ofE. the repens E. plants.repens plants. Table 3. The effect of the laser system using three different lasers for cutting 35-day-old shoots of E. repens (less than 100 mm tall). Each laser was fired approximately 200 times. Some tomatoes were Table 3. The effect of the laser system using three different lasers for cutting 35-day-old shoots of E. also hit and left with traces on the leaves but did not kill the plants. repens (less than 100 mm tall). Each laser was fired approximately 200 times. Some tomatoes were also hit and leftLaser with tracesLaser on the leaves but didDamaged not kill theLaser plants. Average A Complete Distance Energy No.Laser PowerLaser Spot DamagedTomatoes LaserTime AverageDetection CutA Complete of Straw Cut Distance cm EnergyJ № power W spot Mm Tomatoes % Time s detection% of% Straw cm J 1W 0.3Mm 0.30 15 20% 0.5s 76 23% 85 84% − 1 20.3 1 0.30 0.5515−20 15 200.5 1.5 76 2323 23 85 88 85 84 − 2 31 5 0.55 0.8015−20 15 201.5 7 23 623 35 88 85 97 85 3 5 0.80 15−20 − 7 6 35 85 97

If the laser beam did not hit exactly in the middle of the E. repens shoot, the laser beam was split into two and therefore sometimessometimes alsoalso hithit andand damageddamaged thethe tomatotomato plantplant (Figure(Figure 1616).).

Figure 16. Sometimes under the operation, the laser beam was split into two beams as the violet spots Figure 16. Sometimes under the operation, the laser beam was split into two beams as the violet spots show, resultingresulting inin damagedamage ofof cropcrop plants.plants.

The result of this phenomenon is shown on im imagesages taken by an infrared camera (Figure (Figure 17).17). Tomatoes werewere onlyonly exposed to laser irradiation on onee time on the same place during the test period. It is necessary to carry out additional research to understand how much the short-term irradiation may affect the tomatoes. However, we did not notice any detrimental effect of the short-term laser exposure on the tomato plants.

Figure 18 shows a straw and a leaf of E. repens before and after irradiation with a 1 W laser. A leaf cut was completed in less than a second but did not affect the plant much due to regrowth.

Agronomy 2020, 10, 1616 14 of 18

It is necessary to carry out additional research to understand how much the short-term irradiation may aﬀect the tomatoes. However, we did not notice any detrimental eﬀect of the short-term laser exposure on the tomato plants. Agronomy 2020, 10, x FOR PEER REVIEW 14 of 18

Agronomy 2020, 10, x FOR PEER REVIEW 14 of 18

(a) (b) (c)

Figure 17. Infrared images ofof thethe resultresult ofof aa divideddivided laserlaser beam.beam. TheThe threethree imagesimages werewere taken taken with with 10-s 10- intervals.s intervals. (a ()a This) This image image was was taken taken immediately immediately after after the the treatment. treatment. 1. The1. The spot spot where where the the straw straw of the of (a) (b) (c) E.the repens E. repenswas was hit. hit. 2. The2. The damage damage tomato tomato plant: plant: (b ()b image) image taken taken 10 10 s s after after the the firstfirst image;image; ((cc)) imageimage Figuretaken 2020 17. ss afterInfraredafter thethe images firstfirst image.image. of the result of a divided laser beam. The three images were taken with 10- s intervals. (a) This image was taken immediately after the treatment. 1. The spot where the straw of theFigure E. repens 18 shows was hit. a straw 2. The and damage a leaf tomato of E. repensplant: (beforeb) image and taken after 10irradiation s after the first with image; a 1 W (c laser.) image A leaf cut wastaken completed 20 s after the in lessfirst thanimage. a second but did not affect the plant much due to regrowth.

(a) (b) (c) (d)

Figure 18. Elytrigia repens plant before and after irradiation with a 1 W laser for 0.5 s. (a) The straw before the irradiation; (b ) the straw after the irradiation; (c) the leaf of E. repens before the irradiation and (d) after(a )the irradiation. (b) (c) (d) Figure 18. Elytrigia repens plant before and after irradiation with a 1 W laser for 0.5 s. ( a) The straw 4. Discussionbefore the irradiation; ( b) the straw after the irradiation; ( c) the leaf of E. repens before the irradiation andThere ( d) afterwere the some irradiation. challenges with the image processing. Images of the subject in a real environment was needed to train the Haar cascades. The more alike the samples are to what we want 4. Discussion Discussion to recognize, the better the results become. The tracking accuracy based on 224 test images was calculatedThere weretowere 80% some some and challenges needschallenges to with be with improved. the imagethe image processing. Alternatively, processing. Images the ofImages laser the subjectdevice of the in may asubject real have environment in to a drive real wasenvironmentthrough needed the tofield was train moreneeded the than Haar to trainone cascades. time. the Haar In The the cascades. more future, alike it The is the necessary more samples alike to arethe enlarge tosamples what the we arematrix want to what and to recognize, weconsider want totheseveral recognize, better weeds the resultsthe and better several become. the plants. results The tracking Many become. factors accuracy The affect tr basedacking the onprecision accuracy 224 test of images thebased detection wason calculated224 of test the imagesweeds to 80% (e.g.,was and calculatedneedslighting, to bebackground, to improved. 80% and and Alternatively,needs camera to be angle). improved. the laser Negative device Alternatively, images may have (images the to drivelaser without throughdevice a weed) may the field musthave more beto takendrive than throughoneunder time. the the Insame field the conditions. future, more than it is Mounting necessaryone time. In toa lightthe enlarge future, source the it matrixonis necessarythe field and considerequipment to enlarge several thecould matrix weeds ensure and and a considerconstant several plants.severaland uniform Manyweeds lighting factors and several aandffect therebyplants. the precision Many improve factors of the the detection precisionaffect the of precisionof the the weeds weed of the (e.g.,detection. detection lighting, of background, the weeds (e.g., and lighting,cameraHaar angle). background, cascade Negative is andan imagesedge camera detector. (images angle). without OnNegative that abasis, weed)images a must (imagesdecision be taken withoutis made under a whetherweed) the same must the conditions. becascade taken underMountingrecognized the same a the light objectconditions. source in onthe Mounting theimage field or equipment anot. light To source recognize could on ensurethe the field border a equipment constant between and could uniformweed ensure and lighting cropa constant in and an andtherebyimage, uniform there improve mustlighting the be precision anda color thereby contrast of the improve weed between detection.the theprecision weed andof the the weed crop. detection. HaarWe did cascadecascade not consider is is an an edge edgesituations detector. detector. in On which thatOn basis, thatthe weedbasis, a decision anda decision the is made tomatoes whetheris made were thewhether tangled. cascade the recognizedSuch cascade cases recognizedtherequire object another in the the object imageapproach. in or not.the It isimage To difficult recognize or tonot. train the To border therecognize detecting between the of border weedintertwined and between crop plants in weed an with image, and Haar crop there cascade, in must an image,beas acomponents color there contrast must in be betweenthe a imagecolor the contrast that weed belong between and theto weeds crop. the weed cannot and bethe marked. crop. The Haar cascade can only recordWe the did fact not that consider weeds situations and crops in have which become the weed tangled. and A the solution tomatoes would were be tangled. to use deep Such neural cases requirenetworks another like Mask-RCNN, approach. It iswhere difficult objects to train manu theally detecting can be markedof intertwined for identification. plants with Haar cascade, as componentsThe laser beam in the has image to hit that the belongcalculated to weedscoordinates cannot of be the marked. target. TheThe coordinates Haar cascade are cangiven only to recordthe laser, the and fact the that galvanometer weeds and crops adjusts have the become laser so tangled. it hits the A specificsolution coordinate. would be toWe use have deep achieved neural networksa galvanometer like Mask-RCNN, accuracy of where0.1 degrees, objects which manu allyallows can hittingbe marked a target for identification. with an error of 1 mm at a distanceThe oflaser several beam meters. has to hit the calculated coordinates of the target. The coordinates are given to the laser, and the galvanometer adjusts the laser so it hits the specific coordinate. We have achieved a galvanometer accuracy of 0.1 degrees, which allows hitting a target with an error of 1 mm at a distance of several meters.

Agronomy 2020, 10, 1616 15 of 18

We did not consider situations in which the weed and the tomatoes were tangled. Such cases require another approach. It is difficult to train the detecting of intertwined plants with Haar cascade, as components in the image that belong to weeds cannot be marked. The Haar cascade can only record the fact that weeds and crops have become tangled. A solution would be to use deep neural networks like Mask-RCNN, where objects manually can be marked for identification. The laser beam has to hit the calculated coordinates of the target. The coordinates are given to the laser, and the galvanometer adjusts the laser so it hits the specific coordinate. We have achieved a galvanometer accuracy of 0.1 degrees, which allows hitting a target with an error of 1 mm at a distance of several meters. It was sufficient to use a laser power of 1 W to cut E. repens plants with a diameter of more than 2 mm. However, a diameter of 2 mm requires a duration of 40 s making the system inefficient. For weeds with a diameter of more than 2 mm, a 5 W laser was necessary (Table2). Therefore, we decided focusing on the 5 W laser. If laser weeding is going to be used early in the growing season of the crop, laser weeding should take place when weeds only have developed a few leaves for monocots and 2 4 permanent leaves for − dicots, and in that case, all weed stems would be less than 2 mm. In such cases, the laser should focus on leaves and meristems as the stems may be hidden by the leaves [15]. Small plants would be more sensitive to laser compared to the relatively large model plants we have used in this studied. The laser spot diameter is an essential parameter [15] in relation to energy consumption. The larger spot diameter, the more energy is needed (Figure 13, Table2). In the future, the machine vision of the system must be able to control the size of the spot diameter to operate efficiently. We changed the laser spot diameter by changing the distance between the laser and the galvanometer. In the future, we plan to use a laser with focus control. When the laser spot decreases, the spot temperature increases linearly increasing the efficiency of the system. Focusing the laser requires a significant increase in the accuracy of the tracking system, video fixation, and stabilization of the laser position. The diameter of the plant is related to the length (Figure 12) and age of the E. repens plant reflecting the mass fraction of the fluid content in the plant, which directly affects the amount of energy required to cut the grass (Figure 13). The experiments were conducted when plants were relatively large. Optimal weed control of E. repens is obtained just before or when the plants reach the compensation point at the 3 4 leaf stage [23]. After this stage, the net energy flow between the rhizomes and leaves − changes, resulting in increased biomass of the rhizomes and therefore increasing ability to regrow. Splitting of the laser beam was a problem that affected the crop. Sometimes, a part of the laser beam gets outside the view of the camera. In such cases, the system continues to work, and the calculated energy received by the weed became incorrect. Figure 16 shows an example where the laser beam is located both on the E. repens plant and on the crops. By using a laser with a power less than 1 W, the divided energy was insufficient to damage crop plants within a short time. A 1 W laser would probably we sufficient to control small weed plants. There is a need to analyze the influence of the laser beam on the weed after irradiation. If the color of the hit point on the weed is more or less unchanged, the damage has not been sufficient, and the plant will be able to recover (Figures 17 and 18). The number of images used for training the algorithm needs to be increased for reducing the error of identification of weeds and crop. For example, the laser could be programmed to irradiate only if there is a 90% probability that the plant is a weed to avoid harming the crop with the laser (Figures 16 and 17). However, that will reduce weeding efficiency. Figure 18b presents the result of an incorrect calculation of the necessary energy to cut the weed because of the wrong estimation of the plant diameter by the machine vision system resulting in insufficient damage. Figure 18c shows a damaged leaf. However, the damaged leaf did not prevent the plant from recovering, and therefore we decided to focus on irradiation of the straws. Agronomy 2020, 10, 1616 16 of 18

Figure 18 showed that black spots present on the weed were not a good indicator for significant damage to the plant. The best sign is a physical rift of the stem for visual confirmation. The next step will be to conduct experiments in a field with a crop by installing the equipment on a tractor. It will be necessary to upgrade the installation and place it in a protected IP54 enclosure. The Raspberry Pi 3 Model B+ computer will be replaced with a STM32 microcontroller (STMicroelectronics, Genève, Schweiz). A STM32 has an X-CUBE-AI - AI - expansion pack for STM32CubeMX. This extension can work with various deep learning environments such as Caffe, Keras, TensorFlow, Caffe, ConvNetJs, which makes it possible to train with a stationary computer with a Graphics Processing Unit (GPU). After this integration, it can be used to optimize the library for the 32-bit STM32 microcontroller. A telephoto will be connected to the pi cam lens to increase the weed control area, and a camera will be connected to the servomotor to move along the z-axis. In order to develop a low-cost laser weeding system, we used a Raspberry Pi camera 8 MP (price <20 dollars) equipped with an eight-megapixel (8 MP) Sony IMX219 Exmor sensor, which could capture, record, and stream video in 1080 p. The maximum image resolution reached 3280 2464 × pixels. The system could be improved significantly by using a better camera with a resolution of more than 8 MP with autofocus covering a large infested area. In addition, it is necessary to work out protective measures (e.g., shields) to avoid unwanted impact on persons, material and the surrounding environment. Although several groups have studied the effect of laser beams on weeds e.g., [13–15] only a few have developed an automatic device for controlling weeds with laser beams. Xiong et al. [16] developed a robot equipped with machine vision and laser pointers. The robot identified weeds in indoor environments, and the lasers irradiated the weeds at the cotyledon stage. Our system works in a much more complex system with large grass weeds and crop plants partly covering each other. Therefore, the identification of weeds and crops became much more complicated, and the system requires a huge amount of images to develop a machine vision with high precision. In the future, we will focus on controlling small weed plants to reduce the energy consumption and initiate weed control early in the growing season before the competition between the crop and the weeds starts. We consider that developing a prototype of a robot for controlling emerging weeds with laser beams in row crops like maize, sugar beets, and horticultural crops would be an obtainable goal within the next three years.

5. Conclusions A laser weeding system based on rather cheap equipment was created. Machine vision was used to find tall Elytrigia repens plants among tomato plants in a controlled environment. Image processing was complicated by the fact that the weeds and crop were in a similar hue range, and close to each other. We used untransformed images obtained from the RGB camera, as transformations did not improve the separation of weeds and crop. Haar cascades were trained on E. repens located vertically, and three lasers were tested to cut and control E. repens from the side. We found a linear relationship between plant lengths and plant diameters, which was used to estimate the spot diameter necessary to cut an E. repens shoot. The relationship between energy consumption (Joule) for cutting E. repens shoots and shoot diameters followed a S-shaped function, while the relationship between energy consumption (Joule) and the spot diameter of the laser beam (5 W laser) followed an exponential function. The relationship between spot diameter of the laser beam and the effect of the exposure also followed and S-shaped curve. The smaller the internal spot diameter of the laser beam, the better control of the weed. The 5 W laser turned out to be the most promising to cut large E. repens plants, (shoot diameter >2 mm), but it also increased the risk of damaging the crop. A 1 W or 3 W laser would probably be sufficient to control small weed plants.

Author Contributions: I.R. designed the concept, conducted the experiments and wrote the research protocol. C.A. did the statistical analysis of data and wrote the manuscript. Both authors edited and accepted the ﬁnal paper. All authors have read and agreed to the published version of the manuscript. Agronomy 2020, 10, 1616 17 of 18

Funding: This research was funded by South Ural State University, Russia, the University of Copenhagen, Denmark, and the EU–project WeLASER “Sustainable Weed Management in Agriculture with Laser-Based Autonomous Tools,” Grant agreement ID: 101000256 funded under H2020-EU.3.2.1.1. Acknowledgments: We thank Marat Zhanikeev, Tokyo University of Science, for comments and recommendations concerning setup of the laser system. Conﬂicts of Interest: The authors declare no conﬂict of interest. The funders had no role in the design of the study; in the collection, analyses or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

References

1. FAO. AGP—Weeds. 2017. Available online: http://www.fao.org/agriculture/crops/thematicsitemap/theme/ biodiversity/weeds/en/ (accessed on 7 July 2020). 2. Heap, I. Herbicide resistant weeds. In Integrated Weed Management; Pimental, D., Peshin, E., Eds.; Springer: Dordrecht, The Netherlands, 2014; pp. 281–301. 3. Andreasen, C.; Streibig, J.C. Evaluation of changes in weed flora in arable fields of Nordic countries—Based on Danish long-term surveys. Weed Res. 2011, 51, 214–226. [CrossRef] 4. Chauvel, B.; Guillemin, J.-P.; Gasquez, J.; Gauvrit, C. History of chemical weeding from 1944 to 2011 in France: Changes and evolution of herbicide molecules. Crop. Prot. 2012, 42, 220–223. [CrossRef] 5. Duke, S. Why have no new herbicide modes of action appeared in recent years? Pest. Manag. Sci. 2011, 68, 505–512. [CrossRef][PubMed] 6. Funk, C.; Kennedy, B. The New Food Fights: U.S. Public Divides Over Food Science; Differing Views on Benefits and Risks of Organic Foods, GMOs as Americans Report Higher Priority for Healthy Eating; Pew Research Center: Washington DC, USA, 2016. 7. Liebmann, M.; Dyck, E. Crop rotation and intercropping strategies for weed management. Ecol. Appl. 1993, 3, 92–122. [CrossRef][PubMed] 8. Cloutier, D.; Leblanc, M.L. Mechanical Weed Control in Agriculture. In Physical Control Methods in Plant Protection, 1st ed.; Vincent, C., Panneton, B., Fleurat-Lessard, F., Eds.; Springer: Berlin/Heidelberg, Germany, 2001; pp. 191–204. 9. Sivesind, E.; Leblanc, M.; Cloutier, D.; Siguin, P.; Stewart, K. Weed response to flame weeding at different development stages. Weed Res. 2009, 23, 438–443. [CrossRef] 10. Andreasen, C.; Bitarafan, Z.; Fenselau, J.; Glasner, C. Exploiting waste heat from combine harvesters to damage harvested weed seeds and reduce weed infestation. Agriculture 2018, 8, 42. [CrossRef] 11. Bitarafan, Z.; Andreasen, C. Harvest Weed Seed Control: Seed production and retention of Fallopia convolvulus, Sinapis arvensis, Spergula arvensis and Stellaria media at spring oat maturity. Agronomy 2020, 10, 42. [CrossRef] 12. Heisel, T.; Schou, J.; Andreasen, C.; Christensen, S. Using laser to cut and measure thickness of Beta vulgaris L. and Solanum nigrum L. stems. Weed Res. 2002, 42, 242–248. [CrossRef]

13. Heisel, T.; Schou, J.; Christensen, S.; Andreasen, C. Cutting weeds with CO2 laser. Weed Res. 2001, 41, 19–29. [CrossRef] 14. Marx, C.; Barcikowski, S.; Hustedt, M.; Haferkamp, H.; Rath, T. Design and application of a weed damage model for laser-based weed control. Biosyst. Eng. 2012, 113, 148–157. [CrossRef] 15. Mathiassen, K.; Bak, T.; Christensen, S.; Kudsk, P. The effect of laser treatment as a weed control method. Biosyst. Eng. 2006, 95, 497–505. [CrossRef] 16. Xiong, Y.; Ge, Y.; Liang, Y.; Blackmore, S. Development of a prototype robot and fast path-planning algorithm for static laser weeding. Comput. Electron. Agric. 2017, 142, 494–503. [CrossRef] 17. Viola, P.; Jones, M. Robust real-time face detection. Int. J. Comput. Vis. 2004, 57, 137–154. [CrossRef] 18. Ganesh, A. Deep Learning Reading Group: SqueeszeNet. 2016. Available online: https://www.kdnuggets. com/2016/09/deep-learning-reading-group-squeezenet.html (accessed on 20 October 2020). 19. Wrzesień,M.; Treder, W.; Klamkowski, K.; Rudnicki, W. Prediction of the apple scab using machine learning and simple weather stations. Comput. Electron. Agric. 2019, 161, 252–259. [CrossRef] 20. Dorina, B.; Pierre-Benoît, J. Modelling world agriculture as a learning machine? From mainstream models to Agribiom 1.0. Land Use Policy 2009, 96, 103624. [CrossRef] 21. Suchithra, M.; Maya, L. Improving the prediction accuracy of soil nutrient classification by optimizing extreme learning machine parameters. Inf. Process. Agric. 2019, 7, 72–82. [CrossRef] Agronomy 2020, 10, 1616 18 of 18

22. Ritz, C.; Streibig, J.C. Bioassay analysis using R. J. Stat. Softw. 2005, 12, 5. [CrossRef] 23. Håkansson, S. Weeds and Weed Management on Arable Land. An. Ecological Approach, 1st ed.; Cabi: Wallingford, UK, 2003; p. 247.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional aﬃliations.

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).