Google Weed View? Professor trains computer to spot invasive weed

To manage johnsongrass, a noxious weed that crowds out cotton and sickens horses, farmers have tried herbicides, burning and hand-pulling. Now, researchers at University of California, Davis, have developed a more high-tech weapon against the invasive weed: artificial intelligence and machine learning.

Using photos from Google's Street View database, UC Davis researchers have tracked down over 2,000 cases of johnsongrass in the Western United States for a fraction of the cost and time that it would take to do drive-by or other in-person surveys. They call their tool Google Weed View.

The advancement could help land managers easily and quickly survey for other problem plants.

“Once the model is trained, you can just go and run it on millions of images from Google Street View,” said Mohsen Mesgaran, an assistant professor in the Department of Plant Sciences at UC Davis. “We have huge flexibility, and its capability can be scaled up very quickly.”

The technique can easily be extended to other plant species. All that is needed is to label the new item in Street View photos and train the algorithm to identify that object in the images.

By providing location information, Google Weed View also offers an opportunity to examine how climate affects the growth and spread of weeds and invasive plants at very large scales.

“I think it can be both useful for management and for people with interests in more basic questions in ecology,” Mesgaran said.

Johnsongrass patches identified using Google Street View. The yellow boxes were designated by artificial intelligence; the red boxes were drawn by human hand.

A colleague's query

Mesgaran began looking at using Google's photo database of roadways, streets and highways after Kassim Al-Khatib, a professor of Cooperative Extension in the same department, asked if he could survey Western states for johnsongrass.

Al-Khatib studies where johnsongrass grows, ways to manage it and how this perennial has evolved to be so prevalent and resilient. He's also working with scientists at the University of Georgia to decode the genome of johnsongrass, which is one of the top 10 most invasive weeds worldwide.

Johnsongrass can crowd out native plants, harbor pathogens and affect agriculture. It grows up to 7 feet tall with flowers that are green, violet, dark red or purplish brown depending on maturity, according to a UC Statewide Integrated Pest Management Program briefing page.

“Johnsongrass is a major weed not just in California but worldwide,” Al-Khatib said. “It's very difficult to control. It's a problem on vineyards. It's a problem for cultivated crops. It's a problem on orchards.”

Google Weed View allows for rapid, convenient scanning. It is continuously updated via everyday users with compatible cameras and images collected by Google. “Instead of a day of in-person driving, we can use AI to determine if johnsongrass is in a county or not,” Al-Khatib said. 

Johnsongrass identified growing near agricultural land using Google Street View. The yellow boxes were designated by artificial intelligence; the red boxes were drawn by human hand.

Setting the parameters

To find the weeds, Mesgaran went to Google Street View, which hosts billions of panoramic photos. It didn't take long to find johnsongrass.

“The pictures are really good quality,” he said. “You can see plants and flowers.”

Street View's photos offer a 360-degree view, so in his request Mesgaran set parameters, based on street direction (bearing), to only see the side view. He also specified latitude and longitude, and other factors. To train the deep, or machine learning model, he chose Texas, where johnsongrass is prevalent.

A student sorted through over 20,000 images from that request to find pictures with johnsongrass and drew rectangular shapes around the weeds. They located 1,000 images.

The labeled photos were fed into a computer to train a deep learning algorithm capable of identifying johnsongrass in Google's images. The model was run again to capture potentially more images containing johnsongrass. These additional images were then labeled and used to further refine the model. With each iteration, the algorithm learned and became more accurate.

“This deep learning model was trained by these images,” Mesgaran said. “Once we had a semi-working model, we ran it against about 300,000 images.”

For Al-Khatib's request, researchers focused on 84,000 miles of main roads in California, Nevada, Oregon and Washington states. The team discovered 2,000 locations with johnsongrass.

Google Weed View cost less than $2,000 to purchase the images and teach the model. A traditional car survey to cover the same area would cost an estimated $40,000 in gas, hotel, food and other costs.

“In a matter of months, we came up with 2,000 records and I can do it for the whole U.S.,” Mesgaran said.

Next up? The entire United States.

This story was originally published on the UC Davis College of Agricultural and Environmental Sciences news site.