Today was a great day at the ESRI User conference. I attended several sessions covering the use of raster and vector data in big data analyses, the use of python and r for data science, and the use of arcpy to create and modify maps in ArcGIS Pro.

Speaking of big data and arcgis, over the past 5 years I have watched ESRI develop their software from a desktop environment to a distributed computer platform capable of analyzing millions of spatial features in minutes and hours instead of days or weeks. These tools are now accessible to the average users of arcgis without the need to understand the underlying big data frameworks and software. We now have the ability to install tools such as GeoAnalytics Server, the Spatial Temporal Datastore, and ArcGIS Pro and take advantage of big data analytics within ArcGIS Pro. By using these tools we do not have to know how to setup and manage tools like apache spark, elasticsearch, or other tools.

The above tools are used to store and analyze vector data. To analyze multidimensional rasters, we have the ability to use mosaic datasets to store and analyze multiple raster datasets and multiple raster formats. Over the past years I underestimated the capability and the utility of mosaic datasets. These datasets have the ability to calculate custom raster functions on the fly. These datasets can also be shared via ArcGIS Server and ArcGIS Online. I look forward to utilizing the datasets in the future.

One thing to note at the ESRI User Conference is that like last year all of the presentations that I have seen by ESRI staff this year have been on ArcGIS Pro. The ArcGIS Pro software is getting more powerful and ESRI is building more functionality into this new software platform.

I cannot wait to see what day 4 brings tomorrow.

Of all the technical session and talks I went to today the topic that was the most exciting was the “ArcGIS Maps for Adobe Creative Cloud”.  The plugin for Adobe Creative Cloud is a bridge between ArcGIS and Adobe Illustrator and Photoshop.  This plugin allows cartographers and graphics designers to import vector and raster data directly from ArcGIS Online and from shapefiles into Illustrator and Photoshop.  Once these data are in Illustrator and Photoshop the data can be manipulated and styled using the native tools in Illustrator and Photoshop.

This tool fills a need that cartographers have wanted filled for many years.  I foresee these tools as being very important to allow graphics designers within our organization to extend the spatial data that we have created to publications and other materials that UCANR produce.

To download and start using these tools you will need access to Adobe Creative Cloud https://exchange.adobe.com/addons/products/16913 and an ArcGIS for Organizations account http://www.arcgis.com.  If you are a member of the UCANR network and do not have access to ArcGIS Online fill out the following form and we will help you gain access to the ArcGIS Online http://igis.ucanr.edu/resources/esri_software/. 

Notes and stray thoughts:

• ArcGIS Pro 2.0 and ArcGIS 10.5.1 Released in advance of the User Conference this year.  To see what is new in these new releases go to http://pro.arcgis.com/en/pro-app/get-started/whats-new-in-arcgis-pro.htm for ArcGIS Pro 2.0 and http://desktop.arcgis.com/en/arcmap/latest/get-started/introduction/whats-new-in-arcgis.htm#ESRI_SECTION1_76DF146740D047C78D38DC5FFF917CCD for ArcGIS 10.5.1.
• ArcGIS Pro and ArcGIS will be have new releases every 6 months in to the future.
• As always, the ESRI User Conference is a well done production with professional presentation and branding.
• Being at the ESRI Conference is always a nice opportunity to see old friends in the GIS community.

More to Come Tomorrow

Last week we enjoyed attending the CalGIS 2017 conference in Oakland. This year the meeting was co-hosted with LocationCon, so it was larger than usual and had a good mix of participants from government, non-profits, academic and consulting companies. As expected there were a lot of people from California, but we also met a lot of people from other parts of the US. 

The first day of the conference was devoted to workshops, and IGIS gave a half-day version of our workshop on drone technology and data analysis. This was well attended, and one of several sessions focused on drone technology. For us, this was also more preparation for our upcoming Dronecamp at the end of July.

The following day, Andy gave a presentation on some of the issues for scaling up drone capacity within a specific institutional setting like ANR. We discussed some of the issues we've been dealing with, including matching the scale of the data to the scale of science and management questions, outreach and training, regulatory compliance, and tailoring off-the-shelf technology for specific applications and contexts. The Q&A period highlighted a number of common challenges facing many organizations striving to take advantage of drone technology. One of the most common needs is developing institutional level policies to ensure safety and compliance with a dynamic array of federal, state, and local regulations. This discussion reminded us how fortunate ANR is to be backed by the UC Center of Excellence on Unmanned Aircraft System Safety, because many local agencies and public utilities are still trying to figure it out.

Another common theme that came up was management of the massive amount of data that drones can collect, and how to share and find drone data. Managing drone data is challenging because of the sheer volume of data. This makes many traditional strategies platforms unworkable and even cloud based solutions difficult to use because of long transfer times. Like many programs, we started managing drone data by adapting existing tools and established practices from other fields like GIS and remote sensing, which we have been refining as we learn more and as our drone service program grows to include more people and projects. We recently started documenting our data management system in a recent Tech Note (more about that in an upcoming blog), and are currently exploring a new online platform for dissemination in collaboration with ESRI (stay tuned for more info about that also). What became clear at the conference however is that the tools and platforms for drone data management are still catching up, and we have a long way to go before we can reach the capabilities of portals for more traditional GIS data, such as the State of CA Geoportal or even the National Map.

Other highlights from the conference were the many excellent talks, including presentations on using drones to create a very precise digital elevation model of a wetland restoration site, techniques for machine learning classification of aerial imagery, and the US Forest Service's system-wide database called EDW. We also heard about some of the exciting new features of Cal-Adapt, including an API that will dramatically simplify the process of creating decision support tools and other applications that require downscaled climate forecasts. Many of the presentations are available through the conference website, all of which are well worth checking out.

 

As part of our mission to test drone technology and provide mapping services to the ANR community, IGIS recently completed data collection for our largest drone mapping project to date - 3600 acres of the Mojave Desert.

The drone data will be used to create fine resolution vegetation and elevation maps, as part of a desert tortoise research project led by Dr. Brian Todd in the Dept. of Wildlife, Fish, and Conservation Biology at UC Davis. The study seeks to better understand how the threatened desert tortoise navigates in its landscape, which will help biologists and land managers responsible for protecting the desert tortoise make management decisions, such as where to place road fences that don't cut off important resources for this long-lived species. Researchers on this project track the tortoises with GPS and temperature sensors, which they will now be able to overlay on the highly detailed maps of the desert generated from the drone imagery.

How does the desert look to a tortoise?

A Large Study Area

The study area is approximately 3600 acres, which is extremely large for a drone. We mapped the area in two trips in 2016 and 2017, with significant improvements to our mapping methods in between.

In mid-2016, IGIS Academic Coordinator Sean Hogan mapped the first 1200 acres using a 3DR Solo drone flying a consumer grade GoPro camera. GoPro cameras work surprisingly well for making 3D topographic models, thanks to their high resolution, wide field of view, and rapid shutter rate. Hogan programmed the drone to fly a lawnmower pattern with a lot of overlap between images, so every point on the ground is captured in multiple images taken from different angles. The stitching software uses photogrammetric algorithms to recreate the surface geometry, producing a 3D model of the topography.

The Solo+GoPro combo worked well, but the flight time of the Solo, its mediocre GPS receiver, and the lack of an integrated GPS module in the GoPro were limiting factors that required more time for both flying and processing the data.

New Drone Platform for 2017

In May 2017, Hogan returned to the Mojave with IGIS Academic Coordinator Andy Lyons, as well as new equipment and almost a year's worth of additional flight experience. For this campaign we flew our new DJI Matrice 100; a larger drone that can carry two batteries for longer flight times. Hogan also modified the bottom of the Matrice, outfitting it with an additional camera mount, so we were able to fly with a high quality RGB camera as well as the Parrot Sequoia multispectral camera. The combination worked brilliantly, allowing us to capture both high quality RGB images for the 3D surface model, as well as multispectral images for vegetation analysis during each flight. After a day of working out kinks with the cameras and software, we were able to reliably fly eight 100-acre flights per day with minimal downtime between each consecutive mission. Flying at 400 feet, the resolution of the resulting maps will be about 2.25 inches for the RGB orthomosaic and digital elevation model (DEM), and about 4.5 inches for the multispectral products.

Lessons Learned

Some lessons learned for mapping large areas include:

• Mapping large areas with a quadcopter is viable with careful planning
• The Matrice 100 is large enough has enough power to fly 100 acre flights with two cameras on board with two batteries and calm winds
• Heat can dramatically reduce flight time because the air is thinner and equipment less efficient
• Checking your data in the field and having backup plans for weather and equipment glitches are essential
• Having two or more people on the flight crew helps tremendously
• Developing individual mission plans before going to the field reduces time between flights and hence illumination differences between adjacent areas
• The dessert tortoise is a charismatic creature with amazing adaptations, and a great curiosity about the research equipment we were setting up alongside the road: