During this coronavirus lock down, IGIS has set out to revamp its data infrastructure to address our growing needs for big data storage and management moving forward. In particular, over the past few years we have accumulated over a dozen terabytes of drone data and associated mapping products, constituting tens of thousands of project files, and the quantity of this data is only expected to keep growing. Typically this drone data has been processed on a number of local desktop computers, and then backed up onto RAID hard drives or the cloud for cold storage; however, this is far from ideal in terms of consistent organization, versioning and ease of distribution.
As a solution to the problem, IGIS purchased a web server, equipped with multiple virtual machines (for processing, analysis and web services) along with a 30TB RAID data store/repository. The repository was networked to our various IGIS computers and RAID storage devices, so that all of our drone data could be transferred over to it. After much consideration, we settled on a standardized file structure, which could accommodate both datasets from past and future drone projects, with room for growth as needed. A python script was written to automatically generate this file structure, with some metadata inputs for each project, and our previous projects' data were then moved into their appropriate slots in the new structure, while jettisoning unwanted intermediary processing files; freeing up a ton of storage space. It would be correct to assume that this process of moving data was quite time consuming. However, moving forward, it will be easy to automatically set up our projects' file structures right from the inception of every new project, beginning with running the python script in ArcGIS Pro's Jupyter Notebook utility in the field, to eventually be delivered to the server repository down the pipeline, in a nicely organized package (similar to what we would provide to our non-IGIS project collaborators).
That alone is a big step in the right direction, but it gets better. Because all of this data is now in a standardized file structure, with standardized folder naming conventions, scripting our ArcGIS portal to automatically connect with the data via the imager server was only a small step away. With this complete, now any IGIS team member can access our entire post-processed, GIS-ready, drone data inventory of layers via ArcGIS Online or ArcGIS Pro.
Ultimately this has been a big leap forward, in terms of IGIS's informatics infrastructure; to compliment our significantly evolved pipeline for drone data collection and processing, depicted below.
- Author: Andy Lyons
Under the Fly4Fall campaign, amateur drone hobbyists across the globe are invited to take aerial 360 photos with their drone and contribute them to a collection of fall landscapes that will grow over time.
Never taken an aerial 360 photo before? Me either, but fortunately it recently got a whole lot easier with a free iOS app called Hangar 360. The Hangar app flies your DJI drone for you, climbing to the height you program and then taking about 25 photos in a circle at three different angles to the horizon. The whole thing takes about 2 minutes, and you can collect multiple panos per flight. You then land the drone (but don't turn it off just yet!), transfer the photos from the drone to your phone over the WiFi, and then upload the photos to Hangar. Hangar stitches the photos for you in the cloud (also free!), and sends you a link. The results are stunning! See the panoramic photo below of Kearny REC made by IGIS's Robert Johnson earlier this week.
Inspired by citizen science initiatives like the Christmas Bird Count and Project BudBurst, where large numbers of naturalists record observations in a coordinated way, Fly4Fall is part non-professional science project, part art, part community building, and a whole lot of fun. Crutsinger discussed some of the potential science angles in a recent LinkedIn post.
Full instructions can be found at Fly4Fall.com. Currently, the Hangar app only works on iOS, unfortunately, and only with DJI drones (but the list includes most of the popular ones). Android enthusiasts can check out Litchi, which includes similar functionality but costs $25 and you have to process the images on your own (look for tutorials online).
Of course like any drone flight you have the follow the rules - only fly in permitted areas, don't fly directly over people, and be safe!
We look forward to seeing the Fly4Fall panoramas coming in. Feel free to use the comment box below to share your experiences and thoughts!
- Author: Andy Lyons
Drone Mapping California is a moderated email list intended to share news, information, and questions about using drones for mapping and data collection. That covers a lot - technology, training, regulations, hardware, software, analytical techniques, etc. We hope this list will be a channel through which new and seasoned drone operators and researchers can share and grow their knowledge and expertise.
The list has a California focus, but all are welcome. If you are interested in collecting data with drones, please subscribe here! IGIS will administer the list for the foreseeable future, including moderating messages to prevent spam, but we are always open to comments and suggestions.
Top: Matrice 100 with dual RGB and multispectral sensors
Bottom: mNDGI image of a field at Desert Research and Extension Center
Photos by Sean Hogan
A FAA remote pilot license is required to fly drones legally for any non-recreational purpose (which includes basically everything we use drones for in ANR and UC). The 'hard' part of obtaining your drone pilot certificate is passing a 60 question FAA Airman General Knowledge exam, which covers a broad range of topics related to the safe and legal operation of drones in the national airspace. Our efforts to get certified were propelled forward by an excellent FAA exam prep-class offered in early March by UC Merced Extension, and taught by Andreas Anderson, a long-term pilot and graduate of the UC Merced MESA lab.
Our programmatic goal in getting more certified drone pilots is to help serve the growing demand for drone services in the Division, including both flying missions and training. Flying safely and legally however is only the start. Using drones effectively as data collection platforms for research and extension takes a host of other skills and knowledge, including mission planning, flight operations, using the equipment, data management, and select principles of photogrammetry and remote sensing. This is why we encourage everyone in ANR interested in using UAVs for their research or extension programs to attend one of our Drone workshops, such as the upcoming workshops at Kearney REC (April 13-14), UC Berkeley (Apr 2 ), Quincy (June 7-8), or our three-day Dronecamp at the end of July (application deadline April 15, 2017). Need some inspiration how drones might be useful in your work? Check out the current issue of Cal Ag which features a number of applications of drone science for agriculture and natural resources.