- Author: Andy Lyons
The Magic Behind Cal-Adapt: The API
If you're reading this you've probably heard of Cal-Adapt, California's official source for downscaled climate data from the 4th Climate Change Assessment. Most people have engaged with climate data on Cal-Adapt through the website, which provides over a dozen well-designed interactive tools that allow you to create charts and maps of wide range of climate variables, from temperature to snow pack to wildfire risk, for your location of interest. For the intrepid user, the website also has a download tool where you can download the data as CSV or raster files, or the source files in NetCDF format.
Cal-Adapt provides access to a wealth of downscaled climate data for
California, Nevada, and parts of neighboring states
Sample chart from Cal-Adapt.org
A lesser known fact is that all this magic is made possible under the hood by a well-designed open API. An API, or Application Programming Interface, is essentially a common language that allows computer programs to talk to each other. Using the API, the Cal-Adapt website can retrieve data from the Cal-Adapt server, on demand, to make all those beautiful charts and maps.
But guess what? This same API, which is fully open and documented on the Cal-Adapt website, can be used by things other than websites. Any programming language that supports standard web protocols can send specially formatted commands through the API to get climate data directly from Cal-Adapt, without ever having to go through the website. This includes R and Python, the two most popular programming languages for data science, with users in the hundreds of thousands.
Introducing caladaptR
Why is this a big deal, you might ask? We can start with the huge number of R users, particularly in research and academia. The science of climate change impacts and adaptation planning are still young and rapidly evolving, and R is by far the favorite computing environment for researchers. Before caladaptR, a data scientist would need to go through the Cal-Adapt website and download the raw data as rasters or CSV files, one variable at a time, and one zip file at a time. She would then have to import those files into R using the file names as a reminder of which data it contains. This process works, and many people have done, but it's time consuming and klunky. caladaptR replaces all of those manual steps with a few lines of code that can be easily tweaked for new locations, climate variables, and models.
For civilians out there, this is still a big deal because it means adaptation planners and researchers can not only work more efficiently, they can also package up their analyses in user-friendly output formats that R is really good at. This includes markdown reports, as well as tailored decision support tools.
One of R's most popular features is R-Shiny, a package which allows R users to bundle their analyses up into user friendly interactive web sites, or apps. These web apps aren't limited to show pre-canned results, they can actually fetch data "on the fly" and run analyses based on user input. caladaptR is a "Shiny ready" package that includes a widget to show your data download status. A couple of sample Shiny apps are available on the website, including one that computes the projected amount of winter chill in the coming decades, which can help tree crop growers decide which variety of tree nuts to plant, or whether to get out of tree nuts all together.
Not surprisingly, we've recently seen growing interest in climate change impacts, and a growing recognition that data from climate models need to be transformed into measures that make sense to regular folks. Data science platforms like R, that can tap into cloud data and services via APIs, are the kinds of tools that will build the next generation of climate science translation services. With R and caladaptR, analyzing climate impacts and developing decision support tools is well within reach of anyone who has intermediate R skills.
Learning More
For more details, check out the following overview videos. When you're ready to start writing some code, head over to the caladaptR website which has installation instructions, getting started code, sample notebooks, workshop recordings, and long-form articles on some of the more advanced features. You can also attend an upcoming workshop (the next caladaptR workshop will be February 2022) and sign-up for IGIS Office Hours if you have any questions.
- Author: Sean Hogan
- Author: Brandon Stark
- Author: Maggi Kelly
non-recreational use of small unmanned aerial systems (sUAS - aka. drones) was added to the FAA's Code of Federal Regulations (CFR) in mid-2016, there have been a number of updates to the rules over the years, including an Academic Exemption for education and research purposes (which is only apply to institutions of higher education and not other state agencies or conservation groups). The most recent of these updates occurred in January 2021, when the regulations were further amended to allow for the operation of drones during the night and over people in limited circumstances. For some operators, these have been very welcome changes. However, maybe more widely relevant to our community of drone enthusiasts, in June 2021 the FAA announced the requirement of "The Recreational UAS Safety Test" (TRUST) for all recreational pilots; with the objective of ensuring that these pilots are likewise knowledgeable in the FAA's rules and regulations for drone operations.
Amidst these updates and changes to the regulations, a common question that IGIS receives is "what do I need to do to legally fly a drone?" To answer this question, the head of our UC System's Center of Excellence on Unmanned Aircraft System Safety (UASSafety), Dr. Brandon Stark, has provided the following simplified guidelines to help steer people in the right directions, for whether they need the TRUST and/or Part 107 certifications:
- Coursework -> TRUST
- Research in access-controlled field -> TRUST
- Research in public area with people -> Part 107
- Collection of data for another agency -> Part 107
- Promotional media -> Part 107
- Demonstration for students -> TRUST
- Demonstration for non-students -> Part 107
- Inspecting any structure for repairs -> Part 107
- Flying above the FAA's Facility Map Altitude -> Part 107
- Flying above 400 ft (AGL) -> Part 107
- Any operation that could profit the pilot in any way -> Part 107
- Strictly for fun (even if you have a Part 107 certification) -> TRUST
Note - The Academic Exception is not a loophole. Everyone now needs to have either completed a TRUST or Part 107 certification, and preferably both.
For the sake of drone operators everywhere, it is extremely important that we all abide by these rules and regulations; as it could take only one grievous mistake (even if the mistake is unwitting) to legally set back all of the progress that has been made for the allowance of sUAS operations, both within and outside of the UC System.
For more information on the TRUST and Part 107 certifications, please refer to the FAA's official site: https://faadronezone.faa.gov/
For all UC drone operations, please do not forget to file your flights in advance through the UC's UAS Safety App, https://ehs.ucop.edu/drones/. This is not only required by the UC System, but also provides the valuable benefit of insurance coverage for you and your drone in the event of an accident. Additionally, the recently created UC Drones knowledge portal is an incredible resource for a wide range of drone safety and regulation information: https://ucdrones.github.io/
This week I am venturing out of California for a quick trip to Provo Utah and BYU to deliver the Chauncy Harris Lecture. Chauncy Harris was a pioneer of modern urban geography, with critical scholarship in urban form: "The Nature of Cities" and "A Functional Classification of Cities in the United States". He was a geographer, born in Utah, and every year the Geography Department at BYU hold a lecture in his honor. My buddy Ryan Jensen, son of the famous and fabulous John Jensen, invited me to give this year’s talk - so I am going!
I’ll be talking about my “Mapping for Impact” concept - and how we can use spatial data science to help tackle some of the thorny problems facing us today. I am going to highlight some of my lidar + forest work, the VTM historical ecology project, and of course, drones.
Check out this cute poster they put together for the talk.
- Author: Sean Hogan
It was a great pleasure to attend the first in-person event held by ESRI since the start of COVID-19 in 2020, the ESRI Imagery Summit. This event largely focused on ESRI's steady progress towards incorporating more tools for collection, analysis and visualization of remotely sensed data/imagery into both ESRI's online and offline applications.
Among the many topics presented at the summit, there were two topics that excited me the most. The first was “ESRI Image for ArcGIS Online”. This brand new add-on for ArcGIS Online promises to be an increadibly valuable resource for efficiently uploading, analyzing and visualizing your own imagery to ArcGIS Online. Currently, ArcGIS Online requires users to host their own imagery on either a server or in the cloud, which can be prohibitively difficult for some users, and then call this imagery into your ArcGIS Online maps. However, this add-on will allow you to directly upload your data to ArcGIS Online, where it will be hosted in ESRI's Amazon cloud (at a yet to be determined credit fee structure), where it can then very be efficiently analyzed and displayed in ArcGIS Online.
The second topic that excited me was ESRI's Site Scan applications. I say “applications” (plural) because there are two parts to Site Scan, which can be used independently, but can also be used together. One part is a drone flight app (referred to as Site Scan Flight Limited Addition), and one part a cloud processing app. The flight app might be the best drone flight planning app that I have ever seen for quadcopters (almost like Sensefly's e-Motion, my favorite flight planning app, but for quadcopters). The only down side that I can see for the Site Scan flight app is that it is only compatible with a limited number of drones. As for the cloud processing app, it is a little early to tell, but for GIS users I think that it might be the future of drone data stitching apps. In particular, I am excited about how nicely it integrates with ArcGIS Online, via ESRI's cloud storage, which could be incredibly efficient for the data management of ongoing drone projects; being one of the most challenging aspects of regular drone operations.
IGIS will continue to explore/follow these developments as they are forthcoming, and will update the IGIS Blog accordingly.
Sean Hogan
- Author: Sean Hogan
- Author:
Ben Satzman
- Author: Maggi Kelly
July 2021 marks the fifth anniversary of Drone Camp, and thanks to an all-star lineup of presenters and instructors and a fantastic and diverse group of over 255 attendees from all over the world, it was a massive success this year. Initially launched in 2016 by the UC ANR, IGIS Statewide program, DroneCamp has now evolved into a multi-campus and industry collaboration, with a network of drone experts hailing from UC ANR, UC Santa Cruz, UC Berkeley, UC Merced, UC Davis, CSU Monterey Bay, and the Monterey Bay Drone Automation and Robotics Technology corporation. From 2016 to 2019, this event was held in-person at Davis, San Diego, and then in Monterey, CA.; however, in 2020 it was moved online due to COVID-19. The move indoors, to discuss a very outdoor-oriented topic, came with some challenges. We had to shelve our important (and fun) hands-on equipment and flight training, for example. Yet it had rewards. We were able to reach a more diverse and a greater number of participants, and widen the scope of content. Ultimately, 2020 was a great success, but in the process we recognized that it could be even better with additional help from our network of drone expert friends from around the state.
Like last year, we came together online for DroneCamp 2021 in July. Over 255 people joined from around the world to learn about theory, application, regulation, and data processing. We learned about the practical aspects of maintaining safety while flying, we took deep dives into various software workflows, and explored agricultural, forestry, and vegetation mapping examples. This year we hosted 5 Plenary sessions, and had some electrifying plenary talk sessions from cutting-edge scientists from around California: Crashing drones! Precision Agriculture! Citizen Science! Mapping aquatic environments!
Because the overall objective of DroneCamp remains to provide the most practical and comprehensive learning experience possible for attendees, we are working on an in-person, hands-on training day that is being scheduled for October, to be held in Monterey/Marina California. And, as the threat of COVID-19 subsides, additional in-person training sessions will be added around the state of California in the coming year.
DroneCamp Overview:
DroneCamp is designed for a wide range of skill levels and interests, for those who are interested in using drones for anything other than non-recreation use (calling for a part 107 remote pilot's license), from complete beginners with little to no experience in drone technology, to intermediate users who want to learn more advanced data processing and analysis. Between presentations on contemporary applications of drones in environmental and agricultural research, and hands-on data processing and analysis exercises, attendees have the opportunity to fully immerse themselves in the following topics:
-
Flight Skills: Safe Launch and Landings, Basic Operations, Traversing and Avoiding Obstacles, Night Flying
-
Safety and Regulations: Safety and Regulations Overview; How to be a Good Visual Observer; Operating in Controlled Airspace
-
Hardware: Sensors, Platforms and Field Accessories
-
Data Collection: Mission Planning; High Precision Mapping
-
Data Processing: Stitching Drone Images with Pix4D, Agisoft Metashape, ArcGIS Pro, and OpenDroneMap; Analyzing Processed Drone Data in QGIS and ArcGIS Pro
-
Data Analysis and Management: Vegetation Analysis, Vegetation Analysis and Classification in ArcGIS Pro;Analysis of the Intertidal Zone; A case study of data management, from collection to storage and sharing of data outputs.
It was super fun and rewarding, and a great success for ANR and all the other collaborators. We built networks, increased collaboration, learned some very cool technical stuff, and got updated on current regulations, including the fact that you can now renew your 107 license easily here.
Some inspiring quotes from anonymous attendee reviews:
I was so appreciative of the extremely high caliber faculty/instructors that were recruited to give presentations, demos, and use of software. You all worked so well together to impart different pieces of expert knowledge. You all are brilliant and I'm inspired!!!
Loved it. We started off heavy which blew my mind but all the talks were so informative and fascinating. Really appreciate the diverse group of people you gathered together. Just wish it was in person! I would love to meet everyone.
Excellent presentation that allowed those of us with ArcMap experience to see the similarities and differences offered by ArcGis Pro. Again, a wonderful presentation (by a professional) that accounts for all the practical steps involved with data manipulation rolled into a final product.
Once I read through my notes, look at my screen captures and watch some of the presentations again, I will be able to structure my drone classes for my students. I'm developing a drone program from scratch for middle and high school students at a local charter school.
Consider signing up for the in-person training in October, and keep DroneCamp 2022 in your sights! Further information will be coming soon to the DroneCamp website, https://dronecampca.org/
