- Author: Andy Lyons
- Contributor: Jacob Flanagan
- Contributor: Sean Hogan
- Contributor: Maggi Kelly
At the recent conference of the American Association of Geographers (AAG) in Washington DC, IGIS was recognized for its work in developing open source data management tools for drone data. The AAG is the largest association of geographers, GIS specialists, environmental scientists in the world, and its annual conference attracts over 8,500 people and over 6,900 presentations. IGIS researchers Andy Lyons, Jacob Flanagan, and Sean Hogan won second place in a poster competition sponsored by the AAG Remote Sensing Specialty Group and photogrammetry company Pix4D.
Data management involves protocols and file transfer utilities to help you organize data, assess quality, create backups, find and retrieve files, and reproduce workflows. Photogrammetry programs are brilliant at stitching photos into beautiful high resolution imagery, but they don't offer many tools for managing data before or after the processing. This is the gap IGIS is addressing.
IGIS has been working on data management tools for the past couple of years to help us manage the dozens of drone projects we conduct each year on behalf of UC researchers. In keeping with our public mission and research innovation focus, we use open source programming platforms such as Python and R and share our tools on GitHub so anyone can use them.
The poster presented at the AAG conference showcases 5 data management projects. The foundation of all our projects is a directory structure and file naming protocol that encompasses all the types of data in a drone project, including images, ground control points, GIS layers, documentation, intermediate files, and final outputs. Building upon this, the uavimg package for R creates offline HTML catalogs and maps of images, allow for quality control checks in the field and a master catalog to help an analyst find the right set of images for a project. The IGIS Drive Monitor, written in Python, is designed to run on a laptop in the field and automates the process even further by monitoring a USB drive and copying files into the right place automatically. The IGIS Pix4D Controller, also written in Python, runs on a server and automates the next step of the process, launching the Pix4D stitching software when a new set of images is detected, creating a new project, and initiating the stitching process. Finally, the IGIS Drone Data Management Logbook, still being developed, combines functionality of the previous utilities with additional visualization tools for quality control.
Our drone data management utilities are still under development, but they are available currently available for testing (see GitHub links in the poster). If you'd like to be a beta tester, please let us know!). Later this year, we plan on holding a webinar describing how to use these tools and invite feedback from users. Open source software is designed for collaboration, and our ultimate hope is to collaborate with other drone users and programmers facing similar needs. In the meantime, it's great to be recognized by leading experts in the field.
Open Source Tools for Drone Data Management poster, AAG 2019
- Author: Mark Bolda
Great article out of the ubiquitous Wall Street Journal transcribing an interview with Tadashi Yanai, the billionaire behind the speciality clothing manufacturer Fast Retailing Co., who also operates the more than 1,700 Uniqlo stores in Asia and not incidentally oversees tremendous sales over the Web as well. This guy is very successful and we can assume he has a good idea of what's up.
Why I am writing about clothing retailers on this on an ostensibly berry science oriented blog? Here's why. Mr. Yanai refers to the meaning of data, and what to do with it which as my readers know is a pretty big theme here. Data is not the end all for acquisition of knowledge; one actually needs to apply real insight, both from education and experience, to make it work.
Here's what this very successful businessman has to say about it:
"Data would never substitute the merchant. How do you interpret the data? That's the merchant's skill set. You need to uncover the insight that is buried in the data and the merchants need to uncover it. Even if you employ artificial intelligence to help you, the numbers [don't tell] the future."
There you have it. Guess what all, you can't farm from just a computer, you actually need to know the meaning of the numbers [and that takes hard work].
- Author: Mark Bolda
Pretty decent article here from the Capitol Press on how growers are struggling with how to make sense of the really large amounts of data so easily available to them in our increasingly technological age.
The fact of the matter is that reduced computing costs have created an enormous wave of information, and in a recent article in the Wall Street Journal written by Michael Milken and Igor Tulchinsky, caution us to buckle up because this already large tsunami of data "doubles in size every few years". The two authors, while conceding that this is a challenge, also recognize it as an opportunity.
As many are finding, the problem of understanding the meaning of lots and lots of data is deeper than just pressing the whole undifferentiated mass into Google and getting an actionable answer. I would suggest it is rather more a matter of sorting out the unimportant data from the important, and then having the mastery of that body of knowledge to which the data refer and only then be able put it all together to arrive at a good decision.
The book titled "The Signal and the Noise", written by statistician Nate Silver makes some progress on this issue. A lot of the data available to decision makers and prognosticators in a wide range of fields, from weather, to markets, to sports events, to elections, and yes to agriculture is not that useful, is not worth listening to and can be called noise, while those bits that are really useful, the signals, are where we should be spending our time and attention.
Complicating this however is the fact that it's very rarely just one signal that merits our attention, but rather it can be a multiple or even further an interaction of these signals which is most meaningful, and yet not all are as equally important. Take for the example the malnourished plant with a compromised root system. Is the malnourishment truly just the roots, or do we also face some deficiencies in the soil? What of the soil pH or CEC which might be impeding the transmission of these nutrients to the plant? The knowledgeable person is going to know what compromises a root system, what soil nutrient deficiencies look like, what a pH of x means to the whole shebang and weighs its value, pieces the important parts together, discards the rest and then makes the call on how to proceed.
In short, it is a deception to think that simply having access to ever greater amounts data effortlessly bestows upon one the ability to make better and more accurate decisions. Really it takes some accomplishment, experience and quite frankly a lot of hard work as an individual to sort out the signals from the noise, and further be able to put this concert of signals into a comprehensible whole.
![What were the signals and what was the noise for investors to avoid this near 50% schmeissing on this insurance stock? A clue, the big signal isn't even here. What were the signals and what was the noise for investors to avoid this near 50% schmeissing on this insurance stock? A clue, the big signal isn't even here.](/blogs/blogcore/blogfiles/42902.png)
![That about sums it up. That about sums it up.](/blogs/blogcore/blogfiles/43851.jpg)