Wow! It’s been awhile since I posted. That’s because Fall semester was a full one. I’ll update you all on that soon - it basically will be a summary of using Pro for class, which was terrific.
OK on to the workshop.
We started with the inspirational video showcased at 2019 ESRI User Conference, and a video welcome from JD. The theme of the workshop is “Amplify your GIS”, and he introduced the new concept “geospatial infrastructure” also developed in hot-off-the-presses paper with Michael Goodchild. All the ESRI GIS world is referred to as the Esri Geospatial Cloud, and a new key component are the geo-enabled workflows.
What’s New in ArcGIS. There is lots of new stuff in Pro: coding, visualizations, editing workflows; new stuff in AGOL, primarily the new Map Viewer (Beta); and new stuff in the Solutions Workflows. Highlights for me include:
What’s new in ArcGIS Pro – Professional GIS
New extension is LocateXT – unstructured data
New in Editor
Find and replace in the attribute table
Contingent values within Domains
Using Arcade for things like automatic calculation of attributes. They suggest we check it out
New in Visualization tools
Feature binning via the enable Feature Binning tool
Opens a new tab on the ribbon, creates dynamic polygons summarizing features found within. e.g. for large point datasets symbolized in hexagons
Match Layer Symbology to a Style: Quick way to have different styles
The “Calendar Heat Chart” for data with temporal data
You can pie charts to polygon features, changing input and making pie charts variable; and make into 3D
New in Coding/Tools"
In 2.5, you can schedule your geoprocessing tools. WHAT?
You can turn geoprocessing history directly into a script, or Export model to a python file, or Export geoprocessing history directly to python
In 2.5, jupyter notebook can be run directly in Pro (from “Notebooks”) in the Catalog
New Layouts: Import Layout Gallery: 12 default templates within a Gallery. You can add your own.
Printing: Printing is now done in the background; you can print in black and white; you can tile your larger maps.
Password protect your pdfs
New format: AIX files for Adobe Illustrator
Text boxes: you can now change the shape of your text boxes – reshape along a complex boundary, e.g.
What’s new in ArcGIS Online
Map Viewer Beta is the new thing
New Layout, with symbolization by field, by expression, (using Arcade) and filter data; New dot density symbolization choice;
More control over look and field of pop-ups (yay!), including adding text and images. Dynamic changes, yay!
Plus some slick links to Story Maps
What’s new in ArcGIS Solution Workflows
Showed us some great stuff from the Conservation Easement Solutions Workflow – migration from a paper-based solution to a digital workflow. There are many other examples
Use the ArcGIS Solution Deployment Tool
What’s new in ArcGIS for Developers
ESRI continues its work to extend the platform through:
App Builders – Web AppBuilder (new is Experience Builder, which is in beta testing now)
Some other stuff that I wasn’t fast enough to capture. But looks good.
Example GIS workflows
The presenters walked us through a few key sectors using GIS. Some great stuff came out for me to explore, including:
GIS for law enforcement
There is a “Crime Analysis” toolbar, BTW. Free and added to the ribbon. It contains lots of crime-relevant tools and some ways to efficiently update data from tables
The “Import Records Tool” seems very useful for anyone using large tabular data that updates regularly.
Apparently ESRI has quite a few staff/engineers with law enforcement backgrounds who have built most of these tools.
Showed QuickCapture as an easy workflow to capture information about the condition of streets.
Once you have your problem areas, you can route them in AGOL via the “Plan Routes” Tool (uses credits!) in AGOL.
Those routes are used with the “Navigator” tool, and integrated with the “Tracker” app
AutoCad integration: you can now edit an ArcGIS hosted feature service in AutoCad. This is a big deal.
Parcel Editing and Parcel Fabric
Showcased the Parcel Fabric Administration, and Parcel Tasks
Parcel Fabric keeps track of all actions.
Update COGO; updates all the directions, areas, etc.
This is where ESRI has been building the Urban Solution Workflow
We’ve seen this example before at the 2019 ESRI UC; it’s from Boston, discussing planning for a new high rise building and its potential shading of Boston Common.
Lots of stuff to follow up on for IGIS and for projects. Happy 2020.
Day 2 Highlights
Morning sessions included using ArcGIS Pro for Lidar analysis. This was a great overview of working with LAS files within ArcGIS Pro: data formats, including LAS related data structures multipoints and multipatches, data diagnostics to examine data quality and coverage, and basic processing of Lidar data in ArcGIS Pro. Very useful. Thanks to ccrawford. Next Shane and I checked out Data Science in ArcGIS Pro: Using R and Python. This is an update from last year’s session on R and Python in ArcGIS, and again we started with the classic battle of the bands. The python demo showed PySAL, and how to do a quick spatial econometric analysis in PySAL within Pro. Need more time, obvi. The R demo was a nice walk through of data input - vector and raster - via the R-bridge, and showed how easy the framework is. Very useful.
Afternoon sessions included a bunch of stuff, but the highlight was the always funny, always useful, always inspiring wizards of cartography Kenneth Field, John Nelson, and Edie Punt and Mapping with Style. Another great session! Edie discussed the excellent Styles capacity in Pro, which I am aware of but not an expert at. She helpfully pointed out some key things to pay attention to: USE STYLES FOR COLOR CONTROL! because of the new graphics engines used in Pro, TRANSPARENCY is available on all colors, and will be transferred to your PDF! Also, color locking is great! And color brewer schemes are available in Pro Styles. So much love! Interspersed with Edie’s slide were John and Kenneth delivering their usual hilarious take on making beautiful maps: John showed off some nice and creative new published styles: Firefly, Imhof, and Lego; and Kenneth presented a whimsical case study bringing to life a 1930s map of Redlands using the watercolor style from John. See other styles here. Good day.
My wrap-up from a very engaged and provocative 1.5 day workshop on geospatial technology futures, hosted by the CyberGIS Center: “Towards a National Geospatial Software Ecosystem”. First: great group of cool peeps all hyper-engaged in geospatial data, tools, use cases, science, and community. Second: fun to be involved in big-picture thinking on what a geospatial software institute might look like if it was to be built from scratch. Finally, I was on the panel discussing core questions bridging use cases and core technical capabilities, and I share my reflections of the workshop here.
- Question 1. Are there any significant gaps between the use cases and core technical capabilities that GSI should address?
- Training needs: beyond GIS training – “spatial data science” training, for K-12; undergrad; graduate; veterans; professionals
- Easy ways to get access to cloud storage and computation, and for different datasets like UAVs. There are examples like CyVerse (from Tyson Swetnam) and others
- Data integration: Data assimilation, Data fusion, Sensor triangulation.
- Whatever you want to call it – this remains a challenge for geospatial experts and beginners alike. And it is especially a challenge when you work across disciplines (e.g. the work of SESYNC from Mary Shelley and Margaret Palmer, SESYNC, University of Maryland)
- Dynamics: Spatio-temporal and real-time data streams: sensor networks, social media, cube sats
- in space (e.g. the new Antarctic DEM from Paul Morin, University of Minnesota);
- in time (e.g. cubesats, sensor networks; social media);
- in depth?: going under-ground (from Debra Laefer, NYU)
- We love FAIR for data. What about FAIR for tools: make tools Findable, Accessible, Interoperable, and Re-usable
- Question 2: What does the CyberGIS Geographic Software Institute (GSI) need to do to address community needs and contribute to the national CyberInfrastructure ecosystem?
- Link strongly with existing diversity-supporting frameworks: HBCU; community colleges; tribes; networks such as @WomenWhoCode, @LadiesOfLandsat, @BlackGirlsCode, @500womensci, @RLadiesGlobal, etc.
- More of these workshops! Multi-disciplinary meetings of people with tight/packed agendas and make use of workshop attendees between workshops; what can we do to spread the word
- Create GSI Data Institute or Bootcamp or Faculty Education Mentoring Network
- Support standards for data and software standards to promote interoperability
- Support frameworks for data and software discovery and interoperability: FAIR for data; FAIR for tools
Conclusion: Super Fun. Learned a Ton. Plus parting words from Michael Goodchild: It is not location that matters, it is context. Location provides context; context allows integration: with data, between disciplines, between people, between tools. "Let's get above the layers".
As always, the Plenary session was an immersive and emotional showcase of the power of mapping. Running through Monday’s talks was a sense of urgency for we GIS people to save the world. This is what JD calls “societal GIS”, or “embracing the digital transformation and leverage the science of where”. Shane and I had a great time. Some key news from the Plenary:
ESRI is in every K-12 school in the US; JD announced it will be offered to every K-12 school in the world.
The work of Thomas Crowther, Professor of Global Ecosystem Ecology at ETH Zürich (@crowthelab) is inspirational. They estimate 3T trees globally, with room for 1T more. (See paper here.) Gonna be checking out his tree data on the Living Atlas (global maps of tree density, diversity, carbon uptake, and reflectance).
A great demo from JD Irving, a private Canadian forestry, transportation and products company heavy into sustainability and GIS. All there properties are managed using ArcGIS + R.
ESRI is showcasing some key "Solution Configurations" that are bundled software products focused on high-priority areas such as: 1) community engagement ("Hub"); 2) interior spaces ("Indoors") and 3) smart cities ("Urban"). The highlighted snazzy urban planning 3D vis tools will be giving UrbanSim a run for their money. Might we work RUCS2.0 into a "Solution Configuration" for working landscape planning?
Plus some highlights of what I learned overall:
Wow. ESRI's Living Atlas of the World has some amazing resources. Living Atlas is ESRI’s curated web data portal that links seamlessly with Pro. It has tons of data on environment and imagery. Want Sentinel-2 imagery, NAIP, or MODIS thermal? Want global climate and weather data? Want to easily play with Open Street Map or other vector tiles within your GIS project? It is all in the Living Atlas. This will be a game changer for class. Plus TC’s tree data. Gonna be checking this out.
Unstructured data can be added to your workflow now, this is text, etc. This is big.
Offering access to Open Street Map within Pro.
Software updates (mostly about Pro)
Pro is the way to go, but ESRI will continue to support ArcMap “for years to come”
New stuff in ArcGIS Pro related to Image Analysis:
Sensor support has been expanded; plus new formats supported, eg. netcdf. Pro supports mosaic datasets, they call mosaics the optimum data model for image management.
ESRI is now supporting “oriented” imagery - StreetView Imagery, oblique imagery, etc. Easily integrate things like iPhone photos within your Pro project. They call this working in “image space” rather than “map space”.
Ortho Mapping within ESRI has 3 solutions: Drone2Map (stand-alone software), within ArcGIS Pro (using the Image Server license), and OrthoMaker (web interface).
New release of Pro has full motion video support. (Upcoming releases will have more deep learning algorithms, multi-patch editing in stereo, and pixel editing.)
There are so many cool things going on on the imagery front in Pro, makes me excited.
New stuff in ArcGIS Pro in general:
Adding an unstructured data format - e.g. text!
3D editing and 3D voxel support.
Machine Learning is increasingly embedded in ESRI workflows, and when that is not enough, ML is also possible via linkages with external resources (via R, TensorFlow, MXNET, AWS tools, etc.).
ESRI increasingly recognizing that people work in and outside of ESRI software: R-Bridge, Python API, Jupyter Notebooks makes external linkages super easy.
ESRI is working to support cloud-based storage and computing via:
Support via AWS and Azure; Optimizing raster storage and caching in multiple formats; and the ability to point to existing cloud storage
Plus, for your GPS needs:
Trimble Catalyst antenna + ESRI Collector might be the way to go, but it is windows/android specific for now. iOS compatibility is "on a horizon" as of now.
A quick note about ArcGIS online (ESRI's complete mapping and location intelligence platform). It has 6M subscribers (!), making 1B maps a day (!!). (Did I get those numbers correctly?)
Notes for classes/workshops
GIS-stat-analysis-py-tutor on GitHub
ESRI provides many Learning templates for us who are dreading converting all our ArcMap labs to Pro: https://www.esri.com/training/ and
ESRI is also working on providing templated best practice workflows to help teach concepts. They call them, at least in Image Analyst "Imagery workflows". Might be useful in class/workshops.
As always a great conference! The new ESRI terminology might be a useful organizing structure for class: A GIS is a system of:
- Record: storing spatially indexed information
- Insights: via analysis
- Engagement: through mapping and visualization
We've just wrapped up #DroneCamp2018, hosted at beautiful UC San Diego.
This was an expanded version from last year's model, which we held in Davis. We had 52 participants (from all over the world!) who were keen to learn about drones, data analysis, new technology, and drone futures.
Day 1 was a flight day from half our participants: lots of hands-on with takeoffs and landings, and flying a mission;
Day 2 covered drone safety and regulations, with guest talks from Brandon Stark and Dominique Meyer;
Day 3 covered drone data and analysis;
Day 4 was a flight day for Group 2 and a repeat of Day 1.
We had lots of fun taking pics and tweeting: here is our wrapup on Twitter for #DroneCamp2018.