1. 1. NextGen tools for scaling from seeds to traits to ecosystems Tim Brown, Research Fellow, Borevitz Lab ARC Centre for Plant Energy Biology, Australian National University Joel Granados, Chuong Nguyen, Kevin D. Murray, Riyan Cheng, Cristopher Brack, Justin Borevitz,
2. 2. In the next 100 years we will face challenges of unprecedented scale and complexity
3. 3. Humans are causing a 6th mass extinction Since 1500, ~30 -90% species declines globally 1,2 1) Dirzo, Rodolfo, et al. 2014. DOI: 10.1126/science.1251817 2) http://worldoceanreview.com/en/wor-2/fisheries/state-of-fisheries-worldwide/
4. 4. More than 7 billion people on the planet for the next 100 years >10 billion people on the planet by 2050
5. 5. Feeding 10 billion people will be very hard We must grow more food in the next 75 years than all the food produced in human history 1 Requires a 38% greater yield increase over historical gains, every year for the next 40 years2 1) Seeds of Doubt New Yorker, Aug 25, 2014 2) Tester & LeGrange. 2010. Science:327(818)
6. 6. Terraforming To alter the environment of a planet to make it capable of supporting terrestrial life forms. We are currently unterraforming the earth at an exceptionally fast rate To meet the challenges of the coming century we need to restore and re-engineer the environment to support >7 billion people for the next 100 years in the face of climate change, while maintaining biodiversity and ecosystem services These ecological challenges are too hard to be solved with existing data and methods
7. 7. The current resolution of field ecology is very limited Low spatial & time resolution data Limited sensors types Sampling is often manual and subjective Observations not-interoperable or proprietary; little or no data sharing Sample resolution is Forest or field not Tree or Plant Very little data from the last century of ecology is available for reuse This slows our rate of knowledge discovery Tim Brown
8. 8. Research focus How can we use new technology to quantify environmental processes in high resolution for scientists (and the public) in a format that they can use? (1) Lab phenomics Phenotype (plant behaviors) + Genomics (plant genomes) Identifying the genetic basis of plant growth and development (2) Field ecology High resolution monitoring of ecosystems
9. 9. Genotype x Environment = Phenotype Phenotype (and ecosystem function) emerges as a cross-scale interaction between genotypes and the environment they experience. The degree to which we can measure all three components is the degree to which we can understand ecosystem function Requires LOTS of complex data!
10. 10. The challenge Measure everything all the time How do we go from doing the science at the scale of one point per forest to multilayer data cubes for every tree or leaf? 10/ 20
11. 11. This isnt impossible Google didnt exit 17 yrs ago and now it indexes 30 trillion web pages You can now ASK google almost anything and get a pretty good answer 1.8 billion (mostly geolocated) images are uploaded to social media every day (2014; was 500m in 2013)1 We need this level of resolution (and google-like tools) for ecological knowledge 1. Meeker, 2013, 2014
12. 12. Lab vs field phenotyping Lab: High precision measurement and control but low realism youtu.be/d3vUwCbpDk0
13. 13. Lab vs field phenotyping Field: Realistic environment but low precision measurements In the field we have real environments but the complexity (and bad lighting!) reduces our ability to measure things with precision youtu.be/gFnXXT1d_7s
14. 14. Lab phenotyping Normal lab growth conditions arent very natural Kulheim, Agren, and Jansson 2002 Real World Growth Chamber
15. 15. Borevitz Lab Approach Create more natural Lab conditions with precision LEDs and temperature control Measure more precisely in the Field Suzanne Marselis enviro-net.org
16. 16. Growth cabinets with dynamic semi-realistic environmental & lighting conditions 8 & 10-band controllable LED lights to control light spectra, intensity Python scripts control chamber Temp/Humidity in 5 min intervals Grow plants in simulated regional/seasonal conditions & simulate climate E.g. coastal vs inland | Late Spring or Early Fall Expose cryptic phenotypes Repeat environmental conditions Between studies and collaborators Simulate live field site climate SpectralPhenoClimatron (SPC) Spectral response of Heliospectra LEDs. (L4A s20: 10-band)
17. 17. Real-time monitoring, analysis and data visualization Phenotype 2,000 plants in real-time 24/7 2 DSLR cameras / chamber * 7 chambers JPG + RAW imaging every 10 minutes processing server Automated Image analysis pipeline to extract growth data from 150,000 pot images a day Detect color checker Correct color and lens distortion Detect pots Segment each image Leaf detection and tracking Corrected Segmented Original
18. 18. Goal: High Throughput Phenotyping for the masses Most high throughput phenotyping systems cost millions of $$ Our system: Low cost: Off the shelf cameras controlled by $35 linux pcs ( Phenomics 20 years of technical advances have turned genetics into genomics into phenomics and yielded the ability to address fundamental and very complex questions Now state of the art phenomics is pretty high resolution Precision environmental controls 3D time-lapse models of every plant growing with each pixel in all spectra mapped to the 3D data cube model of each plant Genome sequence for every plant Automated bioinformatics pipeline for trait extraction
19. 21. The real world is way more complex than plants in the lab We need equally complex datasets and models to understand real world ecosystems The questions we ask have often been defined more by what data we can get than by what the best question would be
20. 22. So how do we measure everything all the time? Persistent 3D, time-series multi data-layer ecosystem models tracking every tree
21. 23. NextGen field monitoring Within 5-10 years we will have similar data to lab phenomics Automated time-series (weekly/daily) aerial (UAV) scans measure RGB; Hyperspectral; Thermal; LiDAR Centimeter resolution 3D model of every tree on field site Gigapixel imaging to track phenology in every tree/plant, hourly Automated computer visions analysis for change detection LIDAR (laser) scanning DWEL / Zebedee high resolution ground-based 3D scans Dense point clouds of 3D structure Microclimate sensor networks PAR Temp, Humidity Soil moisture @ multiple depths mm resolution dendrometers Full genome for every tree on site ( 1,000 trees Campbell weather stations (baseline data for verification) All data live online in realtime LIDAR: DWEL / Zebedee UAV overflights (bi-weekly/monthly) Georectified image layers High resolution DEM 3D point cloud of site in time-series Total Cost ~ tree data Tree Height; Volume, foliage density RGB color GPS location DEM of site 34/ 20 View 3D model online: http://bit.ly/ARB3Dv1
22. 34. Software outputs DEM and point cloud data Processing script for tree data: GPS, Height, 3D volume, top-down area, RGB phenology data Straight to google maps online
23. 35. Ultra-high resolution ground-based laser DWEL (CSIRO); Echidna (handheld; $25K LiDAR) Multiband Lidar with full point returns ~30 million points in a 50m2 area (vs 5-10 pts/m for aerial) Data: Michael.Schaefer@csiro.au
24. 36. 3D trees rendered from LiDAR data Image: Stu Ramsden, ANU Vislab
25. 37. So what do we do with all this data? The challenge is no longer to gather the data, the challenge is how we do science with the data once we have it A sample is no longer a data point Example: Gigavision data Sample: Camera hardware: 900 images (per hour) Automate stitching into panoramas (5-20,000 tiled images/pano/hr) Need to align time-series images to each other and to the real world (despite hardware failures, camera upgrades) How to visualize a time-series of 22 million images/year? Computer vision analysis Automated feature detection and phenophase detections To Data: Phenophase transition and growth data from 1,000 trees And then how do you even analyze that?!
26. 38. Virtual 3D Arboretum Project Goal: Use modern gaming software to explore new methods for visualizing time-series environmental data Historic and real-time data layers integrated into persistent 3D model of the national arboretum in the Unreal gaming engine Collaboration with ANU Computer Science Dept. TechLauncher students Stuart Ramsden, ANU VISlab
27. 39. Tips for managing big data 1. This is hard! Dont feel bad. No one else has much of an idea how to do it either. 2. Where we are now in dealing with big data is like dealing with numbers before excel 3. Learn to program; hire computer scientists whenever possible 4. Collaborate but dont be afraid to make executive decisions: 1. There are many ways to solve the same problem 5. Make a robust data management plan; dont collect data until you have a plan to organize it 1. BUT dont be afraid to jump in and fail. Learning is iterative and there arent actually consensual solutions to most of these problems yet 6. Virtual machines and cloud storage are your friends 1. Dont manage any hardware you dont have to 2. Convince your university to invest in computing infrastructure and good IT support 7. Share code If you solve a problem give the solution to others!
28. 40. Thanks and Contacts Justin Borevitz Lab Leader Lab web page: http://borevitzlab.anu.edu.au Funding: Arboretum ANU Major Equipment Grant ARC Center of Excellence in Planet Energy Biology | ARC Linkage 2014 Arboretum http://bit.ly/PESA2014 Cris Brack, Albert VanDijk, Justin Borevitz (PESA Project PIs) UAV data: Darrell Burkey, ProUAV 3D site modelling: Pix4D.com / Zac Hatfield Dodds / ANUVR team Dendrometers & site infrastructure Darius Culvenor: Environmental Sensing Systems Mesh sensors: EnviroStatus, Alberta, CA ANUVR Team Zena Wolba; Alex Alex Jansons; Isobel Stobo; David Wai TraitCapture: Chuong Nguyen; Joel Granados; Kevin Murray; Gareth Dunstone; Jiri Fajkus Pip Wilson; Keng Rugrat; Borevitz Lab Contact me: tim.brown@anu.edu.au http://bit.ly/Tim_ANU http://github.com/borevitzlab