All Learning Resources
Data Collection Part 1: How to avoid a spreadsheet mess - Lessons learned from an ecologist
Most scientists have experienced the disappointment of opening an old data file and not fully understanding the contents. During data collection, we frequently optimize ease and efficiency of data entry, producing files that are not well formatted or described for longer term uses, perhaps assuming in the moment that the details of our experiments and observations would be impossible to forget. We can make the best of our sometimes embarrassing data management errors by using them as ‘teachable moments’, opening our dusty file drawers to explore the most common errors, and some quick fixes to improve day-to-day approaches to data.
Data Collection Part 2: Relational databases - Getting the foundation right
Data Sharing and Management within a Large-Scale, Heterogeneous Sensor Network using the CUAHSI Hydrologic Information System
Hydrology researchers are collecting data using in situ sensors at high frequencies, for extended durations, and with spatial distributions that require infrastructure for data storage, management, and sharing. Managing streaming sensor data is challenging, especially in large networks with large numbers of sites and sensors. The availability and utility of these data in addressing scientific questions related to water availability, water quality, and natural disasters relies on effective cyberinfrastructure that facilitates transformation of raw sensor data into usable data products. It also depends on the ability of researchers to share and access the data in useable formats. In this presentation I will describe tools that have been developed for research groups and sites conducting long term monitoring using in situ sensors. Functionality includes the ability to track equipment, deployments, calibrations, and other events related to monitoring site maintenance and to link this information to the observational data that they are collecting, which is imperative in ensuring the quality of sensor-based data products. I will present these tools in the context of a data management and publication workflow case study for the iUTAH (innovative Urban Transitions and Aridregion Hydrosustainability) network of aquatic and terrestrial sensors. iUTAH researchers have developed and deployed an ecohydrologic observatory to monitor Gradients Along Mountain to Urban Transitions (GAMUT). The GAMUT Network measures aspects of water inputs, outputs, and quality along a mountain-to-urban gradient in three watersheds that share common water sources (winter-derived precipitation) but differ in the human and biophysical nature of land-use transitions. GAMUT includes sensors at aquatic and terrestrial sites for real-time monitoring of common meteorological variables, snow accumulation and melt, soil moisture, surface water flow, and surface water quality. I will present the overall workflow we have developed, our use of existing software tools from the CUAHSI Hydrologic Information System, and new software tools that we have deployed for both managing the sensor infrastructure and for storing, managing, and sharing the sensor data.
Metadata: Standards, tools and recommended techniques
How high performance computing is changing the game for scientists, and how to get involved
Best practices for preparing data to share and preserve
Scientists spend considerable time conducting field studies and experiments, analyzing the data collected, and writing research papers, but an often overlooked activity is effectively managing the resulting data. The goal of this webinar is to provide guidance on fundamental data management practices that investigators should perform during the course of data collection to improve the usability of their data sets. Topics covered will include data structure, quality control, and data documentation. In addition, I will briefly discuss data curation practices that are done by archives to ensure that data can be discovered and used in the future. By following the practices, data will be less prone to error, more efficiently structured for analysis, and more readily understandable for any future questions that they might help address.
Data citation and you: Where things stand today
Open data and the USGS Science Data Catalog
Data Rescue: Packaging, Curation, Ingest, and Discovery
Data Conservancy was introduced to Data Rescue Boulder through our long-time partner Ruth Duerr of Ronin Institute. Through our conversations, we recognized that Data Rescue Boulder has a need to process large number of rescued data sets and store them in more permanent homes. We also recognized that Data Conservancy along with Open Science Framework have the software infrastructure to support such activities and bring a selective subset of the rescued data into our own institution repository. We chose the subset of data based on a selection from one of the Johns Hopkins University faculty members.
This video shows one of the pathways through which data could be brought into a Fedora-backed institutional repository using our tools and platforms
Data Conservancy screen cast demonstrating integration between the Data Conservancy Packaging Tool, the Fedora repository, and the Open Science Framework. Resources referenced throughout the screen cast are linked below.
DC Package Tool GUI
DC Package Ingest
- Package Ingest release page
- Fedora API Extension Architecture Home, GitHub, and Docker-based demo
- API-X funding provided by IMLS grant #LG-70-16-0076-16
Fedora OSF Storage Provider
(under development as of April 2017)
Open Principles in Education - Building Bridges, Empowering Communities
This presentation shared experiences from “Geo for All” initiative on the importance of having open principles in education for empowering communities worldwide . Central to “Geo for All” mission is the belief that knowledge is a public good and Open Principles in Education will provide great opportunities for everyone. By combining the potential of free and open software, open data, open standards, open access to research publications, open education resources in Geospatial education and research will enable the creation of sustainable innovation ecosystem . This is key for widening education opportunities, accelerating new discoveries and helping solving global cross disciplinary societal challenges from Climate change mitigation to sustainable cities. Service for the benefit and betterment of humanity is a key fundamental principle of “Geo for All” and we want to contribute and focus our efforts for the United Nations Sustainable Development Goals. We aim to create openness in Geo Education for developing creative and open minds in students which is critical for building open innovation and contributes to building up Open Knowledge for the benefit of the whole society and for our future generations. The bigger aim is to advance STEM education across the world and bring together schools, teachers and students across the world in joint projects and help building international understanding and global peace.
Unidata Data Management Resource Center
In this online resource center, Unidata provides information about evolving data management requirements, techniques, and tools. They walk through common requirements of funding agencies to make it easier for researchers to fulfill them. In addition, they explain how to use some common tools to build a scientific data managment workflow that makes the life of an investigator easier, and enhances access to their work. The resource center provides information about: 1) Agency Requirements, 2) Best Practices, 3) Tools for Managing Data, 4) Data Management Plan Resources, 5) Data Archives, and 6) Scenarios and Use Case.
USGS Data Management Training Modules – the Value of Data Management
This is one of six interactive modules created to help researchers, data stewards, managers and the public gain an understanding of the value of data management in science and provide best practices to perform good data management within their organization. In this module, you will learn how to: 1. Describe the various roles and responsibilities of data management. 2. Explain how data management relates to everyday work and the greater good. 3. Motivate (with examples) why data management is valuable. These basic lessons will provide the foundation for understanding why good data management is worth pursuing.
USGS Data Management Training Modules – Planning for Data Management
This is one of six interactive modules created to help researchers, data stewards, managers and the public gain an understanding of the value of data management in science and provide best practices to perform good data management within their organization. In this module, we will provide an overview of data management plans. First, we will define and describe Data Management Plans, or DMPs. We will then explain the benefits of creating a DMP. Finally, we will provide instructions on how to prepare a DMP, including covering key components common to most DMPs.
USGS Data Management Training Modules – Best Practices for Preparing Science Data to Share
This is one of six interactive modules created to help researchers, data stewards, managers and the public gain an understanding of the value of data management in science and provide best practices to perform good data management within their organization. In this module, you’ll learn:
The importance of maintaining well-managed science data
Nine fundamental practices scientists should implement when preparing data to share
Associated best practices for each data management habit
USGS Data Management Training Modules – Science Data Lifecycle
This is one of six interactive modules created to help researchers, data stewards, managers and the public gain an understanding of the value of data management in science and provide best practices to perform good data management within their organization. By the end of this module, you should be able to answer the following questions… What is a science data lifecycle? Why is a science data lifecycle important and useful? What are the elements of the USGS science data lifecycle, and how are they connected? What are the difference roles and responsibilities? Where do you go if you need more information?
USGS Data Management Training Modules – Planning for Data Management Part II
This is one of six interactive modules created to help researchers, data stewards, managers and the public gain an understanding of the value of data management in science and provide best practices to perform good data management within their organization. By the end of this course you should know the difference between data management plans and project plans; you should know how to use the DMPTool to create a data management plan; and you should understand the basic information that should go into a data management plan.
USGS Data Management Training Modules—Metadata for Research Data
This is one of six interactive modules created to help researchers, data stewards, managers and the public gain an understanding of the value of data management in science and provide best practices to perform good data management within their organization. This module covers metadata for research data. The USGS Data Management Training modules were funded by the USGS Community for Data Integration and the USGS Office of Organizational and Employee Development's Technology Enabled Learning Program in collaboration with Bureau of Land Management, California Digital Library, and Oak Ridge National Laboratory. Special thanks to Jeffrey Morisette, Dept. of the Interior North Central Climate Science Center; Janice Gordon, USGS Core Science Analytics, Synthesis, and Libraries; National Indian Programs Training Center; and Keith Kirk, USGS Office of Science Quality Information.
DataONE Data Management Module 06: Data Protection and Backups
This module covers the difference between data protection, backup, archiving and preservation, best practices for backing up and preserving data.
DataONE Data Management Module 08: Data Citation
This module defines data citation, explains benefits of data citation, and provides examples and best practices for data citation.