All Learning Resources

This tutorial on data warehouse concepts will tell you everything you need to know in performing data warehousing and business intelligence. The various data warehouse concepts explained in this video are:
1. What Is Data Warehousing?
2. Data Warehousing Concepts:
3. OLAP (On-Line Analytical Processing)
4. Types of OLAP Cubes
5. Dimensions, Facts & Measures
6. Data Warehouse Schema
Check our complete Data Warehousing & Business Intelligence playlist here.

This page provides a variety of resources for creating, submitting and using ORNL DAAC data. We provide guidance on preparing a data management plan and properly formatting data for long-term archiving, various access tools and services, and  using and analyzing data in popular software packages.  Resource types include tutorials, code, webinars, help pages, classroom exercises and workshops.  New resources are added as they are made available.

This tutorial is designed to help scientists think about the best way to structure and document their USGS public data releases. The ultimate goal is to present data in a logical and organized manner that enables users to quickly understand the data. The first part of the tutorial describes the general considerations for structuring and documenting a data release, regardless of the platform being used to distribute the data. The second part of the tutorial describes how these general consideration can be implemented in ScienceBase. The tutorial is designed for USGS researchers, data managers, and collaborators, but some of the content may be useful for non-USGS researchers who need some tips for structuring and documenting their data for public distribution.

Emily Fort of the USGS presents an introduction to data management planning and a USGS program experience.

Most scientists have experienced the disappointment of opening an old data file and not fully understanding the contents. During data collection, we frequently optimize ease and efficiency of data entry, producing files that are not well formatted or described for longer term uses, perhaps assuming in the moment that the details of our experiments and observations would be impossible to forget. We can make the best of our sometimes embarrassing data management errors by using them as ‘teachable moments’, opening our dusty file drawers to explore the most common errors, and some quick fixes to improve day-to-day approaches to data.
 

Hydrology researchers are collecting data using in situ sensors at high frequencies, for extended durations, and with spatial distributions that require infrastructure for data storage, management, and sharing. Managing streaming sensor data is challenging, especially in large networks with large numbers of sites and sensors.  The availability and utility of these data in addressing scientific questions related to water availability, water quality, and natural disasters relies on effective cyberinfrastructure that facilitates transformation of raw sensor data into usable data products.  It also depends on the ability of researchers to share and access the data in useable formats.  In this presentation I will describe tools that have been developed for research groups and sites conducting long term monitoring using in situ sensors.  Functionality includes the ability to track equipment, deployments, calibrations, and other events related to monitoring site maintenance and to link this information to the observational data that they are collecting, which is imperative in ensuring the quality of sensor-based data products. I will present these tools in the context of a data management and publication workflow case study for the iUTAH (innovative Urban Transitions and Aridregion Hydrosustainability) network of aquatic and terrestrial sensors.  iUTAH researchers have developed and deployed an ecohydrologic observatory to monitor Gradients Along Mountain to Urban Transitions (GAMUT). The GAMUT Network measures aspects of water inputs, outputs, and quality along a mountain-to-urban gradient in three watersheds that share common water sources (winter-derived precipitation) but differ in the human and biophysical nature of land-use transitions. GAMUT includes sensors at aquatic and terrestrial sites for real-time monitoring of common meteorological variables, snow accumulation and melt, soil moisture, surface water flow, and surface water quality. I will present the overall workflow we have developed, our use of existing software tools from the CUAHSI Hydrologic Information System, and new software tools that we have deployed for both managing the sensor infrastructure and for storing, managing, and sharing the sensor data.

The presentation focuses on USGS/​Pacific Northwest Aquatic Monitoring Partnership's (PNAMP) Monitoring Resources toolset.

Scientists spend considerable time conducting field studies and experiments, analyzing the data collected, and writing research papers, but an often overlooked activity is effectively managing the resulting data. The goal of this webinar is to provide guidance on fundamental data management practices that investigators should perform during the course of data collection to improve the usability of their data sets.  Topics covered will include data structure, quality control, and data documentation. In addition, I will briefly discuss data curation practices that are done by archives to ensure that data can be discovered and used in the future. By following the practices, data will be less prone to error, more efficiently structured for analysis, and more readily understandable for any future questions that they might help address.

Dendro is a collaborative file storage and description platform designed to support users in collecting and describing data, with its roots in research data management. It does not intend to replace existing research data repositories, because it is placed before the moment of deposit in a data repository.  The DENDRO platform is an open-source platform designed to help researchers describe their datasets, fully build on Linked Open Data. Whenever researchers want to publish a dataset, they can export to repositories such as CKAN, DSpace, Invenio, or EUDAT's B2SHARE. 

It is designed to support the work of research groups with collaborative features such as:

File metadata versioning
Permissions management
Editing and rollback
Public/Private/Metadata Only project visibility

You start by creating a “Project”, which is like a Dropbox shared folder. Projects can be private (completely invisible to non-colaborators), metadata-only (only metadata is visible but data is not), and public (everyone can read both data and metadata). Project members can then upload files and folders and describe those resources using domain-specific and generic metadata, so it can suit a broad spectrum of data description needs. The contents of some files that contain data (Excel, CSV, for example) is automatically extracted, as well as text from others (PDF, Word, TXT, etc) to assist discovery.

Dendro provides a flexible data description framework built on Linked Open Data at the core (triple store as), scalable file storage for handling big files, BagIt-represented backups, authentication with ORCID and sharing to practically any repository platform.

Further information about Dendro can be found on its Github repository at:  https://github.com/feup-infolab/dendro.  Documentation and descriptions of Dendro can be found in other languages from the primary URL home page.