Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ Package: metajam
Type: Package
Title: Easily Download Data and Metadata from 'DataONE'
Version: 0.3.2
Date: 2026-06-23
Date: 2026-08-31
Authors@R: c(
person("Julien", "Brun",
email = "julien.brun@alumni.duke.edu",
Expand Down Expand Up @@ -45,7 +45,7 @@ Description: A set of tools to foster the development of reproducible analytical
License: Apache License (== 2.0)
Encoding: UTF-8
Language: en-US
RoxygenNote: 7.3.3
RoxygenNote: 8.1.0
SystemRequirements: Mac OSX: redland (>= 1.0.14) ; Linux: librdf0 (>= 1.0.14),
librdf0-dev (>= 1.0.14)
URL: https://nceas.github.io/metajam/, https://github.com/NCEAS/metajam
Expand Down
2 changes: 1 addition & 1 deletion R/check_version.R
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
#' @export
#'
#' @examples
#' \donttest{
#' \dontrun{
#' # Most data URLs and identifiers work
#' check_version("https://cn.dataone.org/cn/v2/resolve/urn:uuid:a2834e3e-f453-4c2b-8343-99477662b570")
#' check_version("doi:10.18739/A2J09W56F")
Expand Down
2 changes: 1 addition & 1 deletion R/download_ISO_data.R
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ ISO_type <- metadata2 %>% filter(name == "doc.children.MD_Metadata.children.meta
pid <- data_id
data_sys <- suppressMessages(dataone::getSystemMetadata(d1c@mn, pid))

data_name <- data_sys@fileName %|||% ifelse(exists("entity_data"), entity_data$physical$objectName %|||% entity_data$entityName, NA) %|||% data_id
data_name <- data_sys@fileName %|||% data_id
data_name <- gsub("[^a-zA-Z0-9. -]+", "_", data_name) #remove special characters & replace with _
data_extension <- gsub("(.*\\.)([^.]*$)", "\\2", data_name)
data_name <- gsub("\\.[^.]*$", "", data_name) #remove extension
Expand Down
3 changes: 1 addition & 2 deletions R/download_d1_data.R
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
#' @seealso [read_d1_files()] [download_d1_data_pkg()]
#'
#' @examples
#' \dontest{
#' \donttest{
#' download_d1_data("urn:uuid:a2834e3e-f453-4c2b-8343-99477662b570", path = tempdir())
#' download_d1_data(
#' "https://cn.dataone.org/cn/v2/resolve/urn:uuid:a2834e3e-f453-4c2b-8343-99477662b570",
Expand All @@ -31,7 +31,6 @@
#' }

download_d1_data <- function(data_url, path) {
# TODO: add meta_doi to explicitly specify doi

# Silence visible bindings note
entity_data <- eml <- dir_name <- NULL
Expand Down
2 changes: 1 addition & 1 deletion R/download_d1_data_pkg.R
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
#' @examples
#' \donttest{
#' download_d1_data_pkg("doi:10.18739/A2CJ87M3J", tempdir())
#' download_d1_data_pkg("https://doi.org/10.18739/A2CJ87M3J, tempdir())
#' download_d1_data_pkg("https://doi.org/10.18739/A2CJ87M3J", tempdir())
#' }

download_d1_data_pkg <- function(meta_obj, path) {
Expand Down
5 changes: 4 additions & 1 deletion R/tabularize_eml.R
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,10 @@
#' @export
#'
#' @examples
#' eml <- system.file("extdata", "test_data", "SoilMois2012_2017__full_metadata.xml", package = "metajam")
#' eml <- system.file("extdata",
#' "test_data",
#' "SoilMois2012_2017__full_metadata.xml",
#' package = "metajam")
#' tabularize_eml(eml)

tabularize_eml <- function(eml, full = FALSE) {
Expand Down
2 changes: 1 addition & 1 deletion R/utils.R
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
`%|||%` <- function (x, y) {
#based on the purrr/rlang op-null-default
if (is.null(x) || is.na(x)) {
if (is.null(x) || length(x) == 0 || is.na(x)) {
y
}
else {
Expand Down
6 changes: 5 additions & 1 deletion cran-comments.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,8 @@

0 errors | 0 warnings | 1 note

* This is a new release.
* Archived on 2025-12-04 as requires archived package 'dataone'

This is a new release

Addresses previous submission comments about /dontrun and using temporary folders
4 changes: 2 additions & 2 deletions man/check_version.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions man/download_d1_data.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions man/download_d1_data_pkg.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 5 additions & 3 deletions man/tabularize_eml.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

63 changes: 16 additions & 47 deletions vignettes/use02_dataset-single-dataone.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ knitr::opts_chunk$set(collapse = TRUE, comment = "#>")
This vignette aims to showcase a use case using the 2 main functions of `metajam` - `download_d1_data` and `read_d1_files` to download one dataset from the DataOne data repository.


## Note on data url provenance when using download_d1_data.R
## Note on data url provenance when using `download_d1_data()`

There are two parameters required to run the download_d1_data.R function in metajam. One is the data url for the dataset you'd like to download.You can retrieve this by navigating to the data package of interest, right-clicking on the download data button, and selecting Copy Link Address.
There are two parameters required to run the `download_d1_data()` function in metajam. One is the data url for the dataset you'd like to download.You can retrieve this by navigating to the data package of interest, right-clicking on the download data button, and selecting "Copy Link Address".

For several DataOne member nodes (Arctic Data Center, Environmental Data Initiative, and The Knowledge Network for Biocomplexity), metajam users can retrieve the data url from either the 'home' site of the member node or the from the DataOne instance of that same data package. For example, if you wanted to download this dataset:

Expand All @@ -40,30 +40,29 @@ We have not tested metajam's compatibility with the home sites of all DataOne me

We include two examples, one downloading a dataset with metadata in eml (ecological metadata format) and the other downloading a dataset with metadata in ISO (International Organization for Standardization) format.

## Example 1: eml
## Example 1: EML metadata

For the first example, we are using Diatom Community Data from Coweeta LTER, 2005-2019: Kelsey J. Solomon, Rebecca J. Bixby, and Catherine M. Pringle. Environmental Data Initiative. <https://pasta.lternet.edu/package/metadata/eml/edi/858/1>.


## Libraries and constants
### Libraries and constants

```{r libraries, warning=FALSE}
# devtools::install_github("NCEAS/metajam")
library(metajam)

```

```{r constants}
# Directory to save the data set
path_folder <- "Data_coweeta"
path_folder <- file.path(tempdir(),"Data_coweeta")

# URL to download the dataset from DataONE
data_url <- "https://cn.dataone.org/cn/v2/resolve/https%3A%2F%2Fpasta.lternet.edu%2Fpackage%2Fdata%2Feml%2Fedi%2F858%2F1%2F15ad768241d2eeed9f0ba159c2ab8fd5"

```


## Download the dataset
### Download the dataset

```{r download, eval=FALSE}

Expand All @@ -73,62 +72,55 @@ dir.create(path_folder, showWarnings = FALSE)
# Download the dataset and associated metdata
data_folder <- metajam::download_d1_data(data_url, path_folder)



data_folder
```

At this point, you should have the data and the metadata downloaded inside your main directory; `Data_coweeta` in this example. `metajam` organize the files as follow:

- Each dataset is stored a sub-directory named after the package DOI and the file name
- Inside this sub-directory, you will find
- the data: `my_data.csv`
- the data: `CWT_Hemlock_Diatom_Data.csv`
- the raw EML with the naming convention _file name_ + `__full_metadata.xml`: `my_data__full_metadata.xml`
- the package level metadata summary with the naming convention _file name_ + `__summary_metadata.csv`: `my_data__summary_metadata.csv`
- If relevant, the attribute level metadata with the naming convention _file name_ + `__attribute_metadata.csv`: `my_data__attribute_metadata.csv`
- If relevant, the factor level metadata with the naming convention _file name_ + `__attribute_factor_metadata.csv`: my_data`__attribute_factor_metadata.csv`


```{r, out.width="90%", echo=FALSE, fig.align="center", fig.cap="Local file structure of a dataset downloaded by metajam"}
knitr::include_graphics("../man/figures/metajam_v1_folder.png")
```


## Read the data and metadata in your R environment
### Read the data and metadata in your R environment

```{r read_data, eval=FALSE}
# Read all the datasets and their associated metadata in as a named list
coweeta_diatom <- metajam::read_d1_files(data_folder)

```

## Structure of the named list object
### Structure of the named list object

You have now loaded in your R environment one named list object that contains the data `coweeta_diatom$data`, the general (summary) metadata `coweeta_diatom$summary_metadata` - such as title, creators, dates, locations - and the attribute level metadata information `coweeta_diatom$attribute_metadata`, allowing user to get more information, such as units and definitions of your attributes.

## Example 2: iso


## Example 2: ISO metadata

For the second example, we are using Marine bird survey observation and density data from Northern Gulf of Alaska LTER cruises, 2018. Kathy Kuletz, Daniel Cushing, and Elizabeth Labunski. Research Workspace. <https://doi.org/10.24431/rw1k45w>


## Libraries and constants
### Libraries and constants

```{r libraries-2, warning=FALSE}
# devtools::install_github("NCEAS/metajam")
library(metajam)

```

```{r constants-2}
# Directory to save the data set
path_folder <- "Data_alaska"
path_folder <- file.path(tempdir(), "Data_alaska")

# URL to download the dataset from DataONE
data_url <- "https://cn.dataone.org/cn/v2/resolve/4139539e-94e7-49cc-9c7a-5f879e438b16"

```


## Download the dataset
### Download the dataset

```{r download-2, eval=FALSE}

Expand All @@ -138,8 +130,6 @@ dir.create(path_folder, showWarnings = FALSE)
# Download the dataset and associated metdata
data_folder <- metajam::download_d1_data(data_url, path_folder)



```

At this point, you should have the data and the metadata downloaded inside your main directory; `Data_alaska` in this example. `metajam` organize the files as follow:
Expand All @@ -151,24 +141,3 @@ At this point, you should have the data and the metadata downloaded inside your
- the package level metadata summary with the naming convention _file name_ + `__summary_metadata.csv`: `my_data__summary_metadata.csv`


```{r, out.width="90%", echo=FALSE, fig.align="center", fig.cap="Local file structure of a dataset downloaded by metajam"}
knitr::include_graphics("../man/figures/metajam_v1_folder.png")
```


## Read the data and metadata in your R environment

```{r read_data-2, eval=FALSE}
# Read all the datasets and their associated metadata in as a named list
coweeta_diatom <- metajam::read_d1_files(data_folder)

```

## Structure of the named list object

You have now loaded in your R environment one named list object that contains the data `coweeta_diatom$data`, the general (summary) metadata `coweeta_diatom$summary_metadata` - such as title, creators, dates, locations - and the attribute level metadata information `coweeta_diatom$attribute_metadata`, allowing user to get more information, such as units and definitions of your attributes.


```{r, out.width="90%", echo=FALSE, fig.align="center", fig.cap="Structure of the named list object containing tabular metadata and data as loaded by metajam"}
knitr::include_graphics("../man/figures/metajam_v1_named_list.png")
```
2 changes: 1 addition & 1 deletion vignettes/use03_dataset-batch-processing.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ library(stringr)

```{r constants}
# Download the data from DataONE on your local machine
data_folder <- "Data_SEC"
data_folder <- file.path(tempdir(), "Data_SEC")

# Ammonium to Ammoniacal-nitrogen conversion. We will use this conversion later.
coeff_conv_NH4_to_NH4N <- 0.7764676534
Expand Down
Loading