Assesses the content and structure of a dataset object and generates reports of the results. This function can be used to evaluate data structure, presence of specific fields, coherence across elements, and data dictionary formats.
dataset_evaluate(
dataset,
data_dict = NULL,
is_data_dict_mlstr = TRUE,
valueType_guess = TRUE,
taxonomy = NULL,
dataset_name = NULL
)A dataset object.
A list of data frame(s) representing metadata of the input dataset. Automatically generated if not provided.
Whether the input data dictionary should be coerced with specific format restrictions for compatibility with other Maelstrom Research software. TRUE by default.
Whether the output should include a more accurate valueType that could be applied to the dataset. TRUE by default.
An optional data frame identifying a variable classification schema.
A character string specifying the name of the dataset
(used internally in the function dossier_evaluate()).
A list of data frames containing assessment reports.
A data dictionary contains the list of variables in a dataset and metadata
about the variables and can be associated with a dataset. A data dictionary
object is a list of data frame(s) named 'Variables' (required) and
'Categories' (if any). To be usable in any function, the data frame
'Variables' must contain at least the name column, with all unique and
non-missing entries, and the data frame 'Categories' must contain at least
the variable and name columns, with unique combination of
variable and name. The function truncates each cell to a maximum of
10000 characters, to be readable and compatible with Excel.
A dataset is a data table containing variables. A dataset object is a data frame and can be associated with a data dictionary. If no data dictionary is provided with a dataset, a minimum workable data dictionary will be generated as needed within relevant functions. Identifier variable(s) for indexing can be specified by the user. The id values must be non-missing and will be used in functions that require it. If no identifier variable is specified, indexing is handled automatically by the function.
A taxonomy is a classification schema that can be defined for variable
attributes. A taxonomy is usually extracted from an
Opal environment, and a
taxonomy object is a data frame that must contain at least the columns
taxonomy, vocabulary, and terms. Additional details about Opal
taxonomies are
available online.
The object may be specifically formatted to be compatible with additional Maelstrom Research software, in particular Opal environments.
# \donttest{
library(dplyr)
###### Example 1: use madshapR_examples provided by the package
dataset <- madshapR_examples$`dataset_example - errors with data`
data_dict <- madshapR_examples$`data_dictionary_example - errors with data`
eval_dataset <- dataset_evaluate(dataset, data_dict)
#> - DATA DICTIONARY ASSESSMENT: data_dict --------------
#> Assess the standard adequacy of naming
#> Assess the uniqueness and presence of variable names
#> Assess the presence of possible duplicated columns
#> Assess the presence of duplicated rows
#> Assess the presence of empty rows in the data dictionary
#> Assess the presence of empty column in the data dictionary
#> Assess the presence of categories not in the data dictionary
#> Assess the `valueType` column in 'Variables'
#> Assess the completion of `label:en` column in 'Variables'
#> Assess presence and completion of `label:en` column in 'Categories'
#> Assess the logical values of missing column in Categories
#> Generate report
#>
#> - WARNING MESSAGES (if any): --------------------------------------------
#>
#> - DATASET ASSESSMENT: dataset --------------------------
#> Assess the standard adequacy of naming
#> Assess the presence of variable names both in dataset and data dictionary
#> Assess the presence of possible duplicated variable in the dataset
#> Assess the presence of possible duplicated participants
#> Assess the presence of unique value columns in dataset
#> Assess the presence of empty rows in the dataset
#> Assess the presence all empty variable in the dataset
#> Assess the Categories comparison in dataset and data dictionary
#> Assess the `valueType` comparison in dataset and data dictionary
#> Generate report
#>
#> - WARNING MESSAGES (if any): -------------------------------------------------
#>
glimpse(eval_dataset)
#> List of 2
#> $ Data dictionary summary: tibble [9 × 6] (S3: tbl_df/tbl/data.frame)
#> ..$ Index : chr [1:9] "1" "2" "3" "4" ...
#> ..$ Variable name : chr [1:9] "part_id" "gndr" "height" "weight_ms" ...
#> ..$ Variable label : chr [1:9] "id of the participant" "gndr" "height" "weight_ms" ...
#> ..$ Data dictionary valueType : chr [1:9] "text" "boolean" "decimal" "integer" ...
#> ..$ Categories in data dictionary: chr [1:9] NA "[1] Male\n[2] Female\n[-77] Don’t want to answer" NA "[-88] Don’t want to answer\n[-99] Do not remember" ...
#> ..$ Non-valid categories : chr [1:9] NA "\n\n[-77] Don’t want to answer" NA "[-88] Don’t want to answer\n[-99] Do not remember" ...
#> $ Dataset assessment : tibble [15 × 5] (S3: tbl_df/tbl/data.frame)
#> ..$ Index : chr [1:15] "2" "2" "3" "3" ...
#> ..$ Variable name : chr [1:15] "gndr" "gndr" "height" "height" ...
#> ..$ Dataset assessment : chr [1:15] "[INFO] - Variable is defined as categorical in data dictionary but not in dataset." "[ERROR] - valueType in data dictionary is not compatible with dataset values." "[INFO] - Variable has a constant value." "[INFO] - Suggested valueType." ...
#> ..$ Value : chr [1:15] "1 ; 2 ; -77" "boolean" "191" "decimal" ...
#> ..$ Suggested valueType: chr [1:15] NA "text" NA "integer" ...
###### Example 2: Any data frame can be a dataset by definition
eval_iris <- dataset_evaluate(iris)
#> - DATA DICTIONARY ASSESSMENT: data_dict --------------
#> Assess the standard adequacy of naming
#> Assess the uniqueness and presence of variable names
#> Assess the presence of possible duplicated columns
#> Assess the presence of duplicated rows
#> Assess the presence of empty rows in the data dictionary
#> Assess the presence of empty column in the data dictionary
#> Assess the presence of categories not in the data dictionary
#> Assess the `valueType` column in 'Variables'
#> Assess the completion of `label` column in 'Variables'
#> Assess presence and completion of `label` column in 'Categories'
#> Assess the logical values of missing column in Categories
#> Generate report
#>
#> The data dictionary contains no errors/warnings.
#>
#> - WARNING MESSAGES (if any): --------------------------------------------
#>
#> - DATASET ASSESSMENT: iris --------------------------
#> Assess the standard adequacy of naming
#> Assess the presence of variable names both in dataset and data dictionary
#> Assess the presence of possible duplicated variable in the dataset
#> Assess the presence of possible duplicated participants
#> Assess the presence of unique value columns in dataset
#> Assess the presence of empty rows in the dataset
#> Assess the presence all empty variable in the dataset
#> Assess the Categories comparison in dataset and data dictionary
#> Assess the `valueType` comparison in dataset and data dictionary
#> Generate report
#>
#> - WARNING MESSAGES (if any): -------------------------------------------------
#>
glimpse(eval_iris)
#> List of 2
#> $ Data dictionary summary: tibble [5 × 6] (S3: tbl_df/tbl/data.frame)
#> ..$ Index : chr [1:5] "1" "2" "3" "4" ...
#> ..$ Variable name : chr [1:5] "Sepal.Length" "Sepal.Width" "Petal.Length" "Petal.Width" ...
#> ..$ Variable label : chr [1:5] "Sepal.Length" "Sepal.Width" "Petal.Length" "Petal.Width" ...
#> ..$ Data dictionary valueType : chr [1:5] "decimal" "decimal" "decimal" "decimal" ...
#> ..$ Categories in data dictionary: chr [1:5] NA NA NA NA ...
#> ..$ Non-valid categories : chr [1:5] NA NA NA NA ...
#> $ Dataset assessment : tibble [1 × 4] (S3: tbl_df/tbl/data.frame)
#> ..$ Index : chr NA
#> ..$ Variable name : chr "(all)"
#> ..$ Dataset assessment: chr "[INFO] - Duplicated rows."
#> ..$ Value : chr "102 ; 143"
# }