Generate an assessment report and summary of a dataset
Source:R/07-data_summarize.R
dataset_summarize.RdAssesses and summarizes the content and structure of a dataset and generates reports of the results. This function can be used to evaluate data structure, presence of specific fields, coherence across elements, and data dictionary formats, and to summarize additional information about variable distributions and descriptive statistics.
dataset_summarize(
dataset,
data_dict = data_dict_extract(dataset),
group_by = NULL,
taxonomy = NULL,
dataset_name = .dataset_name,
valueType_guess = FALSE,
.dataset_name = NULL
)Arguments
- dataset
A dataset object.
- data_dict
A list of data frame(s) representing metadata of the input dataset. Automatically generated if not provided.
- group_by
A character string identifying the column in the dataset to use as a grouping variable. Elements will be grouped by this column.
- taxonomy
An optional data frame identifying a variable classification schema.
- dataset_name
A character string specifying the name of the dataset (internally used in the function
dossier_evaluate()).- valueType_guess
Whether the output should include a more accurate valueType that could be applied to the dataset. FALSE by default.
- .dataset_name
![[Deprecated]](figures/lifecycle-deprecated.svg)
Value
A list of data frames containing assessment reports and summaries.
Details
A data dictionary contains the list of variables in a dataset and metadata
about the variables and can be associated with a dataset. A data dictionary
object is a list of data frame(s) named 'Variables' (required) and
'Categories' (if any). To be usable in any function, the data frame
'Variables' must contain at least the name column, with all unique and
non-missing entries, and the data frame 'Categories' must contain at least
the variable and name columns, with unique combination of
variable and name. The function truncates each cell to a maximum of
10000 characters, to be readable and compatible with Excel.
A dataset is a data table containing variables. A dataset object is a data frame and can be associated with a data dictionary. If no data dictionary is provided with a dataset, a minimum workable data dictionary will be generated as needed within relevant functions. Identifier variable(s) for indexing can be specified by the user. The id values must be non-missing and will be used in functions that require it. If no identifier variable is specified, indexing is handled automatically by the function.
A taxonomy is a classification schema that can be defined for variable
attributes. A taxonomy is usually extracted from an
Opal environment, and a
taxonomy object is a data frame that must contain at least the columns
taxonomy, vocabulary, and terms. Additional details about Opal
taxonomies are
available online.
The valueType is a declared property of a variable that is required in certain functions to determine handling of the variables. Specifically, valueType refers to the OBiBa data type of a variable. The valueType is specified in a data dictionary in a column 'valueType' and can be associated with variables as attributes. Acceptable valueTypes include 'text', 'integer', 'decimal', 'boolean', datetime', 'date'. The full list of OBiBa valueType possibilities and their correspondence with R data types are available using valueType_list. The valueType can be used to coerce the variable to the corresponding data type.
See also
Examples
{
# use madshapR_DEMO provided by the package
library(dplyr)
#' ###### Example : Any data frame can be summarized
dataset <- iris['Sepal.Width']
glimpse(dataset_summarize(dataset))
}
#> - DATA DICTIONARY ASSESSMENT: data_dict --------------
#> Assess the standard adequacy of naming
#> Assess the uniqueness of variable names
#> Assess the presence of possible duplicated columns
#> Assess the presence of empty rows in the data dictionary
#> Assess the presence of empty columns in the data dictionary
#> Assess the completion of `label(:xx)` column in 'Variables'
#> Assess the `valueType` column in 'Variables'
#> Generate report
#>
#> The data dictionary contains no error/warning.
#>
#> - WARNING MESSAGES (if any): --------------------------------------------
#>
#> - DATASET ASSESSMENT: dataset --------------------------
#> Assess the standard adequacy of naming
#> Assess the presence of variable names both in dataset and data dictionary
#> Assess the presence of possible duplicated variable in the dataset
#> Assess the presence of duplicated participants in the dataset
#> Assess the presence of unique value columns in dataset
#> Assess the presence of empty rows in the data dictionary
#> Assess the presence all NA(s) of columns in the data dictionary
#> Assess the presence of categories not in the data dictionary
#> Generate report
#>
#> - WARNING MESSAGES (if any): -------------------------------------------------
#>
#> - DATASET SUMMARIZE: dataset --------------------------
#> Summarize the data type of each variable across the dataset
#> Summarize information for all variables
#> Summarize information for numerical variables
#> Summarize information for text variables
#> Summarize information for date variables
#> Summarize information for datetime variables
#> Summarize information for categorical variables
#> Summarize global information (Overview)
#> Generate report
#> List of 4
#> $ Overview : tibble [15 × 2] (S3: tbl_df/tbl/data.frame)
#> ..$ Quality control of dataset: chr [1:15] "Date" "Name of the dataset" " Identifier Variable" " Variables" ...
#> ..$ (all) : chr [1:15] "2024-04-23" "dataset" "" " " ...
#> $ Dataset assessment : tibble [18 × 4] (S3: tbl_df/tbl/data.frame)
#> ..$ index in data dict. : chr [1:18] NA NA NA NA ...
#> ..$ name : chr [1:18] NA NA NA NA ...
#> ..$ Quality assessment comment: chr [1:18] "[INFO] - possible duplicated row values" "[INFO] - possible duplicated row values" "[INFO] - possible duplicated row values" "[INFO] - possible duplicated row values" ...
#> ..$ value : chr [1:18] "63 ; 69 ; 120" "42 ; 54 ; 88 ; 94" "58 ; 81 ; 82" "70 ; 73 ; 90 ; 99 ; 107 ; [...]" ...
#> $ Variables summary (all) : tibble [1 × 13] (S3: tbl_df/tbl/data.frame)
#> ..$ index in data dict. : chr "1"
#> ..$ name : chr "Sepal.Width"
#> ..$ Quality assessment comment : chr NA
#> ..$ label : chr "Sepal.Width"
#> ..$ Data Dictionary valueType : chr "decimal"
#> ..$ Estimated dataset valueType : chr "decimal"
#> ..$ Actual dataset valueType : chr "decimal"
#> ..$ Categorical variable : chr "no"
#> ..$ Categories in data dictionary: chr NA
#> ..$ Total number of observations : chr "150"
#> ..$ Nb. distinct values : chr "23"
#> ..$ % total Valid values : chr "100"
#> ..$ % NA : chr "0"
#> $ Numerical variable summary: tibble [1 × 22] (S3: tbl_df/tbl/data.frame)
#> ..$ index in data dict. : chr "1"
#> ..$ name : chr "Sepal.Width"
#> ..$ Quality assessment comment : chr NA
#> ..$ label : chr "Sepal.Width"
#> ..$ Data Dictionary valueType : chr "decimal"
#> ..$ Estimated dataset valueType : chr "decimal"
#> ..$ Actual dataset valueType : chr "decimal"
#> ..$ Categorical variable : chr "no"
#> ..$ Categories in data dictionary : chr NA
#> ..$ Total number of observations : chr "150"
#> ..$ Nb. distinct values : chr "23"
#> ..$ % total Valid values : chr "100"
#> ..$ % NA : chr "0"
#> ..$ % Valid categorical values (if applicable) : chr NA
#> ..$ % Missing categorical values (if applicable): chr NA
#> ..$ MIN : chr "2"
#> ..$ Q1 : chr "2.8"
#> ..$ MEDIAN : chr "3"
#> ..$ Q3 : chr "3.3"
#> ..$ MAX : chr "4.4"
#> ..$ MEAN : chr "3.05733333333333"
#> ..$ STDEV : chr "0.435866284936698"