Data Assistant
A Data Assistant is a pre-configured utility that simplifies the creation of ExpectationsA verifiable assertion about data.. A Data Assistant can help you determine a starting point when working with a large, new, or complex dataset by asking questions and then building a list of relevant MetricsA computed attribute of data such as the mean of a column. from the answers to those questions. Branching question paths based on your responses ensure that additional, relevant Metrics are not missed. The result is a comprehensive collection of Metrics that can be saved, reviewed as graphical plots, or used by the Data Assistant to generate a set of proposed Expectations.
Data Assistants allow you to introspect multiple BatchesA selection of records from a Data Asset. and create an Expectation SuiteA collection of verifiable assertions about data. from the aggregated Metrics of those Batches. They provide convenient, visual representations of the generated Expectations to assist with identifying outliers in the corresponding parameters. They can be accessed from your Data ContextThe primary entry point for a Great Expectations deployment, with configurations and methods for all supporting components., and provide a good starting point for building Expectations or performing initial data exploration.
Relationships to other objects
A Data Assistant implements a pre-configured ProfilerGenerates Metrics and candidate Expectations from data. in order to gather Metrics and propose an Expectation Suite based on the introspection of the Batch or Batches contained in a provided Batch RequestProvided to a Datasource in order to create a Batch..
Use cases
Data Assistants are an ideal starting point for creating Expectations. When you're working with unfamiliar data, a Data Assistant can provide an overview by introspecting the data and generating a series of relevant Expectations using estimated parameters for you to review. When you use the "flag_outliers" value for the estimation parameter, your generated Expectations have parameters that disregard values that the Data Assistant identifies as outliers. To create a graphical representation of the generated Expectations, use the Data Assistant's plot_metrics() method. Reviewing the Data Assistant's results can help you identify outliers in your data.
When you're working with familiar, good data, a Data Assistant can use the "exact" value for the estimation parameter to provide comprehensive Expectations that reflect the values found in the provided data.
Profiling
To provide a representative analysis of the provided data, a Data Assistant can automatically process multiple Batches from a single Batch Request.
Multi-Batch introspection
Data Assistants leverage the ability to process multiple Batches from a single Batch Request to provide a representative analysis of the provided data. With previous Profilers you would only be able to introspect a single Batch at a time. This meant that the Expectation Suite generated would only reflect a single Batch. If you had many Batches of data that you wanted to build inter-related Expectations for, you would have needed to run each Batch individually and then manually compare and update the Expectation parameters that were generated. With a Data Assistant, that process is automated. You can provide a Data Assistant multiple Batches and get back Expectations that have parameters based on, for instance, the mean or median value of a column on a per-Batch basis.
Visual plots for Metrics
When working in a Jupyter Notebook you can use the plot_metrics() method of a Data Assistant's result object to generate a visual representation of your Expectations, the values that were assigned to their parameters, and the Metrics that informed those values. This assists in exploratory data analysis and fine-tuning your Expectations, while providing complete transparency into the information used by the Data Assistant to build your Expectations.
Data Assistants can be accessed from your Data Context. To select a Data Assistant in a Jupyter Notebook, enter context.assistants. and use code completion. All Data Assistants have a run(...) method that takes in a Batch Request and numerous optional parameters, and then loads the results into an Expectation Suite for future use.
To access the Onboarding Data Assistant, use context.assistants.onboarding.
Configure
Data Assistants are pre-configured. You provide the Batch Request, and some optional parameters in the Data Assistant's run(...) method.
Related documentation
To learn more about working with the Onboarding Data Assistant, see How to create an Expectation Suite with the Onboarding Data Assistant.