Copyright	(c) 2013-2023 Brendan Hay
License	Mozilla Public License, v. 2.0.
Maintainer	Brendan Hay
Stability	auto-generated
Portability	non-portable (GHC extensions)
Safe Haskell	Safe-Inferred
Language	Haskell2010

Amazonka.Evidently.GetExperimentResults

Contents

Creating a Request
Request Lenses
Destructuring the Response
Response Lenses

Description

Retrieves the results of a running or completed experiment. No results are available until there have been 100 events for each variation and at least 10 minutes have passed since the start of the experiment. To increase the statistical power, Evidently performs an additional offline p-value analysis at the end of the experiment. Offline p-value analysis can detect statistical significance in some cases where the anytime p-values used during the experiment do not find statistical significance.

Experiment results are available up to 63 days after the start of the experiment. They are not available after that because of CloudWatch data retention policies.

Synopsis

data GetExperimentResults = GetExperimentResults' {
- baseStat :: Maybe ExperimentBaseStat
- endTime :: Maybe POSIX
- period :: Maybe Natural
- reportNames :: Maybe [ExperimentReportName]
- resultStats :: Maybe [ExperimentResultRequestType]
- startTime :: Maybe POSIX
- experiment :: Text
- metricNames :: NonEmpty Text
- project :: Text
- treatmentNames :: NonEmpty Text
}
newGetExperimentResults :: Text -> NonEmpty Text -> Text -> NonEmpty Text -> GetExperimentResults
getExperimentResults_baseStat :: Lens' GetExperimentResults (Maybe ExperimentBaseStat)
getExperimentResults_endTime :: Lens' GetExperimentResults (Maybe UTCTime)
getExperimentResults_period :: Lens' GetExperimentResults (Maybe Natural)
getExperimentResults_reportNames :: Lens' GetExperimentResults (Maybe [ExperimentReportName])
getExperimentResults_resultStats :: Lens' GetExperimentResults (Maybe [ExperimentResultRequestType])
getExperimentResults_startTime :: Lens' GetExperimentResults (Maybe UTCTime)
getExperimentResults_experiment :: Lens' GetExperimentResults Text
getExperimentResults_metricNames :: Lens' GetExperimentResults (NonEmpty Text)
getExperimentResults_project :: Lens' GetExperimentResults Text
getExperimentResults_treatmentNames :: Lens' GetExperimentResults (NonEmpty Text)
data GetExperimentResultsResponse = GetExperimentResultsResponse' {
- details :: Maybe Text
- reports :: Maybe [ExperimentReport]
- resultsData :: Maybe [ExperimentResultsData]
- timestamps :: Maybe [POSIX]
- httpStatus :: Int
}
newGetExperimentResultsResponse :: Int -> GetExperimentResultsResponse
getExperimentResultsResponse_details :: Lens' GetExperimentResultsResponse (Maybe Text)
getExperimentResultsResponse_reports :: Lens' GetExperimentResultsResponse (Maybe [ExperimentReport])
getExperimentResultsResponse_resultsData :: Lens' GetExperimentResultsResponse (Maybe [ExperimentResultsData])
getExperimentResultsResponse_timestamps :: Lens' GetExperimentResultsResponse (Maybe [UTCTime])
getExperimentResultsResponse_httpStatus :: Lens' GetExperimentResultsResponse Int

Creating a Request

data GetExperimentResults Source #

See: newGetExperimentResults smart constructor.

Constructors

GetExperimentResults'

Fields

baseStat :: Maybe ExperimentBaseStat
The statistic used to calculate experiment results. Currently the only valid value is mean, which uses the mean of the collected values as the statistic.
endTime :: Maybe POSIX
The date and time that the experiment ended, if it is completed. This must be no longer than 30 days after the experiment start time.
period :: Maybe Natural
In seconds, the amount of time to aggregate results together.
reportNames :: Maybe [ExperimentReportName]
The names of the report types that you want to see. Currently, BayesianInference is the only valid value.
resultStats :: Maybe [ExperimentResultRequestType]
The statistics that you want to see in the returned results.
- PValue specifies to use p-values for the results. A p-value is used in hypothesis testing to measure how often you are willing to make a mistake in rejecting the null hypothesis. A general practice is to reject the null hypothesis and declare that the results are statistically significant when the p-value is less than 0.05.
- ConfidenceInterval specifies a confidence interval for the results. The confidence interval represents the range of values for the chosen metric that is likely to contain the true difference between the baseStat of a variation and the baseline. Evidently returns the 95% confidence interval.
- TreatmentEffect is the difference in the statistic specified by the baseStat parameter between each variation and the default variation.
- BaseStat returns the statistical values collected for the metric for each variation. The statistic uses the same statistic specified in the baseStat parameter. Therefore, if baseStat is mean, this returns the mean of the values collected for each variation.
startTime :: Maybe POSIX
The date and time that the experiment started.
experiment :: Text
The name of the experiment to retrieve the results of.
metricNames :: NonEmpty Text
The names of the experiment metrics that you want to see the results of.
project :: Text
The name or ARN of the project that contains the experiment that you want to see the results of.
treatmentNames :: NonEmpty Text
The names of the experiment treatments that you want to see the results for.

Instances

Instances details

ToJSON GetExperimentResults Source #
Instance details Defined in Amazonka.Evidently.GetExperimentResults Methods toJSON :: GetExperimentResults -> Value # toEncoding :: GetExperimentResults -> Encoding # toJSONList :: [GetExperimentResults] -> Value # toEncodingList :: [GetExperimentResults] -> Encoding #
ToHeaders GetExperimentResults Source #
Instance details Defined in Amazonka.Evidently.GetExperimentResults Methods toHeaders :: GetExperimentResults -> [Header] #
ToPath GetExperimentResults Source #
Instance details Defined in Amazonka.Evidently.GetExperimentResults Methods toPath :: GetExperimentResults -> ByteString #
ToQuery GetExperimentResults Source #
Instance details Defined in Amazonka.Evidently.GetExperimentResults Methods toQuery :: GetExperimentResults -> QueryString #
AWSRequest GetExperimentResults Source #
Instance details Defined in Amazonka.Evidently.GetExperimentResults Associated Types type AWSResponse GetExperimentResults # Methods request :: (Service -> Service) -> GetExperimentResults -> Request GetExperimentResults # response :: MonadResource m => (ByteStringLazy -> IO ByteStringLazy) -> Service -> Proxy GetExperimentResults -> ClientResponse ClientBody -> m (Either Error (ClientResponse (AWSResponse GetExperimentResults))) #
Generic GetExperimentResults Source #
Instance details Defined in Amazonka.Evidently.GetExperimentResults Associated Types type Rep GetExperimentResults :: Type -> Type # Methods from :: GetExperimentResults -> Rep GetExperimentResults x # to :: Rep GetExperimentResults x -> GetExperimentResults #
Read GetExperimentResults Source #
Instance details Defined in Amazonka.Evidently.GetExperimentResults Methods readsPrec :: Int -> ReadS GetExperimentResults # readList :: ReadS [GetExperimentResults] # readPrec :: ReadPrec GetExperimentResults # readListPrec :: ReadPrec [GetExperimentResults] #
Show GetExperimentResults Source #
Instance details Defined in Amazonka.Evidently.GetExperimentResults Methods showsPrec :: Int -> GetExperimentResults -> ShowS # show :: GetExperimentResults -> String # showList :: [GetExperimentResults] -> ShowS #
NFData GetExperimentResults Source #
Instance details Defined in Amazonka.Evidently.GetExperimentResults Methods rnf :: GetExperimentResults -> () #
Eq GetExperimentResults Source #
Instance details Defined in Amazonka.Evidently.GetExperimentResults Methods (==) :: GetExperimentResults -> GetExperimentResults -> Bool # (/=) :: GetExperimentResults -> GetExperimentResults -> Bool #
Hashable GetExperimentResults Source #
Instance details Defined in Amazonka.Evidently.GetExperimentResults Methods hashWithSalt :: Int -> GetExperimentResults -> Int # hash :: GetExperimentResults -> Int #
type AWSResponse GetExperimentResults Source #
Instance details Defined in Amazonka.Evidently.GetExperimentResults type AWSResponse GetExperimentResults = GetExperimentResultsResponse
type Rep GetExperimentResults Source #
Instance details Defined in Amazonka.Evidently.GetExperimentResults type Rep GetExperimentResults = D1 ('MetaData "GetExperimentResults" "Amazonka.Evidently.GetExperimentResults" "amazonka-evidently-2.0-Es2fJk98R0DAUPYaVkZSYC" 'False) (C1 ('MetaCons "GetExperimentResults'" 'PrefixI 'True) (((S1 ('MetaSel ('Just "baseStat") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe ExperimentBaseStat)) :: S1 ('MetaSel ('Just "endTime") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe POSIX))) :: (S1 ('MetaSel ('Just "period") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Natural)) :: (S1 ('MetaSel ('Just "reportNames") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe [ExperimentReportName])) :: S1 ('MetaSel ('Just "resultStats") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe [ExperimentResultRequestType]))))) :: ((S1 ('MetaSel ('Just "startTime") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe POSIX)) :: S1 ('MetaSel ('Just "experiment") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 Text)) :: (S1 ('MetaSel ('Just "metricNames") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (NonEmpty Text)) :: (S1 ('MetaSel ('Just "project") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 Text) :*: S1 ('MetaSel ('Just "treatmentNames") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (NonEmpty Text)))))))

newGetExperimentResults Source #

Arguments

:: Text	`$sel:experiment:GetExperimentResults'`
-> NonEmpty Text	`$sel:metricNames:GetExperimentResults'`
-> Text	`GetExperimentResults`
-> NonEmpty Text	`$sel:treatmentNames:GetExperimentResults'`
-> GetExperimentResults

Create a value of GetExperimentResults with all optional fields omitted.

Use generic-lens or optics to modify other optional fields.

The following record fields are available, with the corresponding lenses provided for backwards compatibility:

$sel:baseStat:GetExperimentResults', getExperimentResults_baseStat - The statistic used to calculate experiment results. Currently the only valid value is mean, which uses the mean of the collected values as the statistic.

GetExperimentResults, getExperimentResults_endTime - The date and time that the experiment ended, if it is completed. This must be no longer than 30 days after the experiment start time.

$sel:period:GetExperimentResults', getExperimentResults_period - In seconds, the amount of time to aggregate results together.

$sel:reportNames:GetExperimentResults', getExperimentResults_reportNames - The names of the report types that you want to see. Currently, BayesianInference is the only valid value.

$sel:resultStats:GetExperimentResults', getExperimentResults_resultStats - The statistics that you want to see in the returned results.

PValue specifies to use p-values for the results. A p-value is used in hypothesis testing to measure how often you are willing to make a mistake in rejecting the null hypothesis. A general practice is to reject the null hypothesis and declare that the results are statistically significant when the p-value is less than 0.05.
ConfidenceInterval specifies a confidence interval for the results. The confidence interval represents the range of values for the chosen metric that is likely to contain the true difference between the baseStat of a variation and the baseline. Evidently returns the 95% confidence interval.
TreatmentEffect is the difference in the statistic specified by the baseStat parameter between each variation and the default variation.
BaseStat returns the statistical values collected for the metric for each variation. The statistic uses the same statistic specified in the baseStat parameter. Therefore, if baseStat is mean, this returns the mean of the values collected for each variation.

GetExperimentResults, getExperimentResults_startTime - The date and time that the experiment started.

$sel:experiment:GetExperimentResults', getExperimentResults_experiment - The name of the experiment to retrieve the results of.

$sel:metricNames:GetExperimentResults', getExperimentResults_metricNames - The names of the experiment metrics that you want to see the results of.

GetExperimentResults, getExperimentResults_project - The name or ARN of the project that contains the experiment that you want to see the results of.

$sel:treatmentNames:GetExperimentResults', getExperimentResults_treatmentNames - The names of the experiment treatments that you want to see the results for.

Request Lenses

getExperimentResults_baseStat :: Lens' GetExperimentResults (Maybe ExperimentBaseStat) Source #

The statistic used to calculate experiment results. Currently the only valid value is mean, which uses the mean of the collected values as the statistic.

getExperimentResults_endTime :: Lens' GetExperimentResults (Maybe UTCTime) Source #

The date and time that the experiment ended, if it is completed. This must be no longer than 30 days after the experiment start time.

getExperimentResults_period :: Lens' GetExperimentResults (Maybe Natural) Source #

In seconds, the amount of time to aggregate results together.

getExperimentResults_reportNames :: Lens' GetExperimentResults (Maybe [ExperimentReportName]) Source #

The names of the report types that you want to see. Currently, BayesianInference is the only valid value.

getExperimentResults_resultStats :: Lens' GetExperimentResults (Maybe [ExperimentResultRequestType]) Source #

The statistics that you want to see in the returned results.

PValue specifies to use p-values for the results. A p-value is used in hypothesis testing to measure how often you are willing to make a mistake in rejecting the null hypothesis. A general practice is to reject the null hypothesis and declare that the results are statistically significant when the p-value is less than 0.05.
ConfidenceInterval specifies a confidence interval for the results. The confidence interval represents the range of values for the chosen metric that is likely to contain the true difference between the baseStat of a variation and the baseline. Evidently returns the 95% confidence interval.
TreatmentEffect is the difference in the statistic specified by the baseStat parameter between each variation and the default variation.
BaseStat returns the statistical values collected for the metric for each variation. The statistic uses the same statistic specified in the baseStat parameter. Therefore, if baseStat is mean, this returns the mean of the values collected for each variation.

getExperimentResults_startTime :: Lens' GetExperimentResults (Maybe UTCTime) Source #

The date and time that the experiment started.

getExperimentResults_experiment :: Lens' GetExperimentResults Text Source #

The name of the experiment to retrieve the results of.

getExperimentResults_metricNames :: Lens' GetExperimentResults (NonEmpty Text) Source #

The names of the experiment metrics that you want to see the results of.

getExperimentResults_project :: Lens' GetExperimentResults Text Source #

The name or ARN of the project that contains the experiment that you want to see the results of.

getExperimentResults_treatmentNames :: Lens' GetExperimentResults (NonEmpty Text) Source #

The names of the experiment treatments that you want to see the results for.

Destructuring the Response

data GetExperimentResultsResponse Source #

See: newGetExperimentResultsResponse smart constructor.

Constructors

GetExperimentResultsResponse'

Fields

details :: Maybe Text
If the experiment doesn't yet have enough events to provide valid results, this field is returned with the message Not enough events to generate results. If there are enough events to provide valid results, this field is not returned.
reports :: Maybe [ExperimentReport]
An array of structures that include the reports that you requested.
resultsData :: Maybe [ExperimentResultsData]
An array of structures that include experiment results including metric names and values.
timestamps :: Maybe [POSIX]
The timestamps of each result returned.
httpStatus :: Int
The response's http status code.

Instances

Instances details

Generic GetExperimentResultsResponse Source #
Instance details Defined in Amazonka.Evidently.GetExperimentResults Associated Types type Rep GetExperimentResultsResponse :: Type -> Type # Methods from :: GetExperimentResultsResponse -> Rep GetExperimentResultsResponse x # to :: Rep GetExperimentResultsResponse x -> GetExperimentResultsResponse #
Read GetExperimentResultsResponse Source #
Instance details Defined in Amazonka.Evidently.GetExperimentResults Methods readsPrec :: Int -> ReadS GetExperimentResultsResponse # readList :: ReadS [GetExperimentResultsResponse] # readPrec :: ReadPrec GetExperimentResultsResponse # readListPrec :: ReadPrec [GetExperimentResultsResponse] #
Show GetExperimentResultsResponse Source #
Instance details Defined in Amazonka.Evidently.GetExperimentResults Methods showsPrec :: Int -> GetExperimentResultsResponse -> ShowS # show :: GetExperimentResultsResponse -> String # showList :: [GetExperimentResultsResponse] -> ShowS #
NFData GetExperimentResultsResponse Source #
Instance details Defined in Amazonka.Evidently.GetExperimentResults Methods rnf :: GetExperimentResultsResponse -> () #
Eq GetExperimentResultsResponse Source #
Instance details Defined in Amazonka.Evidently.GetExperimentResults Methods (==) :: GetExperimentResultsResponse -> GetExperimentResultsResponse -> Bool # (/=) :: GetExperimentResultsResponse -> GetExperimentResultsResponse -> Bool #
type Rep GetExperimentResultsResponse Source #
Instance details Defined in Amazonka.Evidently.GetExperimentResults type Rep GetExperimentResultsResponse = D1 ('MetaData "GetExperimentResultsResponse" "Amazonka.Evidently.GetExperimentResults" "amazonka-evidently-2.0-Es2fJk98R0DAUPYaVkZSYC" 'False) (C1 ('MetaCons "GetExperimentResultsResponse'" 'PrefixI 'True) ((S1 ('MetaSel ('Just "details") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Text)) :: S1 ('MetaSel ('Just "reports") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe [ExperimentReport]))) :: (S1 ('MetaSel ('Just "resultsData") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe [ExperimentResultsData])) :: (S1 ('MetaSel ('Just "timestamps") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe [POSIX])) :: S1 ('MetaSel ('Just "httpStatus") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 Int)))))

newGetExperimentResultsResponse Source #

Arguments

:: Int	`$sel:httpStatus:GetExperimentResultsResponse'`
-> GetExperimentResultsResponse

Create a value of GetExperimentResultsResponse with all optional fields omitted.

Use generic-lens or optics to modify other optional fields.

The following record fields are available, with the corresponding lenses provided for backwards compatibility:

GetExperimentResultsResponse, getExperimentResultsResponse_details - If the experiment doesn't yet have enough events to provide valid results, this field is returned with the message Not enough events to generate results. If there are enough events to provide valid results, this field is not returned.

$sel:reports:GetExperimentResultsResponse', getExperimentResultsResponse_reports - An array of structures that include the reports that you requested.

$sel:resultsData:GetExperimentResultsResponse', getExperimentResultsResponse_resultsData - An array of structures that include experiment results including metric names and values.

$sel:timestamps:GetExperimentResultsResponse', getExperimentResultsResponse_timestamps - The timestamps of each result returned.

$sel:httpStatus:GetExperimentResultsResponse', getExperimentResultsResponse_httpStatus - The response's http status code.

Response Lenses

getExperimentResultsResponse_details :: Lens' GetExperimentResultsResponse (Maybe Text) Source #

If the experiment doesn't yet have enough events to provide valid results, this field is returned with the message Not enough events to generate results. If there are enough events to provide valid results, this field is not returned.

getExperimentResultsResponse_reports :: Lens' GetExperimentResultsResponse (Maybe [ExperimentReport]) Source #

An array of structures that include the reports that you requested.

getExperimentResultsResponse_resultsData :: Lens' GetExperimentResultsResponse (Maybe [ExperimentResultsData]) Source #

An array of structures that include experiment results including metric names and values.

getExperimentResultsResponse_timestamps :: Lens' GetExperimentResultsResponse (Maybe [UTCTime]) Source #

The timestamps of each result returned.

getExperimentResultsResponse_httpStatus :: Lens' GetExperimentResultsResponse Int Source #

The response's http status code.