Decision rule

Formal definition

Examples of decision rules

• An estimator is a decision rule used for estimating a parameter. In this case the set of actions is the parameter space, and a loss function details the cost of the discrepancy between the true value of the parameter and the estimated value. For example, in a linear model with a single scalar parameter, the domain of may extend over . An associated decision rule for estimating from some observed data might be, "choose the value of the, say, that minimizes the sum of squared error between some observed responses and responses predicted from the corresponding covariates given that you chose." Thus, the cost function is the sum of squared error, and one would aim to minimize this cost. Once the cost function is defined, could be chosen, for instance, using some optimization algorithm.
• Out of sample prediction in regression and classification models.