Use this optimization step or heuristic to study an explicitly defined objective and constraints with reproducible parameters, initialization, and stopping rules.

Parameter

Required number input.

Gradient

Required number input.

Squared-gradient average

Required number input.

Learning rate

Required number input.

Decay

Required number input.

Epsilon

Required number input.

Calculate one scalar RMSProp parameter update and its moving squared-gradient state. The tool evaluates the supplied inputs together and returns the named outputs below; it does not infer omitted operating conditions or change the units shown.^[1]

RMSProp step

The resulting rmsprop step returned as an object.

• Convergence and solution quality depend on smoothness, convexity, scaling, gradients, conditioning, hyperparameters, randomness, constraints, and implementation details. Non-convex methods need not find a global optimum.
• Use finite inputs in the displayed units and preserve more precision than the final presentation requires. Independently verify safety-critical, financial, compliance, or production decisions.

Scale variables, monitor objective and constraint residuals, compare starts and algorithms, and verify small instances or local optimality with an independent solver.

• Adagrad Optimizer Step Calculator

  Calculate one scalar Adagrad parameter update and its accumulated gradient state.

• AdaDelta Optimizer Step Calculator

  Calculate one scalar AdaDelta parameter update and its accumulated state.

• Adam Optimizer Step Calculator

  Calculate one scalar Adam parameter update with bias-corrected moments.