Use this optimization step or heuristic to study an explicitly defined objective and constraints with reproducible parameters, initialization, and stopping rules.

Parameter

Required number input.

Gradient

Required number input.

First moment

Required number input.

Second moment

Required number input.

Step

Required integer input.

Learning rate

Required number input.

Beta 1

Required number input.

Beta 2

Required number input.

Epsilon

Required number input.

Calculate one scalar Adam parameter update with bias-corrected moments. The tool evaluates the supplied inputs together and returns the named outputs below; it does not infer omitted operating conditions or change the units shown.^[1]

Adam step

The resulting adam step returned as an object.

• Convergence and solution quality depend on smoothness, convexity, scaling, gradients, conditioning, hyperparameters, randomness, constraints, and implementation details. Non-convex methods need not find a global optimum.
• Use finite inputs in the displayed units and preserve more precision than the final presentation requires. Independently verify safety-critical, financial, compliance, or production decisions.

Scale variables, monitor objective and constraint residuals, compare starts and algorithms, and verify small instances or local optimality with an independent solver.

• Nadam Optimizer Step

  Apply a bias-corrected Nesterov Adam update from caller-supplied first and second moments and the one-based iteration.

• AdaDelta Optimizer Step Calculator

  Calculate one scalar AdaDelta parameter update and its accumulated state.

• Adagrad Optimizer Step Calculator

  Calculate one scalar Adagrad parameter update and its accumulated gradient state.