fit_improve
- exception LineSearchWarning[source]
Bases:
RuntimeWarning
- fmin_bfgs_f(f_g, x0, B0=None, M=2, gtol=1e-05, Delta=10.0, maxiter=None, callback=None, norm_ord=inf, **_kwargs)[source]
test BFGS with nonmonote line search
- line_search_nonmonote(f, myfprime, xk, pk, gfk=None, old_fval=None, fk=None, old_old_fval=None, args=(), c1=0.5, maxiter=10)[source]
- line_search_wolfe2(f, myfprime, xk, pk, gfk=None, fk=None, old_fval=None, old_old_fval=None, args=(), c1=0.0001, c2=0.9, amax=None, extra_condition=None, maxiter=10)[source]
Find alpha that satisfies strong Wolfe conditions.
- Parameters:
f (callable f(x,*args)) – Objective function.
myfprime (callable f'(x,*args)) – Objective function gradient.
xk (ndarray) – Starting point.
pk (ndarray) – Search direction.
gfk (ndarray, optional) – Gradient value for x=xk (xk being the current parameter estimate). Will be recomputed if omitted.
old_fval (float, optional) – Function value for x=xk. Will be recomputed if omitted.
old_old_fval (float, optional) – Function value for the point preceding x=xk
args (tuple, optional) – Additional arguments passed to objective function.
c1 (float, optional) – Parameter for Armijo condition rule.
c2 (float, optional) – Parameter for curvature condition rule.
amax (float, optional) – Maximum step size
extra_condition (callable, optional) – A callable of the form
extra_condition(alpha, x, f, g)
returning a boolean. Arguments are the proposed stepalpha
and the correspondingx
,f
andg
values. The line search accepts the value ofalpha
only if this callable returnsTrue
. If the callable returnsFalse
for the step length, the algorithm will continue with new iterates. The callable is only called for iterates satisfying the strong Wolfe conditions.maxiter (int, optional) – Maximum number of iterations to perform
- Returns:
alpha (float or None) – Alpha for which
x_new = x0 + alpha * pk
, or None if the line search algorithm did not converge.fc (int) – Number of function evaluations made.
gc (int) – Number of gradient evaluations made.
new_fval (float or None) – New function value
f(x_new)=f(x0+alpha*pk)
, or None if the line search algorithm did not converge.old_fval (float) – Old function value
f(x0)
.new_slope (float or None) – The local slope along the search direction at the new value
<myfprime(x_new), pk>
, or None if the line search algorithm did not converge.
Notes
Uses the line search algorithm to enforce strong Wolfe conditions. See Wright and Nocedal, ‘Numerical Optimization’, 1999, pg. 59-60.
For the zoom phase it uses an algorithm by […].
- minimize(fun, x0, args=(), method=None, jac=None, hess=None, hessp=None, bounds=None, constraints=(), tol=None, callback=None, options=None)[source]
- scalar_search_wolfe2(phi, derphi, phi0=None, old_phi0=None, derphi0=None, c1=0.0001, c2=0.9, amax=None, extra_condition=None, maxiter=10)[source]
Find alpha that satisfies strong Wolfe conditions.
alpha > 0 is assumed to be a descent direction.
- Parameters:
phi (callable phi(alpha)) – Objective scalar function.
derphi (callable phi'(alpha)) – Objective function derivative. Returns a scalar.
phi0 (float, optional) – Value of phi at 0
old_phi0 (float, optional) – Value of phi at previous point
derphi0 (float, optional) – Value of derphi at 0
c1 (float, optional) – Parameter for Armijo condition rule.
c2 (float, optional) – Parameter for curvature condition rule.
amax (float, optional) – Maximum step size
extra_condition (callable, optional) – A callable of the form
extra_condition(alpha, phi_value)
returning a boolean. The line search accepts the value ofalpha
only if this callable returnsTrue
. If the callable returnsFalse
for the step length, the algorithm will continue with new iterates. The callable is only called for iterates satisfying the strong Wolfe conditions.maxiter (int, optional) – Maximum number of iterations to perform
- Returns:
alpha_star (float or None) – Best alpha, or None if the line search algorithm did not converge.
phi_star (float) – phi at alpha_star
phi0 (float) – phi at 0
derphi_star (float or None) – derphi at alpha_star, or None if the line search algorithm did not converge.
Notes
Uses the line search algorithm to enforce strong Wolfe conditions. See Wright and Nocedal, ‘Numerical Optimization’, 1999, pg. 59-60.
For the zoom phase it uses an algorithm by […].