L-BFGS-B supports ftol #47
Description
Hello,
In the optimizer wrapper code, there is a check to warn users that ftol will be ignored when using L-BFGS-B.
class opt_lbfgsb(Optimizer):
# ..
def opt(self, x_init, f_fp=None, f=None, fp=None):
# ....
if self.ftol is not None:
print("WARNING: l-bfgs-b doesn't have an ftol arg, so I'm going to ignore it")
However, l-bfgs-b does support ftol as a termination condition - the argument isn't called ftol, however. It is called factr, and it is ftol divided by machine epsilon.
https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin_l_bfgs_b.html
factr float, optional
The iteration stops when (f^k - f^{k+1})/max{|f^k|,|f^{k+1}|,1} <= factr * eps, where eps is the machine precision, which is automatically generated by the code. Typical values for factr are: 1e12 for low accuracy; 1e7 for moderate accuracy; 10.0 for extremely high accuracy. See Notes for relationship to ftol, which is exposed (instead of factr) by the scipy.optimize.minimize interface to L-BFGS-B.
The SciPy code inside f_min_l_bfgs_b() converts factr into ftol using the following formula.
'ftol': factr * np.finfo(float).eps,
However, this formula could be applied in reverse, to find the value of factr from ftol.
factr = ftol / np.finfo(float).eps,
This would allow users to supply ftol to control the termination condition of the minimizer. If it is not supplied, factr will default to 1e7, which is an ftol of approx 2.22-09.