o "j)@s(ddlZd ddZ    d dd ZdS) Nc Cs|dur |\}}n||kr||fn||f\}}||d||||} | d||} | dkrj| } ||krN||||| | ||d| } n||||| | ||d| } tt| ||S||dS)a]Cubic interpolation between (x1, f1, g1) and (x2, f2, g2). Use two points and their gradient to determine a cubic function and get the minimun point between them in the cubic curve. Reference: Jorge Nocedal, Stephen J. Wright, Numerical Optimization, Second Edition, 2006. pp59: formula 3.59 Args: x1, f1, g1: point1's position, value and gradient. x2, f2, g2: point2's position, value and gradient. bounds: bounds of interpolation area Returns: min_pos: the minimun point between the specified points in the cubic curve. Nrg@)sqrtminmax) x1f1g1Zx2f2g2boundsZ xmin_boundZ xmax_boundZd1Z d2_squareZd2Zmin_posr n/var/www/html/Deteccion_Ine/venv/lib/python3.10/site-packages/paddle/incubate/optimizer/line_search_dygraph.py_cubic_interpolates *( r-C6??& .>c ! Cs|} |}||||\} } d}t| |}tjd|jd|||f\}}}}d}d}|| kr| ||||ksD|dkrW| |krW||g}|| g}|| g}||g}nkt|| |krm|g}| g}| g}d}nU|dkr||g}|| g}|| g}||g}n>|d||}|d}|}t||||| |||fd}|}| }| }|}||||\} } |d7}| |}|d7}|| ks2|| krd|g}|| g}|| g}d}|d|d krd nd \}}|s|| krt|d|d| | krnt|d|d|d|d|d|d}d t|t|} tt|||t|| kr`|s:|t|ks:|t|kr]t|t|t|t|krTt|| }nt|| }d}nd}nd}||||\} } |d7}| |}|d7}| ||||ks| ||kr|||<| ||<| ||<|||<|d|dkrd nd \}}nEt|| |krd}n%|||||dkr||||<||||<||||<||||<|||<| ||<| ||<|||<|s|| ks||}||} ||} | | ||fS) a4 Implements of line search algorithm that satisfies the strong Wolfe conditions using double zoom. Reference: Jorge Nocedal, Stephen J. Wright, Numerical Optimization, Second Edition, 2006. pp60: Algorithm 3.5 (Line Search Algorithm). Args: obj_func: the objective function to minimize. ```` accepts a multivariate input and returns a scalar. xk (Tensor): the starting point of the iterates. alpha (Scalar): the initial step size. d (Tensor): search direction. loss (scalar): the initial loss grad (Tensor): the initial grad c1 (Scalar): parameter for sufficient decrease condition. c2 (Scalar): parameter for curvature condition. tolerance_change (Scalar): terminates if the change of function value/position/parameter between two iterations is smaller than this value. max_ls(int): max iteration of line search. alpha_max (float): max step length. Returns: loss_new (Scaler): loss of obj_func at final alpha. grad_new, (Tensor): derivative of obj_func at final alpha. alpha(Tensor): optimal step length, or 0. if the line search algorithm did not converge. ls_func_evals (Scaler): number of objective function called in line search process. Following summarizes the essentials of the strong Wolfe line search algorithm. Some notations used in the description: - `func` denotes the objective function. - `obi_func` is a function of step size alpha, restricting `obj_func` on a line. obi_func = func(xk + alpha * d), where xk is the position of k'th iterate, d is the line search direction(decent direction), and a is the step size. - alpha : substitute of alpha - a1 is alpha of last iteration, which is alpha_(i-1). - a2 is alpha of current iteration, which is alpha_i. - a_lo is alpha in left position when calls zoom, which is alpha_low. - a_hi is alpha in right position when calls zoom, which is alpha_high. Line Search Algorithm: repeat Compute obi_func(a2) and derphi(a2). 1. If obi_func(a2) > obi_func(0) + c_1 * a2 * obi_func'(0) or [obi_func(a2) >= obi_func(a1) and i > 1], alpha= zoom(a1, a2) and stop; 2. If |obi_func'(a2)| <= -c_2 * obi_func'(0), alpha= a2 and stop; 3. If obi_func'(a2) >= 0, alpha= zoom(a2, a1) and stop; a1 = a2 a2 = min(2 * a2, a2) i = i + 1 end(repeat) zoom(a_lo, a_hi) Algorithm: repeat aj = cubic_interpolation(a_lo, a_hi) Compute obi_func(aj) and derphi(aj). 1. If obi_func(aj) > obi_func(0) + c_1 * aj * obi_func'(0) or obi_func(aj) >= obi_func(a_lo), then a_hi <- aj; 2. 2.1. If |obi_func'(aj)| <= -c_2 * obi_func'(0), then alpha= a2 and stop; 2.2. If obi_func'(aj) * (a2 - a1) >= 0, then a_hi = a_lo a_lo = aj; end(repeat) r)dtypeFTg{Gz? )r )rr)rrg?) absrclonepaddledotZ to_tensorrrr)!Zobj_funcZxkalphadZlossZgradZgtdc1c2Ztolerance_changeZmax_lsZd_normZloss_newZgrad_newZ ls_func_evalsZgtd_newZt_prevZf_prevZg_prevZgtd_prevdoneZls_iterbracketZ bracket_fZ bracket_gZ bracket_gtdZmin_stepZmax_steptmpZinsuf_progressZlow_posZhigh_posepsr r r _strong_wolfe6s V      3 "         L r$)N)rrrr)rrr$r r r rs ,