o "j_@sddlZddlZddlmZddlmZmZd$ddZd$ddZdd Z d d Z Gd d d Z GdddZ GdddZ Gddde ZGddde ZddZddZd$ddZddZddZd$d d!Zd"d#ZdS)%N) framework)primapiutilscCsdt|||tstst|t|}}t|tjr!||n||}t |||t |||fS)aComputes the Vector-Jacobian product, a functional form of reverse mode automatic differentiation. Warning: This API is in beta, the signatures could be changed in future version. Args: func(Callable): A function that takes ``xs`` as inputs parameter and returns a sequence of Tensors or a Tensor. xs(Tensor|Sequence[Tensor]): Used as positional arguments to evaluate ``func``. ``xs`` is accepted as one Tensor or a sequence of Tensors. v(Tensor|Sequence[Tensor]|None, optional): The cotangent vector invovled in the VJP computation. ``v`` matches the size and shape of ``func`` 's output. Defaults to None, which is equivalent to all ones the same size of ``func`` 's output. Returns: output(tuple): - func_out(Tensor|tuple[Tensor]): The output of ``func(xs)`` . - vjp(Tensor|tuple[Tensor]): The vjp result. Examples: .. code-block:: python >>> import paddle >>> def func(x): ... return paddle.matmul(x, x) ... >>> x = paddle.ones(shape=[2, 2], dtype='float32') >>> _, vjp_result = paddle.incubate.autograd.vjp(func, x) >>> print(vjp_result) Tensor(shape=[2, 2], dtype=float32, place=Place(cpu), stop_gradient=False, [[4., 4.], [4., 4.]]) >>> v = paddle.to_tensor([[1.0, 0.0], [0.0, 0.0]]) >>> _, vjp_result = paddle.incubate.autograd.vjp(func, x, v) >>> print(vjp_result) Tensor(shape=[2, 2], dtype=float32, place=Place(cpu), stop_gradient=False, [[2., 1.], [1., 0.]]) ) _check_inputsrin_dygraph_moder prim_enabled _separate isinstancetypingSequence_check_v_shape_gradfuncxsvysrd/var/www/html/Deteccion_Ine/venv/lib/python3.10/site-packages/paddle/incubate/autograd/functional.pyvjps . rcCst|||tstst|t|}}t|tjr!||n||}t ||ts;tr;|t |||fS|t |||fS)a# Computes the Jacobian-Vector product for a function at the given inputs and a vector in the tangent space induced by the inputs. Warning: This API is in beta, the signatures could be changed in future version. Args: func(Callable): The ``func`` takes as input a Tensor or a Sequence of Tensors and returns a Tensor or a Sequence of Tensors. xs(Tensor|Sequence[Tensor]): Used as positional arguments to evaluate ``func``. The ``xs`` is accepted as one Tensor or a Sequence of Tensors. v(Tensor|Sequence[Tensor]|None, Optional): The tangent vector invovled in the JVP computation. The ``v`` matches the size and shape of ``xs`` . Default value is None and in this case is equivalent to all ones the same size of ``xs`` . Returns: output(tuple): - func_out(Tensor|tuple[Tensor]): The output of ``func(xs)`` . - jvp(Tensor|tuple[Tensor]): The jvp result. Examples: .. code-block:: python >>> import paddle >>> def func(x): ... return paddle.matmul(x, x) ... >>> x = paddle.ones(shape=[2, 2], dtype='float32') >>> _, jvp_result = paddle.incubate.autograd.jvp(func, x) >>> print(jvp_result) Tensor(shape=[2, 2], dtype=float32, place=Place(gpu:0), stop_gradient=False, [[4., 4.], [4., 4.]]) >>> v = paddle.to_tensor([[1.0, 0.0], [0.0, 0.0]]) >>> _, jvp_result = paddle.incubate.autograd.jvp(func, x, v) >>> print(jvp_result) Tensor(shape=[2, 2], dtype=float32, place=Place(gpu:0), stop_gradient=False, [[2., 1.], [1., 0.]]) ) rrrrrrr r r r rZ forward_grad_double_backward_trickrrrrjvpPs 1 rcCs t|}t|||}t|||S)zDouble backward trick for computing ``jvp`` by ``vjp`` see details: https://j-towns.github.io/2017/06/12/A-new-trick.html )_zeros_like_with_gradr )rrrZys_gradxs_gradrrrrs  rcCsLt|tjst|}d|_|Sg}|D]}t|}d|_||q|S)zaCreate a zero or zeros sequence Tensor like ``xs`` with a flag ``stop_graident=False`` . F)r r r paddle zeros_like stop_gradientappend)rrxyrrrrs    rc@.eZdZdZd ddZddZeddZd S) Jacobiana? Computes the Jacobian matrix of a given function. If the function has multiple inputs and multiple outputs, during internal implementation, all input tensors are concatenated after being flatten, the batch dimension is retained, and the output is subject to the same processing rules. Once the Jacobian ``J`` is constructed, you can use a multidimensional index to retrieve the submatrix of ``J``, as same as slicing a Tensor. The submatrix is lazily evaluated along row axis, and will be cached once evaluated. For examples, supposing ``is_batched=True``, you can retrieve the submatrix by following methods: * J[:], retrieving the full matrix. * J[:, :, j], retrieving the partial derivatives w.r.t. the j'th input variable. * J[:, i, :], retrieving the partial derivatives w.r.t. the i'th output variable. * J[:, i, j], retrieving the partial derivatives w.r.t. the i'th output variable and the j'th input variable. Notes: Eclipsis index is not supported currently. Warning: This API is in beta, the signatures could be changed in future version. Args: func (Callable): A python function that takes a Tensor or a sequence of Tensors as inputs(the first dimension is batch size) and returns a Tensor a sequence of Tensors. xs (Tensor|Sequence[Tensor]): The input to the function ``func`` . is_batched (bool): If true, the first axis is batch axis. Defaults to False. Returns: Jacobian (Object): A python object retains the Jacobian matrix. Examples: .. code-block:: python >>> import paddle >>> def func(x, y): ... return paddle.matmul(x, y) ... >>> x = paddle.to_tensor([[1., 2.], [3., 4.]]) >>> J = paddle.incubate.autograd.Jacobian(func, [x, x]) >>> print(J[:, :]) Tensor(shape=[4, 8], dtype=float32, place=Place(cpu), stop_gradient=False, [[1., 3., 0., 0., 1., 0., 2., 0.], [2., 4., 0., 0., 0., 1., 0., 2.], [0., 0., 1., 3., 3., 0., 4., 0.], [0., 0., 2., 4., 0., 3., 0., 4.]]) >>> print(J[0, :]) Tensor(shape=[8], dtype=float32, place=Place(cpu), stop_gradient=False, [1., 3., 0., 0., 1., 0., 2., 0.]) >>> print(J[:, 0]) Tensor(shape=[4], dtype=float32, place=Place(cpu), stop_gradient=False, [1., 2., 0., 0.]) FcCs$|s t|||_dSt|||_dSN)_JacobianNoBatch _jacobian_JacobianBatchFirst)selfrr is_batchedrrr__init__szJacobian.__init__cC |j|Sr")r$r&indexesrrr __getitem__ zJacobian.__getitem__cC|jjS)z'The shape of flattened Jacobian matrix.)r$shaper&rrrr/zJacobian.shapeNF__name__ __module__ __qualname____doc__r(r,propertyr/rrrrr!s  Gr!c@r ) Hessiana Computes the Hessian matrix with a given ``func`` with respect to ``xs`` . If the function has multiple inputs, during internal implementation, all input tensors are concatenated after being flatten, the batch dimension is retained. The Hessian submatrix is lazily evaluated, and can be retrieved with a multidimensional indexes. See details ``Jacobian`` . Warning: This API is in beta, the signatures could be changed in future version. Args: func (Callable): A python function that takes a Tensor or a Tensor sequence as inputs and returns a Tensor with shape ``[batch_size, 1]`` with batch or ``[1]`` without batch. xs (Tensor|Sequence(Tensor)): The input Tensor or Tensor sequence of the function ``func``. is_batched (bool): If true, the first axis is batch axis. Defaults to False. Returns: Hessian (Object): A python object retains the Hessian matrix. Examples: .. code-block:: python >>> import paddle >>> def reducer(x): ... return paddle.sum(x * x) ... >>> x = paddle.rand([2, 2]) >>> h = paddle.incubate.autograd.Hessian(reducer, x) >>> print(h[:]) Tensor(shape=[4, 4], dtype=float32, place=CPUPlace(), stop_gradient=False, [[2., 0., 0., 0.], [0., 2., 0., 0.], [0., 0., 2., 0.], [0., 0., 0., 2.]]) Fcs"fdd}t||d|_dS)Ncsdt|d}r|jddkss|jddkrtdr*|dddddfS|dddfS)Nr'rzeThe function given to Hessian shoud return as single element Tensor or batched single element Tensor.)r!r/ RuntimeError)rZjacrr'rr _jac_func2s*z#Hessian.__init__.._jac_funcr:)r!symbolic)r&rrr'r>rr=rr(1s zHessian.__init__cCr)r")r?r*rrrr,>r-zHessian.__getitem__cCr.)z&The shape of flattened Hessian matrix.)r?r/r0rrrr/Ar1z Hessian.shapeNr2r3rrrrr9s  / r9c@sbeZdZdZddZeddZeddZdd Zd d Z dd dZ ddZ ddZ ddZ dS) _JacobianaThe base class for computing Jacobian matrix. ``_Jacobian`` implementes the core logic of multidimensional index and lazy evaluation for Jacobian matrix, subclass only need to overwrite following methods: * ``_lazy_axis()``, return the axis along which will be lazy evaluating. * ``_flatten(xs)``, flattens the inputs ``xs``. * ``_evaluate(index)``, evaluates one slice along ``_lazy_axis`` . Notes: Because currently PaddlePaddle only support reverse differentiation by ``paddle.grad``, so lazy evaluation is only supported along the row of Jacobian matrix, which means that slicing along row will get better performance. cCsfts tr ||_nt||_|t|j|_|t|j|_ |t|j|_ i|_ dSr") rrrr_xsr as_tensorsZ_ys_flatten _flatten_xs _flatten_ys_cacher&rrrrrr(\s  z_Jacobian.__init__cCtr"NotImplementedErrorr0rrrr/hz_Jacobian.shapecCrH)z "The axis of lazily evaluated.rIr0rrr _lazy_axislsz_Jacobian._lazy_axiscCs0||j}t|tr |fStt|j|j|jSr")rLr inttuplerangestartstopstep)r&r+idxrrr _lazy_indexesqs z_Jacobian._lazy_indexescCrHr"rIr&rrrrrCysz_Jacobian._flattenrcCsJ||j}t|tr dntd|d}|d|j|f||jddSNrr;)rLr rMslice)r&r+Zlazy_axis_sizerSZshifted_lazy_axis_idxrrr_shifted_indexes|s  z_Jacobian._shifted_indexescst|j}t|jtr(|dj|jdd}|j|S|}tj}t||j<t j fdd|Djd |}| |t|S)Nr;csg|]}|qSr)_cached_evaluate.0ir0rr sz)_Jacobian.__getitem__..)Zaxis) _multi_indexr/r rLrMrYrTlistlenrconcatreshaperX)r&r+Z other_indexesZ lazy_indexesr/Zpart_jacrr0rr,s"   z_Jacobian.__getitem__cCs,|j|}|dur||}||j|<|Sr")rFget _evaluate)r&krrrrrYs   z_Jacobian._cached_evaluatecCrH)z&Evaluate one slice at along lazy axis.rI)r&indexrrrrdrKz_Jacobian._evaluateN)r)r4r5r6r7r(r8r/rLrTrCrXr,rYrdrrrrr@Gs     r@cHeZdZdZfddZeddZeddZdd Zd d Z Z S) r#zCompute Jacobian matrix without batch dimension. Suppose the mapping is :math:`f: R^M o R^N`, the output shape is ``(N, M)`` . ct||dSr"superr(rG __class__rrr(z_JacobianNoBatch.__init__cCs|jjd|jjdfSNr)rEr/rDr0rrrr/sz_JacobianNoBatch.shapecCdSrnrr0rrrrLrKz_JacobianNoBatch._lazy_axiscCsttdd|DS)Ncss|]}|dVqdS))N)rbr[rrrr sz,_JacobianNoBatch._flatten..)rrarNrUrrrrCsz_JacobianNoBatch._flattencCs|t|j||jSr"rCr rErAr&Z row_indexrrrrds z_JacobianNoBatch._evaluate r4r5r6r7r(r8r/rLrCrd __classcell__rrrkrr#s   r#crg) r%zCompute Jacobian matrix with batch at first axis. Suppose the mapping is :math:`f: R^{B,M} o R^{B,N}`, the output shape is ``(B, N, M)`` . crhr"rirGrkrrr(rmz_JacobianBatchFirst.__init__cCs"|jjd|jjd|jjdfSrV)rDr/rEr0rrrr/s   z_JacobianBatchFirst.shapecCro)Nr;rr0rrrrLrKz_JacobianBatchFirst._lazy_axiscCs ttddt|DdS)Ncss$|] }||jddfVqdS)rrpN)rbr/rqrrrrrs"z/_JacobianBatchFirst._flatten..r;)rrarNrrBrUrrrrCsz_JacobianBatchFirst._flattencCs |t|jdd|f|jSr"rsrtrrrrds z_JacobianBatchFirst._evaluaterurrrkrr%s   r%cCst|tjr|n|f}tdd|Drtd|tdddft|t|}g}t|D][\}}t|trnt|jp=d|j pC|||j pGd}| t|jdkrX|j||n|j|j dkrf|j ||n|j |j q/t|t r| |dkr|||n|q/t d|dt|S) aaA tool for parsing N-dimensional index into a standard format. Currently supporting following input format: * ([positive|negative|slice], ...), the right-most elements can be omited. The standard format after converted is slice tuple which contains N elements: * ([positive|slice], ..., [positive|slice]) Notes: Ellipsis indexes such as ``(..., i), (i, ...)`` is not supported. Args: indexes (tuple): The input indexes. shape (tuple): The input shape. Returns: tuple: The standard format index as the above description. css|] }t|ttVqdSr")r typeEllipsisrZrrrrrsz_multi_index..z*Ellipsis index currently is not supported.rNr;zNot supported index type .)r r r any IndexErrorrWr` enumeraterPrQrRrrM TypeErrorrN)r+r/Zpositive_indexesr\rfrrrr^s*"  r^csH|durt}j|_|St|tjr"tfddt|DS|S)Nc3s"|] \}}t||VqdSr")_replace_none_with_zero_tensor)r[r\rrefsrrrrs z1_replace_none_with_zero_tensor..)rrrr r r rNr|)rrrrrr~s   r~cCsjtr'tj|||ddd}t|tjjjr&t|tjr&t |dkr&|d}n tj j |||}t ||S)a^A gradient function that can be used in dynamic graph and static graph. The ``grad`` combines ``paddle.grad`` used in dynamic graph and ``paddle.static.gradients`` used in static graph, and do following changes: * The ``allow_unused`` flag is removed and set defaults to true internally, none in outputs will be replaced by zero tensor. * The ``create_graph`` flag is removed and set defaults to true internally, only makes sense in dynamic graph. * When xs is a single Tensor, ``paddle.grad`` returns a list which only contains one Tensor. It may confuse users, thus in this case we improve to return a single Tensor in _grad interface. Args: ys (Tensor|Sequence[Tensor]): The output tensor or tensor sequence of the graph to compute gradients. xs (Tensor|Sequence[Tensor]): The input tensor or tensor sequence of the graph to compute gradients. The returned values of this API are the gradients of inputs . v (Tensor|Sequence[Tensor]|None,optional): The initial gradient values of outputs . If grad_outputs is None, the initial gradient values of outputs would be Tensors filled with 1; if grad_outputs is not None, it must have the same length as outputs , and in this case, the initial gradient value of the i-th outputs would be: (1) a Tensor filled with 1 when the i-th element of grad_outputs is None; (2) the i-th element of grad_outputs when the i-th element of grad_outputs is a Tensor. Default None. Returns: Tensor|tuple[Tensor]: Tensor or a tuple of Tensors, whose length is the same as the Tensor number inside inputs, and the i-th returned Tensor is the sum of gradients of outputs with respect to the i-th inputs. T)Z create_graphZ allow_unusedr) rrrZgradr baseVariabler r r`ZincubateZautogradr~)rrrrrrrr s#   r cCs&t|tjrtdd|DSt|S)a ``_separate`` separates ``xs`` from the computation graph through ``clone`` or ``deteach`` . Interally, ``paddle.grad(xs, ys)`` is stateful API implemented based on computional graph, which will reduce gradients along all path from ys to xs. However, funcional autograd API such as ``vjp``, ``jvp`` is stateless, and only compute gradients with a given ``func`` . For example, given a ``func`` :math:`y0=f(x0)`, supposing forward path is: ``x0 -> y0``, ``x0 -> x1 -> y0`` . ``paddle.grad(y0, x0)`` will reduce gradients along ``y0->x0`` and ``y0->x1->x0``, and ``vjp`` only need reduce along ``y0->x0``. So, it's needed to clone or detach xs for breaking the dependencies with other variables. Examples: .. code-block:: python >>> import paddle >>> from paddle.incubate.autograd.functional import _separate >>> def func(x, y): ... return x * y ... >>> x = paddle.ones((1,)) >>> x.stop_gradient = False >>> y = func(x, x) >>> print(paddle.grad(y, x)) [Tensor(shape=[1], dtype=float32, place=Place(gpu:0), stop_gradient=True, [2.])] >>> x1, x2 = _separate((x, x)) >>> y = func(x1, x2) >>> print(paddle.grad(y, x1)) [Tensor(shape=[1], dtype=float32, place=Place(gpu:0), stop_gradient=True, [1.])] css|]}t|VqdSr")_single_separaterqrrrrrsz_separate..)r r r rNr)rrrrrSs ,rcCs.|dur|S|jst|S|}d|_|S)NF)rrZassigndetach)rrrrrs rcCst|stdt|dt|tjtjfs!tdt|dt|tjr4tdd|Ds4tdt|tjtjtdfsJtdt|dt|tjr]tdd|Ds_tddSdS) Nz$Expected 'fun' is Callable, but got ryz3Expected 'xs' is a Tensor|Sequence[Tensor],but got cs|] }t|tjVqdSr"r rrrqrrrrr  z _check_inputs..z&All elements of 'xs' shoule be Tensor.z6Expected 'v' is Tensor|Sequence[Tensor]|None, but got csrr"r)r[errrrrr) callabler}rwr rrr r all)rrrrrrrs*rc Cs|durdSt|t|}}t|t|kr(tdt|dt|dtt||D]\}\}}|j|jkrKtd|d|jd|jdq/dS)Nz6The argument v is a tuple of invalid length:should be z but got ryzThe v[z] has invalid shape: should be )rrBr`r<r|zipr/)rrrfZ element_vZ element_refrrrr s, r r")r rZ paddle.baserZpaddle.incubate.autogradrrrrrrr!r9r@r#r%r^r~r rrrr rrrrs(   :? WFa0 32