o "j@s^ddlZddlmZGdddZddZdd ZGd d d ejjZGd d d ej j Z dS)N) CUDAGraphc@seZdZddZdS)CUDAGraphContextcCs&d|_||_t|_t|_||_dS)Nr)steplayerr forward_graphbackward_graphnum_warmup_stepsselfrr r f/var/www/html/Deteccion_Ine/venv/lib/python3.10/site-packages/paddle/device/cuda/cuda_graphed_layer.py__init__s  zCUDAGraphContext.__init__N)__name__ __module__ __qualname__rr r r r rs rcCs$t|tjr|}|j|_|S|SN) isinstancepaddleTensordetachZ stop_gradient)xZ x_detachedr r r rs rcCt|tjr |jS|SrrrrZgradrr r r get_grad&s rc@s$eZdZeddZeddZdS)_CUDAGraphedLayercGs:dd|D}|j|jkr1t |j|}Wdn1s!wY|||||S|j|jkrr||_|j t|j|j}Wdn1sTwY|j |j ||_ |||j||St |j|D]\}}t|tjr||dqx|j |j }|||j||S)NcSsg|]}t|qSr )r.0rr r r 0sz-_CUDAGraphedLayer.forward..T)rr rZ enable_gradrZsave_for_backwardrZ args_staticr capture_begin capture_endreplayy_staticziprrcopy_)ctxcontextargsyZx_staicrr r r forward.s4           z_CUDAGraphedLayer.forwardcs|\}}}|j|jkr||n,|j|jkr3||_|j|j|j|j|j n |j |d|j ddt fdd|D}|jd7_|S)NTcSrrrrr r r r_sz,_CUDAGraphedLayer.backward..get_gradc3s|]}|VqdSrr rrr r bsz-_CUDAGraphedLayer.backward..r) Z saved_tensorrr backwardZ dy_staticrr r#r!r"r%tuple)r&Zdyr'r(r)Z args_gradr r+r r-Os       z_CUDAGraphedLayer.backwardN)rrr staticmethodr*r-r r r r r-s   rcs4eZdZdZddejjffdd ZddZZ S) CUDAGraphedLayera CUDAGraphedLayer: A PaddlePaddle Layer to convert an eager mode model to utilize CUDA Graphs. CUDA Graphs provide a way to capture kernel-level operations of a model and play them back efficiently, allowing for potential speedups in repetitive computations, such as those during training iterations. This layer is a wrapper that enables the usage of CUDA Graphs with PaddlePaddle models. Overview: - The layer encapsulates another layer (the model to be converted). - During the first few (num_warmup_steps) iterations, the layer operates in eager mode without any CUDA Graphs. - After the warmup steps, the layer captures the forward and backward computations and replays them using CUDA Graphs in subsequent iterations. Usage: model = Model() graphed_model = CUDAGraphedLayer(model) Parameters: - layer (paddle.nn.Layer): The PaddlePaddle model/layer to be converted. - num_warmup_steps (int): The number of iterations before the CUDA Graph capture begins. Default is 3. Notes: - Restrictions: * CPU-GPU Synchronization: Operations that synchronize the CPU with the GPU, like device to host transfers, are not allowed. * CPU Work: Any operations on the CPU within the captured graph are not recorded. * Memory Address (Pointer) Consistency: Replays consistently read from and write to identical virtual memory addresses. * Dynamic Operations: - Control Flow: Dynamic control flows, especially those based on CPU data like if/else statements, are prohibited. - Tensor Shapes: Dynamic tensor shapes are not supported. - Allowed Operations: * CUDA RNG Operations: CUDA-based Random Number Generation operations are allowed. rcstt|||_dSr)superrrr'r  __class__r r rs zCUDAGraphedLayer.__init__cGstj|jg|RSr)rapplyr')r r(r r r r*szCUDAGraphedLayer.forward)r1) rrr__doc__rnnLayerrr* __classcell__r r r3r r0gs%r0) rZgraphsrrrrZautogradZPyLayerrr7r8r0r r r r s   :