test-gpt2-kernels - TF32 Mode is turned OFF. test-gpt2 - [State] Batch Size: 4 Sequence Length: 64 GPT2 - Allocated 221 MiB for activations test-gpt2 - Printing 10 Expected vs Actual Logits: -43.431618, -43.431801 -39.836346, -39.836525 -43.065910, -43.066086 -42.828045, -42.828217 -43.529541, -43.529724 -44.318398, -44.318581 -41.227425, -41.227596 -41.270760, -41.270927 -42.541393, -42.541595 -42.394997, -42.395184 ---------------- Logits OK! (max err 0.000069, RMSE 0.000002) ---------------- _________________________ All tests passed! _________________________ Collecting data... Generating '/tmp/nsys-report-e25f.qdstrm' [1/8] [0% ] report7.nsys-rep [1/8] [0% ] report7.nsys-rep [1/8] [=16% ] report7.nsys-rep [1/8] [=====31% ] report7.nsys-rep [1/8] [=====32% ] report7.nsys-rep [1/8] [====25% ] report7.nsys-rep [1/8] [======35% ] report7.nsys-rep [1/8] [=======36% ] report7.nsys-rep [1/8] [=======37% ] report7.nsys-rep [1/8] [=======38% ] report7.nsys-rep [1/8] [=======39% ] report7.nsys-rep [1/8] [========40% ] report7.nsys-rep [1/8] [========41% ] report7.nsys-rep [1/8] [========42% ] report7.nsys-rep [1/8] [==================78% ] report7.nsys-rep [1/8] [=====================86% ] report7.nsys-rep [1/8] [========================100%] report7.nsys-rep [1/8] [========================100%] report7.nsys-rep [2/8] [0% ] report7.sqlite [2/8] [1% ] report7.sqlite [2/8] [2% ] report7.sqlite [2/8] [3% ] report7.sqlite [2/8] [4% ] report7.sqlite [2/8] [5% ] report7.sqlite [2/8] [6% ] report7.sqlite [2/8] [7% ] report7.sqlite [2/8] [8% ] report7.sqlite [2/8] [9% ] report7.sqlite [2/8] [10% ] report7.sqlite [2/8] [11% ] report7.sqlite [2/8] [12% ] report7.sqlite [2/8] [13% ] report7.sqlite [2/8] [14% ] report7.sqlite [2/8] [=15% ] report7.sqlite [2/8] [=16% ] report7.sqlite [2/8] [=17% ] report7.sqlite [2/8] [==18% ] report7.sqlite [2/8] [==19% ] report7.sqlite [2/8] [==20% ] report7.sqlite [2/8] [==21% ] report7.sqlite [2/8] [===22% ] report7.sqlite [2/8] [===23% ] report7.sqlite [2/8] [===24% ] report7.sqlite [2/8] [====25% ] report7.sqlite [2/8] [====26% ] report7.sqlite [2/8] [====27% ] report7.sqlite [2/8] [====28% ] report7.sqlite [2/8] [=====29% ] report7.sqlite [2/8] [=====30% ] report7.sqlite [2/8] [=====31% ] report7.sqlite [2/8] [=====32% ] report7.sqlite [2/8] [======33% ] report7.sqlite [2/8] [======34% ] report7.sqlite [2/8] [======35% ] report7.sqlite [2/8] [=======36% ] report7.sqlite [2/8] [=======37% ] report7.sqlite [2/8] [=======38% ] report7.sqlite [2/8] [=======39% ] report7.sqlite [2/8] [========40% ] report7.sqlite [2/8] [========41% ] report7.sqlite [2/8] [========42% ] report7.sqlite [2/8] [=========43% ] report7.sqlite [2/8] [=========44% ] report7.sqlite [2/8] [=========45% ] report7.sqlite [2/8] [=========46% ] report7.sqlite [2/8] [==========47% ] report7.sqlite [2/8] [==========48% ] report7.sqlite [2/8] [==========49% ] report7.sqlite [2/8] [===========50% ] report7.sqlite [2/8] [===========51% ] report7.sqlite [2/8] [===========52% ] report7.sqlite [2/8] [===========53% ] report7.sqlite [2/8] [============54% ] report7.sqlite [2/8] [============55% ] report7.sqlite [2/8] [============56% ] report7.sqlite [2/8] [============57% ] report7.sqlite [2/8] [=============58% ] report7.sqlite [2/8] [=============59% ] report7.sqlite [2/8] [=============60% ] report7.sqlite [2/8] [==============61% ] report7.sqlite [2/8] [==============62% ] report7.sqlite [2/8] [==============63% ] report7.sqlite [2/8] [==============64% ] report7.sqlite [2/8] [===============65% ] report7.sqlite [2/8] [===============66% ] report7.sqlite [2/8] [===============67% ] report7.sqlite [2/8] [================68% ] report7.sqlite [2/8] [================69% ] report7.sqlite [2/8] [================70% ] report7.sqlite [2/8] [================71% ] report7.sqlite [2/8] [=================72% ] report7.sqlite [2/8] [=================73% ] report7.sqlite [2/8] [=================74% ] report7.sqlite [2/8] [==================75% ] report7.sqlite [2/8] [==================76% ] report7.sqlite [2/8] [==================77% ] report7.sqlite [2/8] [==================78% ] report7.sqlite [2/8] [===================79% ] report7.sqlite [2/8] [===================80% ] report7.sqlite [2/8] [===================81% ] report7.sqlite [2/8] [===================82% ] report7.sqlite [2/8] [====================83% ] report7.sqlite [2/8] [====================84% ] report7.sqlite [2/8] [====================85% ] report7.sqlite [2/8] [=====================86% ] report7.sqlite [2/8] [=====================87% ] report7.sqlite [2/8] [=====================88% ] report7.sqlite [2/8] [=====================89% ] report7.sqlite [2/8] [======================90% ] report7.sqlite [2/8] [======================91% ] report7.sqlite [2/8] [======================92% ] report7.sqlite [2/8] [=======================93% ] report7.sqlite [2/8] [=======================94% ] report7.sqlite [2/8] [=======================95% ] report7.sqlite [2/8] [=======================96% ] report7.sqlite [2/8] [========================97% ] report7.sqlite [2/8] [========================98% ] report7.sqlite [2/8] [========================99% ] report7.sqlite [2/8] [========================100%] report7.sqlite [2/8] [========================100%] report7.sqlite [3/8] Executing 'nvtx_sum' stats report [4/8] Executing 'osrt_sum' stats report Time (%) Total Time (ns) Num Calls Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name -------- --------------- --------- ---------- ---------- -------- --------- ----------- ---------------------- 62.0 680717261 17 40042191.8 10065417.0 1532 100136863 44760103.1 poll 14.8 162110818 38 4266074.2 6131.0 3496 161029552 26117864.7 fopen 11.0 120373004 6 20062167.3 89457.5 6833 95159843 38120924.5 fread 10.4 114685549 591 194053.4 23404.0 1032 22556281 980200.7 ioctl 0.8 9116342 1 9116342.0 9116342.0 9116342 9116342 0.0 pthread_cond_broadcast 0.4 3892842 32 121651.3 2144.0 1353 3504445 619781.4 fclose 0.2 2358791 25 94351.6 8787.0 6392 1637323 324262.9 mmap64 0.2 1996705 1 1996705.0 1996705.0 1996705 1996705 0.0 pthread_cond_wait 0.2 1950609 10 195060.9 27181.0 22051 798313 285177.5 write 0.1 591677 17 34804.5 5981.0 1893 395630 94748.6 mmap 0.0 362288 2 181144.0 181144.0 171561 190727 13552.4 pthread_create 0.0 322858 55 5870.1 5861.0 2314 14327 2395.2 open64 0.0 76734 1 76734.0 76734.0 76734 76734 0.0 fgets 0.0 62457 1 62457.0 62457.0 62457 62457 0.0 fflush 0.0 40205 6 6700.8 6922.5 3527 9608 2269.3 open 0.0 38922 8 4865.3 5074.5 2765 7153 1707.3 munmap 0.0 19787 3 6595.7 7844.0 3657 8286 2554.5 pipe2 0.0 18904 10 1890.4 1623.0 1012 3516 882.0 read 0.0 15960 2 7980.0 7980.0 5100 10860 4072.9 socket 0.0 11101 1 11101.0 11101.0 11101 11101 0.0 connect 0.0 6472 1 6472.0 6472.0 6472 6472 0.0 fopen64 0.0 3417 2 1708.5 1708.5 1393 2024 446.2 fwrite 0.0 1593 1 1593.0 1593.0 1593 1593 0.0 bind [5/8] Executing 'cuda_api_sum' stats report Time (%) Total Time (ns) Num Calls Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name -------- --------------- --------- ---------- ---------- -------- -------- ----------- --------------------------------- 61.4 51907511 3 17302503.7 15637119.0 21119 36249273 18171403.8 cudaMemcpy 18.9 15960434 4 3990108.5 3249843.5 1031059 8429688 3147630.5 cuLibraryLoadData 6.1 5177442 12 431453.5 6161.5 191 1837587 621479.2 cudaFree 5.7 4819182 1347 3577.7 3086.0 2835 335788 9243.6 cudaLaunchKernel 2.9 2489386 1 2489386.0 2489386.0 2489386 2489386 0.0 cudaFreeHost 1.7 1448309 1 1448309.0 1448309.0 1448309 1448309 0.0 cudaGetDeviceProperties_v2_v12000 1.6 1383518 1 1383518.0 1383518.0 1383518 1383518 0.0 cudaMallocHost 0.9 766724 6 127787.3 123215.5 3447 318305 118859.5 cudaMalloc 0.2 160077 838 191.0 151.0 70 6683 242.2 cuGetProcAddress_v2 0.2 139420 48 2904.6 2500.0 2184 13164 1581.6 cudaMemsetAsync 0.2 137165 48 2857.6 2690.0 1974 12383 1519.2 cudaEventRecord 0.0 35015 1 35015.0 35015.0 35015 35015 0.0 cudaMemset 0.0 30347 4 7586.8 7734.0 6913 7966 469.4 cuLibraryGetKernel 0.0 27345 240 113.9 90.0 69 1262 81.3 cuStreamGetCaptureInfo_v2 0.0 17219 18 956.6 300.0 260 7834 1819.2 cudaEventCreateWithFlags 0.0 12852 4 3213.0 2549.5 991 6762 2485.5 cudaDeviceSynchronize 0.0 8116 18 450.9 301.0 211 2675 566.0 cudaEventDestroy 0.0 5371 4 1342.8 1343.0 1172 1513 139.5 cuInit 0.0 3317 1 3317.0 3317.0 3317 3317 0.0 cudaEventQuery 0.0 1322 3 440.7 180.0 130 1012 495.4 cuModuleGetLoadingMode 0.0 872 2 436.0 436.0 211 661 318.2 cudaGetDriverEntryPoint_v11030 [6/8] Executing 'cuda_gpu_kern_sum' stats report Time (%) Total Time (ns) Instances Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name -------- --------------- --------- -------- -------- -------- -------- ----------- ---------------------------------------------------------------------------------------------------- 25.6 4083613 25 163344.5 163744.0 158464 166208 2004.3 layernorm_forward_kernel(float *, float *, float *, const float *, const float *, const float *, in… 22.8 3644222 576 6326.8 6336.0 6144 6592 79.1 ampere_sgemm_32x128_nn 16.8 2675550 25 107022.0 78560.0 60384 1000383 186333.0 ampere_sgemm_128x64_tn 15.9 2535902 576 4402.6 4384.0 4352 4864 60.7 ampere_sgemm_32x32_sliced1x4_tn 10.0 1591582 12 132631.8 133168.0 129568 134783 1430.5 ampere_sgemm_64x64_tn 2.4 390976 12 32581.3 32528.0 32384 32928 170.9 ampere_sgemm_64x32_sliced1x4_tn 1.4 223072 12 18589.3 18560.0 18528 18848 90.0 gelu_forward_kernel(float *, const float *, int) 1.4 219968 48 4582.7 4288.0 2848 7616 1622.8 bias_add_kernel(float *, const float *, int, int) 1.4 217344 12 18112.0 18000.0 17920 19200 353.5 softmax_forward_kernel(float *, float, const float *, int, int) 0.9 142080 24 5920.0 5952.0 5632 6240 265.2 residual_forward_kernel(float *, float *, float *, int) 0.6 89376 1 89376.0 89376.0 89376 89376 0.0 encoder_forward_kernel(float *, const int *, const float *, const float *, int, int, int) 0.5 84511 12 7042.6 7008.0 6976 7392 112.9 permute_kernel(float *, float *, float *, const float *, int, int, int, int) 0.4 70944 12 5912.0 5888.0 5824 6016 62.7 unpermute_kernel(float *, float *, int, int, int, int) [7/8] Executing 'cuda_gpu_mem_time_sum' stats report Time (%) Total Time (ns) Count Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Operation -------- --------------- ----- ---------- ---------- -------- -------- ----------- ---------------------------- 81.3 36158470 2 18079235.0 18079235.0 832 36157638 25566722.7 [CUDA memcpy Host-to-Device] 17.6 7821051 1 7821051.0 7821051.0 7821051 7821051 0.0 [CUDA memcpy Device-to-Host] 1.1 473504 49 9663.3 1184.0 480 407616 58036.0 [CUDA memset] [8/8] Executing 'cuda_gpu_mem_size_sum' stats report Total (MB) Count Avg (MB) Med (MB) Min (MB) Max (MB) StdDev (MB) Operation ---------- ----- -------- -------- -------- -------- ----------- ---------------------------- 497.905 2 248.952 248.952 0.001 497.904 352.070 [CUDA memcpy Host-to-Device] 232.407 49 4.743 0.000 0.000 232.392 33.199 [CUDA memset] 51.511 1 51.511 51.511 51.511 51.511 0.000 [CUDA memcpy Device-to-Host] Generated: /u/asingh41/sp26_ece408_Temperature1.5/report7.nsys-rep /u/asingh41/sp26_ece408_Temperature1.5/report7.sqlite