Paste your logs.

Built for Minecraft & Hytale

Unknown Log

6692 lines

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 1, GPU 72 (MiB)

11:20:19 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

11:20:20 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +731, GPU +2, now: CPU 20115, GPU 1344 (MiB)

11:20:20 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

11:20:20 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

11:20:20 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

11:20:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

11:20:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

11:20:21 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

11:20:21 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

11:20:21 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2404ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

11:20:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

11:20:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

11:20:21 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

11:20:21 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.778871 seconds.

11:20:21 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 125 MiB

11:20:21 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

11:20:21 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

11:20:21 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 1, GPU 86 (MiB)

11:20:21 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

11:20:22 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +675, GPU +2, now: CPU 20076, GPU 1358 (MiB)

11:20:22 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

11:20:22 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

11:20:22 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

11:20:22 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

11:20:22 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

11:20:22 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

11:20:22 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

11:20:22 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3523ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

11:20:22 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

11:20:22 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

11:20:22 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

11:20:22 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.79518 seconds.

11:20:22 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 125 MiB

11:20:23 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

11:20:23 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

11:20:23 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 2, GPU 99 (MiB)

11:20:23 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

11:20:23 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +715, GPU +0, now: CPU 20108, GPU 1372 (MiB)

11:20:23 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

11:20:24 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

11:20:24 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

11:20:24 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

11:20:24 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

11:20:24 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

11:20:24 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

11:20:24 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2645ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

11:20:24 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

11:20:24 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

11:20:24 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

11:20:24 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.801743 seconds.

11:20:24 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 125 MiB

11:20:24 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

11:20:24 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

11:20:24 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 2, GPU 112 (MiB)

11:20:25 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

11:20:25 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +749, GPU +0, now: CPU 20110, GPU 1386 (MiB)

11:20:25 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

11:20:26 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

11:20:26 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

11:20:26 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

11:20:26 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

11:20:26 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

11:20:26 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

11:20:26 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2596ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

11:20:26 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

11:20:26 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

11:20:26 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

11:20:26 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.821087 seconds.

11:20:26 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 133 MiB

11:20:26 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

11:20:26 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

11:20:26 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 2, GPU 125 (MiB)

11:20:27 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

11:20:27 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +741, GPU +2, now: CPU 20114, GPU 1402 (MiB)

100

11:20:27 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

101

11:20:28 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

102

11:20:28 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

103

11:20:28 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

104

11:20:28 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

105

11:20:28 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

106

11:20:28 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

107

11:20:28 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2935ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

108

11:20:28 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

109

11:20:28 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

110

11:20:28 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

111

11:20:28 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.828896 seconds.

112

11:20:28 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 146 MiB

113

11:20:28 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

114

11:20:28 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

115

11:20:28 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

116

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

117

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

118

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

119

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

120

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

121

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 2, GPU 138 (MiB)

122

11:20:28 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

123

11:20:29 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +715, GPU +2, now: CPU 20125, GPU 1416 (MiB)

124

11:20:29 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

125

11:20:30 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

126

11:20:30 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

127

11:20:30 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

128

11:20:30 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

129

11:20:30 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

130

11:20:30 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

131

11:20:30 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2517ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

132

11:20:30 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

133

11:20:30 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

134

11:20:30 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

135

11:20:30 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.780999 seconds.

136

11:20:30 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 159 MiB

137

11:20:30 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

138

11:20:30 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

139

11:20:30 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

140

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

141

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

142

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

143

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

144

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

145

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 3, GPU 151 (MiB)

146

11:20:30 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

147

11:20:31 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +749, GPU +0, now: CPU 20115, GPU 1430 (MiB)

148

11:20:31 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

149

11:20:31 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

150

11:20:32 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 10 inputs and 6 output network tensors.

151

11:20:32 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

152

11:20:32 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

153

11:20:32 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

154

11:20:32 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

155

11:20:32 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2858ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

156

11:20:32 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

157

11:20:32 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

158

11:20:32 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

159

11:20:32 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.773004 seconds.

160

11:20:32 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 171 MiB

161

11:20:32 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

162

11:20:32 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

163

11:20:32 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

164

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

165

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

166

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

167

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

168

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

169

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 3, GPU 164 (MiB)

170

11:20:32 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

171

11:20:33 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +728, GPU +0, now: CPU 20107, GPU 1444 (MiB)

172

11:20:33 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

173

11:20:34 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

174

11:20:34 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 7 inputs and 8 output network tensors.

175

11:20:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 379968 bytes

176

11:20:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

177

11:20:34 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

178

11:20:34 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 80 steps to complete.

179

11:20:34 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.6269ms to assign 5 blocks to 80 nodes requiring 10485760 bytes.

180

11:20:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

181

11:20:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 6769344 bytes

182

11:20:34 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

183

11:20:34 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.35131 seconds.

184

11:20:34 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 183 MiB

185

11:20:34 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

186

11:20:34 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

187

11:20:34 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

188

INFO: [Torch-TensorRT] - Loaded engine size: 7 MiB

189

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

190

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

191

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

192

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

193

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 4, GPU 181 (MiB)

194

11:20:35 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

195

11:20:35 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +717, GPU +2, now: CPU 20119, GPU 1464 (MiB)

196

11:20:35 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

197

11:20:36 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

198

11:20:36 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

199

11:20:36 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

200

11:20:36 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

201

11:20:36 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

202

11:20:36 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

203

11:20:36 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3196ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

204

11:20:36 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

205

11:20:36 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

206

11:20:36 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

207

11:20:36 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.766223 seconds.

208

11:20:36 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 202 MiB

209

11:20:36 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

210

11:20:36 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

211

11:20:36 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

212

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

213

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

214

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

215

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

216

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

217

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 4, GPU 194 (MiB)

218

11:20:36 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

219

11:20:37 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +718, GPU +2, now: CPU 20118, GPU 1478 (MiB)

220

11:20:37 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

221

11:20:38 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

222

11:20:38 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

223

11:20:38 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

224

11:20:38 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

225

11:20:38 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

226

11:20:38 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

227

11:20:38 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3123ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

228

11:20:38 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

229

11:20:38 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

230

11:20:38 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

231

11:20:38 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.766578 seconds.

232

11:20:38 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 215 MiB

233

11:20:38 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

234

11:20:38 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

235

11:20:38 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

236

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

237

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

238

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

239

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

240

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

241

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 4, GPU 207 (MiB)

242

11:20:38 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

243

11:20:39 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +713, GPU +2, now: CPU 20129, GPU 1492 (MiB)

244

11:20:39 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

245

11:20:39 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

246

11:20:40 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

247

11:20:40 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

248

11:20:40 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

249

11:20:40 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

250

11:20:40 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

251

11:20:40 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2769ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

252

11:20:40 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

253

11:20:40 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

254

11:20:40 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

255

11:20:40 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.736688 seconds.

256

11:20:40 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 228 MiB

257

11:20:40 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

258

11:20:40 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

259

11:20:40 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

260

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

261

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

262

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

263

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

264

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

265

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 5, GPU 220 (MiB)

266

11:20:40 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

267

11:20:41 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +736, GPU +0, now: CPU 20132, GPU 1506 (MiB)

268

11:20:41 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

269

11:20:41 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

270

11:20:41 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

271

11:20:42 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

272

11:20:42 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

273

11:20:42 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

274

11:20:42 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

275

11:20:42 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2479ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

276

11:20:42 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

277

11:20:42 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

278

11:20:42 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

279

11:20:42 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.754449 seconds.

280

11:20:42 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 241 MiB

281

11:20:42 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

282

11:20:42 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

283

11:20:42 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

284

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

285

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

286

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

287

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

288

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

289

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 5, GPU 233 (MiB)

290

11:20:42 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

291

11:20:43 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +727, GPU +0, now: CPU 20153, GPU 1520 (MiB)

292

11:20:43 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

293

11:20:43 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

294

11:20:43 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

295

11:20:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

296

11:20:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

297

11:20:43 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

298

11:20:43 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

299

11:20:43 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3089ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

300

11:20:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

301

11:20:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

302

11:20:43 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

303

11:20:43 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.77584 seconds.

304

11:20:43 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 254 MiB

305

11:20:44 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

306

11:20:44 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

307

11:20:44 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

308

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

309

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

310

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

311

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

312

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

313

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 5, GPU 247 (MiB)

314

11:20:44 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

315

11:20:44 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +704, GPU +2, now: CPU 20148, GPU 1536 (MiB)

316

11:20:45 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

317

11:20:45 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

318

11:20:45 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

319

11:20:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

320

11:20:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

321

11:20:45 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

322

11:20:45 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

323

11:20:45 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3358ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

324

11:20:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

325

11:20:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

326

11:20:45 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

327

11:20:45 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.765545 seconds.

328

11:20:45 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 268 MiB

329

11:20:45 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

330

11:20:45 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

331

11:20:45 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

332

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

333

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

334

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

335

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

336

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

337

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 6, GPU 260 (MiB)

338

11:20:46 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

339

11:20:46 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +729, GPU +2, now: CPU 20149, GPU 1550 (MiB)

340

11:20:46 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

341

11:20:47 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

342

11:20:47 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

343

11:20:47 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

344

11:20:47 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

345

11:20:47 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

346

11:20:47 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

347

11:20:47 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2889ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

348

11:20:47 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

349

11:20:47 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

350

11:20:47 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

351

11:20:47 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.766297 seconds.

352

11:20:47 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 281 MiB

353

11:20:47 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

354

11:20:47 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

355

11:20:47 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

356

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

357

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

358

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

359

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

360

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

361

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 6, GPU 273 (MiB)

362

11:20:47 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

363

11:20:48 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +716, GPU +2, now: CPU 20159, GPU 1568 (MiB)

364

11:20:48 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

365

11:20:48 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

366

11:20:49 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 6 output network tensors.

367

11:20:49 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

368

11:20:49 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

369

11:20:49 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

370

11:20:49 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

371

11:20:49 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2508ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

372

11:20:49 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

373

11:20:49 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

374

11:20:49 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

375

11:20:49 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.74689 seconds.

376

11:20:49 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 293 MiB

377

11:20:49 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

378

11:20:49 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

379

11:20:49 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

380

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

381

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

382

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

383

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

384

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

385

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 6, GPU 286 (MiB)

386

11:20:49 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

387

11:20:50 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +743, GPU +0, now: CPU 20160, GPU 1582 (MiB)

388

11:20:50 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

389

11:20:51 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

390

11:20:51 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 8 inputs and 8 output network tensors.

391

11:20:51 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 379968 bytes

392

11:20:51 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

393

11:20:51 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

394

11:20:51 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 81 steps to complete.

395

11:20:51 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.5298ms to assign 6 blocks to 81 nodes requiring 10486272 bytes.

396

11:20:51 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

397

11:20:51 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 7441088 bytes

398

11:20:51 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

399

11:20:51 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.26093 seconds.

400

11:20:51 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 305 MiB

401

11:20:51 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

402

11:20:51 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

403

11:20:51 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

404

INFO: [Torch-TensorRT] - Loaded engine size: 8 MiB

405

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

406

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

407

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

408

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

409

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 7, GPU 303 (MiB)

410

11:20:52 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

411

11:20:52 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +765, GPU +0, now: CPU 20210, GPU 1600 (MiB)

412

11:20:52 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

413

11:20:53 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

414

11:20:53 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

415

11:20:53 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

416

11:20:53 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

417

11:20:53 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

418

11:20:53 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

419

11:20:53 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

420

11:20:53 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3229ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

421

11:20:53 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

422

11:20:53 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

423

11:20:53 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

424

11:20:53 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.81125 seconds.

425

11:20:53 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 324 MiB

426

11:20:53 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

427

11:20:53 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

428

11:20:53 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

429

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

430

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

431

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

432

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

433

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

434

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 7, GPU 316 (MiB)

435

11:20:53 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

436

11:20:54 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +744, GPU +2, now: CPU 20195, GPU 1616 (MiB)

437

11:20:54 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

438

11:20:55 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

439

11:20:55 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

440

11:20:55 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

441

11:20:55 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

442

11:20:55 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

443

11:20:55 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

444

11:20:55 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

445

11:20:55 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3399ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

446

11:20:55 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

447

11:20:55 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

448

11:20:55 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

449

11:20:55 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.805102 seconds.

450

11:20:55 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 337 MiB

451

11:20:55 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

452

11:20:55 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

453

11:20:55 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

454

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

455

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

456

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

457

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

458

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

459

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 8, GPU 330 (MiB)

460

11:20:55 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

461

11:20:56 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +684, GPU +2, now: CPU 20198, GPU 1630 (MiB)

462

11:20:56 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

463

11:20:56 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

464

11:20:57 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

465

11:20:57 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

466

11:20:57 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

467

11:20:57 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

468

11:20:57 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

469

11:20:57 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

470

11:20:57 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.319ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

471

11:20:57 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

472

11:20:57 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

473

11:20:57 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

474

11:20:57 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.78612 seconds.

475

11:20:57 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 351 MiB

476

11:20:57 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

477

11:20:57 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

478

11:20:57 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

479

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

480

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

481

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

482

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

483

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

484

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 8, GPU 343 (MiB)

485

11:20:57 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

486

11:20:58 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +749, GPU +2, now: CPU 20206, GPU 1644 (MiB)

487

11:20:58 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

488

11:20:58 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

489

11:20:58 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

490

11:20:58 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

491

11:20:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

492

11:20:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

493

11:20:59 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

494

11:20:59 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

495

11:20:59 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3791ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

496

11:20:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

497

11:20:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

498

11:20:59 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

499

11:20:59 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.781787 seconds.

500

11:20:59 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 364 MiB

501

11:20:59 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

502

11:20:59 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

503

11:20:59 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

504

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

505

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

506

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

507

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

508

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

509

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 8, GPU 356 (MiB)

510

11:20:59 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

511

11:21:00 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +696, GPU +0, now: CPU 20214, GPU 1658 (MiB)

512

11:21:00 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

513

11:21:00 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

514

11:21:00 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

515

11:21:00 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

516

11:21:00 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

517

11:21:00 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

518

11:21:00 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

519

11:21:00 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

520

11:21:00 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2631ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

521

11:21:00 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

522

11:21:01 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

523

11:21:01 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

524

11:21:01 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.77001 seconds.

525

11:21:01 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 377 MiB

526

11:21:01 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

527

11:21:01 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

528

11:21:01 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

529

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

530

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

531

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

532

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

533

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

534

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 8, GPU 369 (MiB)

535

11:21:01 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

536

11:21:02 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +751, GPU +0, now: CPU 20219, GPU 1672 (MiB)

537

11:21:02 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

538

11:21:02 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

539

11:21:02 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

540

11:21:02 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

541

11:21:02 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

542

11:21:02 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

543

11:21:02 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

544

11:21:02 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

545

11:21:02 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3085ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

546

11:21:02 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

547

11:21:02 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

548

11:21:02 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

549

11:21:02 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.78596 seconds.

550

11:21:02 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 390 MiB

551

11:21:02 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

552

11:21:02 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

553

11:21:02 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

554

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

555

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

556

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

557

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

558

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

559

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 9, GPU 383 (MiB)

560

11:21:03 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

561

11:21:03 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +707, GPU +2, now: CPU 20226, GPU 1688 (MiB)

562

11:21:03 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

563

11:21:04 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

564

11:21:04 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

565

11:21:04 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

566

11:21:04 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

567

11:21:04 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

568

11:21:04 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

569

11:21:04 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

570

11:21:04 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3007ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

571

11:21:04 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

572

11:21:04 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

573

11:21:04 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

574

11:21:04 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.791843 seconds.

575

11:21:04 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 404 MiB

576

11:21:04 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

577

11:21:04 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

578

11:21:04 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

579

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

580

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

581

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

582

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

583

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

584

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 9, GPU 396 (MiB)

585

11:21:04 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

586

11:21:05 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +715, GPU +2, now: CPU 20232, GPU 1702 (MiB)

587

11:21:05 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

588

11:21:06 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

589

11:21:06 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 6 output network tensors.

590

11:21:06 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

591

11:21:06 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

592

11:21:06 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

593

11:21:06 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

594

11:21:06 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3023ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

595

11:21:06 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

596

11:21:06 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

597

11:21:06 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

598

11:21:06 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.783161 seconds.

599

11:21:06 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 416 MiB

600

11:21:06 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

601

11:21:06 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

602

11:21:06 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

603

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

604

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

605

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

606

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

607

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

608

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 9, GPU 409 (MiB)

609

11:21:06 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

610

11:21:07 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +750, GPU +0, now: CPU 20250, GPU 1716 (MiB)

611

11:21:07 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

612

11:21:08 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

613

11:21:08 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 10 inputs and 6 output network tensors.

614

11:21:09 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 465008 bytes

615

11:21:09 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

616

11:21:09 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

617

11:21:09 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 108 steps to complete.

618

11:21:09 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 1.3912ms to assign 7 blocks to 108 nodes requiring 22052864 bytes.

619

11:21:09 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

620

11:21:09 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 10556224 bytes

621

11:21:09 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

622

11:21:09 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.84104 seconds.

623

11:21:09 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 428 MiB

624

11:21:09 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

625

11:21:09 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

626

11:21:09 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

627

INFO: [Torch-TensorRT] - Loaded engine size: 11 MiB

628

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

629

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

630

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

631

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

632

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 10, GPU 440 (MiB)

633

11:21:09 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

634

11:21:10 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +746, GPU +0, now: CPU 20260, GPU 1750 (MiB)

635

11:21:10 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

636

11:21:10 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

637

11:21:11 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

638

11:21:11 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

639

11:21:11 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

640

11:21:11 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

641

11:21:11 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

642

11:21:11 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

643

11:21:11 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.8445ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

644

11:21:11 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

645

11:21:11 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

646

11:21:11 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

647

11:21:11 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.19394 seconds.

648

11:21:11 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 459 MiB

649

11:21:11 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

650

11:21:11 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

651

11:21:11 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

652

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

653

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

654

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

655

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

656

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

657

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +21, now: CPU 11, GPU 466 (MiB)

658

11:21:12 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

659

11:21:12 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +752, GPU +4, now: CPU 20272, GPU 1782 (MiB)

660

11:21:12 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

661

11:21:13 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

662

11:21:13 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

663

11:21:13 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

664

11:21:14 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

665

11:21:14 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

666

11:21:14 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

667

11:21:14 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

668

11:21:14 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.8879ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

669

11:21:14 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

670

11:21:14 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

671

11:21:14 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

672

11:21:14 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.18345 seconds.

673

11:21:14 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 485 MiB

674

11:21:14 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

675

11:21:14 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

676

11:21:14 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

677

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

678

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

679

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

680

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

681

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

682

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 11, GPU 492 (MiB)

683

11:21:14 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

684

11:21:15 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +757, GPU +4, now: CPU 20282, GPU 1810 (MiB)

685

11:21:15 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

686

11:21:15 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

687

11:21:15 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

688

11:21:15 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

689

11:21:16 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

690

11:21:16 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

691

11:21:16 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

692

11:21:16 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

693

11:21:16 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.8397ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

694

11:21:16 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

695

11:21:16 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

696

11:21:16 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

697

11:21:16 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.15668 seconds.

698

11:21:16 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 511 MiB

699

11:21:16 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

700

11:21:16 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

701

11:21:16 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

702

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

703

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

704

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

705

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

706

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

707

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +21, now: CPU 12, GPU 518 (MiB)

708

11:21:16 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

709

11:21:17 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +744, GPU +0, now: CPU 20291, GPU 1838 (MiB)

710

11:21:17 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

711

11:21:17 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

712

11:21:18 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

713

11:21:18 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

714

11:21:18 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

715

11:21:18 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

716

11:21:18 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

717

11:21:18 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

718

11:21:18 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 1.1741ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

719

11:21:18 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

720

11:21:18 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

721

11:21:18 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

722

11:21:18 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.17524 seconds.

723

11:21:18 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 537 MiB

724

11:21:18 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

725

11:21:18 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

726

11:21:18 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

727

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

728

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

729

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

730

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

731

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

732

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 12, GPU 544 (MiB)

733

11:21:18 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

734

11:21:19 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +703, GPU +0, now: CPU 20317, GPU 1866 (MiB)

735

11:21:19 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

736

11:21:19 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

737

11:21:20 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

738

11:21:20 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

739

11:21:20 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

740

11:21:20 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

741

11:21:20 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

742

11:21:20 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

743

11:21:20 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 1.0181ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

744

11:21:20 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

745

11:21:20 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

746

11:21:20 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

747

11:21:20 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.15952 seconds.

748

11:21:20 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 563 MiB

749

11:21:20 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

750

11:21:20 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

751

11:21:20 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

752

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

753

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

754

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

755

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

756

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

757

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 12, GPU 571 (MiB)

758

11:21:21 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

759

11:21:21 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +771, GPU +2, now: CPU 20358, GPU 1898 (MiB)

760

11:21:21 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

761

11:21:22 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

762

11:21:22 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

763

11:21:22 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

764

11:21:23 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

765

11:21:23 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

766

11:21:23 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

767

11:21:23 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

768

11:21:23 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9751ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

769

11:21:23 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

770

11:21:23 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

771

11:21:23 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

772

11:21:23 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.18046 seconds.

773

11:21:23 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 590 MiB

774

11:21:23 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

775

11:21:23 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

776

11:21:23 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

777

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

778

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

779

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

780

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

781

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

782

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 13, GPU 597 (MiB)

783

11:21:23 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

784

11:21:24 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +720, GPU +2, now: CPU 20328, GPU 1926 (MiB)

785

11:21:24 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

786

11:21:24 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

787

11:21:24 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

788

11:21:24 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

789

11:21:25 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

790

11:21:25 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

791

11:21:25 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

792

11:21:25 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

793

11:21:25 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.881ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

794

11:21:25 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

795

11:21:25 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

796

11:21:25 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

797

11:21:25 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.15464 seconds.

798

11:21:25 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 616 MiB

799

11:21:25 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

800

11:21:25 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

801

11:21:25 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

802

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

803

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

804

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

805

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

806

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

807

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 13, GPU 623 (MiB)

808

11:21:25 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

809

11:21:26 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +723, GPU +0, now: CPU 20340, GPU 1954 (MiB)

810

11:21:26 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

811

11:21:26 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

812

11:21:27 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

813

11:21:27 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 14 inputs and 4 output network tensors.

814

11:21:27 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

815

11:21:27 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

816

11:21:27 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

817

11:21:27 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 77 steps to complete.

818

11:21:27 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9905ms to assign 8 blocks to 77 nodes requiring 22053376 bytes.

819

11:21:27 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

820

11:21:27 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

821

11:21:27 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

822

11:21:27 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.18474 seconds.

823

11:21:27 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 641 MiB

824

11:21:27 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

825

11:21:27 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

826

11:21:27 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

827

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

828

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

829

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

830

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

831

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

832

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 14, GPU 649 (MiB)

833

11:21:27 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

834

11:21:28 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +708, GPU +2, now: CPU 20354, GPU 1984 (MiB)

835

11:21:28 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

836

11:21:29 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 1 output network tensors.

837

11:21:29 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 263984 bytes

838

11:21:29 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

839

11:21:29 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 0 bytes

840

11:21:29 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 70 steps to complete.

841

11:21:29 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.6134ms to assign 8 blocks to 70 nodes requiring 21495808 bytes.

842

11:21:29 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 21495808 bytes

843

11:21:29 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 4384288 bytes

844

11:21:29 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.15378 seconds.

845

11:21:29 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 23 MiB, GPU 658 MiB

846

11:21:29 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

847

11:21:29 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 1454467 bytes of compilation cache.

848

11:21:29 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 5053 timing cache entries

849

INFO: [Torch-TensorRT] - Loaded engine size: 5 MiB

850

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +20, now: CPU 14, GPU 673 (MiB)

851

11:21:30 torch_tensorrt._compile WARNING: Provided model is a torch.fx.GraphModule and retrace is False, inputs or arg_inputs is not necessary during save.

852

11:21:32 py.warnings WARNING: torch_tensorrt\dynamo\_exporter.py:396: UserWarning: Attempted to insert a get_attr Node with no underlying reference in the owning GraphModule! Call GraphModule.add_submodule to add the necessary submodule, GraphModule.add_parameter to add the necessary Parameter, or nn.Module.register_buffer to add the necessary buffer

853

engine_node = gm.graph.get_attr(engine_name)

854

855

11:21:32 py.warnings WARNING: torch\export\exported_program.py:1681: UserWarning: Unable to execute the generated python source code from the graph. The graph module will no longer be directly callable, but you can still run the ExportedProgram, and if needed, you can run the graph module eagerly using torch.fx.Interpreter.

856

warnings.warn(

857

858

W0126 11:21:32.310000 15968 D:\Program Files\jasna\torch\export\pt2_archive\_package.py:586] Expect archive file to be a file ending in .pt2, or is a buffer. Instead got {model_weights\lada_mosaic_restoration_model_generic_v1.2_clip10.trt_fp16.win.engine}

859

Compiling BasicVSR++ model (TensorRT workspace_size=9.42 GB). For large clip length > 100 this can take even few hours.

860

11:21:37 py.warnings WARNING: torch_tensorrt\dynamo\utils.py:307: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.detach().clone() or sourceTensor.detach().clone().requires_grad_(True), rather than torch.tensor(sourceTensor).

861

torch.tensor(inputs),

862

863

11:24:54 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +136, GPU +0, now: CPU 20569, GPU 1330 (MiB)

864

11:25:04 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

865

11:25:15 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

866

11:30:04 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 1 inputs and 1525 output network tensors.

867

11:30:07 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 676576 bytes

868

11:30:07 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

869

11:30:07 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 5783552 bytes

870

11:30:07 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 205 steps to complete.

871

11:30:07 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 25.9975ms to assign 29 blocks to 205 nodes requiring 313214976 bytes.

872

11:30:07 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 313213952 bytes

873

11:30:07 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 10785602 bytes

874

11:30:07 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

875

11:30:07 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 303.043 seconds.

876

11:30:07 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

877

11:30:07 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

878

11:30:07 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

879

11:30:07 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

880

INFO: [Torch-TensorRT] - Loaded engine size: 19 MiB

881

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

882

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 7

883

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 8

884

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

885

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +298, now: CPU 1, GPU 308 (MiB)

886

11:30:09 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

887

11:30:10 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +771, GPU +0, now: CPU 20386, GPU 1654 (MiB)

888

11:30:10 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

889

11:30:10 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

890

11:30:10 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

891

11:30:10 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

892

11:30:10 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

893

11:30:10 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

894

11:30:10 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

895

11:30:10 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3145ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

896

11:30:10 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

897

11:30:10 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

898

11:30:10 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

899

11:30:10 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.772236 seconds.

900

11:30:10 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

901

11:30:11 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

902

11:30:11 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

903

11:30:11 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

904

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

905

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

906

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

907

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

908

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

909

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 1, GPU 322 (MiB)

910

11:30:11 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

911

11:30:12 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +721, GPU +0, now: CPU 20372, GPU 1668 (MiB)

912

11:30:12 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

913

11:30:12 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

914

11:30:12 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

915

11:30:13 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

916

11:30:13 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

917

11:30:13 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

918

11:30:13 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

919

11:30:13 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2878ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

920

11:30:13 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

921

11:30:13 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

922

11:30:13 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

923

11:30:13 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.783385 seconds.

924

11:30:13 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

925

11:30:13 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

926

11:30:13 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

927

11:30:13 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

928

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

929

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

930

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

931

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

932

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

933

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 1, GPU 335 (MiB)

934

11:30:13 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

935

11:30:14 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +740, GPU +0, now: CPU 20381, GPU 1682 (MiB)

936

11:30:14 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

937

11:30:15 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

938

11:30:15 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

939

11:30:15 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

940

11:30:15 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

941

11:30:15 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

942

11:30:15 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

943

11:30:15 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2507ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

944

11:30:15 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

945

11:30:15 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

946

11:30:15 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

947

11:30:15 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.774809 seconds.

948

11:30:15 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

949

11:30:15 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

950

11:30:15 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

951

11:30:15 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

952

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

953

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

954

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

955

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

956

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

957

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 2, GPU 348 (MiB)

958

11:30:16 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

959

11:30:17 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +743, GPU +0, now: CPU 20385, GPU 1696 (MiB)

960

11:30:17 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

961

11:30:17 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

962

11:30:17 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

963

11:30:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

964

11:30:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

965

11:30:17 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

966

11:30:17 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

967

11:30:17 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3185ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

968

11:30:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

969

11:30:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

970

11:30:17 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

971

11:30:17 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.777895 seconds.

972

11:30:17 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

973

11:30:18 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

974

11:30:18 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

975

11:30:18 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

976

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

977

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

978

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

979

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

980

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

981

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 2, GPU 361 (MiB)

982

11:30:18 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

983

11:30:19 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +724, GPU +0, now: CPU 20410, GPU 1710 (MiB)

984

11:30:19 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

985

11:30:19 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

986

11:30:19 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

987

11:30:20 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

988

11:30:20 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

989

11:30:20 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

990

11:30:20 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

991

11:30:20 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3223ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

992

11:30:20 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

993

11:30:20 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

994

11:30:20 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

995

11:30:20 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.773337 seconds.

996

11:30:20 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

997

11:30:20 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

998

11:30:20 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

999

11:30:20 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1000

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1001

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1002

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1003

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1004

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1005

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 2, GPU 374 (MiB)

1006

11:30:20 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1007

11:30:21 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +750, GPU +0, now: CPU 20404, GPU 1724 (MiB)

1008

11:30:21 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1009

11:30:22 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1010

11:30:22 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1011

11:30:22 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1012

11:30:22 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1013

11:30:22 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1014

11:30:22 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1015

11:30:22 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3195ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1016

11:30:22 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1017

11:30:22 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1018

11:30:22 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1019

11:30:22 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.792165 seconds.

1020

11:30:22 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1021

11:30:22 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1022

11:30:22 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1023

11:30:22 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1024

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1025

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1026

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1027

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1028

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1029

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 2, GPU 387 (MiB)

1030

11:30:23 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1031

11:30:24 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +731, GPU +0, now: CPU 20422, GPU 1738 (MiB)

1032

11:30:24 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1033

11:30:24 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1034

11:30:24 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1035

11:30:24 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1036

11:30:24 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1037

11:30:24 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1038

11:30:24 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1039

11:30:24 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2533ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1040

11:30:24 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1041

11:30:24 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1042

11:30:24 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1043

11:30:24 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.779684 seconds.

1044

11:30:24 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1045

11:30:25 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1046

11:30:25 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1047

11:30:25 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1048

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1049

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1050

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1051

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1052

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1053

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 3, GPU 400 (MiB)

1054

11:30:25 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1055

11:30:26 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +693, GPU +0, now: CPU 20422, GPU 1752 (MiB)

1056

11:30:26 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1057

11:30:26 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1058

11:30:26 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1059

11:30:27 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1060

11:30:27 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1061

11:30:27 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1062

11:30:27 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1063

11:30:27 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3479ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1064

11:30:27 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1065

11:30:27 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1066

11:30:27 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1067

11:30:27 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.782922 seconds.

1068

11:30:27 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1069

11:30:27 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1070

11:30:27 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1071

11:30:27 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1072

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1073

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1074

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1075

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1076

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1077

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 3, GPU 413 (MiB)

1078

11:30:27 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1079

11:30:28 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +743, GPU +2, now: CPU 20437, GPU 1768 (MiB)

1080

11:30:28 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1081

11:30:29 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1082

11:30:29 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1083

11:30:29 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1084

11:30:29 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1085

11:30:29 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1086

11:30:29 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1087

11:30:29 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3585ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1088

11:30:29 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1089

11:30:29 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1090

11:30:29 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1091

11:30:29 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.780015 seconds.

1092

11:30:29 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1093

11:30:29 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1094

11:30:29 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1095

11:30:29 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1096

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1097

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1098

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1099

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1100

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1101

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 3, GPU 426 (MiB)

1102

11:30:30 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1103

11:30:30 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +708, GPU +2, now: CPU 20440, GPU 1782 (MiB)

1104

11:30:31 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1105

11:30:31 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1106

11:30:31 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1107

11:30:31 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1108

11:30:31 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1109

11:30:31 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1110

11:30:31 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1111

11:30:31 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2333ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1112

11:30:31 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1113

11:30:31 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1114

11:30:31 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1115

11:30:31 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.808921 seconds.

1116

11:30:31 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1117

11:30:31 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1118

11:30:32 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1119

11:30:32 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1120

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1121

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1122

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1123

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1124

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1125

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 4, GPU 440 (MiB)

1126

11:30:32 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1127

11:30:33 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +772, GPU +2, now: CPU 20459, GPU 1796 (MiB)

1128

11:30:33 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1129

11:30:33 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1130

11:30:33 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1131

11:30:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1132

11:30:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1133

11:30:34 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1134

11:30:34 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1135

11:30:34 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3337ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1136

11:30:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1137

11:30:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1138

11:30:34 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1139

11:30:34 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.80434 seconds.

1140

11:30:34 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1141

11:30:34 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1142

11:30:34 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1143

11:30:34 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1144

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1145

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1146

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1147

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1148

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1149

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 4, GPU 453 (MiB)

1150

11:30:34 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1151

11:30:35 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +692, GPU +0, now: CPU 20462, GPU 1810 (MiB)

1152

11:30:35 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1153

11:30:36 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1154

11:30:36 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1155

11:30:36 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1156

11:30:36 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1157

11:30:36 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1158

11:30:36 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1159

11:30:36 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.4044ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1160

11:30:36 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1161

11:30:36 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1162

11:30:36 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1163

11:30:36 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.784557 seconds.

1164

11:30:36 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1165

11:30:36 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1166

11:30:36 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1167

11:30:36 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1168

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1169

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1170

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1171

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1172

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1173

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 4, GPU 466 (MiB)

1174

11:30:37 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1175

11:30:37 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +795, GPU +0, now: CPU 20499, GPU 1824 (MiB)

1176

11:30:37 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1177

11:30:38 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1178

11:30:38 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1179

11:30:38 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1180

11:30:38 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1181

11:30:38 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1182

11:30:38 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1183

11:30:38 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2568ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1184

11:30:38 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1185

11:30:38 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1186

11:30:38 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1187

11:30:38 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.788323 seconds.

1188

11:30:38 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1189

11:30:38 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1190

11:30:38 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1191

11:30:38 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1192

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1193

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1194

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1195

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1196

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1197

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 5, GPU 479 (MiB)

1198

11:30:39 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1199

11:30:40 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +758, GPU +2, now: CPU 20452, GPU 1840 (MiB)

1200

11:30:40 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1201

11:30:40 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1202

11:30:40 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1203

11:30:41 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1204

11:30:41 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1205

11:30:41 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1206

11:30:41 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1207

11:30:41 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3169ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1208

11:30:41 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1209

11:30:41 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1210

11:30:41 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1211

11:30:41 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.790677 seconds.

1212

11:30:41 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1213

11:30:41 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1214

11:30:41 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1215

11:30:41 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1216

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1217

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1218

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1219

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1220

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1221

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 5, GPU 492 (MiB)

1222

11:30:41 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1223

11:30:42 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +756, GPU +2, now: CPU 20455, GPU 1854 (MiB)

1224

11:30:42 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1225

11:30:42 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1226

11:30:42 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1227

11:30:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1228

11:30:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1229

11:30:43 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1230

11:30:43 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1231

11:30:43 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2691ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1232

11:30:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1233

11:30:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1234

11:30:43 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1235

11:30:43 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.788309 seconds.

1236

11:30:43 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1237

11:30:43 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1238

11:30:43 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1239

11:30:43 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1240

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1241

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1242

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1243

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1244

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1245

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 5, GPU 505 (MiB)

1246

11:30:43 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1247

11:30:44 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +738, GPU +0, now: CPU 20462, GPU 1868 (MiB)

1248

11:30:44 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1249

11:30:45 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1250

11:30:45 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1251

11:30:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1252

11:30:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1253

11:30:45 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1254

11:30:45 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1255

11:30:45 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2459ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1256

11:30:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1257

11:30:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1258

11:30:45 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1259

11:30:45 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.769873 seconds.

1260

11:30:45 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1261

11:30:45 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1262

11:30:45 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1263

11:30:45 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1264

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1265

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1266

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1267

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1268

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1269

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 6, GPU 518 (MiB)

1270

11:30:46 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1271

11:30:46 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +756, GPU +0, now: CPU 20467, GPU 1882 (MiB)

1272

11:30:47 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1273

11:30:47 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1274

11:30:47 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1275

11:30:47 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1276

11:30:47 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1277

11:30:47 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1278

11:30:47 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1279

11:30:47 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2556ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1280

11:30:47 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1281

11:30:47 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1282

11:30:47 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1283

11:30:47 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.777338 seconds.

1284

11:30:47 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1285

11:30:47 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1286

11:30:47 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1287

11:30:47 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1288

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1289

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1290

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1291

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1292

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1293

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 6, GPU 531 (MiB)

1294

11:30:48 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1295

11:30:49 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +749, GPU +2, now: CPU 20480, GPU 1898 (MiB)

1296

11:30:49 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1297

11:30:49 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1298

11:30:49 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1299

11:30:50 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1300

11:30:50 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1301

11:30:50 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1302

11:30:50 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1303

11:30:50 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3585ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1304

11:30:50 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1305

11:30:50 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1306

11:30:50 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1307

11:30:50 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.792508 seconds.

1308

11:30:50 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1309

11:30:50 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1310

11:30:50 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1311

11:30:50 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1312

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1313

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1314

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1315

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1316

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1317

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 6, GPU 544 (MiB)

1318

11:30:50 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1319

11:30:51 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +743, GPU +2, now: CPU 20507, GPU 1912 (MiB)

1320

11:30:51 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1321

11:30:51 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1322

11:30:51 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1323

11:30:52 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1324

11:30:52 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1325

11:30:52 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1326

11:30:52 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1327

11:30:52 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2503ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1328

11:30:52 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1329

11:30:52 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1330

11:30:52 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1331

11:30:52 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.79676 seconds.

1332

11:30:52 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1333

11:30:52 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1334

11:30:52 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1335

11:30:52 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1336

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1337

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1338

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1339

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1340

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1341

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 7, GPU 558 (MiB)

1342

11:30:52 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1343

11:30:53 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +769, GPU +2, now: CPU 20507, GPU 1926 (MiB)

1344

11:30:53 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1345

11:30:54 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1346

11:30:54 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1347

11:30:54 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1348

11:30:54 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1349

11:30:54 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1350

11:30:54 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1351

11:30:54 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2554ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1352

11:30:54 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1353

11:30:54 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1354

11:30:54 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1355

11:30:54 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.774476 seconds.

1356

11:30:54 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1357

11:30:54 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1358

11:30:54 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1359

11:30:54 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1360

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1361

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1362

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1363

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1364

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1365

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 7, GPU 571 (MiB)

1366

11:30:55 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1367

11:30:55 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +725, GPU +0, now: CPU 20514, GPU 1940 (MiB)

1368

11:30:55 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1369

11:30:56 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1370

11:30:56 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1371

11:30:56 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1372

11:30:56 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1373

11:30:56 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1374

11:30:56 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1375

11:30:56 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2715ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1376

11:30:56 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1377

11:30:56 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1378

11:30:56 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1379

11:30:56 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.781695 seconds.

1380

11:30:56 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1381

11:30:56 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1382

11:30:56 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1383

11:30:56 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1384

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1385

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1386

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1387

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1388

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1389

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 7, GPU 584 (MiB)

1390

11:30:57 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1391

11:30:58 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +693, GPU +0, now: CPU 20527, GPU 1954 (MiB)

1392

11:30:58 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1393

11:30:58 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1394

11:30:58 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1395

11:30:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1396

11:30:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1397

11:30:59 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1398

11:30:59 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1399

11:30:59 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3644ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1400

11:30:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1401

11:30:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1402

11:30:59 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1403

11:30:59 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.805517 seconds.

1404

11:30:59 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1405

11:30:59 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1406

11:30:59 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1407

11:30:59 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1408

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1409

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1410

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1411

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1412

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1413

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 8, GPU 597 (MiB)

1414

11:30:59 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1415

11:31:00 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +763, GPU +2, now: CPU 20535, GPU 1970 (MiB)

1416

11:31:00 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1417

11:31:00 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1418

11:31:00 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1419

11:31:01 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1420

11:31:01 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1421

11:31:01 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1422

11:31:01 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1423

11:31:01 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2941ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1424

11:31:01 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1425

11:31:01 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1426

11:31:01 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1427

11:31:01 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.792818 seconds.

1428

11:31:01 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1429

11:31:01 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1430

11:31:01 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1431

11:31:01 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1432

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1433

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1434

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1435

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1436

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1437

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 8, GPU 610 (MiB)

1438

11:31:01 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1439

11:31:02 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +697, GPU +2, now: CPU 20521, GPU 1984 (MiB)

1440

11:31:02 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1441

11:31:03 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1442

11:31:03 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1443

11:31:03 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1444

11:31:03 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1445

11:31:03 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1446

11:31:03 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1447

11:31:03 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2579ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1448

11:31:03 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1449

11:31:03 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1450

11:31:03 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1451

11:31:03 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.789292 seconds.

1452

11:31:03 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1453

11:31:03 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1454

11:31:03 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1455

11:31:03 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1456

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1457

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1458

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1459

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1460

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1461

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 8, GPU 623 (MiB)

1462

11:31:04 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1463

11:31:04 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +691, GPU +4, now: CPU 20528, GPU 2000 (MiB)

1464

11:31:04 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1465

11:31:05 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1466

11:31:05 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1467

11:31:05 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1468

11:31:05 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1469

11:31:05 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1470

11:31:05 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1471

11:31:05 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2564ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1472

11:31:05 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1473

11:31:05 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1474

11:31:05 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1475

11:31:05 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.776042 seconds.

1476

11:31:05 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1477

11:31:05 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1478

11:31:05 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1479

11:31:05 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1480

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1481

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1482

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1483

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1484

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1485

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 9, GPU 636 (MiB)

1486

11:31:06 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1487

11:31:07 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +695, GPU +2, now: CPU 20537, GPU 2014 (MiB)

1488

11:31:07 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1489

11:31:07 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1490

11:31:07 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1491

11:31:07 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1492

11:31:07 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1493

11:31:07 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1494

11:31:07 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1495

11:31:07 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2781ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1496

11:31:07 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1497

11:31:07 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1498

11:31:07 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1499

11:31:07 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.822033 seconds.

1500

11:31:07 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1501

11:31:08 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1502

11:31:08 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1503

11:31:08 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1504

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1505

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1506

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1507

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1508

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1509

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 9, GPU 649 (MiB)

1510

11:31:08 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1511

11:31:09 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +679, GPU +0, now: CPU 20545, GPU 2028 (MiB)

1512

11:31:09 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1513

11:31:09 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1514

11:31:09 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1515

11:31:10 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1516

11:31:10 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1517

11:31:10 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1518

11:31:10 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1519

11:31:10 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2603ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1520

11:31:10 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1521

11:31:10 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1522

11:31:10 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1523

11:31:10 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.799066 seconds.

1524

11:31:10 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1525

11:31:10 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1526

11:31:10 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1527

11:31:10 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1528

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1529

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1530

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1531

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1532

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1533

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 9, GPU 662 (MiB)

1534

11:31:10 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1535

11:31:11 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +690, GPU +2, now: CPU 20557, GPU 2044 (MiB)

1536

11:31:11 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1537

11:31:11 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1538

11:31:12 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1539

11:31:12 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1540

11:31:12 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1541

11:31:12 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1542

11:31:12 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1543

11:31:12 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3496ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1544

11:31:12 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1545

11:31:12 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1546

11:31:12 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1547

11:31:12 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.780956 seconds.

1548

11:31:12 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1549

11:31:12 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1550

11:31:12 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1551

11:31:12 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1552

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1553

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1554

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1555

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1556

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1557

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 10, GPU 675 (MiB)

1558

11:31:12 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1559

11:31:13 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +768, GPU +2, now: CPU 20561, GPU 2058 (MiB)

1560

11:31:13 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1561

11:31:14 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1562

11:31:14 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1563

11:31:14 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1564

11:31:14 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1565

11:31:14 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1566

11:31:14 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1567

11:31:14 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2855ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1568

11:31:14 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1569

11:31:14 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1570

11:31:14 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1571

11:31:14 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.771354 seconds.

1572

11:31:14 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1573

11:31:14 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1574

11:31:14 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1575

11:31:14 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1576

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1577

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1578

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1579

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1580

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1581

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 10, GPU 689 (MiB)

1582

11:31:15 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1583

11:31:16 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +753, GPU +0, now: CPU 20565, GPU 2072 (MiB)

1584

11:31:16 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1585

11:31:16 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1586

11:31:16 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1587

11:31:16 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1588

11:31:16 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1589

11:31:16 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1590

11:31:16 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1591

11:31:16 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2545ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1592

11:31:16 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1593

11:31:16 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1594

11:31:16 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1595

11:31:16 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.7968 seconds.

1596

11:31:16 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1597

11:31:17 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1598

11:31:17 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1599

11:31:17 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1600

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1601

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1602

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1603

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1604

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1605

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 10, GPU 702 (MiB)

1606

11:31:17 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1607

11:31:18 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +673, GPU +0, now: CPU 20572, GPU 2086 (MiB)

1608

11:31:18 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1609

11:31:18 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1610

11:31:18 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1611

11:31:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1612

11:31:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1613

11:31:19 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1614

11:31:19 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1615

11:31:19 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3281ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1616

11:31:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1617

11:31:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1618

11:31:19 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1619

11:31:19 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.793167 seconds.

1620

11:31:19 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1621

11:31:19 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1622

11:31:19 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1623

11:31:19 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1624

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1625

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1626

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1627

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1628

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1629

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 10, GPU 715 (MiB)

1630

11:31:19 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1631

11:31:20 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +751, GPU +2, now: CPU 20573, GPU 2102 (MiB)

1632

11:31:20 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1633

11:31:20 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1634

11:31:21 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1635

11:31:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1636

11:31:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1637

11:31:21 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1638

11:31:21 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1639

11:31:21 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2619ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1640

11:31:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1641

11:31:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1642

11:31:21 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1643

11:31:21 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.777087 seconds.

1644

11:31:21 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1645

11:31:21 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1646

11:31:21 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1647

11:31:21 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1648

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1649

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1650

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1651

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1652

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1653

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 11, GPU 728 (MiB)

1654

11:31:21 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1655

11:31:22 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +727, GPU +2, now: CPU 20580, GPU 2116 (MiB)

1656

11:31:22 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1657

11:31:23 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1658

11:31:23 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1659

11:31:23 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1660

11:31:23 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1661

11:31:23 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1662

11:31:23 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1663

11:31:23 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2674ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1664

11:31:23 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1665

11:31:23 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1666

11:31:23 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1667

11:31:23 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.786908 seconds.

1668

11:31:23 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1669

11:31:23 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1670

11:31:23 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1671

11:31:23 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1672

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1673

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1674

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1675

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1676

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1677

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 11, GPU 741 (MiB)

1678

11:31:24 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1679

11:31:24 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +642, GPU +2, now: CPU 20589, GPU 2130 (MiB)

1680

11:31:24 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1681

11:31:25 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1682

11:31:25 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1683

11:31:25 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1684

11:31:25 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1685

11:31:25 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1686

11:31:25 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1687

11:31:25 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2649ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1688

11:31:25 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1689

11:31:25 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1690

11:31:25 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1691

11:31:25 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.808719 seconds.

1692

11:31:25 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1693

11:31:25 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1694

11:31:26 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1695

11:31:26 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1696

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1697

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1698

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1699

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1700

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1701

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 11, GPU 754 (MiB)

1702

11:31:26 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1703

11:31:27 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +739, GPU +0, now: CPU 20600, GPU 2144 (MiB)

1704

11:31:27 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1705

11:31:27 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1706

11:31:27 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1707

11:31:28 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1708

11:31:28 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1709

11:31:28 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1710

11:31:28 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1711

11:31:28 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2797ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1712

11:31:28 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1713

11:31:28 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1714

11:31:28 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1715

11:31:28 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.826237 seconds.

1716

11:31:28 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1717

11:31:28 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1718

11:31:28 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1719

11:31:28 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1720

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1721

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1722

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1723

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1724

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1725

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 12, GPU 767 (MiB)

1726

11:31:28 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1727

11:31:29 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +714, GPU +0, now: CPU 20637, GPU 2158 (MiB)

1728

11:31:29 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1729

11:31:29 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1730

11:31:29 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1731

11:31:30 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1732

11:31:30 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1733

11:31:30 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1734

11:31:30 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1735

11:31:30 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3608ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1736

11:31:30 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1737

11:31:30 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1738

11:31:30 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1739

11:31:30 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.772014 seconds.

1740

11:31:30 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1741

11:31:30 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1742

11:31:30 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1743

11:31:30 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1744

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1745

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1746

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1747

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1748

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1749

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 12, GPU 780 (MiB)

1750

11:31:30 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1751

11:31:31 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +687, GPU +2, now: CPU 20618, GPU 2174 (MiB)

1752

11:31:31 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1753

11:31:31 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1754

11:31:32 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1755

11:31:32 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1756

11:31:32 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1757

11:31:32 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1758

11:31:32 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1759

11:31:32 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3854ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1760

11:31:32 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1761

11:31:32 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1762

11:31:32 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1763

11:31:32 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.776168 seconds.

1764

11:31:32 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1765

11:31:32 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1766

11:31:32 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1767

11:31:32 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1768

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1769

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1770

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1771

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1772

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1773

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 12, GPU 793 (MiB)

1774

11:31:32 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1775

11:31:33 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +715, GPU +2, now: CPU 20628, GPU 2188 (MiB)

1776

11:31:33 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1777

11:31:34 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1778

11:31:34 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1779

11:31:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1780

11:31:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1781

11:31:34 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1782

11:31:34 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1783

11:31:34 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.5147ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1784

11:31:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1785

11:31:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1786

11:31:34 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1787

11:31:34 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.789424 seconds.

1788

11:31:34 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1789

11:31:34 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1790

11:31:34 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1791

11:31:34 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1792

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1793

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1794

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1795

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1796

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1797

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 13, GPU 807 (MiB)

1798

11:31:35 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1799

11:31:36 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +699, GPU +0, now: CPU 20623, GPU 2202 (MiB)

1800

11:31:36 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1801

11:31:36 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1802

11:31:36 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1803

11:31:36 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1804

11:31:36 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1805

11:31:36 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1806

11:31:36 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1807

11:31:36 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2839ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1808

11:31:36 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1809

11:31:36 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1810

11:31:36 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1811

11:31:36 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.805197 seconds.

1812

11:31:36 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1813

11:31:37 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1814

11:31:37 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1815

11:31:37 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1816

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1817

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1818

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1819

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1820

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1821

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 13, GPU 820 (MiB)

1822

11:31:37 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1823

11:31:38 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +694, GPU +0, now: CPU 20640, GPU 2216 (MiB)

1824

11:31:38 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1825

11:31:38 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1826

11:31:38 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1827

11:31:39 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1828

11:31:39 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1829

11:31:39 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1830

11:31:39 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1831

11:31:39 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2499ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1832

11:31:39 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1833

11:31:39 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1834

11:31:39 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1835

11:31:39 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.777555 seconds.

1836

11:31:39 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1837

11:31:39 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1838

11:31:39 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1839

11:31:39 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1840

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1841

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1842

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1843

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1844

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1845

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 13, GPU 833 (MiB)

1846

11:31:39 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1847

11:31:40 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +693, GPU +2, now: CPU 20642, GPU 2232 (MiB)

1848

11:31:40 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1849

11:31:40 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1850

11:31:41 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1851

11:31:41 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1852

11:31:41 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1853

11:31:41 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1854

11:31:41 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1855

11:31:41 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3825ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1856

11:31:41 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1857

11:31:41 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1858

11:31:41 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1859

11:31:41 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.784509 seconds.

1860

11:31:41 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1861

11:31:41 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1862

11:31:41 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1863

11:31:41 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1864

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1865

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1866

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1867

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1868

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1869

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 14, GPU 846 (MiB)

1870

11:31:41 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1871

11:31:42 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +696, GPU +2, now: CPU 20644, GPU 2246 (MiB)

1872

11:31:42 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1873

11:31:43 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1874

11:31:43 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1875

11:31:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1876

11:31:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1877

11:31:43 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1878

11:31:43 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1879

11:31:43 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2558ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1880

11:31:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1881

11:31:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1882

11:31:43 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1883

11:31:43 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.824534 seconds.

1884

11:31:43 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1885

11:31:43 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1886

11:31:43 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1887

11:31:43 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1888

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1889

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1890

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1891

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1892

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1893

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 14, GPU 859 (MiB)

1894

11:31:44 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1895

11:31:45 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +694, GPU +2, now: CPU 20649, GPU 2260 (MiB)

1896

11:31:45 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1897

11:31:45 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1898

11:31:45 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1899

11:31:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1900

11:31:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1901

11:31:45 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1902

11:31:45 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1903

11:31:45 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.301ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1904

11:31:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1905

11:31:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1906

11:31:45 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1907

11:31:45 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.820281 seconds.

1908

11:31:45 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1909

11:31:46 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1910

11:31:46 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1911

11:31:46 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1912

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1913

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1914

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1915

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1916

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1917

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 14, GPU 872 (MiB)

1918

11:31:46 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1919

11:31:47 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +763, GPU +0, now: CPU 20659, GPU 2274 (MiB)

1920

11:31:47 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1921

11:31:47 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1922

11:31:47 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1923

11:31:48 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1924

11:31:48 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1925

11:31:48 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1926

11:31:48 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1927

11:31:48 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2405ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1928

11:31:48 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1929

11:31:48 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1930

11:31:48 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1931

11:31:48 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.799448 seconds.

1932

11:31:48 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1933

11:31:48 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1934

11:31:48 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1935

11:31:48 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1936

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1937

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1938

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1939

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1940

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1941

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 15, GPU 885 (MiB)

1942

11:31:48 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1943

11:31:49 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +701, GPU +0, now: CPU 20674, GPU 2288 (MiB)

1944

11:31:49 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1945

11:31:49 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1946

11:31:50 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1947

11:31:50 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1948

11:31:50 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1949

11:31:50 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1950

11:31:50 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1951

11:31:50 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.374ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1952

11:31:50 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1953

11:31:50 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1954

11:31:50 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1955

11:31:50 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.782011 seconds.

1956

11:31:50 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1957

11:31:50 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1958

11:31:50 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1959

11:31:50 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1960

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1961

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1962

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1963

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1964

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1965

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 15, GPU 898 (MiB)

1966

11:31:50 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1967

11:31:51 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +765, GPU +2, now: CPU 20674, GPU 2304 (MiB)

1968

11:31:51 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1969

11:31:52 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1970

11:31:52 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1971

11:31:52 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1972

11:31:52 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1973

11:31:52 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1974

11:31:52 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1975

11:31:52 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3684ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

1976

11:31:52 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

1977

11:31:52 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

1978

11:31:52 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

1979

11:31:52 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.80534 seconds.

1980

11:31:52 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

1981

11:31:52 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

1982

11:31:52 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

1983

11:31:52 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

1984

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

1985

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

1986

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

1987

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

1988

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

1989

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 15, GPU 911 (MiB)

1990

11:31:53 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

1991

11:31:54 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +746, GPU +2, now: CPU 20683, GPU 2318 (MiB)

1992

11:31:54 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

1993

11:31:54 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

1994

11:31:54 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

1995

11:31:54 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

1996

11:31:54 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

1997

11:31:54 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

1998

11:31:54 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

1999

11:31:54 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3203ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2000

11:31:54 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2001

11:31:54 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

2002

11:31:54 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2003

11:31:54 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.806716 seconds.

2004

11:31:54 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2005

11:31:55 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2006

11:31:55 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2007

11:31:55 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2008

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2009

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2010

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2011

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2012

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2013

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 16, GPU 925 (MiB)

2014

11:31:55 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2015

11:31:56 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +687, GPU +0, now: CPU 20686, GPU 2332 (MiB)

2016

11:31:56 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2017

11:31:56 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2018

11:31:56 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

2019

11:31:57 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

2020

11:31:57 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2021

11:31:57 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2022

11:31:57 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2023

11:31:57 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3158ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2024

11:31:57 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2025

11:31:57 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

2026

11:31:57 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2027

11:31:57 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.783436 seconds.

2028

11:31:57 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2029

11:31:57 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2030

11:31:57 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2031

11:31:57 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2032

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2033

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2034

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2035

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2036

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2037

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 16, GPU 938 (MiB)

2038

11:31:57 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2039

11:31:58 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +737, GPU +0, now: CPU 20692, GPU 2346 (MiB)

2040

11:31:58 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2041

11:31:58 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2042

11:31:59 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

2043

11:31:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

2044

11:31:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2045

11:31:59 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2046

11:31:59 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2047

11:31:59 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2993ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2048

11:31:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2049

11:31:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

2050

11:31:59 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2051

11:31:59 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.781284 seconds.

2052

11:31:59 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2053

11:31:59 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2054

11:31:59 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2055

11:31:59 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2056

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2057

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2058

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2059

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2060

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2061

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 16, GPU 951 (MiB)

2062

11:31:59 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2063

11:32:00 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +736, GPU +2, now: CPU 20706, GPU 2362 (MiB)

2064

11:32:00 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2065

11:32:01 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2066

11:32:01 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

2067

11:32:01 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

2068

11:32:01 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2069

11:32:01 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2070

11:32:01 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2071

11:32:01 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2595ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2072

11:32:01 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2073

11:32:01 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

2074

11:32:01 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2075

11:32:01 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.810191 seconds.

2076

11:32:01 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2077

11:32:01 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2078

11:32:01 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2079

11:32:01 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2080

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2081

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2082

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2083

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2084

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2085

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 17, GPU 964 (MiB)

2086

11:32:02 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2087

11:32:02 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +720, GPU +2, now: CPU 20726, GPU 2376 (MiB)

2088

11:32:03 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2089

11:32:03 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2090

11:32:03 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

2091

11:32:03 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

2092

11:32:03 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2093

11:32:03 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2094

11:32:03 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2095

11:32:03 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3529ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2096

11:32:03 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2097

11:32:03 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

2098

11:32:03 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2099

11:32:03 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.800981 seconds.

2100

11:32:03 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2101

11:32:03 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2102

11:32:04 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2103

11:32:04 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2104

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2105

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2106

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2107

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2108

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2109

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 17, GPU 977 (MiB)

2110

11:32:04 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2111

11:32:05 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +723, GPU +2, now: CPU 20711, GPU 2390 (MiB)

2112

11:32:05 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2113

11:32:05 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2114

11:32:05 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

2115

11:32:06 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

2116

11:32:06 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2117

11:32:06 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2118

11:32:06 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2119

11:32:06 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2714ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2120

11:32:06 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2121

11:32:06 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

2122

11:32:06 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2123

11:32:06 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.79744 seconds.

2124

11:32:06 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2125

11:32:06 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2126

11:32:06 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2127

11:32:06 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2128

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2129

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2130

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2131

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2132

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2133

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 17, GPU 990 (MiB)

2134

11:32:06 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2135

11:32:07 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +721, GPU +0, now: CPU 20726, GPU 2404 (MiB)

2136

11:32:07 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2137

11:32:07 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2138

11:32:07 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

2139

11:32:08 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

2140

11:32:08 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2141

11:32:08 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2142

11:32:08 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2143

11:32:08 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3019ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2144

11:32:08 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2145

11:32:08 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

2146

11:32:08 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2147

11:32:08 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.78428 seconds.

2148

11:32:08 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2149

11:32:08 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2150

11:32:08 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2151

11:32:08 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2152

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2153

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2154

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2155

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2156

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2157

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 18, GPU 1003 (MiB)

2158

11:32:08 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2159

11:32:09 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +717, GPU +0, now: CPU 20739, GPU 2418 (MiB)

2160

11:32:09 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2161

11:32:09 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2162

11:32:10 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

2163

11:32:10 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

2164

11:32:10 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2165

11:32:10 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2166

11:32:10 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2167

11:32:10 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2456ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2168

11:32:10 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2169

11:32:10 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

2170

11:32:10 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2171

11:32:10 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.789375 seconds.

2172

11:32:10 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2173

11:32:10 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2174

11:32:10 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2175

11:32:10 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2176

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2177

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2178

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2179

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2180

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2181

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 18, GPU 1016 (MiB)

2182

11:32:10 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2183

11:32:11 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +709, GPU +2, now: CPU 20739, GPU 2434 (MiB)

2184

11:32:11 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2185

11:32:12 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2186

11:32:12 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

2187

11:32:12 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

2188

11:32:12 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2189

11:32:12 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2190

11:32:12 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2191

11:32:12 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2771ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2192

11:32:12 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2193

11:32:12 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

2194

11:32:12 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2195

11:32:12 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.806644 seconds.

2196

11:32:12 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2197

11:32:12 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2198

11:32:12 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2199

11:32:12 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2200

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2201

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2202

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2203

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2204

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2205

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 18, GPU 1029 (MiB)

2206

11:32:13 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2207

11:32:14 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +716, GPU +2, now: CPU 20746, GPU 2448 (MiB)

2208

11:32:14 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2209

11:32:14 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2210

11:32:14 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

2211

11:32:14 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

2212

11:32:14 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2213

11:32:14 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2214

11:32:14 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2215

11:32:14 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2667ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2216

11:32:14 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2217

11:32:14 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

2218

11:32:14 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2219

11:32:14 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.798407 seconds.

2220

11:32:14 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2221

11:32:15 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2222

11:32:15 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2223

11:32:15 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2224

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2225

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2226

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2227

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2228

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2229

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 18, GPU 1042 (MiB)

2230

11:32:15 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2231

11:32:16 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +695, GPU +2, now: CPU 20748, GPU 2462 (MiB)

2232

11:32:16 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2233

11:32:16 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2234

11:32:16 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 7 output network tensors.

2235

11:32:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

2236

11:32:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2237

11:32:17 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2238

11:32:17 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2239

11:32:17 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3337ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2240

11:32:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2241

11:32:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

2242

11:32:17 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2243

11:32:17 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.782285 seconds.

2244

11:32:17 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2245

11:32:17 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2246

11:32:17 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2247

11:32:17 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2248

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2249

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2250

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2251

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2252

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2253

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 19, GPU 1056 (MiB)

2254

11:32:17 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2255

11:32:18 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +691, GPU +0, now: CPU 20769, GPU 2476 (MiB)

2256

11:32:18 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2257

11:32:18 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2258

11:32:18 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 10 inputs and 6 output network tensors.

2259

11:32:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

2260

11:32:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2261

11:32:19 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2262

11:32:19 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2263

11:32:19 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2844ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2264

11:32:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2265

11:32:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3255488 bytes

2266

11:32:19 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2267

11:32:19 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.799389 seconds.

2268

11:32:19 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2269

11:32:19 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2270

11:32:19 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2271

11:32:19 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2272

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2273

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2274

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2275

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2276

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2277

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 19, GPU 1069 (MiB)

2278

11:32:19 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2279

11:32:20 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +712, GPU +0, now: CPU 20764, GPU 2490 (MiB)

2280

11:32:20 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2281

11:32:21 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2282

11:32:21 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 7 inputs and 8 output network tensors.

2283

11:32:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 379968 bytes

2284

11:32:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2285

11:32:21 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2286

11:32:21 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 80 steps to complete.

2287

11:32:21 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.5901ms to assign 5 blocks to 80 nodes requiring 10485760 bytes.

2288

11:32:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2289

11:32:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 6769344 bytes

2290

11:32:21 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2291

11:32:21 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.30935 seconds.

2292

11:32:21 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2293

11:32:22 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2294

11:32:22 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2295

11:32:22 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2296

INFO: [Torch-TensorRT] - Loaded engine size: 7 MiB

2297

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2298

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2299

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2300

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2301

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 20, GPU 1085 (MiB)

2302

11:32:22 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2303

11:32:23 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +686, GPU +2, now: CPU 20779, GPU 2508 (MiB)

2304

11:32:23 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2305

11:32:23 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2306

11:32:23 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2307

11:32:24 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2308

11:32:24 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2309

11:32:24 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2310

11:32:24 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2311

11:32:24 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2676ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2312

11:32:24 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2313

11:32:24 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2314

11:32:24 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2315

11:32:24 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.778818 seconds.

2316

11:32:24 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2317

11:32:24 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2318

11:32:24 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2319

11:32:24 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2320

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2321

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2322

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2323

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2324

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2325

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 20, GPU 1098 (MiB)

2326

11:32:24 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2327

11:32:25 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +708, GPU +2, now: CPU 20792, GPU 2522 (MiB)

2328

11:32:25 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2329

11:32:25 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2330

11:32:26 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2331

11:32:26 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2332

11:32:26 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2333

11:32:26 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2334

11:32:26 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2335

11:32:26 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3608ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2336

11:32:26 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2337

11:32:26 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2338

11:32:26 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2339

11:32:26 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.742906 seconds.

2340

11:32:26 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2341

11:32:26 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2342

11:32:26 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2343

11:32:26 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2344

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2345

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2346

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2347

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2348

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2349

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 20, GPU 1111 (MiB)

2350

11:32:26 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2351

11:32:27 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +725, GPU +0, now: CPU 20798, GPU 2536 (MiB)

2352

11:32:27 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2353

11:32:27 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2354

11:32:28 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2355

11:32:28 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2356

11:32:28 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2357

11:32:28 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2358

11:32:28 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2359

11:32:28 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3037ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2360

11:32:28 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2361

11:32:28 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2362

11:32:28 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2363

11:32:28 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.758761 seconds.

2364

11:32:28 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2365

11:32:28 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2366

11:32:28 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2367

11:32:28 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2368

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2369

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2370

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2371

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2372

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2373

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 21, GPU 1125 (MiB)

2374

11:32:28 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2375

11:32:29 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +704, GPU +0, now: CPU 20806, GPU 2550 (MiB)

2376

11:32:29 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2377

11:32:30 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2378

11:32:30 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2379

11:32:30 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2380

11:32:30 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2381

11:32:30 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2382

11:32:30 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2383

11:32:30 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2526ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2384

11:32:30 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2385

11:32:30 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2386

11:32:30 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2387

11:32:30 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.776177 seconds.

2388

11:32:30 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2389

11:32:30 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2390

11:32:30 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2391

11:32:30 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2392

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2393

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2394

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2395

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2396

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2397

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 21, GPU 1138 (MiB)

2398

11:32:31 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2399

11:32:31 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +693, GPU +2, now: CPU 20818, GPU 2566 (MiB)

2400

11:32:32 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2401

11:32:32 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2402

11:32:32 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2403

11:32:32 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2404

11:32:32 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2405

11:32:32 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2406

11:32:32 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2407

11:32:32 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3357ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2408

11:32:32 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2409

11:32:32 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2410

11:32:32 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2411

11:32:32 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.753887 seconds.

2412

11:32:32 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2413

11:32:32 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2414

11:32:32 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2415

11:32:32 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2416

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2417

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2418

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2419

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2420

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2421

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 21, GPU 1151 (MiB)

2422

11:32:33 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2423

11:32:34 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +705, GPU +2, now: CPU 20810, GPU 2582 (MiB)

2424

11:32:34 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2425

11:32:34 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2426

11:32:34 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2427

11:32:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2428

11:32:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2429

11:32:34 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2430

11:32:34 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2431

11:32:34 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3837ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2432

11:32:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2433

11:32:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2434

11:32:34 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2435

11:32:34 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.742253 seconds.

2436

11:32:34 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2437

11:32:35 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2438

11:32:35 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2439

11:32:35 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2440

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2441

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2442

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2443

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2444

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2445

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 22, GPU 1164 (MiB)

2446

11:32:35 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2447

11:32:36 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +676, GPU +2, now: CPU 20819, GPU 2596 (MiB)

2448

11:32:36 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2449

11:32:36 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2450

11:32:36 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2451

11:32:36 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2452

11:32:36 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2453

11:32:36 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2454

11:32:36 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2455

11:32:36 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3175ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2456

11:32:36 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2457

11:32:36 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2458

11:32:36 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2459

11:32:36 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.746737 seconds.

2460

11:32:36 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2461

11:32:37 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2462

11:32:37 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2463

11:32:37 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2464

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2465

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2466

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2467

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2468

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2469

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 22, GPU 1177 (MiB)

2470

11:32:37 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2471

11:32:38 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +737, GPU +2, now: CPU 20830, GPU 2610 (MiB)

2472

11:32:38 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2473

11:32:38 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2474

11:32:38 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2475

11:32:39 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2476

11:32:39 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2477

11:32:39 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2478

11:32:39 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2479

11:32:39 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3474ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2480

11:32:39 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2481

11:32:39 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2482

11:32:39 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2483

11:32:39 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.76344 seconds.

2484

11:32:39 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2485

11:32:39 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2486

11:32:39 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2487

11:32:39 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2488

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2489

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2490

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2491

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2492

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2493

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 22, GPU 1191 (MiB)

2494

11:32:39 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2495

11:32:40 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +723, GPU +0, now: CPU 20834, GPU 2624 (MiB)

2496

11:32:40 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2497

11:32:40 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2498

11:32:40 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2499

11:32:41 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2500

11:32:41 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2501

11:32:41 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2502

11:32:41 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2503

11:32:41 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2688ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2504

11:32:41 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2505

11:32:41 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2506

11:32:41 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2507

11:32:41 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.766495 seconds.

2508

11:32:41 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2509

11:32:41 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2510

11:32:41 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2511

11:32:41 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2512

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2513

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2514

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2515

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2516

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2517

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 23, GPU 1204 (MiB)

2518

11:32:41 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2519

11:32:42 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +747, GPU +0, now: CPU 20834, GPU 2638 (MiB)

2520

11:32:42 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2521

11:32:42 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2522

11:32:43 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2523

11:32:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2524

11:32:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2525

11:32:43 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2526

11:32:43 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2527

11:32:43 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2585ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2528

11:32:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2529

11:32:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2530

11:32:43 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2531

11:32:43 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.749057 seconds.

2532

11:32:43 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2533

11:32:43 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2534

11:32:43 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2535

11:32:43 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2536

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2537

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2538

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2539

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2540

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2541

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 23, GPU 1217 (MiB)

2542

11:32:43 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2543

11:32:44 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +759, GPU +2, now: CPU 20862, GPU 2654 (MiB)

2544

11:32:44 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2545

11:32:45 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2546

11:32:45 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2547

11:32:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2548

11:32:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2549

11:32:45 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2550

11:32:45 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2551

11:32:45 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3743ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2552

11:32:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2553

11:32:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2554

11:32:45 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2555

11:32:45 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.746284 seconds.

2556

11:32:45 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2557

11:32:45 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2558

11:32:45 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2559

11:32:45 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2560

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2561

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2562

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2563

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2564

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2565

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 23, GPU 1230 (MiB)

2566

11:32:46 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2567

11:32:46 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +763, GPU +2, now: CPU 20845, GPU 2668 (MiB)

2568

11:32:46 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2569

11:32:47 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2570

11:32:47 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2571

11:32:47 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2572

11:32:47 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2573

11:32:47 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2574

11:32:47 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2575

11:32:47 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2442ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2576

11:32:47 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2577

11:32:47 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2578

11:32:47 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2579

11:32:47 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.78174 seconds.

2580

11:32:47 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2581

11:32:47 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2582

11:32:47 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2583

11:32:47 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2584

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2585

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2586

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2587

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2588

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2589

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 24, GPU 1243 (MiB)

2590

11:32:48 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2591

11:32:49 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +728, GPU +2, now: CPU 20854, GPU 2682 (MiB)

2592

11:32:49 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2593

11:32:49 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2594

11:32:49 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2595

11:32:49 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2596

11:32:49 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2597

11:32:49 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2598

11:32:49 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2599

11:32:49 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2473ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2600

11:32:49 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2601

11:32:49 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2602

11:32:49 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2603

11:32:49 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.759011 seconds.

2604

11:32:49 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2605

11:32:49 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2606

11:32:50 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2607

11:32:50 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2608

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2609

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2610

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2611

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2612

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2613

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 24, GPU 1256 (MiB)

2614

11:32:50 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2615

11:32:51 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +730, GPU +0, now: CPU 20861, GPU 2696 (MiB)

2616

11:32:51 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2617

11:32:51 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2618

11:32:51 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2619

11:32:52 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2620

11:32:52 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2621

11:32:52 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2622

11:32:52 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2623

11:32:52 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3552ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2624

11:32:52 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2625

11:32:52 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2626

11:32:52 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2627

11:32:52 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.78135 seconds.

2628

11:32:52 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2629

11:32:52 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2630

11:32:52 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2631

11:32:52 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2632

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2633

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2634

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2635

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2636

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2637

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 24, GPU 1270 (MiB)

2638

11:32:52 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2639

11:32:53 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +722, GPU +0, now: CPU 20874, GPU 2710 (MiB)

2640

11:32:53 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2641

11:32:53 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2642

11:32:53 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2643

11:32:54 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2644

11:32:54 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2645

11:32:54 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2646

11:32:54 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2647

11:32:54 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2391ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2648

11:32:54 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2649

11:32:54 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2650

11:32:54 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2651

11:32:54 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.759597 seconds.

2652

11:32:54 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2653

11:32:54 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2654

11:32:54 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2655

11:32:54 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2656

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2657

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2658

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2659

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2660

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2661

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 25, GPU 1283 (MiB)

2662

11:32:54 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2663

11:32:55 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +715, GPU +2, now: CPU 20889, GPU 2726 (MiB)

2664

11:32:55 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2665

11:32:55 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2666

11:32:55 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2667

11:32:56 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2668

11:32:56 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2669

11:32:56 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2670

11:32:56 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2671

11:32:56 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2479ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2672

11:32:56 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2673

11:32:56 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2674

11:32:56 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2675

11:32:56 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.760831 seconds.

2676

11:32:56 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2677

11:32:56 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2678

11:32:56 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2679

11:32:56 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2680

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2681

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2682

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2683

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2684

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2685

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 25, GPU 1296 (MiB)

2686

11:32:56 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2687

11:32:57 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +730, GPU +2, now: CPU 20893, GPU 2740 (MiB)

2688

11:32:57 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2689

11:32:57 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2690

11:32:58 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2691

11:32:58 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2692

11:32:58 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2693

11:32:58 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2694

11:32:58 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2695

11:32:58 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2548ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2696

11:32:58 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2697

11:32:58 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2698

11:32:58 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2699

11:32:58 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.779468 seconds.

2700

11:32:58 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2701

11:32:58 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2702

11:32:58 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2703

11:32:58 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2704

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2705

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2706

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2707

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2708

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2709

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 25, GPU 1309 (MiB)

2710

11:32:58 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2711

11:32:59 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +690, GPU +2, now: CPU 20884, GPU 2754 (MiB)

2712

11:32:59 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2713

11:33:00 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2714

11:33:00 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2715

11:33:00 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2716

11:33:00 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2717

11:33:00 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2718

11:33:00 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2719

11:33:00 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.352ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2720

11:33:00 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2721

11:33:00 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2722

11:33:00 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2723

11:33:00 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.767588 seconds.

2724

11:33:00 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2725

11:33:00 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2726

11:33:00 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2727

11:33:00 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2728

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2729

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2730

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2731

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2732

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2733

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 26, GPU 1322 (MiB)

2734

11:33:00 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2735

11:33:01 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +676, GPU +0, now: CPU 20895, GPU 2768 (MiB)

2736

11:33:01 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2737

11:33:02 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2738

11:33:02 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2739

11:33:02 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2740

11:33:02 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2741

11:33:02 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2742

11:33:02 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2743

11:33:02 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2929ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2744

11:33:02 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2745

11:33:02 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2746

11:33:02 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2747

11:33:02 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.753895 seconds.

2748

11:33:02 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1347 MiB

2749

11:33:02 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2750

11:33:02 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2751

11:33:02 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2752

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2753

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2754

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2755

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2756

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2757

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 26, GPU 1335 (MiB)

2758

11:33:03 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2759

11:33:03 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +730, GPU +0, now: CPU 20906, GPU 2782 (MiB)

2760

11:33:03 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2761

11:33:04 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2762

11:33:04 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2763

11:33:04 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2764

11:33:04 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2765

11:33:04 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2766

11:33:04 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2767

11:33:04 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3304ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2768

11:33:04 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2769

11:33:04 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2770

11:33:04 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2771

11:33:04 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.737212 seconds.

2772

11:33:04 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1356 MiB

2773

11:33:04 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2774

11:33:04 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2775

11:33:04 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2776

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2777

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2778

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2779

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2780

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2781

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 26, GPU 1349 (MiB)

2782

11:33:05 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2783

11:33:05 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +676, GPU +2, now: CPU 20899, GPU 2798 (MiB)

2784

11:33:06 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2785

11:33:06 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2786

11:33:06 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2787

11:33:06 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2788

11:33:06 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2789

11:33:06 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2790

11:33:06 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2791

11:33:06 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2707ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2792

11:33:06 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2793

11:33:06 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2794

11:33:06 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2795

11:33:06 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.766261 seconds.

2796

11:33:06 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1370 MiB

2797

11:33:06 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2798

11:33:07 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2799

11:33:07 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2800

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2801

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2802

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2803

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2804

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2805

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 26, GPU 1362 (MiB)

2806

11:33:07 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2807

11:33:08 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +761, GPU +2, now: CPU 20907, GPU 2816 (MiB)

2808

11:33:08 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2809

11:33:08 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2810

11:33:08 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2811

11:33:08 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2812

11:33:08 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2813

11:33:08 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2814

11:33:08 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2815

11:33:08 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3447ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2816

11:33:08 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2817

11:33:08 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2818

11:33:08 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2819

11:33:08 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.76883 seconds.

2820

11:33:08 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1383 MiB

2821

11:33:09 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2822

11:33:09 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2823

11:33:09 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2824

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2825

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2826

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2827

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2828

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2829

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 27, GPU 1375 (MiB)

2830

11:33:09 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2831

11:33:10 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +713, GPU +2, now: CPU 20908, GPU 2830 (MiB)

2832

11:33:10 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2833

11:33:10 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2834

11:33:10 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2835

11:33:11 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2836

11:33:11 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2837

11:33:11 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2838

11:33:11 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2839

11:33:11 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2619ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2840

11:33:11 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2841

11:33:11 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2842

11:33:11 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2843

11:33:11 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.745195 seconds.

2844

11:33:11 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1396 MiB

2845

11:33:11 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2846

11:33:11 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2847

11:33:11 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2848

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2849

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2850

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2851

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2852

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2853

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 27, GPU 1388 (MiB)

2854

11:33:11 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2855

11:33:12 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +744, GPU +0, now: CPU 20918, GPU 2844 (MiB)

2856

11:33:12 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2857

11:33:12 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2858

11:33:12 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2859

11:33:13 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2860

11:33:13 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2861

11:33:13 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2862

11:33:13 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2863

11:33:13 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2989ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2864

11:33:13 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2865

11:33:13 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2866

11:33:13 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2867

11:33:13 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.756001 seconds.

2868

11:33:13 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1409 MiB

2869

11:33:13 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2870

11:33:13 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2871

11:33:13 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2872

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2873

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2874

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2875

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2876

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2877

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 27, GPU 1401 (MiB)

2878

11:33:13 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2879

11:33:14 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +731, GPU +0, now: CPU 20927, GPU 2858 (MiB)

2880

11:33:14 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2881

11:33:14 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2882

11:33:14 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2883

11:33:15 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2884

11:33:15 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2885

11:33:15 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2886

11:33:15 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2887

11:33:15 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2641ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2888

11:33:15 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2889

11:33:15 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2890

11:33:15 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2891

11:33:15 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.776097 seconds.

2892

11:33:15 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1422 MiB

2893

11:33:15 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2894

11:33:15 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2895

11:33:15 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2896

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2897

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2898

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2899

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2900

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2901

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 28, GPU 1415 (MiB)

2902

11:33:15 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2903

11:33:16 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +729, GPU +2, now: CPU 20937, GPU 2874 (MiB)

2904

11:33:16 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2905

11:33:16 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2906

11:33:17 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2907

11:33:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2908

11:33:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2909

11:33:17 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2910

11:33:17 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2911

11:33:17 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2389ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2912

11:33:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2913

11:33:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2914

11:33:17 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2915

11:33:17 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.7718 seconds.

2916

11:33:17 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1436 MiB

2917

11:33:17 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2918

11:33:17 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2919

11:33:17 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2920

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2921

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2922

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2923

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2924

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2925

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 28, GPU 1428 (MiB)

2926

11:33:17 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2927

11:33:18 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +729, GPU +2, now: CPU 20949, GPU 2888 (MiB)

2928

11:33:18 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2929

11:33:18 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2930

11:33:19 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2931

11:33:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2932

11:33:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2933

11:33:19 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2934

11:33:19 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2935

11:33:19 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2674ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2936

11:33:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2937

11:33:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2938

11:33:19 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2939

11:33:19 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.762101 seconds.

2940

11:33:19 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1449 MiB

2941

11:33:19 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2942

11:33:19 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2943

11:33:19 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2944

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2945

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2946

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2947

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2948

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2949

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 28, GPU 1441 (MiB)

2950

11:33:19 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2951

11:33:20 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +735, GPU +2, now: CPU 20967, GPU 2902 (MiB)

2952

11:33:20 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2953

11:33:21 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2954

11:33:21 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2955

11:33:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2956

11:33:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2957

11:33:21 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2958

11:33:21 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2959

11:33:21 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2765ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2960

11:33:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2961

11:33:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2962

11:33:21 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2963

11:33:21 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.746998 seconds.

2964

11:33:21 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1462 MiB

2965

11:33:21 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2966

11:33:21 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2967

11:33:21 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2968

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2969

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2970

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2971

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2972

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2973

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 29, GPU 1454 (MiB)

2974

11:33:21 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2975

11:33:22 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +701, GPU +0, now: CPU 20968, GPU 2916 (MiB)

2976

11:33:22 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

2977

11:33:23 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

2978

11:33:23 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

2979

11:33:23 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

2980

11:33:23 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

2981

11:33:23 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

2982

11:33:23 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

2983

11:33:23 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3681ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

2984

11:33:23 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

2985

11:33:23 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

2986

11:33:23 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

2987

11:33:23 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.779276 seconds.

2988

11:33:23 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 29 MiB, GPU 1475 MiB

2989

11:33:23 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

2990

11:33:23 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

2991

11:33:23 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

2992

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

2993

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

2994

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

2995

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

2996

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

2997

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 29, GPU 1467 (MiB)

2998

11:33:24 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

2999

11:33:24 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +688, GPU +0, now: CPU 20965, GPU 2930 (MiB)

3000

11:33:24 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3001

11:33:25 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3002

11:33:25 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3003

11:33:25 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3004

11:33:25 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3005

11:33:25 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3006

11:33:25 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3007

11:33:25 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.4689ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3008

11:33:25 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3009

11:33:25 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3010

11:33:25 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3011

11:33:25 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.772938 seconds.

3012

11:33:25 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 30 MiB, GPU 1488 MiB

3013

11:33:25 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3014

11:33:25 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3015

11:33:25 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3016

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3017

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3018

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3019

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3020

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3021

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 29, GPU 1480 (MiB)

3022

11:33:26 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3023

11:33:27 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +743, GPU +0, now: CPU 20972, GPU 2944 (MiB)

3024

11:33:27 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3025

11:33:27 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3026

11:33:27 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3027

11:33:27 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3028

11:33:27 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3029

11:33:27 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3030

11:33:27 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3031

11:33:27 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2445ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3032

11:33:27 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3033

11:33:27 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3034

11:33:27 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3035

11:33:27 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.776694 seconds.

3036

11:33:27 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 30 MiB, GPU 1501 MiB

3037

11:33:28 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3038

11:33:28 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3039

11:33:28 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3040

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3041

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3042

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3043

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3044

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3045

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 30, GPU 1494 (MiB)

3046

11:33:28 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3047

11:33:29 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +692, GPU +2, now: CPU 20980, GPU 2960 (MiB)

3048

11:33:29 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3049

11:33:29 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3050

11:33:29 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3051

11:33:29 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3052

11:33:29 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3053

11:33:29 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3054

11:33:29 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3055

11:33:29 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2632ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3056

11:33:29 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3057

11:33:29 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3058

11:33:29 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3059

11:33:29 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.750652 seconds.

3060

11:33:29 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 30 MiB, GPU 1515 MiB

3061

11:33:30 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3062

11:33:30 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3063

11:33:30 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3064

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3065

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3066

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3067

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3068

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3069

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 30, GPU 1507 (MiB)

3070

11:33:30 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3071

11:33:31 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +684, GPU +2, now: CPU 20998, GPU 2974 (MiB)

3072

11:33:31 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3073

11:33:31 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3074

11:33:31 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3075

11:33:32 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3076

11:33:32 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3077

11:33:32 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3078

11:33:32 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3079

11:33:32 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2392ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3080

11:33:32 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3081

11:33:32 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3082

11:33:32 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3083

11:33:32 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.754517 seconds.

3084

11:33:32 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 30 MiB, GPU 1528 MiB

3085

11:33:32 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3086

11:33:32 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3087

11:33:32 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3088

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3089

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3090

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3091

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3092

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3093

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 30, GPU 1520 (MiB)

3094

11:33:32 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3095

11:33:33 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +744, GPU +0, now: CPU 21002, GPU 2988 (MiB)

3096

11:33:33 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3097

11:33:33 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3098

11:33:33 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3099

11:33:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3100

11:33:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3101

11:33:34 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3102

11:33:34 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3103

11:33:34 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2384ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3104

11:33:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3105

11:33:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3106

11:33:34 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3107

11:33:34 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.765388 seconds.

3108

11:33:34 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 31 MiB, GPU 1541 MiB

3109

11:33:34 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3110

11:33:34 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3111

11:33:34 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3112

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3113

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3114

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3115

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3116

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3117

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 31, GPU 1533 (MiB)

3118

11:33:34 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3119

11:33:35 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +768, GPU +0, now: CPU 21008, GPU 3002 (MiB)

3120

11:33:35 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3121

11:33:35 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3122

11:33:35 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3123

11:33:36 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3124

11:33:36 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3125

11:33:36 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3126

11:33:36 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3127

11:33:36 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2743ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3128

11:33:36 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3129

11:33:36 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3130

11:33:36 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3131

11:33:36 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.771732 seconds.

3132

11:33:36 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 31 MiB, GPU 1554 MiB

3133

11:33:36 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3134

11:33:36 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3135

11:33:36 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3136

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3137

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3138

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3139

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3140

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3141

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 31, GPU 1546 (MiB)

3142

11:33:36 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3143

11:33:37 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +767, GPU +0, now: CPU 21021, GPU 3016 (MiB)

3144

11:33:37 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3145

11:33:37 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3146

11:33:38 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3147

11:33:38 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3148

11:33:38 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3149

11:33:38 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3150

11:33:38 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3151

11:33:38 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2453ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3152

11:33:38 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3153

11:33:38 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3154

11:33:38 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3155

11:33:38 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.747778 seconds.

3156

11:33:38 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 31 MiB, GPU 1567 MiB

3157

11:33:38 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3158

11:33:38 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3159

11:33:38 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3160

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3161

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3162

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3163

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3164

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3165

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 31, GPU 1560 (MiB)

3166

11:33:38 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3167

11:33:39 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +721, GPU +2, now: CPU 21029, GPU 3032 (MiB)

3168

11:33:39 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3169

11:33:39 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3170

11:33:40 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3171

11:33:40 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3172

11:33:40 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3173

11:33:40 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3174

11:33:40 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3175

11:33:40 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2385ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3176

11:33:40 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3177

11:33:40 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3178

11:33:40 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3179

11:33:40 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.761317 seconds.

3180

11:33:40 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 32 MiB, GPU 1581 MiB

3181

11:33:40 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3182

11:33:40 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3183

11:33:40 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3184

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3185

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3186

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3187

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3188

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3189

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 32, GPU 1573 (MiB)

3190

11:33:40 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3191

11:33:41 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +753, GPU +2, now: CPU 21049, GPU 3048 (MiB)

3192

11:33:41 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3193

11:33:42 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3194

11:33:42 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3195

11:33:42 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3196

11:33:42 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3197

11:33:42 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3198

11:33:42 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3199

11:33:42 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3186ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3200

11:33:42 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3201

11:33:42 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3202

11:33:42 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3203

11:33:42 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.77937 seconds.

3204

11:33:42 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 32 MiB, GPU 1594 MiB

3205

11:33:42 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3206

11:33:42 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3207

11:33:42 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3208

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3209

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3210

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3211

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3212

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3213

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 32, GPU 1586 (MiB)

3214

11:33:42 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3215

11:33:43 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +751, GPU +4, now: CPU 21051, GPU 3064 (MiB)

3216

11:33:43 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3217

11:33:44 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3218

11:33:44 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3219

11:33:44 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3220

11:33:44 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3221

11:33:44 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3222

11:33:44 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3223

11:33:44 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3989ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3224

11:33:44 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3225

11:33:44 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3226

11:33:44 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3227

11:33:44 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.766876 seconds.

3228

11:33:44 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 32 MiB, GPU 1607 MiB

3229

11:33:44 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3230

11:33:44 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3231

11:33:44 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3232

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3233

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3234

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3235

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3236

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3237

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 32, GPU 1599 (MiB)

3238

11:33:45 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3239

11:33:45 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +724, GPU +0, now: CPU 21051, GPU 3078 (MiB)

3240

11:33:45 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3241

11:33:46 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3242

11:33:46 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3243

11:33:46 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3244

11:33:46 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3245

11:33:46 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3246

11:33:46 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3247

11:33:46 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.4749ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3248

11:33:46 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3249

11:33:46 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3250

11:33:46 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3251

11:33:46 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.750972 seconds.

3252

11:33:46 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 33 MiB, GPU 1620 MiB

3253

11:33:46 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3254

11:33:46 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3255

11:33:46 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3256

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3257

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3258

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3259

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3260

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3261

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 33, GPU 1612 (MiB)

3262

11:33:47 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3263

11:33:47 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +721, GPU +0, now: CPU 21061, GPU 3092 (MiB)

3264

11:33:47 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3265

11:33:48 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3266

11:33:48 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3267

11:33:48 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3268

11:33:48 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3269

11:33:48 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3270

11:33:48 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3271

11:33:48 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2478ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3272

11:33:48 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3273

11:33:48 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3274

11:33:48 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3275

11:33:48 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.752388 seconds.

3276

11:33:48 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 33 MiB, GPU 1633 MiB

3277

11:33:48 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3278

11:33:48 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3279

11:33:48 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3280

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3281

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3282

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3283

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3284

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3285

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 33, GPU 1625 (MiB)

3286

11:33:49 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3287

11:33:50 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +760, GPU +2, now: CPU 21077, GPU 3108 (MiB)

3288

11:33:50 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3289

11:33:50 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3290

11:33:50 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3291

11:33:50 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3292

11:33:50 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3293

11:33:50 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3294

11:33:50 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3295

11:33:50 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2625ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3296

11:33:50 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3297

11:33:50 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3298

11:33:50 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3299

11:33:50 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.755423 seconds.

3300

11:33:50 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 33 MiB, GPU 1646 MiB

3301

11:33:50 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3302

11:33:51 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3303

11:33:51 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3304

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3305

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3306

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3307

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3308

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3309

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 33, GPU 1639 (MiB)

3310

11:33:51 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3311

11:33:52 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +706, GPU +2, now: CPU 21049, GPU 3122 (MiB)

3312

11:33:52 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3313

11:33:52 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3314

11:33:52 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3315

11:33:52 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3316

11:33:52 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3317

11:33:52 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3318

11:33:52 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3319

11:33:52 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2871ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3320

11:33:52 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3321

11:33:52 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3322

11:33:52 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3323

11:33:52 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.770516 seconds.

3324

11:33:52 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 34 MiB, GPU 1660 MiB

3325

11:33:53 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3326

11:33:53 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3327

11:33:53 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3328

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3329

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3330

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3331

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3332

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3333

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 34, GPU 1652 (MiB)

3334

11:33:53 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3335

11:33:54 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +686, GPU +2, now: CPU 21063, GPU 3136 (MiB)

3336

11:33:54 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3337

11:33:54 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3338

11:33:54 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3339

11:33:55 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3340

11:33:55 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3341

11:33:55 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3342

11:33:55 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3343

11:33:55 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3223ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3344

11:33:55 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3345

11:33:55 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3346

11:33:55 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3347

11:33:55 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.762317 seconds.

3348

11:33:55 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 34 MiB, GPU 1673 MiB

3349

11:33:55 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3350

11:33:55 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3351

11:33:55 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3352

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3353

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3354

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3355

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3356

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3357

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 34, GPU 1665 (MiB)

3358

11:33:55 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3359

11:33:56 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +737, GPU +0, now: CPU 21066, GPU 3150 (MiB)

3360

11:33:56 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3361

11:33:56 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3362

11:33:56 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3363

11:33:57 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3364

11:33:57 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3365

11:33:57 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3366

11:33:57 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3367

11:33:57 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3081ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3368

11:33:57 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3369

11:33:57 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3370

11:33:57 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3371

11:33:57 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.737436 seconds.

3372

11:33:57 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 34 MiB, GPU 1686 MiB

3373

11:33:57 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3374

11:33:57 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3375

11:33:57 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3376

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3377

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3378

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3379

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3380

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3381

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 34, GPU 1678 (MiB)

3382

11:33:57 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3383

11:33:58 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +686, GPU +0, now: CPU 21072, GPU 3164 (MiB)

3384

11:33:58 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3385

11:33:58 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3386

11:33:58 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3387

11:33:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3388

11:33:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3389

11:33:59 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3390

11:33:59 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3391

11:33:59 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3577ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3392

11:33:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3393

11:33:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3394

11:33:59 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3395

11:33:59 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.762681 seconds.

3396

11:33:59 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 35 MiB, GPU 1699 MiB

3397

11:33:59 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3398

11:33:59 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3399

11:33:59 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3400

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3401

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3402

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3403

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3404

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3405

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 34, GPU 1691 (MiB)

3406

11:33:59 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3407

11:34:00 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +684, GPU +2, now: CPU 21082, GPU 3180 (MiB)

3408

11:34:00 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3409

11:34:00 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3410

11:34:00 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3411

11:34:01 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3412

11:34:01 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3413

11:34:01 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3414

11:34:01 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3415

11:34:01 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3367ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3416

11:34:01 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3417

11:34:01 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3418

11:34:01 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3419

11:34:01 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.755612 seconds.

3420

11:34:01 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 35 MiB, GPU 1712 MiB

3421

11:34:01 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3422

11:34:01 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3423

11:34:01 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3424

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3425

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3426

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3427

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3428

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3429

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 35, GPU 1704 (MiB)

3430

11:34:01 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3431

11:34:02 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +681, GPU +2, now: CPU 21088, GPU 3194 (MiB)

3432

11:34:02 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3433

11:34:02 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3434

11:34:02 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3435

11:34:03 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3436

11:34:03 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3437

11:34:03 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3438

11:34:03 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3439

11:34:03 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2982ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3440

11:34:03 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3441

11:34:03 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3442

11:34:03 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3443

11:34:03 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.744209 seconds.

3444

11:34:03 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 35 MiB, GPU 1725 MiB

3445

11:34:03 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3446

11:34:03 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3447

11:34:03 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3448

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3449

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3450

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3451

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3452

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3453

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 35, GPU 1718 (MiB)

3454

11:34:03 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3455

11:34:04 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +784, GPU +2, now: CPU 21128, GPU 3208 (MiB)

3456

11:34:04 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3457

11:34:04 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3458

11:34:05 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3459

11:34:05 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3460

11:34:05 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3461

11:34:05 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3462

11:34:05 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3463

11:34:05 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3722ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3464

11:34:05 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3465

11:34:05 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3466

11:34:05 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3467

11:34:05 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.740475 seconds.

3468

11:34:05 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 36 MiB, GPU 1739 MiB

3469

11:34:05 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3470

11:34:05 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3471

11:34:05 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3472

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3473

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3474

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3475

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3476

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3477

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 35, GPU 1731 (MiB)

3478

11:34:05 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3479

11:34:06 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +757, GPU +0, now: CPU 21100, GPU 3222 (MiB)

3480

11:34:06 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3481

11:34:06 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3482

11:34:07 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3483

11:34:07 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3484

11:34:07 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3485

11:34:07 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3486

11:34:07 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3487

11:34:07 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.4315ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3488

11:34:07 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3489

11:34:07 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3490

11:34:07 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3491

11:34:07 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.740235 seconds.

3492

11:34:07 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 36 MiB, GPU 1752 MiB

3493

11:34:07 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3494

11:34:07 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3495

11:34:07 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3496

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3497

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3498

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3499

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3500

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3501

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 36, GPU 1744 (MiB)

3502

11:34:07 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3503

11:34:08 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +704, GPU +0, now: CPU 21108, GPU 3236 (MiB)

3504

11:34:08 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3505

11:34:08 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3506

11:34:09 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3507

11:34:09 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3508

11:34:09 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3509

11:34:09 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3510

11:34:09 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3511

11:34:09 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2446ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3512

11:34:09 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3513

11:34:09 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3514

11:34:09 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3515

11:34:09 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.759288 seconds.

3516

11:34:09 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 36 MiB, GPU 1765 MiB

3517

11:34:09 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3518

11:34:09 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3519

11:34:09 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3520

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3521

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3522

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3523

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3524

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3525

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 36, GPU 1757 (MiB)

3526

11:34:09 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3527

11:34:10 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +707, GPU +2, now: CPU 21109, GPU 3252 (MiB)

3528

11:34:10 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3529

11:34:11 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3530

11:34:11 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3531

11:34:11 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3532

11:34:11 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3533

11:34:11 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3534

11:34:11 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3535

11:34:11 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2689ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3536

11:34:11 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3537

11:34:11 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3538

11:34:11 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3539

11:34:11 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.747775 seconds.

3540

11:34:11 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 37 MiB, GPU 1778 MiB

3541

11:34:11 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3542

11:34:11 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3543

11:34:11 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3544

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3545

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3546

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3547

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3548

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3549

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 36, GPU 1770 (MiB)

3550

11:34:11 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3551

11:34:12 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +755, GPU +2, now: CPU 21114, GPU 3266 (MiB)

3552

11:34:12 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3553

11:34:13 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3554

11:34:13 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3555

11:34:13 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3556

11:34:13 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3557

11:34:13 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3558

11:34:13 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3559

11:34:13 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2746ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3560

11:34:13 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3561

11:34:13 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3562

11:34:13 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3563

11:34:13 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.768565 seconds.

3564

11:34:13 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 37 MiB, GPU 1791 MiB

3565

11:34:13 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3566

11:34:13 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3567

11:34:13 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3568

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3569

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3570

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3571

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3572

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3573

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 37, GPU 1784 (MiB)

3574

11:34:14 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3575

11:34:14 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +726, GPU +2, now: CPU 21143, GPU 3284 (MiB)

3576

11:34:14 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3577

11:34:15 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3578

11:34:15 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3579

11:34:15 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3580

11:34:15 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3581

11:34:15 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3582

11:34:15 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3583

11:34:15 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3471ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3584

11:34:15 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3585

11:34:15 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3586

11:34:15 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3587

11:34:15 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.748818 seconds.

3588

11:34:15 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 37 MiB, GPU 1805 MiB

3589

11:34:15 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3590

11:34:15 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3591

11:34:15 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3592

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3593

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3594

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3595

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3596

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3597

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 37, GPU 1797 (MiB)

3598

11:34:16 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3599

11:34:16 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +743, GPU +0, now: CPU 21173, GPU 3298 (MiB)

3600

11:34:16 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3601

11:34:17 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3602

11:34:17 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3603

11:34:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3604

11:34:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3605

11:34:17 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3606

11:34:17 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3607

11:34:17 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2763ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3608

11:34:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3609

11:34:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3610

11:34:17 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3611

11:34:17 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.743119 seconds.

3612

11:34:17 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 38 MiB, GPU 1818 MiB

3613

11:34:17 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3614

11:34:17 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3615

11:34:17 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3616

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3617

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3618

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3619

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3620

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3621

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 37, GPU 1810 (MiB)

3622

11:34:18 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3623

11:34:18 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +763, GPU +0, now: CPU 21147, GPU 3312 (MiB)

3624

11:34:19 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3625

11:34:19 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3626

11:34:19 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3627

11:34:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3628

11:34:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3629

11:34:19 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3630

11:34:19 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3631

11:34:19 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.5116ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3632

11:34:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3633

11:34:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3634

11:34:19 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3635

11:34:19 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.770793 seconds.

3636

11:34:19 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 38 MiB, GPU 1831 MiB

3637

11:34:19 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3638

11:34:20 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3639

11:34:20 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3640

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3641

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3642

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3643

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3644

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3645

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 38, GPU 1823 (MiB)

3646

11:34:20 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3647

11:34:21 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +737, GPU +0, now: CPU 21174, GPU 3326 (MiB)

3648

11:34:21 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3649

11:34:21 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3650

11:34:21 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 7 output network tensors.

3651

11:34:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3652

11:34:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3653

11:34:21 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3654

11:34:21 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3655

11:34:21 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2816ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3656

11:34:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3657

11:34:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3658

11:34:21 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3659

11:34:21 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.773257 seconds.

3660

11:34:21 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 38 MiB, GPU 1844 MiB

3661

11:34:22 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3662

11:34:22 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3663

11:34:22 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3664

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3665

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3666

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3667

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3668

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3669

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 38, GPU 1836 (MiB)

3670

11:34:22 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3671

11:34:23 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +699, GPU +2, now: CPU 21156, GPU 3342 (MiB)

3672

11:34:23 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3673

11:34:23 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3674

11:34:23 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 11 inputs and 6 output network tensors.

3675

11:34:23 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202416 bytes

3676

11:34:23 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3677

11:34:23 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3678

11:34:23 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3679

11:34:23 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2727ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3680

11:34:23 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3681

11:34:23 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3329216 bytes

3682

11:34:23 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3683

11:34:23 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.74992 seconds.

3684

11:34:23 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 39 MiB, GPU 1856 MiB

3685

11:34:24 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3686

11:34:24 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3687

11:34:24 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3688

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3689

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3690

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3691

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3692

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3693

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 38, GPU 1849 (MiB)

3694

11:34:24 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3695

11:34:25 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +764, GPU +2, now: CPU 21165, GPU 3356 (MiB)

3696

11:34:25 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3697

11:34:25 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3698

11:34:26 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 8 inputs and 8 output network tensors.

3699

11:34:26 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 379968 bytes

3700

11:34:26 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3701

11:34:26 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3702

11:34:26 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 81 steps to complete.

3703

11:34:26 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.5708ms to assign 6 blocks to 81 nodes requiring 10486272 bytes.

3704

11:34:26 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3705

11:34:26 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 7441088 bytes

3706

11:34:26 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3707

11:34:26 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.2799 seconds.

3708

11:34:26 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 41 MiB, GPU 1868 MiB

3709

11:34:26 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3710

11:34:26 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3711

11:34:26 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3712

INFO: [Torch-TensorRT] - Loaded engine size: 8 MiB

3713

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3714

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3715

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3716

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3717

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 39, GPU 1866 (MiB)

3718

11:34:27 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3719

11:34:27 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +679, GPU +0, now: CPU 21183, GPU 3374 (MiB)

3720

11:34:27 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3721

11:34:28 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3722

11:34:28 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

3723

11:34:28 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

3724

11:34:28 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

3725

11:34:28 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3726

11:34:28 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3727

11:34:28 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3728

11:34:28 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3692ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3729

11:34:28 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3730

11:34:28 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

3731

11:34:28 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3732

11:34:28 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.806101 seconds.

3733

11:34:28 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 41 MiB, GPU 1887 MiB

3734

11:34:28 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3735

11:34:28 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3736

11:34:28 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3737

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3738

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3739

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3740

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3741

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3742

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 39, GPU 1880 (MiB)

3743

11:34:29 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3744

11:34:29 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +721, GPU +0, now: CPU 21196, GPU 3388 (MiB)

3745

11:34:29 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3746

11:34:30 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3747

11:34:30 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

3748

11:34:30 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

3749

11:34:30 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

3750

11:34:30 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3751

11:34:30 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3752

11:34:30 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3753

11:34:30 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2697ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3754

11:34:30 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3755

11:34:30 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

3756

11:34:30 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3757

11:34:30 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.802252 seconds.

3758

11:34:30 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 41 MiB, GPU 1901 MiB

3759

11:34:30 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3760

11:34:31 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3761

11:34:31 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3762

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3763

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3764

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3765

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3766

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3767

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 40, GPU 1893 (MiB)

3768

11:34:31 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3769

11:34:32 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +762, GPU +2, now: CPU 21197, GPU 3404 (MiB)

3770

11:34:32 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3771

11:34:32 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3772

11:34:32 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

3773

11:34:32 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

3774

11:34:32 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

3775

11:34:32 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3776

11:34:32 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3777

11:34:32 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3778

11:34:32 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2907ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3779

11:34:32 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3780

11:34:32 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

3781

11:34:32 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3782

11:34:32 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.776578 seconds.

3783

11:34:32 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 41 MiB, GPU 1914 MiB

3784

11:34:32 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3785

11:34:33 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3786

11:34:33 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3787

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3788

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3789

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3790

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3791

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3792

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 40, GPU 1906 (MiB)

3793

11:34:33 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3794

11:34:34 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +701, GPU +2, now: CPU 21204, GPU 3418 (MiB)

3795

11:34:34 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3796

11:34:34 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3797

11:34:34 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

3798

11:34:34 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

3799

11:34:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

3800

11:34:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3801

11:34:34 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3802

11:34:34 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3803

11:34:34 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3282ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3804

11:34:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3805

11:34:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

3806

11:34:34 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3807

11:34:34 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.77301 seconds.

3808

11:34:34 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 41 MiB, GPU 1927 MiB

3809

11:34:35 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3810

11:34:35 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3811

11:34:35 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3812

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3813

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3814

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3815

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3816

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3817

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 40, GPU 1919 (MiB)

3818

11:34:35 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3819

11:34:36 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +695, GPU +2, now: CPU 21207, GPU 3432 (MiB)

3820

11:34:36 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3821

11:34:36 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3822

11:34:36 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

3823

11:34:36 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

3824

11:34:36 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

3825

11:34:36 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3826

11:34:36 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3827

11:34:36 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3828

11:34:36 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2528ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3829

11:34:36 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3830

11:34:36 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

3831

11:34:36 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3832

11:34:36 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.784786 seconds.

3833

11:34:36 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 41 MiB, GPU 1940 MiB

3834

11:34:37 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3835

11:34:37 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3836

11:34:37 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3837

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3838

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3839

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3840

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3841

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3842

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 41, GPU 1933 (MiB)

3843

11:34:37 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3844

11:34:38 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +671, GPU +0, now: CPU 21224, GPU 3446 (MiB)

3845

11:34:38 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3846

11:34:38 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3847

11:34:38 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

3848

11:34:38 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

3849

11:34:39 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

3850

11:34:39 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3851

11:34:39 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3852

11:34:39 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3853

11:34:39 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2814ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3854

11:34:39 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3855

11:34:39 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

3856

11:34:39 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3857

11:34:39 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.802234 seconds.

3858

11:34:39 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 41 MiB, GPU 1954 MiB

3859

11:34:39 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3860

11:34:39 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3861

11:34:39 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3862

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3863

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3864

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3865

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3866

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3867

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 41, GPU 1946 (MiB)

3868

11:34:39 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3869

11:34:40 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +650, GPU +0, now: CPU 21226, GPU 3460 (MiB)

3870

11:34:40 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3871

11:34:40 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3872

11:34:40 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

3873

11:34:40 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

3874

11:34:41 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

3875

11:34:41 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3876

11:34:41 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3877

11:34:41 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3878

11:34:41 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3107ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3879

11:34:41 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3880

11:34:41 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

3881

11:34:41 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3882

11:34:41 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.780278 seconds.

3883

11:34:41 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 41 MiB, GPU 1967 MiB

3884

11:34:41 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3885

11:34:41 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3886

11:34:41 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3887

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3888

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3889

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3890

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3891

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3892

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 41, GPU 1959 (MiB)

3893

11:34:41 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3894

11:34:42 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +658, GPU +2, now: CPU 21234, GPU 3476 (MiB)

3895

11:34:42 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3896

11:34:42 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3897

11:34:42 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

3898

11:34:42 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

3899

11:34:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

3900

11:34:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3901

11:34:43 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3902

11:34:43 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3903

11:34:43 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2844ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3904

11:34:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3905

11:34:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

3906

11:34:43 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3907

11:34:43 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.781316 seconds.

3908

11:34:43 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 42 MiB, GPU 1980 MiB

3909

11:34:43 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3910

11:34:43 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3911

11:34:43 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3912

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3913

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3914

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3915

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3916

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3917

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 42, GPU 1972 (MiB)

3918

11:34:43 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3919

11:34:44 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +744, GPU +2, now: CPU 21241, GPU 3490 (MiB)

3920

11:34:44 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3921

11:34:44 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3922

11:34:44 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

3923

11:34:44 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

3924

11:34:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

3925

11:34:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3926

11:34:45 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3927

11:34:45 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3928

11:34:45 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.4865ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3929

11:34:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3930

11:34:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

3931

11:34:45 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3932

11:34:45 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.775158 seconds.

3933

11:34:45 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 42 MiB, GPU 1993 MiB

3934

11:34:45 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3935

11:34:45 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3936

11:34:45 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3937

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3938

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3939

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3940

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3941

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3942

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 42, GPU 1986 (MiB)

3943

11:34:45 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3944

11:34:46 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +759, GPU +0, now: CPU 21252, GPU 3504 (MiB)

3945

11:34:46 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3946

11:34:46 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3947

11:34:46 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

3948

11:34:46 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

3949

11:34:47 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

3950

11:34:47 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3951

11:34:47 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3952

11:34:47 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3953

11:34:47 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.4417ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3954

11:34:47 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3955

11:34:47 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

3956

11:34:47 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3957

11:34:47 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.807132 seconds.

3958

11:34:47 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 42 MiB, GPU 2007 MiB

3959

11:34:47 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3960

11:34:47 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3961

11:34:47 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3962

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3963

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3964

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3965

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3966

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3967

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 42, GPU 1999 (MiB)

3968

11:34:47 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3969

11:34:48 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +683, GPU +0, now: CPU 21264, GPU 3520 (MiB)

3970

11:34:48 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3971

11:34:48 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3972

11:34:48 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

3973

11:34:48 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

3974

11:34:49 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

3975

11:34:49 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

3976

11:34:49 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

3977

11:34:49 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

3978

11:34:49 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3162ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

3979

11:34:49 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

3980

11:34:49 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

3981

11:34:49 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

3982

11:34:49 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.788939 seconds.

3983

11:34:49 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 43 MiB, GPU 2020 MiB

3984

11:34:49 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

3985

11:34:49 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

3986

11:34:49 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

3987

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

3988

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

3989

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

3990

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

3991

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

3992

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 42, GPU 2012 (MiB)

3993

11:34:49 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

3994

11:34:50 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +752, GPU +2, now: CPU 21276, GPU 3536 (MiB)

3995

11:34:50 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

3996

11:34:50 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

3997

11:34:50 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

3998

11:34:50 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

3999

11:34:51 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4000

11:34:51 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4001

11:34:51 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4002

11:34:51 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4003

11:34:51 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.441ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4004

11:34:51 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4005

11:34:51 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4006

11:34:51 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4007

11:34:51 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.780624 seconds.

4008

11:34:51 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 43 MiB, GPU 2033 MiB

4009

11:34:51 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4010

11:34:51 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4011

11:34:51 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4012

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4013

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4014

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4015

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4016

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4017

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 43, GPU 2025 (MiB)

4018

11:34:51 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4019

11:34:52 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +745, GPU +2, now: CPU 21275, GPU 3550 (MiB)

4020

11:34:52 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4021

11:34:52 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4022

11:34:53 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4023

11:34:53 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4024

11:34:53 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4025

11:34:53 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4026

11:34:53 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4027

11:34:53 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4028

11:34:53 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2633ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4029

11:34:53 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4030

11:34:53 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4031

11:34:53 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4032

11:34:53 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.781615 seconds.

4033

11:34:53 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 43 MiB, GPU 2046 MiB

4034

11:34:53 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4035

11:34:53 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4036

11:34:53 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4037

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4038

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4039

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4040

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4041

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4042

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 43, GPU 2039 (MiB)

4043

11:34:53 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4044

11:34:54 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +671, GPU +2, now: CPU 21291, GPU 3564 (MiB)

4045

11:34:54 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4046

11:34:54 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4047

11:34:55 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4048

11:34:55 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4049

11:34:55 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4050

11:34:55 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4051

11:34:55 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4052

11:34:55 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4053

11:34:55 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2908ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4054

11:34:55 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4055

11:34:55 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4056

11:34:55 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4057

11:34:55 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.821516 seconds.

4058

11:34:55 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 44 MiB, GPU 2060 MiB

4059

11:34:55 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4060

11:34:55 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4061

11:34:55 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4062

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4063

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4064

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4065

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4066

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4067

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 43, GPU 2052 (MiB)

4068

11:34:55 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4069

11:34:56 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +731, GPU +2, now: CPU 21298, GPU 3580 (MiB)

4070

11:34:56 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4071

11:34:56 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4072

11:34:57 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4073

11:34:57 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4074

11:34:57 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4075

11:34:57 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4076

11:34:57 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4077

11:34:57 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4078

11:34:57 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2534ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4079

11:34:57 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4080

11:34:57 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4081

11:34:57 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4082

11:34:57 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.806626 seconds.

4083

11:34:57 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 44 MiB, GPU 2073 MiB

4084

11:34:57 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4085

11:34:57 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4086

11:34:57 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4087

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4088

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4089

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4090

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4091

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4092

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 44, GPU 2065 (MiB)

4093

11:34:57 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4094

11:34:58 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +723, GPU +0, now: CPU 21306, GPU 3594 (MiB)

4095

11:34:58 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4096

11:34:59 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4097

11:34:59 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4098

11:34:59 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4099

11:34:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4100

11:34:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4101

11:34:59 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4102

11:34:59 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4103

11:34:59 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3382ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4104

11:34:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4105

11:34:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4106

11:34:59 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4107

11:34:59 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.80535 seconds.

4108

11:34:59 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 44 MiB, GPU 2086 MiB

4109

11:34:59 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4110

11:34:59 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4111

11:34:59 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4112

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4113

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4114

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4115

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4116

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4117

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 44, GPU 2078 (MiB)

4118

11:34:59 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4119

11:35:00 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +698, GPU +2, now: CPU 21325, GPU 3610 (MiB)

4120

11:35:00 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4121

11:35:01 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4122

11:35:01 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4123

11:35:01 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4124

11:35:01 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4125

11:35:01 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4126

11:35:01 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4127

11:35:01 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4128

11:35:01 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.4009ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4129

11:35:01 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4130

11:35:01 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4131

11:35:01 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4132

11:35:01 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.800396 seconds.

4133

11:35:01 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 45 MiB, GPU 2099 MiB

4134

11:35:01 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4135

11:35:01 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4136

11:35:01 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4137

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4138

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4139

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4140

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4141

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4142

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 44, GPU 2092 (MiB)

4143

11:35:02 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4144

11:35:02 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +709, GPU +2, now: CPU 21318, GPU 3624 (MiB)

4145

11:35:02 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4146

11:35:03 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4147

11:35:03 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4148

11:35:03 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4149

11:35:03 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4150

11:35:03 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4151

11:35:03 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4152

11:35:03 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4153

11:35:03 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3443ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4154

11:35:03 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4155

11:35:03 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4156

11:35:03 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4157

11:35:03 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.797801 seconds.

4158

11:35:03 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 45 MiB, GPU 2113 MiB

4159

11:35:03 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4160

11:35:03 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4161

11:35:03 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4162

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4163

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4164

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4165

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4166

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4167

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 45, GPU 2105 (MiB)

4168

11:35:04 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4169

11:35:04 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +711, GPU +2, now: CPU 21331, GPU 3638 (MiB)

4170

11:35:04 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4171

11:35:05 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4172

11:35:05 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4173

11:35:05 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4174

11:35:05 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4175

11:35:05 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4176

11:35:05 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4177

11:35:05 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4178

11:35:05 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3186ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4179

11:35:05 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4180

11:35:05 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4181

11:35:05 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4182

11:35:05 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.829447 seconds.

4183

11:35:05 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 45 MiB, GPU 2126 MiB

4184

11:35:05 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4185

11:35:05 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4186

11:35:05 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4187

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4188

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4189

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4190

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4191

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4192

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 45, GPU 2118 (MiB)

4193

11:35:06 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4194

11:35:07 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +784, GPU +0, now: CPU 21602, GPU 3652 (MiB)

4195

11:35:07 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4196

11:35:07 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4197

11:35:07 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4198

11:35:07 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4199

11:35:08 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4200

11:35:08 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4201

11:35:08 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4202

11:35:08 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4203

11:35:08 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.383ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4204

11:35:08 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4205

11:35:08 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4206

11:35:08 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4207

11:35:08 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.02068 seconds.

4208

11:35:08 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 46 MiB, GPU 2139 MiB

4209

11:35:08 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4210

11:35:08 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4211

11:35:08 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4212

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4213

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4214

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4215

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4216

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4217

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 45, GPU 2131 (MiB)

4218

11:35:08 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4219

11:35:09 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +399, GPU +0, now: CPU 21674, GPU 3666 (MiB)

4220

11:35:09 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4221

11:35:10 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4222

11:35:10 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4223

11:35:10 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4224

11:35:10 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4225

11:35:10 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4226

11:35:10 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4227

11:35:10 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4228

11:35:10 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.4361ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4229

11:35:10 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4230

11:35:10 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4231

11:35:10 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4232

11:35:10 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.910561 seconds.

4233

11:35:10 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 46 MiB, GPU 2152 MiB

4234

11:35:10 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4235

11:35:10 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4236

11:35:10 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4237

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4238

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4239

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4240

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4241

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4242

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 46, GPU 2145 (MiB)

4243

11:35:11 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4244

11:35:11 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +693, GPU +2, now: CPU 21660, GPU 3682 (MiB)

4245

11:35:11 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4246

11:35:12 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4247

11:35:12 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4248

11:35:12 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4249

11:35:12 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4250

11:35:12 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4251

11:35:12 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4252

11:35:12 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4253

11:35:12 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.4101ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4254

11:35:12 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4255

11:35:12 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4256

11:35:12 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4257

11:35:12 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.872268 seconds.

4258

11:35:12 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 46 MiB, GPU 2166 MiB

4259

11:35:12 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4260

11:35:13 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4261

11:35:13 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4262

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4263

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4264

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4265

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4266

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4267

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 46, GPU 2158 (MiB)

4268

11:35:13 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4269

11:35:14 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +647, GPU +2, now: CPU 21667, GPU 3696 (MiB)

4270

11:35:14 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4271

11:35:14 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4272

11:35:14 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4273

11:35:14 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4274

11:35:14 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4275

11:35:14 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4276

11:35:14 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4277

11:35:14 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4278

11:35:14 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2506ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4279

11:35:14 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4280

11:35:14 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4281

11:35:14 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4282

11:35:14 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.837217 seconds.

4283

11:35:14 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 46 MiB, GPU 2179 MiB

4284

11:35:15 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4285

11:35:15 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4286

11:35:15 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4287

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4288

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4289

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4290

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4291

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4292

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 46, GPU 2171 (MiB)

4293

11:35:15 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4294

11:35:16 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +645, GPU +0, now: CPU 21685, GPU 3710 (MiB)

4295

11:35:16 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4296

11:35:16 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4297

11:35:16 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4298

11:35:16 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4299

11:35:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4300

11:35:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4301

11:35:17 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4302

11:35:17 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4303

11:35:17 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3742ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4304

11:35:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4305

11:35:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4306

11:35:17 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4307

11:35:17 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.809133 seconds.

4308

11:35:17 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 47 MiB, GPU 2192 MiB

4309

11:35:17 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4310

11:35:17 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4311

11:35:17 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4312

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4313

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4314

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4315

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4316

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4317

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 47, GPU 2184 (MiB)

4318

11:35:17 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4319

11:35:18 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +683, GPU +0, now: CPU 21697, GPU 3724 (MiB)

4320

11:35:18 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4321

11:35:18 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4322

11:35:18 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4323

11:35:18 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4324

11:35:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4325

11:35:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4326

11:35:19 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4327

11:35:19 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4328

11:35:19 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2573ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4329

11:35:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4330

11:35:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4331

11:35:19 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4332

11:35:19 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.795466 seconds.

4333

11:35:19 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 47 MiB, GPU 2205 MiB

4334

11:35:19 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4335

11:35:19 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4336

11:35:19 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4337

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4338

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4339

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4340

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4341

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4342

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 47, GPU 2198 (MiB)

4343

11:35:19 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4344

11:35:20 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +715, GPU +2, now: CPU 21702, GPU 3740 (MiB)

4345

11:35:20 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4346

11:35:20 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4347

11:35:20 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4348

11:35:20 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4349

11:35:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4350

11:35:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4351

11:35:21 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4352

11:35:21 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4353

11:35:21 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2532ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4354

11:35:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4355

11:35:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4356

11:35:21 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4357

11:35:21 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.774385 seconds.

4358

11:35:21 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 47 MiB, GPU 2219 MiB

4359

11:35:21 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4360

11:35:21 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4361

11:35:21 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4362

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4363

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4364

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4365

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4366

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4367

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 47, GPU 2211 (MiB)

4368

11:35:21 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4369

11:35:22 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +744, GPU +2, now: CPU 21710, GPU 3758 (MiB)

4370

11:35:22 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4371

11:35:22 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4372

11:35:22 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4373

11:35:22 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4374

11:35:23 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4375

11:35:23 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4376

11:35:23 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4377

11:35:23 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4378

11:35:23 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2699ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4379

11:35:23 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4380

11:35:23 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4381

11:35:23 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4382

11:35:23 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.801723 seconds.

4383

11:35:23 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 48 MiB, GPU 2232 MiB

4384

11:35:23 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4385

11:35:23 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4386

11:35:23 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4387

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4388

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4389

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4390

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4391

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4392

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 48, GPU 2224 (MiB)

4393

11:35:23 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4394

11:35:24 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +689, GPU +2, now: CPU 21720, GPU 3772 (MiB)

4395

11:35:24 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4396

11:35:24 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4397

11:35:24 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4398

11:35:24 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4399

11:35:25 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4400

11:35:25 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4401

11:35:25 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4402

11:35:25 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4403

11:35:25 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3218ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4404

11:35:25 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4405

11:35:25 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4406

11:35:25 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4407

11:35:25 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.801397 seconds.

4408

11:35:25 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 48 MiB, GPU 2245 MiB

4409

11:35:25 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4410

11:35:25 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4411

11:35:25 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4412

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4413

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4414

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4415

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4416

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4417

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 48, GPU 2237 (MiB)

4418

11:35:25 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4419

11:35:26 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +746, GPU +0, now: CPU 21732, GPU 3786 (MiB)

4420

11:35:26 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4421

11:35:26 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4422

11:35:27 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4423

11:35:27 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4424

11:35:27 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4425

11:35:27 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4426

11:35:27 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4427

11:35:27 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4428

11:35:27 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2701ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4429

11:35:27 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4430

11:35:27 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4431

11:35:27 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4432

11:35:27 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.79552 seconds.

4433

11:35:27 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 48 MiB, GPU 2258 MiB

4434

11:35:27 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4435

11:35:27 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4436

11:35:27 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4437

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4438

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4439

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4440

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4441

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4442

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 48, GPU 2251 (MiB)

4443

11:35:27 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4444

11:35:28 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +736, GPU +0, now: CPU 21736, GPU 3800 (MiB)

4445

11:35:28 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4446

11:35:28 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4447

11:35:29 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4448

11:35:29 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4449

11:35:29 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4450

11:35:29 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4451

11:35:29 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4452

11:35:29 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4453

11:35:29 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.276ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4454

11:35:29 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4455

11:35:29 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4456

11:35:29 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4457

11:35:29 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.770373 seconds.

4458

11:35:29 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 49 MiB, GPU 2272 MiB

4459

11:35:29 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4460

11:35:29 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4461

11:35:29 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4462

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4463

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4464

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4465

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4466

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4467

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 49, GPU 2264 (MiB)

4468

11:35:29 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4469

11:35:30 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +697, GPU +2, now: CPU 21741, GPU 3816 (MiB)

4470

11:35:30 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4471

11:35:30 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4472

11:35:31 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4473

11:35:31 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4474

11:35:31 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4475

11:35:31 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4476

11:35:31 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4477

11:35:31 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4478

11:35:31 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3914ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4479

11:35:31 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4480

11:35:31 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4481

11:35:31 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4482

11:35:31 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.800709 seconds.

4483

11:35:31 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 49 MiB, GPU 2285 MiB

4484

11:35:31 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4485

11:35:31 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4486

11:35:31 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4487

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4488

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4489

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4490

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4491

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4492

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 49, GPU 2277 (MiB)

4493

11:35:31 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4494

11:35:32 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +702, GPU +2, now: CPU 21733, GPU 3830 (MiB)

4495

11:35:32 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4496

11:35:33 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4497

11:35:33 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4498

11:35:33 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4499

11:35:33 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4500

11:35:33 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4501

11:35:33 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4502

11:35:33 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4503

11:35:33 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2616ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4504

11:35:33 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4505

11:35:33 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4506

11:35:33 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4507

11:35:33 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.812707 seconds.

4508

11:35:33 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 49 MiB, GPU 2298 MiB

4509

11:35:33 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4510

11:35:33 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4511

11:35:33 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4512

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4513

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4514

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4515

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4516

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4517

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 49, GPU 2290 (MiB)

4518

11:35:33 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4519

11:35:34 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +713, GPU +0, now: CPU 21735, GPU 3844 (MiB)

4520

11:35:34 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4521

11:35:35 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4522

11:35:35 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4523

11:35:35 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4524

11:35:35 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4525

11:35:35 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4526

11:35:35 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4527

11:35:35 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4528

11:35:35 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.4313ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4529

11:35:35 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4530

11:35:35 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4531

11:35:35 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4532

11:35:35 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.792615 seconds.

4533

11:35:35 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 50 MiB, GPU 2311 MiB

4534

11:35:35 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4535

11:35:35 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4536

11:35:35 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4537

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4538

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4539

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4540

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4541

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4542

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 50, GPU 2304 (MiB)

4543

11:35:36 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4544

11:35:36 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +757, GPU +0, now: CPU 21745, GPU 3858 (MiB)

4545

11:35:36 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4546

11:35:37 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4547

11:35:37 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4548

11:35:37 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4549

11:35:37 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4550

11:35:37 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4551

11:35:37 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4552

11:35:37 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4553

11:35:37 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.42ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4554

11:35:37 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4555

11:35:37 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4556

11:35:37 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4557

11:35:37 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.801002 seconds.

4558

11:35:37 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 50 MiB, GPU 2325 MiB

4559

11:35:37 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4560

11:35:37 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4561

11:35:37 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4562

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4563

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4564

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4565

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4566

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4567

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 50, GPU 2317 (MiB)

4568

11:35:38 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4569

11:35:38 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +689, GPU +2, now: CPU 21754, GPU 3874 (MiB)

4570

11:35:38 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4571

11:35:39 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4572

11:35:39 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4573

11:35:39 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4574

11:35:39 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4575

11:35:39 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4576

11:35:39 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4577

11:35:39 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4578

11:35:39 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3915ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4579

11:35:39 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4580

11:35:39 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4581

11:35:39 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4582

11:35:39 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.78126 seconds.

4583

11:35:39 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 50 MiB, GPU 2338 MiB

4584

11:35:39 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4585

11:35:39 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4586

11:35:39 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4587

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4588

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4589

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4590

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4591

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4592

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 50, GPU 2330 (MiB)

4593

11:35:40 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4594

11:35:40 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +712, GPU +2, now: CPU 21761, GPU 3888 (MiB)

4595

11:35:41 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4596

11:35:41 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4597

11:35:41 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4598

11:35:41 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4599

11:35:41 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4600

11:35:41 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4601

11:35:41 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4602

11:35:41 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4603

11:35:41 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3056ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4604

11:35:41 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4605

11:35:41 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4606

11:35:41 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4607

11:35:41 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.796555 seconds.

4608

11:35:41 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 51 MiB, GPU 2351 MiB

4609

11:35:41 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4610

11:35:42 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4611

11:35:42 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4612

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4613

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4614

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4615

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4616

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4617

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 50, GPU 2343 (MiB)

4618

11:35:42 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4619

11:35:43 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +671, GPU +2, now: CPU 21774, GPU 3902 (MiB)

4620

11:35:43 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4621

11:35:43 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4622

11:35:43 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4623

11:35:43 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4624

11:35:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4625

11:35:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4626

11:35:43 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4627

11:35:43 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4628

11:35:43 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2643ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4629

11:35:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4630

11:35:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4631

11:35:43 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4632

11:35:43 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.815333 seconds.

4633

11:35:43 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 51 MiB, GPU 2364 MiB

4634

11:35:44 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4635

11:35:44 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4636

11:35:44 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4637

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4638

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4639

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4640

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4641

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4642

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 51, GPU 2357 (MiB)

4643

11:35:44 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4644

11:35:45 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +671, GPU +0, now: CPU 21782, GPU 3916 (MiB)

4645

11:35:45 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4646

11:35:45 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4647

11:35:45 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4648

11:35:45 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4649

11:35:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4650

11:35:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4651

11:35:45 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4652

11:35:45 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4653

11:35:45 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3319ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4654

11:35:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4655

11:35:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4656

11:35:45 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4657

11:35:45 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.792801 seconds.

4658

11:35:45 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 51 MiB, GPU 2378 MiB

4659

11:35:46 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4660

11:35:46 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4661

11:35:46 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4662

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4663

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4664

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4665

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4666

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4667

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 51, GPU 2370 (MiB)

4668

11:35:46 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4669

11:35:47 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +701, GPU +0, now: CPU 21789, GPU 3930 (MiB)

4670

11:35:47 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4671

11:35:47 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4672

11:35:47 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4673

11:35:47 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4674

11:35:48 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4675

11:35:48 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4676

11:35:48 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4677

11:35:48 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4678

11:35:48 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.4095ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4679

11:35:48 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4680

11:35:48 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4681

11:35:48 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4682

11:35:48 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.792316 seconds.

4683

11:35:48 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 52 MiB, GPU 2391 MiB

4684

11:35:48 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4685

11:35:48 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4686

11:35:48 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4687

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4688

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4689

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4690

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4691

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4692

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 51, GPU 2383 (MiB)

4693

11:35:48 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4694

11:35:49 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +787, GPU +2, now: CPU 21828, GPU 3946 (MiB)

4695

11:35:49 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4696

11:35:49 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4697

11:35:49 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4698

11:35:49 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4699

11:35:50 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4700

11:35:50 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4701

11:35:50 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4702

11:35:50 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4703

11:35:50 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3116ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4704

11:35:50 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4705

11:35:50 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4706

11:35:50 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4707

11:35:50 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.793936 seconds.

4708

11:35:50 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 52 MiB, GPU 2404 MiB

4709

11:35:50 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4710

11:35:50 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4711

11:35:50 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4712

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4713

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4714

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4715

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4716

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4717

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 52, GPU 2396 (MiB)

4718

11:35:50 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4719

11:35:51 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +770, GPU +2, now: CPU 21818, GPU 3960 (MiB)

4720

11:35:51 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4721

11:35:51 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4722

11:35:51 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4723

11:35:51 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4724

11:35:52 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4725

11:35:52 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4726

11:35:52 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4727

11:35:52 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4728

11:35:52 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.412ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4729

11:35:52 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4730

11:35:52 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4731

11:35:52 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4732

11:35:52 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.807368 seconds.

4733

11:35:52 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 52 MiB, GPU 2417 MiB

4734

11:35:52 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4735

11:35:52 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4736

11:35:52 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4737

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4738

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4739

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4740

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4741

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4742

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 52, GPU 2410 (MiB)

4743

11:35:52 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4744

11:35:53 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +721, GPU +0, now: CPU 21831, GPU 3974 (MiB)

4745

11:35:53 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4746

11:35:53 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4747

11:35:53 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4748

11:35:53 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4749

11:35:54 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4750

11:35:54 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4751

11:35:54 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4752

11:35:54 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4753

11:35:54 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2989ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4754

11:35:54 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4755

11:35:54 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4756

11:35:54 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4757

11:35:54 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.798048 seconds.

4758

11:35:54 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 53 MiB, GPU 2431 MiB

4759

11:35:54 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4760

11:35:54 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4761

11:35:54 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4762

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4763

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4764

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4765

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4766

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4767

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 52, GPU 2423 (MiB)

4768

11:35:54 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4769

11:35:55 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +767, GPU +0, now: CPU 21857, GPU 3990 (MiB)

4770

11:35:55 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4771

11:35:55 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4772

11:35:55 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4773

11:35:55 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4774

11:35:56 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4775

11:35:56 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4776

11:35:56 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4777

11:35:56 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4778

11:35:56 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.4283ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4779

11:35:56 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4780

11:35:56 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4781

11:35:56 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4782

11:35:56 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.772758 seconds.

4783

11:35:56 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 53 MiB, GPU 2444 MiB

4784

11:35:56 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4785

11:35:56 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4786

11:35:56 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4787

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4788

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4789

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4790

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4791

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4792

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 53, GPU 2436 (MiB)

4793

11:35:56 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4794

11:35:57 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +747, GPU +0, now: CPU 21837, GPU 4004 (MiB)

4795

11:35:57 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4796

11:35:57 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4797

11:35:58 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4798

11:35:58 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4799

11:35:58 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4800

11:35:58 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4801

11:35:58 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4802

11:35:58 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4803

11:35:58 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2676ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4804

11:35:58 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4805

11:35:58 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4806

11:35:58 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4807

11:35:58 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.786534 seconds.

4808

11:35:58 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 53 MiB, GPU 2457 MiB

4809

11:35:58 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4810

11:35:58 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4811

11:35:58 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4812

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4813

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4814

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4815

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4816

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4817

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 53, GPU 2449 (MiB)

4818

11:35:58 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4819

11:35:59 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +711, GPU +2, now: CPU 21840, GPU 4020 (MiB)

4820

11:35:59 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4821

11:35:59 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4822

11:36:00 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4823

11:36:00 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4824

11:36:00 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4825

11:36:00 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4826

11:36:00 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4827

11:36:00 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4828

11:36:00 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.4089ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4829

11:36:00 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4830

11:36:00 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4831

11:36:00 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4832

11:36:00 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.791999 seconds.

4833

11:36:00 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 54 MiB, GPU 2470 MiB

4834

11:36:00 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4835

11:36:00 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4836

11:36:00 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4837

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4838

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4839

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4840

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4841

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4842

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 53, GPU 2463 (MiB)

4843

11:36:00 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4844

11:36:01 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +719, GPU +2, now: CPU 21857, GPU 4034 (MiB)

4845

11:36:01 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4846

11:36:02 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4847

11:36:02 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4848

11:36:02 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4849

11:36:02 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4850

11:36:02 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4851

11:36:02 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4852

11:36:02 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4853

11:36:02 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.374ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4854

11:36:02 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4855

11:36:02 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4856

11:36:02 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4857

11:36:02 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.789329 seconds.

4858

11:36:02 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 54 MiB, GPU 2484 MiB

4859

11:36:02 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4860

11:36:02 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4861

11:36:02 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4862

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4863

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4864

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4865

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4866

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4867

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 54, GPU 2476 (MiB)

4868

11:36:02 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4869

11:36:03 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +720, GPU +0, now: CPU 21865, GPU 4048 (MiB)

4870

11:36:03 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4871

11:36:04 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4872

11:36:04 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4873

11:36:04 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4874

11:36:04 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4875

11:36:04 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4876

11:36:04 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4877

11:36:04 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4878

11:36:04 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2504ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4879

11:36:04 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4880

11:36:04 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4881

11:36:04 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4882

11:36:04 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.773563 seconds.

4883

11:36:04 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 54 MiB, GPU 2497 MiB

4884

11:36:04 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4885

11:36:04 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4886

11:36:04 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4887

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4888

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4889

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4890

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4891

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4892

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 54, GPU 2489 (MiB)

4893

11:36:04 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4894

11:36:05 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +687, GPU +0, now: CPU 21872, GPU 4062 (MiB)

4895

11:36:05 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4896

11:36:06 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4897

11:36:06 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4898

11:36:06 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4899

11:36:06 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4900

11:36:06 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4901

11:36:06 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4902

11:36:06 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4903

11:36:06 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2474ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4904

11:36:06 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4905

11:36:06 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4906

11:36:06 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4907

11:36:06 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.793052 seconds.

4908

11:36:06 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 54 MiB, GPU 2510 MiB

4909

11:36:06 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4910

11:36:06 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4911

11:36:06 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4912

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4913

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4914

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4915

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4916

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4917

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 54, GPU 2502 (MiB)

4918

11:36:06 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4919

11:36:07 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +699, GPU +2, now: CPU 21880, GPU 4078 (MiB)

4920

11:36:07 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4921

11:36:08 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4922

11:36:08 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4923

11:36:08 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4924

11:36:08 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4925

11:36:08 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4926

11:36:08 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4927

11:36:08 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4928

11:36:08 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3195ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4929

11:36:08 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4930

11:36:08 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4931

11:36:08 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4932

11:36:08 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.852992 seconds.

4933

11:36:08 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 55 MiB, GPU 2523 MiB

4934

11:36:08 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4935

11:36:08 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4936

11:36:08 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4937

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4938

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4939

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4940

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4941

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4942

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 55, GPU 2516 (MiB)

4943

11:36:09 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4944

11:36:09 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +759, GPU +2, now: CPU 21918, GPU 4092 (MiB)

4945

11:36:09 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4946

11:36:10 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4947

11:36:10 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4948

11:36:10 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4949

11:36:10 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4950

11:36:10 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4951

11:36:10 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4952

11:36:10 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4953

11:36:10 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3688ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4954

11:36:10 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4955

11:36:10 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4956

11:36:10 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4957

11:36:10 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.798494 seconds.

4958

11:36:10 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 55 MiB, GPU 2537 MiB

4959

11:36:10 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4960

11:36:10 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4961

11:36:10 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4962

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4963

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4964

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4965

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4966

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4967

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 55, GPU 2529 (MiB)

4968

11:36:11 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4969

11:36:11 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +674, GPU +2, now: CPU 21899, GPU 4106 (MiB)

4970

11:36:12 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4971

11:36:12 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4972

11:36:12 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4973

11:36:12 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4974

11:36:12 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

4975

11:36:12 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

4976

11:36:12 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

4977

11:36:12 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

4978

11:36:12 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3289ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

4979

11:36:12 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

4980

11:36:12 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

4981

11:36:12 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

4982

11:36:12 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.787708 seconds.

4983

11:36:12 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 55 MiB, GPU 2550 MiB

4984

11:36:12 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

4985

11:36:13 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

4986

11:36:13 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

4987

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

4988

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

4989

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

4990

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

4991

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

4992

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 55, GPU 2542 (MiB)

4993

11:36:13 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

4994

11:36:14 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +681, GPU +0, now: CPU 21919, GPU 4120 (MiB)

4995

11:36:14 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

4996

11:36:14 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

4997

11:36:14 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

4998

11:36:14 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

4999

11:36:14 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

5000

11:36:14 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5001

11:36:14 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5002

11:36:14 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

5003

11:36:14 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3497ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

5004

11:36:14 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

5005

11:36:14 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

5006

11:36:14 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5007

11:36:14 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.806489 seconds.

5008

11:36:14 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 56 MiB, GPU 2563 MiB

5009

11:36:15 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5010

11:36:15 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5011

11:36:15 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5012

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

5013

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5014

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5015

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5016

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5017

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 56, GPU 2555 (MiB)

5018

11:36:15 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5019

11:36:16 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +722, GPU +0, now: CPU 21926, GPU 4134 (MiB)

5020

11:36:16 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5021

11:36:16 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5022

11:36:16 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5023

11:36:16 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

5024

11:36:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

5025

11:36:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5026

11:36:17 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5027

11:36:17 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

5028

11:36:17 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2716ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

5029

11:36:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

5030

11:36:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

5031

11:36:17 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5032

11:36:17 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.820182 seconds.

5033

11:36:17 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 56 MiB, GPU 2576 MiB

5034

11:36:17 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5035

11:36:17 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5036

11:36:17 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5037

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

5038

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5039

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5040

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5041

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5042

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 56, GPU 2569 (MiB)

5043

11:36:17 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5044

11:36:18 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +636, GPU +2, now: CPU 21960, GPU 4150 (MiB)

5045

11:36:18 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5046

11:36:18 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5047

11:36:18 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5048

11:36:18 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

5049

11:36:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

5050

11:36:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5051

11:36:19 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5052

11:36:19 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

5053

11:36:19 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3219ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

5054

11:36:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

5055

11:36:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

5056

11:36:19 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5057

11:36:19 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.815865 seconds.

5058

11:36:19 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 56 MiB, GPU 2590 MiB

5059

11:36:19 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5060

11:36:19 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5061

11:36:19 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5062

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

5063

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5064

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5065

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5066

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5067

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 56, GPU 2582 (MiB)

5068

11:36:19 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5069

11:36:20 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +617, GPU +4, now: CPU 21924, GPU 4166 (MiB)

5070

11:36:20 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5071

11:36:20 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5072

11:36:20 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5073

11:36:20 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

5074

11:36:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

5075

11:36:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5076

11:36:21 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5077

11:36:21 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

5078

11:36:21 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2855ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

5079

11:36:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

5080

11:36:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

5081

11:36:21 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5082

11:36:21 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.815258 seconds.

5083

11:36:21 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 57 MiB, GPU 2603 MiB

5084

11:36:21 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5085

11:36:21 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5086

11:36:21 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5087

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

5088

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5089

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5090

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5091

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5092

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 57, GPU 2595 (MiB)

5093

11:36:21 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5094

11:36:22 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +735, GPU +0, now: CPU 21951, GPU 4180 (MiB)

5095

11:36:22 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5096

11:36:22 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5097

11:36:23 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5098

11:36:23 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

5099

11:36:23 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

5100

11:36:23 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5101

11:36:23 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5102

11:36:23 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

5103

11:36:23 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3113ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

5104

11:36:23 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

5105

11:36:23 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

5106

11:36:23 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5107

11:36:23 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.778967 seconds.

5108

11:36:23 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 57 MiB, GPU 2616 MiB

5109

11:36:23 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5110

11:36:23 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5111

11:36:23 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5112

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

5113

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5114

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5115

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5116

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5117

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 57, GPU 2608 (MiB)

5118

11:36:23 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5119

11:36:24 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +743, GPU +0, now: CPU 21943, GPU 4194 (MiB)

5120

11:36:24 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5121

11:36:24 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5122

11:36:25 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5123

11:36:25 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 13 inputs and 7 output network tensors.

5124

11:36:25 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

5125

11:36:25 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5126

11:36:25 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5127

11:36:25 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

5128

11:36:25 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.3611ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

5129

11:36:25 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

5130

11:36:25 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

5131

11:36:25 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5132

11:36:25 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.776116 seconds.

5133

11:36:25 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 57 MiB, GPU 2629 MiB

5134

11:36:25 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5135

11:36:25 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5136

11:36:25 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5137

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

5138

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5139

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5140

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5141

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5142

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 57, GPU 2622 (MiB)

5143

11:36:25 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5144

11:36:26 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +750, GPU +2, now: CPU 21949, GPU 4210 (MiB)

5145

11:36:26 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5146

11:36:27 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5147

11:36:27 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 12 inputs and 6 output network tensors.

5148

11:36:27 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 202064 bytes

5149

11:36:27 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5150

11:36:27 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5151

11:36:27 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 46 steps to complete.

5152

11:36:27 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.2705ms to assign 6 blocks to 46 nodes requiring 10486272 bytes.

5153

11:36:27 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 10485760 bytes

5154

11:36:27 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 3402944 bytes

5155

11:36:27 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5156

11:36:27 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 0.777158 seconds.

5157

11:36:27 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 58 MiB, GPU 2642 MiB

5158

11:36:27 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5159

11:36:27 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5160

11:36:27 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5161

INFO: [Torch-TensorRT] - Loaded engine size: 4 MiB

5162

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5163

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5164

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5165

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5166

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +10, now: CPU 58, GPU 2635 (MiB)

5167

11:36:27 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5168

11:36:28 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +689, GPU +2, now: CPU 21962, GPU 4228 (MiB)

5169

11:36:28 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5170

11:36:29 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5171

11:36:29 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 10 inputs and 6 output network tensors.

5172

11:36:30 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 465008 bytes

5173

11:36:30 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5174

11:36:30 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5175

11:36:30 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 108 steps to complete.

5176

11:36:30 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 1.3674ms to assign 7 blocks to 108 nodes requiring 22052864 bytes.

5177

11:36:30 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5178

11:36:30 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 10556224 bytes

5179

11:36:30 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5180

11:36:30 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.71404 seconds.

5181

11:36:30 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 62 MiB, GPU 2654 MiB

5182

11:36:30 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5183

11:36:30 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5184

11:36:30 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5185

INFO: [Torch-TensorRT] - Loaded engine size: 11 MiB

5186

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5187

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5188

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5189

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5190

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 58, GPU 2666 (MiB)

5191

11:36:31 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5192

11:36:31 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +732, GPU +2, now: CPU 21993, GPU 4262 (MiB)

5193

11:36:31 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5194

11:36:32 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5195

11:36:32 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5196

11:36:32 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5197

11:36:33 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5198

11:36:33 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5199

11:36:33 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5200

11:36:33 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5201

11:36:33 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 1.036ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5202

11:36:33 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5203

11:36:33 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5204

11:36:33 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5205

11:36:33 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.17449 seconds.

5206

11:36:33 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 62 MiB, GPU 2685 MiB

5207

11:36:33 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5208

11:36:33 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5209

11:36:33 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5210

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5211

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5212

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5213

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5214

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5215

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +21, now: CPU 59, GPU 2692 (MiB)

5216

11:36:33 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5217

11:36:34 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +665, GPU +0, now: CPU 22001, GPU 4290 (MiB)

5218

11:36:34 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5219

11:36:34 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5220

11:36:35 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5221

11:36:35 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5222

11:36:35 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5223

11:36:35 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5224

11:36:35 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5225

11:36:35 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5226

11:36:35 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9036ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5227

11:36:35 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5228

11:36:35 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5229

11:36:35 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5230

11:36:35 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.1844 seconds.

5231

11:36:35 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 62 MiB, GPU 2711 MiB

5232

11:36:35 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5233

11:36:35 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5234

11:36:35 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5235

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5236

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5237

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5238

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5239

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5240

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 59, GPU 2718 (MiB)

5241

11:36:36 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5242

11:36:36 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +752, GPU +2, now: CPU 22020, GPU 4320 (MiB)

5243

11:36:37 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5244

11:36:37 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5245

11:36:37 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5246

11:36:37 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5247

11:36:38 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5248

11:36:38 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5249

11:36:38 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5250

11:36:38 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5251

11:36:38 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 1.0064ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5252

11:36:38 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5253

11:36:38 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5254

11:36:38 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5255

11:36:38 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.18526 seconds.

5256

11:36:38 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 62 MiB, GPU 2737 MiB

5257

11:36:38 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5258

11:36:38 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5259

11:36:38 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5260

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5261

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5262

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5263

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5264

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5265

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +21, now: CPU 60, GPU 2744 (MiB)

5266

11:36:38 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5267

11:36:39 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +690, GPU +2, now: CPU 22047, GPU 4348 (MiB)

5268

11:36:39 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5269

11:36:39 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5270

11:36:40 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5271

11:36:40 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5272

11:36:40 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5273

11:36:40 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5274

11:36:40 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5275

11:36:40 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5276

11:36:40 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.8431ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5277

11:36:40 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5278

11:36:40 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5279

11:36:40 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5280

11:36:40 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.1751 seconds.

5281

11:36:40 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 62 MiB, GPU 2763 MiB

5282

11:36:40 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5283

11:36:40 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5284

11:36:40 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5285

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5286

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5287

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5288

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5289

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5290

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 60, GPU 2770 (MiB)

5291

11:36:41 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5292

11:36:42 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +746, GPU +0, now: CPU 22058, GPU 4376 (MiB)

5293

11:36:42 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5294

11:36:42 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5295

11:36:42 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5296

11:36:42 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5297

11:36:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5298

11:36:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5299

11:36:43 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5300

11:36:43 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5301

11:36:43 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9498ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5302

11:36:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5303

11:36:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5304

11:36:43 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5305

11:36:43 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.20883 seconds.

5306

11:36:43 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 62 MiB, GPU 2789 MiB

5307

11:36:43 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5308

11:36:43 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5309

11:36:43 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5310

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5311

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5312

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5313

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5314

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5315

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +21, now: CPU 61, GPU 2796 (MiB)

5316

11:36:43 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5317

11:36:44 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +635, GPU +0, now: CPU 22074, GPU 4404 (MiB)

5318

11:36:44 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5319

11:36:44 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5320

11:36:45 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5321

11:36:45 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5322

11:36:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5323

11:36:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5324

11:36:45 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5325

11:36:45 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5326

11:36:45 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9086ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5327

11:36:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5328

11:36:45 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5329

11:36:45 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5330

11:36:45 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.17568 seconds.

5331

11:36:45 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 62 MiB, GPU 2815 MiB

5332

11:36:45 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5333

11:36:46 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5334

11:36:46 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5335

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5336

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5337

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5338

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5339

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5340

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 61, GPU 2822 (MiB)

5341

11:36:46 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5342

11:36:47 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +734, GPU +2, now: CPU 22072, GPU 4434 (MiB)

5343

11:36:47 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5344

11:36:47 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5345

11:36:47 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5346

11:36:47 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5347

11:36:48 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5348

11:36:48 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5349

11:36:48 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5350

11:36:48 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5351

11:36:48 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.957ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5352

11:36:48 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5353

11:36:48 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5354

11:36:48 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5355

11:36:48 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.19627 seconds.

5356

11:36:48 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 62 MiB, GPU 2841 MiB

5357

11:36:48 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5358

11:36:48 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5359

11:36:48 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5360

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5361

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5362

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5363

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5364

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5365

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 61, GPU 2848 (MiB)

5366

11:36:48 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5367

11:36:49 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +671, GPU +2, now: CPU 22091, GPU 4462 (MiB)

5368

11:36:49 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5369

11:36:50 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5370

11:36:50 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5371

11:36:50 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5372

11:36:50 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5373

11:36:50 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5374

11:36:50 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5375

11:36:50 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5376

11:36:50 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.997ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5377

11:36:50 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5378

11:36:50 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5379

11:36:50 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5380

11:36:50 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.15277 seconds.

5381

11:36:50 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 62 MiB, GPU 2867 MiB

5382

11:36:51 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5383

11:36:51 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5384

11:36:51 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5385

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5386

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5387

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5388

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5389

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5390

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 62, GPU 2874 (MiB)

5391

11:36:51 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5392

11:36:52 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +699, GPU +0, now: CPU 22096, GPU 4490 (MiB)

5393

11:36:52 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5394

11:36:52 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5395

11:36:52 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5396

11:36:52 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5397

11:36:53 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5398

11:36:53 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5399

11:36:53 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5400

11:36:53 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5401

11:36:53 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 1.004ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5402

11:36:53 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5403

11:36:53 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5404

11:36:53 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5405

11:36:53 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.1706 seconds.

5406

11:36:53 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 62 MiB, GPU 2893 MiB

5407

11:36:53 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5408

11:36:53 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5409

11:36:53 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5410

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5411

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5412

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5413

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5414

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5415

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 62, GPU 2900 (MiB)

5416

11:36:53 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5417

11:36:54 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +765, GPU +2, now: CPU 22137, GPU 4520 (MiB)

5418

11:36:54 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5419

11:36:55 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5420

11:36:55 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5421

11:36:55 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5422

11:36:56 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5423

11:36:56 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5424

11:36:56 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5425

11:36:56 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5426

11:36:56 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9831ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5427

11:36:56 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5428

11:36:56 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5429

11:36:56 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5430

11:36:56 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.21686 seconds.

5431

11:36:56 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 63 MiB, GPU 2919 MiB

5432

11:36:56 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5433

11:36:56 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5434

11:36:56 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5435

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5436

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5437

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5438

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5439

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5440

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 63, GPU 2926 (MiB)

5441

11:36:56 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5442

11:36:57 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +742, GPU +2, now: CPU 22134, GPU 4548 (MiB)

5443

11:36:57 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5444

11:36:57 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5445

11:36:58 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5446

11:36:58 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5447

11:36:58 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5448

11:36:58 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5449

11:36:58 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5450

11:36:58 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5451

11:36:58 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.876ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5452

11:36:58 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5453

11:36:58 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5454

11:36:58 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5455

11:36:58 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.17543 seconds.

5456

11:36:58 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 63 MiB, GPU 2945 MiB

5457

11:36:58 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5458

11:36:58 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5459

11:36:58 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5460

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5461

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5462

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5463

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5464

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5465

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +22, now: CPU 63, GPU 2953 (MiB)

5466

11:36:59 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5467

11:36:59 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +749, GPU +0, now: CPU 22145, GPU 4576 (MiB)

5468

11:37:00 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5469

11:37:00 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5470

11:37:00 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5471

11:37:00 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5472

11:37:01 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5473

11:37:01 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5474

11:37:01 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5475

11:37:01 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5476

11:37:01 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9271ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5477

11:37:01 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5478

11:37:01 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5479

11:37:01 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5480

11:37:01 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.19795 seconds.

5481

11:37:01 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 64 MiB, GPU 2972 MiB

5482

11:37:01 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5483

11:37:01 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5484

11:37:01 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5485

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5486

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5487

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5488

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5489

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5490

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +21, now: CPU 64, GPU 2979 (MiB)

5491

11:37:01 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5492

11:37:02 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +658, GPU +0, now: CPU 22157, GPU 4604 (MiB)

5493

11:37:02 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5494

11:37:02 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5495

11:37:03 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5496

11:37:03 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5497

11:37:03 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5498

11:37:03 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5499

11:37:03 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5500

11:37:03 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5501

11:37:03 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 1.1465ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5502

11:37:03 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5503

11:37:03 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5504

11:37:03 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5505

11:37:03 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.24391 seconds.

5506

11:37:03 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 64 MiB, GPU 2998 MiB

5507

11:37:03 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5508

11:37:04 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5509

11:37:04 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5510

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5511

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5512

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5513

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5514

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5515

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 64, GPU 3005 (MiB)

5516

11:37:04 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5517

11:37:05 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +757, GPU +2, now: CPU 22165, GPU 4634 (MiB)

5518

11:37:05 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5519

11:37:05 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5520

11:37:05 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5521

11:37:05 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5522

11:37:06 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5523

11:37:06 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5524

11:37:06 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5525

11:37:06 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5526

11:37:06 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9022ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5527

11:37:06 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5528

11:37:06 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5529

11:37:06 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5530

11:37:06 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.24343 seconds.

5531

11:37:06 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 65 MiB, GPU 3024 MiB

5532

11:37:06 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5533

11:37:06 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5534

11:37:06 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5535

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5536

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5537

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5538

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5539

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5540

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +21, now: CPU 65, GPU 3031 (MiB)

5541

11:37:06 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5542

11:37:07 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +692, GPU +2, now: CPU 22188, GPU 4662 (MiB)

5543

11:37:07 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5544

11:37:08 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5545

11:37:08 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5546

11:37:08 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5547

11:37:08 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5548

11:37:08 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5549

11:37:08 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5550

11:37:08 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5551

11:37:08 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9465ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5552

11:37:08 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5553

11:37:09 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5554

11:37:09 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5555

11:37:09 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.19247 seconds.

5556

11:37:09 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 65 MiB, GPU 3050 MiB

5557

11:37:09 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5558

11:37:09 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5559

11:37:09 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5560

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5561

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5562

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5563

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5564

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5565

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 65, GPU 3057 (MiB)

5566

11:37:09 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5567

11:37:10 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +712, GPU +0, now: CPU 22196, GPU 4692 (MiB)

5568

11:37:10 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5569

11:37:10 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5570

11:37:11 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5571

11:37:11 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5572

11:37:11 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5573

11:37:11 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5574

11:37:11 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5575

11:37:11 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5576

11:37:11 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 1.1846ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5577

11:37:11 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5578

11:37:11 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5579

11:37:11 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5580

11:37:11 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.17277 seconds.

5581

11:37:11 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 66 MiB, GPU 3076 MiB

5582

11:37:11 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5583

11:37:11 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5584

11:37:11 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5585

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5586

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5587

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5588

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5589

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5590

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +21, now: CPU 66, GPU 3083 (MiB)

5591

11:37:12 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5592

11:37:12 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +644, GPU +0, now: CPU 22211, GPU 4720 (MiB)

5593

11:37:12 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5594

11:37:13 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5595

11:37:13 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5596

11:37:13 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5597

11:37:14 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5598

11:37:14 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5599

11:37:14 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5600

11:37:14 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5601

11:37:14 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.8921ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5602

11:37:14 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5603

11:37:14 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5604

11:37:14 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5605

11:37:14 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.22135 seconds.

5606

11:37:14 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 66 MiB, GPU 3102 MiB

5607

11:37:14 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5608

11:37:14 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5609

11:37:14 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5610

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5611

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5612

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5613

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5614

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5615

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 66, GPU 3109 (MiB)

5616

11:37:14 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5617

11:37:15 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +641, GPU +2, now: CPU 22234, GPU 4750 (MiB)

5618

11:37:15 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5619

11:37:15 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5620

11:37:16 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5621

11:37:16 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5622

11:37:16 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5623

11:37:16 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5624

11:37:16 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5625

11:37:16 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5626

11:37:16 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.976ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5627

11:37:16 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5628

11:37:16 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5629

11:37:16 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5630

11:37:16 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.2075 seconds.

5631

11:37:16 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 67 MiB, GPU 3128 MiB

5632

11:37:16 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5633

11:37:17 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5634

11:37:17 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5635

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5636

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5637

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5638

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5639

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5640

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 66, GPU 3135 (MiB)

5641

11:37:17 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5642

11:37:18 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +657, GPU +2, now: CPU 22242, GPU 4778 (MiB)

5643

11:37:18 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5644

11:37:18 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5645

11:37:18 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5646

11:37:18 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5647

11:37:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5648

11:37:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5649

11:37:19 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5650

11:37:19 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5651

11:37:19 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9288ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5652

11:37:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5653

11:37:19 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5654

11:37:19 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5655

11:37:19 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.15463 seconds.

5656

11:37:19 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 67 MiB, GPU 3154 MiB

5657

11:37:19 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5658

11:37:19 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5659

11:37:19 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5660

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5661

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5662

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5663

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5664

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5665

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 67, GPU 3161 (MiB)

5666

11:37:19 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5667

11:37:20 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +683, GPU +0, now: CPU 22241, GPU 4806 (MiB)

5668

11:37:20 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5669

11:37:20 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5670

11:37:21 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5671

11:37:21 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5672

11:37:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5673

11:37:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5674

11:37:21 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5675

11:37:21 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5676

11:37:21 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 1.1027ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5677

11:37:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5678

11:37:21 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5679

11:37:21 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5680

11:37:21 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.17118 seconds.

5681

11:37:21 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 67 MiB, GPU 3180 MiB

5682

11:37:21 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5683

11:37:22 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5684

11:37:22 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5685

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5686

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5687

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5688

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5689

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5690

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 67, GPU 3187 (MiB)

5691

11:37:22 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5692

11:37:23 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +641, GPU +2, now: CPU 22253, GPU 4836 (MiB)

5693

11:37:23 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5694

11:37:23 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5695

11:37:23 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5696

11:37:23 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5697

11:37:24 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5698

11:37:24 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5699

11:37:24 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5700

11:37:24 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5701

11:37:24 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9737ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5702

11:37:24 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5703

11:37:24 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5704

11:37:24 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5705

11:37:24 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.18777 seconds.

5706

11:37:24 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 68 MiB, GPU 3206 MiB

5707

11:37:24 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5708

11:37:24 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5709

11:37:24 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5710

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5711

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5712

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5713

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5714

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5715

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 68, GPU 3213 (MiB)

5716

11:37:24 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5717

11:37:25 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +704, GPU +2, now: CPU 22269, GPU 4864 (MiB)

5718

11:37:25 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5719

11:37:26 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5720

11:37:26 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5721

11:37:26 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5722

11:37:26 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5723

11:37:26 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5724

11:37:26 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5725

11:37:26 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5726

11:37:26 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 1.2039ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5727

11:37:26 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5728

11:37:26 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5729

11:37:26 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5730

11:37:26 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.14868 seconds.

5731

11:37:26 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 68 MiB, GPU 3232 MiB

5732

11:37:27 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5733

11:37:27 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5734

11:37:27 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5735

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5736

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5737

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5738

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5739

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5740

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 68, GPU 3239 (MiB)

5741

11:37:27 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5742

11:37:28 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +738, GPU +0, now: CPU 22278, GPU 4892 (MiB)

5743

11:37:28 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5744

11:37:28 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5745

11:37:28 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5746

11:37:28 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5747

11:37:29 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5748

11:37:29 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5749

11:37:29 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5750

11:37:29 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5751

11:37:29 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9417ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5752

11:37:29 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5753

11:37:29 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5754

11:37:29 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5755

11:37:29 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.14767 seconds.

5756

11:37:29 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 69 MiB, GPU 3258 MiB

5757

11:37:29 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5758

11:37:29 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5759

11:37:29 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5760

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5761

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5762

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5763

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5764

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5765

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 69, GPU 3265 (MiB)

5766

11:37:29 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5767

11:37:30 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +660, GPU +0, now: CPU 22292, GPU 4920 (MiB)

5768

11:37:30 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5769

11:37:31 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5770

11:37:31 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5771

11:37:31 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5772

11:37:31 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5773

11:37:31 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5774

11:37:31 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5775

11:37:31 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5776

11:37:31 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9035ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5777

11:37:31 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5778

11:37:31 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5779

11:37:31 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5780

11:37:31 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.18571 seconds.

5781

11:37:31 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 69 MiB, GPU 3284 MiB

5782

11:37:32 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5783

11:37:32 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5784

11:37:32 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5785

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5786

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5787

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5788

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5789

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5790

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 69, GPU 3291 (MiB)

5791

11:37:32 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5792

11:37:33 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +642, GPU +2, now: CPU 22307, GPU 4950 (MiB)

5793

11:37:33 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5794

11:37:33 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5795

11:37:34 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5796

11:37:34 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5797

11:37:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5798

11:37:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5799

11:37:34 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5800

11:37:34 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5801

11:37:34 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9784ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5802

11:37:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5803

11:37:34 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5804

11:37:34 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5805

11:37:34 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.18256 seconds.

5806

11:37:34 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 70 MiB, GPU 3310 MiB

5807

11:37:34 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5808

11:37:34 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5809

11:37:34 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5810

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5811

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5812

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5813

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5814

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5815

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +21, now: CPU 70, GPU 3317 (MiB)

5816

11:37:34 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5817

11:37:35 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +666, GPU +2, now: CPU 22318, GPU 4978 (MiB)

5818

11:37:35 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5819

11:37:36 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5820

11:37:36 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5821

11:37:36 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5822

11:37:37 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5823

11:37:37 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5824

11:37:37 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5825

11:37:37 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5826

11:37:37 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9248ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5827

11:37:37 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5828

11:37:37 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5829

11:37:37 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5830

11:37:37 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.17373 seconds.

5831

11:37:37 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 70 MiB, GPU 3336 MiB

5832

11:37:37 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5833

11:37:37 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5834

11:37:37 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5835

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5836

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5837

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5838

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5839

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5840

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 70, GPU 3343 (MiB)

5841

11:37:37 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5842

11:37:38 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +683, GPU +0, now: CPU 22328, GPU 5006 (MiB)

5843

11:37:38 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5844

11:37:38 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5845

11:37:39 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5846

11:37:39 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5847

11:37:39 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5848

11:37:39 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5849

11:37:39 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5850

11:37:39 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5851

11:37:39 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 1.0803ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5852

11:37:39 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5853

11:37:39 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5854

11:37:39 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5855

11:37:39 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.16226 seconds.

5856

11:37:39 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 71 MiB, GPU 3362 MiB

5857

11:37:39 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5858

11:37:39 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5859

11:37:39 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5860

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5861

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5862

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5863

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5864

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5865

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +21, now: CPU 71, GPU 3369 (MiB)

5866

11:37:40 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5867

11:37:40 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +645, GPU +0, now: CPU 22338, GPU 5034 (MiB)

5868

11:37:40 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5869

11:37:41 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5870

11:37:41 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5871

11:37:41 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5872

11:37:42 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5873

11:37:42 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5874

11:37:42 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5875

11:37:42 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5876

11:37:42 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.8659ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5877

11:37:42 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5878

11:37:42 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5879

11:37:42 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5880

11:37:42 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.20222 seconds.

5881

11:37:42 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 71 MiB, GPU 3388 MiB

5882

11:37:42 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5883

11:37:42 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5884

11:37:42 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5885

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5886

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5887

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5888

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5889

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5890

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 71, GPU 3395 (MiB)

5891

11:37:42 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5892

11:37:43 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +713, GPU +2, now: CPU 22358, GPU 5064 (MiB)

5893

11:37:43 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5894

11:37:43 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5895

11:37:44 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5896

11:37:44 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5897

11:37:44 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5898

11:37:44 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5899

11:37:44 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5900

11:37:44 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5901

11:37:44 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9493ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5902

11:37:44 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5903

11:37:44 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5904

11:37:44 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5905

11:37:44 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.1768 seconds.

5906

11:37:44 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 72 MiB, GPU 3414 MiB

5907

11:37:44 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5908

11:37:44 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5909

11:37:44 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5910

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5911

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5912

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5913

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5914

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5915

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +21, now: CPU 72, GPU 3421 (MiB)

5916

11:37:45 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5917

11:37:45 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +766, GPU +2, now: CPU 22373, GPU 5092 (MiB)

5918

11:37:46 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5919

11:37:46 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5920

11:37:46 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5921

11:37:46 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5922

11:37:47 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5923

11:37:47 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5924

11:37:47 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5925

11:37:47 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5926

11:37:47 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 1.0537ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5927

11:37:47 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5928

11:37:47 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5929

11:37:47 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5930

11:37:47 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.15499 seconds.

5931

11:37:47 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 72 MiB, GPU 3440 MiB

5932

11:37:47 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5933

11:37:47 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5934

11:37:47 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5935

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5936

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5937

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5938

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5939

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5940

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 72, GPU 3448 (MiB)

5941

11:37:47 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5942

11:37:48 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +670, GPU +0, now: CPU 22375, GPU 5120 (MiB)

5943

11:37:48 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5944

11:37:48 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5945

11:37:49 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5946

11:37:49 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5947

11:37:49 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5948

11:37:49 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5949

11:37:49 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5950

11:37:49 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5951

11:37:49 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.8779ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5952

11:37:49 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5953

11:37:49 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5954

11:37:49 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5955

11:37:49 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.2353 seconds.

5956

11:37:49 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 72 MiB, GPU 3467 MiB

5957

11:37:49 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5958

11:37:50 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5959

11:37:50 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5960

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5961

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5962

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5963

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5964

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5965

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 72, GPU 3474 (MiB)

5966

11:37:50 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5967

11:37:51 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +653, GPU +2, now: CPU 22387, GPU 5154 (MiB)

5968

11:37:51 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5969

11:37:51 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5970

11:37:51 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5971

11:37:51 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5972

11:37:52 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5973

11:37:52 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5974

11:37:52 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

5975

11:37:52 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

5976

11:37:52 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.8734ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

5977

11:37:52 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

5978

11:37:52 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

5979

11:37:52 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

5980

11:37:52 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.18723 seconds.

5981

11:37:52 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 73 MiB, GPU 3493 MiB

5982

11:37:52 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

5983

11:37:52 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

5984

11:37:52 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

5985

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

5986

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

5987

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

5988

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

5989

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

5990

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 73, GPU 3500 (MiB)

5991

11:37:52 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

5992

11:37:53 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +726, GPU +2, now: CPU 22454, GPU 5182 (MiB)

5993

11:37:53 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

5994

11:37:53 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

5995

11:37:54 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

5996

11:37:54 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

5997

11:37:54 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

5998

11:37:54 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

5999

11:37:54 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

6000

11:37:54 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

6001

11:37:54 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9913ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

6002

11:37:54 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

6003

11:37:54 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

6004

11:37:54 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

6005

11:37:54 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.16901 seconds.

6006

11:37:54 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 73 MiB, GPU 3519 MiB

6007

11:37:54 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

6008

11:37:55 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

6009

11:37:55 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

6010

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

6011

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

6012

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

6013

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

6014

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

6015

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 73, GPU 3526 (MiB)

6016

11:37:55 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

6017

11:37:56 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +696, GPU +2, now: CPU 22437, GPU 5212 (MiB)

6018

11:37:56 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

6019

11:37:56 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

6020

11:37:56 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

6021

11:37:56 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

6022

11:37:57 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

6023

11:37:57 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

6024

11:37:57 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

6025

11:37:57 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

6026

11:37:57 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9282ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

6027

11:37:57 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

6028

11:37:57 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

6029

11:37:57 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

6030

11:37:57 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.16349 seconds.

6031

11:37:57 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 74 MiB, GPU 3545 MiB

6032

11:37:57 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

6033

11:37:57 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

6034

11:37:57 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

6035

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

6036

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

6037

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

6038

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

6039

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

6040

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 74, GPU 3552 (MiB)

6041

11:37:57 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

6042

11:37:58 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +669, GPU +2, now: CPU 22467, GPU 5240 (MiB)

6043

11:37:58 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

6044

11:37:59 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

6045

11:37:59 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

6046

11:37:59 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

6047

11:37:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

6048

11:37:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

6049

11:37:59 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

6050

11:37:59 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

6051

11:37:59 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9102ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

6052

11:37:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

6053

11:37:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

6054

11:37:59 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

6055

11:37:59 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.18309 seconds.

6056

11:37:59 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 74 MiB, GPU 3571 MiB

6057

11:38:00 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

6058

11:38:00 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

6059

11:38:00 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

6060

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

6061

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

6062

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

6063

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

6064

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

6065

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 74, GPU 3578 (MiB)

6066

11:38:00 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

6067

11:38:01 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +654, GPU +2, now: CPU 22451, GPU 5270 (MiB)

6068

11:38:01 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

6069

11:38:01 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

6070

11:38:01 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

6071

11:38:01 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

6072

11:38:02 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

6073

11:38:02 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

6074

11:38:02 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

6075

11:38:02 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

6076

11:38:02 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 1.0152ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

6077

11:38:02 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

6078

11:38:02 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

6079

11:38:02 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

6080

11:38:02 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.17419 seconds.

6081

11:38:02 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 75 MiB, GPU 3597 MiB

6082

11:38:02 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

6083

11:38:02 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

6084

11:38:02 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

6085

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

6086

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

6087

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

6088

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

6089

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

6090

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +21, now: CPU 75, GPU 3604 (MiB)

6091

11:38:02 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

6092

11:38:03 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +654, GPU +2, now: CPU 22479, GPU 5298 (MiB)

6093

11:38:03 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

6094

11:38:04 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

6095

11:38:04 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

6096

11:38:04 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

6097

11:38:05 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

6098

11:38:05 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

6099

11:38:05 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

6100

11:38:05 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

6101

11:38:05 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9839ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

6102

11:38:05 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

6103

11:38:05 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

6104

11:38:05 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

6105

11:38:05 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.15909 seconds.

6106

11:38:05 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 75 MiB, GPU 3623 MiB

6107

11:38:05 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

6108

11:38:05 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

6109

11:38:05 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

6110

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

6111

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

6112

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

6113

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

6114

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

6115

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 75, GPU 3630 (MiB)

6116

11:38:05 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

6117

11:38:06 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +653, GPU +0, now: CPU 22467, GPU 5326 (MiB)

6118

11:38:06 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

6119

11:38:06 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

6120

11:38:07 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

6121

11:38:07 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

6122

11:38:07 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

6123

11:38:07 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

6124

11:38:07 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

6125

11:38:07 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

6126

11:38:07 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9677ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

6127

11:38:07 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

6128

11:38:07 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

6129

11:38:07 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

6130

11:38:07 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.18144 seconds.

6131

11:38:07 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 76 MiB, GPU 3649 MiB

6132

11:38:07 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

6133

11:38:07 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

6134

11:38:07 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

6135

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

6136

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

6137

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

6138

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

6139

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

6140

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +21, now: CPU 76, GPU 3656 (MiB)

6141

11:38:08 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

6142

11:38:08 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +657, GPU +2, now: CPU 22482, GPU 5356 (MiB)

6143

11:38:08 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

6144

11:38:09 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

6145

11:38:09 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

6146

11:38:09 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

6147

11:38:10 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

6148

11:38:10 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

6149

11:38:10 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

6150

11:38:10 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

6151

11:38:10 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 1.0062ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

6152

11:38:10 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

6153

11:38:10 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

6154

11:38:10 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

6155

11:38:10 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.18165 seconds.

6156

11:38:10 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 76 MiB, GPU 3675 MiB

6157

11:38:10 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

6158

11:38:10 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

6159

11:38:10 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

6160

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

6161

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

6162

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

6163

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

6164

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

6165

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 76, GPU 3682 (MiB)

6166

11:38:10 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

6167

11:38:11 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +726, GPU +2, now: CPU 22526, GPU 5384 (MiB)

6168

11:38:11 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

6169

11:38:11 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

6170

11:38:12 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

6171

11:38:12 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

6172

11:38:12 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

6173

11:38:12 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

6174

11:38:12 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

6175

11:38:12 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

6176

11:38:12 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.8918ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

6177

11:38:12 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

6178

11:38:12 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

6179

11:38:12 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

6180

11:38:12 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.17225 seconds.

6181

11:38:12 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 77 MiB, GPU 3701 MiB

6182

11:38:12 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

6183

11:38:12 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

6184

11:38:12 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

6185

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

6186

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

6187

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

6188

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

6189

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

6190

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +21, now: CPU 77, GPU 3708 (MiB)

6191

11:38:13 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

6192

11:38:13 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +737, GPU +0, now: CPU 22512, GPU 5412 (MiB)

6193

11:38:14 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

6194

11:38:14 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

6195

11:38:14 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

6196

11:38:14 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

6197

11:38:15 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

6198

11:38:15 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

6199

11:38:15 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

6200

11:38:15 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

6201

11:38:15 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9287ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

6202

11:38:15 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

6203

11:38:15 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

6204

11:38:15 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

6205

11:38:15 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.1435 seconds.

6206

11:38:15 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 77 MiB, GPU 3727 MiB

6207

11:38:15 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

6208

11:38:15 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

6209

11:38:15 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

6210

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

6211

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

6212

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

6213

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

6214

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

6215

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 77, GPU 3734 (MiB)

6216

11:38:15 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

6217

11:38:16 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +675, GPU +0, now: CPU 22522, GPU 5440 (MiB)

6218

11:38:16 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

6219

11:38:16 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

6220

11:38:17 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

6221

11:38:17 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

6222

11:38:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

6223

11:38:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

6224

11:38:17 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

6225

11:38:17 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

6226

11:38:17 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9564ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

6227

11:38:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

6228

11:38:17 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

6229

11:38:17 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

6230

11:38:17 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.18917 seconds.

6231

11:38:17 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 77 MiB, GPU 3753 MiB

6232

11:38:17 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

6233

11:38:18 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

6234

11:38:18 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

6235

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

6236

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

6237

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

6238

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

6239

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

6240

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 77, GPU 3760 (MiB)

6241

11:38:18 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

6242

11:38:19 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +702, GPU +2, now: CPU 22523, GPU 5470 (MiB)

6243

11:38:19 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

6244

11:38:19 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

6245

11:38:19 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

6246

11:38:19 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

6247

11:38:20 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

6248

11:38:20 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

6249

11:38:20 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

6250

11:38:20 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

6251

11:38:20 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9518ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

6252

11:38:20 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

6253

11:38:20 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

6254

11:38:20 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

6255

11:38:20 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.19079 seconds.

6256

11:38:20 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 78 MiB, GPU 3779 MiB

6257

11:38:20 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

6258

11:38:20 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

6259

11:38:20 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

6260

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

6261

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

6262

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

6263

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

6264

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

6265

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 78, GPU 3786 (MiB)

6266

11:38:20 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

6267

11:38:21 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +656, GPU +2, now: CPU 22540, GPU 5498 (MiB)

6268

11:38:21 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

6269

11:38:21 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

6270

11:38:22 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

6271

11:38:22 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

6272

11:38:22 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

6273

11:38:22 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

6274

11:38:22 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

6275

11:38:22 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

6276

11:38:22 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.966ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

6277

11:38:22 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

6278

11:38:22 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

6279

11:38:22 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

6280

11:38:22 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.17571 seconds.

6281

11:38:22 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 78 MiB, GPU 3805 MiB

6282

11:38:23 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

6283

11:38:23 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

6284

11:38:23 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

6285

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

6286

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

6287

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

6288

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

6289

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

6290

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 78, GPU 3812 (MiB)

6291

11:38:23 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

6292

11:38:24 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +702, GPU +0, now: CPU 22555, GPU 5526 (MiB)

6293

11:38:24 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

6294

11:38:24 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

6295

11:38:24 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

6296

11:38:24 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

6297

11:38:25 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

6298

11:38:25 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

6299

11:38:25 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

6300

11:38:25 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

6301

11:38:25 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9176ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

6302

11:38:25 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

6303

11:38:25 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

6304

11:38:25 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

6305

11:38:25 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.18119 seconds.

6306

11:38:25 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 79 MiB, GPU 3831 MiB

6307

11:38:25 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

6308

11:38:25 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

6309

11:38:25 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

6310

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

6311

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

6312

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

6313

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

6314

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

6315

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 79, GPU 3838 (MiB)

6316

11:38:25 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

6317

11:38:26 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +679, GPU +0, now: CPU 22573, GPU 5554 (MiB)

6318

11:38:26 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

6319

11:38:27 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

6320

11:38:27 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

6321

11:38:27 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

6322

11:38:28 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

6323

11:38:28 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

6324

11:38:28 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

6325

11:38:28 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

6326

11:38:28 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 1.2195ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

6327

11:38:28 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

6328

11:38:28 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

6329

11:38:28 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

6330

11:38:28 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.22356 seconds.

6331

11:38:28 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 79 MiB, GPU 3857 MiB

6332

11:38:28 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

6333

11:38:28 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

6334

11:38:28 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

6335

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

6336

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

6337

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

6338

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

6339

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

6340

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 79, GPU 3864 (MiB)

6341

11:38:28 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

6342

11:38:29 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +716, GPU +2, now: CPU 22598, GPU 5584 (MiB)

6343

11:38:29 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

6344

11:38:29 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

6345

11:38:30 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

6346

11:38:30 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

6347

11:38:30 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

6348

11:38:30 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

6349

11:38:30 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

6350

11:38:30 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

6351

11:38:30 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9023ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

6352

11:38:30 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

6353

11:38:30 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

6354

11:38:30 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

6355

11:38:30 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.18565 seconds.

6356

11:38:30 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 80 MiB, GPU 3883 MiB

6357

11:38:30 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

6358

11:38:30 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

6359

11:38:30 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

6360

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

6361

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

6362

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

6363

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

6364

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

6365

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +21, now: CPU 80, GPU 3890 (MiB)

6366

11:38:31 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

6367

11:38:31 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +634, GPU +2, now: CPU 22626, GPU 5614 (MiB)

6368

11:38:32 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

6369

11:38:32 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

6370

11:38:32 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

6371

11:38:32 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

6372

11:38:33 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

6373

11:38:33 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

6374

11:38:33 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

6375

11:38:33 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

6376

11:38:33 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9358ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

6377

11:38:33 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

6378

11:38:33 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

6379

11:38:33 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

6380

11:38:33 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.16656 seconds.

6381

11:38:33 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 80 MiB, GPU 3909 MiB

6382

11:38:33 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

6383

11:38:33 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

6384

11:38:33 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

6385

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

6386

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

6387

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

6388

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

6389

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

6390

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +22, now: CPU 80, GPU 3917 (MiB)

6391

11:38:33 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

6392

11:38:34 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +681, GPU +0, now: CPU 22616, GPU 5642 (MiB)

6393

11:38:34 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

6394

11:38:34 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

6395

11:38:35 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

6396

11:38:35 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

6397

11:38:35 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

6398

11:38:35 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

6399

11:38:35 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

6400

11:38:35 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

6401

11:38:35 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 1.1982ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

6402

11:38:35 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

6403

11:38:35 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

6404

11:38:35 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

6405

11:38:35 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.21197 seconds.

6406

11:38:35 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 81 MiB, GPU 3936 MiB

6407

11:38:35 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

6408

11:38:36 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

6409

11:38:36 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

6410

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

6411

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

6412

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

6413

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

6414

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

6415

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +21, now: CPU 81, GPU 3943 (MiB)

6416

11:38:36 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

6417

11:38:37 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +680, GPU +2, now: CPU 22624, GPU 5672 (MiB)

6418

11:38:37 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

6419

11:38:37 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

6420

11:38:37 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

6421

11:38:37 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

6422

11:38:38 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

6423

11:38:38 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

6424

11:38:38 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

6425

11:38:38 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

6426

11:38:38 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9712ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

6427

11:38:38 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

6428

11:38:38 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

6429

11:38:38 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

6430

11:38:38 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.21286 seconds.

6431

11:38:38 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 81 MiB, GPU 3962 MiB

6432

11:38:38 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

6433

11:38:38 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

6434

11:38:38 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

6435

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

6436

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

6437

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

6438

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

6439

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

6440

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 81, GPU 3969 (MiB)

6441

11:38:38 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

6442

11:38:39 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +669, GPU +2, now: CPU 22637, GPU 5700 (MiB)

6443

11:38:39 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

6444

11:38:40 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

6445

11:38:40 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

6446

11:38:40 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

6447

11:38:41 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

6448

11:38:41 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

6449

11:38:41 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

6450

11:38:41 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

6451

11:38:41 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.891ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

6452

11:38:41 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

6453

11:38:41 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

6454

11:38:41 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

6455

11:38:41 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.16772 seconds.

6456

11:38:41 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 82 MiB, GPU 3988 MiB

6457

11:38:41 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

6458

11:38:41 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

6459

11:38:41 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

6460

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

6461

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

6462

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

6463

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

6464

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

6465

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +21, now: CPU 82, GPU 3995 (MiB)

6466

11:38:41 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

6467

11:38:42 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +712, GPU +0, now: CPU 22650, GPU 5728 (MiB)

6468

11:38:42 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

6469

11:38:42 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

6470

11:38:43 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

6471

11:38:43 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

6472

11:38:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

6473

11:38:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

6474

11:38:43 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

6475

11:38:43 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

6476

11:38:43 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 1.1571ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

6477

11:38:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

6478

11:38:43 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

6479

11:38:43 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

6480

11:38:43 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.21077 seconds.

6481

11:38:43 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 82 MiB, GPU 4014 MiB

6482

11:38:43 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

6483

11:38:43 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

6484

11:38:43 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

6485

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

6486

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

6487

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

6488

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

6489

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

6490

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 82, GPU 4021 (MiB)

6491

11:38:44 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

6492

11:38:45 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +686, GPU +0, now: CPU 22671, GPU 5756 (MiB)

6493

11:38:45 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

6494

11:38:45 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

6495

11:38:45 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

6496

11:38:45 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

6497

11:38:46 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

6498

11:38:46 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

6499

11:38:46 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

6500

11:38:46 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

6501

11:38:46 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.8734ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

6502

11:38:46 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

6503

11:38:46 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

6504

11:38:46 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

6505

11:38:46 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.23482 seconds.

6506

11:38:46 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 82 MiB, GPU 4040 MiB

6507

11:38:46 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

6508

11:38:46 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

6509

11:38:46 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

6510

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

6511

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

6512

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

6513

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

6514

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

6515

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 82, GPU 4047 (MiB)

6516

11:38:46 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

6517

11:38:47 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +708, GPU +2, now: CPU 22682, GPU 5786 (MiB)

6518

11:38:47 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

6519

11:38:48 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

6520

11:38:48 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

6521

11:38:48 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

6522

11:38:49 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

6523

11:38:49 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

6524

11:38:49 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

6525

11:38:49 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

6526

11:38:49 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.855ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

6527

11:38:49 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

6528

11:38:49 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

6529

11:38:49 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

6530

11:38:49 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.19669 seconds.

6531

11:38:49 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 83 MiB, GPU 4066 MiB

6532

11:38:49 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

6533

11:38:49 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

6534

11:38:49 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

6535

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

6536

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

6537

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

6538

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

6539

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

6540

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 83, GPU 4073 (MiB)

6541

11:38:49 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

6542

11:38:50 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +641, GPU +2, now: CPU 22709, GPU 5814 (MiB)

6543

11:38:50 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

6544

11:38:50 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

6545

11:38:51 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

6546

11:38:51 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

6547

11:38:51 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

6548

11:38:51 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

6549

11:38:51 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

6550

11:38:51 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

6551

11:38:51 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 1.068ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

6552

11:38:51 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

6553

11:38:51 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

6554

11:38:51 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

6555

11:38:51 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.17987 seconds.

6556

11:38:51 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 83 MiB, GPU 4092 MiB

6557

11:38:51 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

6558

11:38:51 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

6559

11:38:51 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

6560

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

6561

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

6562

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

6563

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

6564

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

6565

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 83, GPU 4099 (MiB)

6566

11:38:52 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

6567

11:38:53 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +675, GPU +0, now: CPU 22730, GPU 5842 (MiB)

6568

11:38:53 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

6569

11:38:53 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

6570

11:38:53 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

6571

11:38:53 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

6572

11:38:54 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

6573

11:38:54 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

6574

11:38:54 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

6575

11:38:54 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

6576

11:38:54 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9106ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

6577

11:38:54 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

6578

11:38:54 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

6579

11:38:54 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

6580

11:38:54 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.17217 seconds.

6581

11:38:54 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 84 MiB, GPU 4118 MiB

6582

11:38:54 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

6583

11:38:54 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

6584

11:38:54 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

6585

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

6586

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

6587

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

6588

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

6589

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

6590

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 84, GPU 4125 (MiB)

6591

11:38:54 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

6592

11:38:55 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +671, GPU +0, now: CPU 22728, GPU 5870 (MiB)

6593

11:38:55 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

6594

11:38:56 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

6595

11:38:56 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

6596

11:38:56 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 15 inputs and 6 output network tensors.

6597

11:38:56 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

6598

11:38:56 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

6599

11:38:56 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

6600

11:38:56 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 76 steps to complete.

6601

11:38:56 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.9462ms to assign 8 blocks to 76 nodes requiring 22053376 bytes.

6602

11:38:56 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

6603

11:38:56 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

6604

11:38:56 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

6605

11:38:56 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.15496 seconds.

6606

11:38:56 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 84 MiB, GPU 4144 MiB

6607

11:38:57 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

6608

11:38:57 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

6609

11:38:57 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

6610

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

6611

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

6612

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

6613

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

6614

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

6615

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 84, GPU 4151 (MiB)

6616

11:38:57 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

6617

11:38:58 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +694, GPU +2, now: CPU 22764, GPU 5900 (MiB)

6618

11:38:58 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

6619

11:38:58 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine build.

6620

11:38:58 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

6621

11:38:58 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 14 inputs and 4 output network tensors.

6622

11:38:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 287424 bytes

6623

11:38:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

6624

11:38:59 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 1572864 bytes

6625

11:38:59 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 77 steps to complete.

6626

11:38:59 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 1.0881ms to assign 8 blocks to 77 nodes requiring 22053376 bytes.

6627

11:38:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 22052864 bytes

6628

11:38:59 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 5264578 bytes

6629

11:38:59 torch_tensorrt [TensorRT Conversion Context] INFO: Compiler backend is used during engine execution.

6630

11:38:59 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 1.12775 seconds.

6631

11:38:59 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 85 MiB, GPU 4169 MiB

6632

11:38:59 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

6633

11:38:59 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

6634

11:38:59 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 6746 timing cache entries

6635

INFO: [Torch-TensorRT] - Loaded engine size: 6 MiB

6636

INFO: [Torch-TensorRT] - [MS] Running engine with multi stream info

6637

INFO: [Torch-TensorRT] - [MS] Number of aux streams is 1

6638

INFO: [Torch-TensorRT] - [MS] Number of total worker streams is 2

6639

INFO: [Torch-TensorRT] - [MS] The main stream provided by execute/enqueue calls is the first worker stream

6640

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +21, now: CPU 85, GPU 4177 (MiB)

6641

11:38:59 torch_tensorrt [TensorRT Conversion Context] WARNING: WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

6642

11:39:00 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageChange] Init builder kernel library: CPU +732, GPU +2, now: CPU 22801, GPU 5928 (MiB)

6643

11:39:00 torch_tensorrt [TensorRT Conversion Context] INFO: Global timing cache in use. Profiling results in this builder pass will be stored.

6644

11:39:07 torch_tensorrt [TensorRT Conversion Context] INFO: [GraphReduction] The approximate region cut reduction algorithm is called.

6645

11:39:07 torch_tensorrt [TensorRT Conversion Context] INFO: Detected 65 inputs and 1 output network tensors.

6646

11:39:07 torch_tensorrt [TensorRT Conversion Context] INFO: Total Host Persistent Memory: 263984 bytes

6647

11:39:07 torch_tensorrt [TensorRT Conversion Context] INFO: Total Device Persistent Memory: 0 bytes

6648

11:39:07 torch_tensorrt [TensorRT Conversion Context] INFO: Max Scratch Memory: 0 bytes

6649

11:39:07 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Started assigning block shifts. This will take 70 steps to complete.

6650

11:39:07 torch_tensorrt [TensorRT Conversion Context] INFO: [BlockAssignment] Algorithm ShiftNTopDown took 0.5761ms to assign 8 blocks to 70 nodes requiring 21495808 bytes.

6651

11:39:07 torch_tensorrt [TensorRT Conversion Context] INFO: Total Activation Memory: 21495808 bytes

6652

11:39:07 torch_tensorrt [TensorRT Conversion Context] INFO: Total Weights Memory: 4384288 bytes

6653

11:39:07 torch_tensorrt [TensorRT Conversion Context] INFO: Engine generation completed in 6.54131 seconds.

6654

11:39:07 torch_tensorrt [TensorRT Conversion Context] INFO: [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 85 MiB, GPU 4750 MiB

6655

11:39:07 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 27 bytes of code generator cache.

6656

11:39:07 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 4534990 bytes of compilation cache.

6657

11:39:07 torch_tensorrt [TensorRT Conversion Context] INFO: Serialized 7408 timing cache entries

6658

INFO: [Torch-TensorRT] - Loaded engine size: 5 MiB

6659

INFO: [Torch-TensorRT] - [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +21, now: CPU 85, GPU 4202 (MiB)

6660

11:39:07 torch_tensorrt._compile WARNING: Provided model is a torch.fx.GraphModule and retrace is False, inputs or arg_inputs is not necessary during save.

6661

11:39:21 py.warnings WARNING: torch_tensorrt\dynamo\_exporter.py:396: UserWarning: Attempted to insert a get_attr Node with no underlying reference in the owning GraphModule! Call GraphModule.add_submodule to add the necessary submodule, GraphModule.add_parameter to add the necessary Parameter, or nn.Module.register_buffer to add the necessary buffer

6662

engine_node = gm.graph.get_attr(engine_name)

6663

6664

11:39:21 py.warnings WARNING: torch\export\exported_program.py:1681: UserWarning: Unable to execute the generated python source code from the graph. The graph module will no longer be directly callable, but you can still run the ExportedProgram, and if needed, you can run the graph module eagerly using torch.fx.Interpreter.

6665

warnings.warn(

6666

6667

W0126 11:39:21.575000 15968 D:\Program Files\jasna\torch\export\pt2_archive\_package.py:586] Expect archive file to be a file ending in .pt2, or is a buffer. Instead got {model_weights\lada_mosaic_restoration_model_generic_v1.2_clip60.trt_fp16.win.engine}

6668

Compiling model_weights\rfdetr-v2.onnx to model_weights\rfdetr-v2.bs4.fp16.win.engine

6669

[01/26/2026-11:40:07] [TRT] [W] WARNING The logger passed into createInferBuilder differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead.

6670

[01/26/2026-11:40:08] [TRT] [I] [MemUsageChange] Init builder kernel library: CPU +746, GPU +0, now: CPU 18995, GPU 6608 (MiB)

6671

[01/26/2026-11:40:08] [TRT] [I] ----------------------------------------------------------------

6672

[01/26/2026-11:40:08] [TRT] [I] ONNX IR version: 0.0.8

6673

[01/26/2026-11:40:08] [TRT] [I] Opset version: 17

6674

[01/26/2026-11:40:08] [TRT] [I] Producer name: pytorch

6675

[01/26/2026-11:40:08] [TRT] [I] Producer version: 2.8.0

6676

[01/26/2026-11:40:08] [TRT] [I] Domain:

6677

[01/26/2026-11:40:08] [TRT] [I] Model version: 0

6678

[01/26/2026-11:40:08] [TRT] [I] Doc string:

6679

[01/26/2026-11:40:08] [TRT] [I] ----------------------------------------------------------------

6680

[01/26/2026-11:40:08] [TRT] [I] Local timing cache in use. Profiling results in this builder pass will not be stored.

6681

[01/26/2026-11:40:10] [TRT] [I] Compiler backend is used during engine build.

6682

[01/26/2026-11:42:08] [TRT] [I] Detected 1 inputs and 3 output network tensors.

6683

[01/26/2026-11:42:09] [TRT] [I] Total Host Persistent Memory: 87456 bytes

6684

[01/26/2026-11:42:09] [TRT] [I] Total Device Persistent Memory: 0 bytes

6685

[01/26/2026-11:42:09] [TRT] [I] Max Scratch Memory: 214361088 bytes

6686

[01/26/2026-11:42:09] [TRT] [I] [BlockAssignment] Started assigning block shifts. This will take 60 steps to complete.

6687

[01/26/2026-11:42:09] [TRT] [I] [BlockAssignment] Algorithm ShiftNTopDown took 1.4255ms to assign 8 blocks to 60 nodes requiring 461825024 bytes.

6688

[01/26/2026-11:42:09] [TRT] [I] Total Activation Memory: 461825024 bytes

6689

[01/26/2026-11:42:09] [TRT] [I] Total Weights Memory: 69964928 bytes

6690

[01/26/2026-11:42:09] [TRT] [I] Compiler backend is used during engine execution.

6691

[01/26/2026-11:42:09] [TRT] [I] Engine generation completed in 120.625 seconds.

6692

[01/26/2026-11:42:09] [TRT] [I] [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 171 MiB, GPU 8404 MiB

This log will be saved for 3 months from its last view.

Report abuse