Execution of Inference Workloads on Hikey970 with layer splitting

Hello,
I am currently working on executing inference workloads on Hikey970. I am trying to split the layers of a network amongst CPU and GPU, and run the workloads to reduce inference latency. I am following the repo attached below to run the models with CPU and GPU utilization.
https://github.com/adityagupta1089/ComputeLibrary.git
Could you guys help me understand how I can split the layers of the network and assign them to CPU and GPU?
Is there any API specific for CPU and GPU in ARM-CL?
Thanks.

Hi @Shraddhaa1
The graph api in ACL is experimental and does not support that level of granularity to specify the backend for each individual layer.
You could experiment with the functions interface which lets you mix GPU and CPU kernels, please see the example: https://github.com/ARM-software/ComputeLibrary/blob/master/examples/neoncl_scale_median_gaussian.cpp
Hope this helps.

Hello Sir,
Thank you for the reply. That really helped.
Have a good day.
Sincerely,
Shraddha Dahal
On Fri, Mar 26, 2021 at 5:44 AM Pablo Marquez Tello <
@.***> wrote:

Closed #882 https://github.com/ARM-software/ComputeLibrary/issues/882.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ARM-software/ComputeLibrary/issues/882#event-4513101747,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ARRDMC2KQYG5FVP7OF54Z53TFRQQVANCNFSM4ZHEPDJQ
.

Hello Sir,
I had contacted you a few weeks ago regarding the Execution of Inference
Workloads on Hikey970 with layer splitting. Could you please help me know
if there is any function specific in ARM-CL which would let me calculate
inference time taken by each layer of a neural network?
Thank you.
Sincerely,
Shraddha Dahal
On Fri, Apr 9, 2021 at 12:06 PM Shraddha Dahal @.***>
wrote:

Hello Sir,
Thank you for the reply. That really helped.
Have a good day.
Sincerely,
Shraddha Dahal
On Fri, Mar 26, 2021 at 5:44 AM Pablo Marquez Tello <
@.***> wrote:

Closed #882 https://github.com/ARM-software/ComputeLibrary/issues/882.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ARM-software/ComputeLibrary/issues/882#event-4513101747,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ARRDMC2KQYG5FVP7OF54Z53TFRQQVANCNFSM4ZHEPDJQ
.

Hi Shraddha,
If you build ACL with the option benchmark_examples=1 you can then
run the network and use the instruments to see how much time each kernel is taking:
@.***_hikey_dev:~/tmp/pabtel01# LD_LIBRARY_PATH=$LD_LIBRARY_PATH:./acl_neon+cl_release/ ./benchmark_graph_mobilenet --instruments=OPENCL_TIMESTAMPS_MS --example_args='--target=CL'
This will produce output like below:
OpenCLTimestamps/Now OpenCL:    AVG=1121307039 ms
OpenCLTimestamps/Now Wall clock:    AVG=1623931494169279 us
OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/gemm_mm_floating_point_f32_bifrost GWS[8,3136,1] LWS[4,1,1] #2:    AVG=1121306920 ms
OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/im2col3x3_nhwc GWS[2,12544,1] #1:    AVG=1121306919 ms
OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/reshape_to_columns GWS[3,3,3] #0:    AVG=1121306918 ms
OpenCLTimestamps/[end]Conv2d_10_depthwise/depthwise+Conv2d_10_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1 GWS[512,7,7] #36:    AVG=1121306986 ms
OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t GWS[128,64,1] #40:    AVG=1121306995 ms
OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt GWS[128,40,1] #39:    AVG=1121306989 ms
OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t GWS[128,128,1] #38:    AVG=1121306988 ms
OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/reshape_to_columns GWS[512,1,1] #37:    AVG=1121306988 ms
OpenCLTimestamps/[end]Conv2d_11_depthwise/depthwise+Conv2d_11_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1 GWS[512,7,7] #41:    AVG=1121306995 ms
OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t GWS[128,64,1] #45:    AVG=1121307003 ms
OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt GWS[128,40,1] #44:    AVG=1121306998 ms
OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t GWS[128,128,1] #43:    AVG=1121306997 ms
OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/reshape_to_columns GWS[512,1,1] #42:    AVG=1121306997 ms
OpenCLTimestamps/[end]Conv2d_12_depthwise/depthwise+Conv2d_12_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc GWS[512,7,7] #46:    AVG=1121307004 ms
OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t GWS[256,16,1] #50:    AVG=1121307012 ms
OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt GWS[128,10,1] #49:    AVG=1121307008 ms
OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t GWS[256,128,1] #48:    AVG=1121307008 ms
OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/reshape_to_columns GWS[512,1,1] #47:    AVG=1121307007 ms
OpenCLTimestamps/[end]Conv2d_13_depthwise/depthwise+Conv2d_13_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1 GWS[1024,4,4] #51:    AVG=1121307012 ms
OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t GWS[256,16,1] #55:    AVG=1121307025 ms
OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt GWS[256,10,1] #54:    AVG=1121307018 ms
OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t GWS[256,256,1] #53:    AVG=1121307018 ms
OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/reshape_to_columns GWS[1024,1,1] #52:    AVG=1121307016 ms
OpenCLTimestamps/[end]Conv2d_1_depthwise/depthwise+Conv2d_1_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1 GWS[32,56,56] #3:    AVG=1121306921 ms
OpenCLTimestamps/[end]Conv2d_1_pointwise/Conv2D+Conv2d_1_pointwise/BatchNorm/gemm_mm_floating_point_f32_bifrost GWS[16,28,112] LWS[4,1,1] #5:    AVG=1121306924 ms
OpenCLTimestamps/[end]Conv2d_1_pointwise/Conv2D+Conv2d_1_pointwise/BatchNorm/reshape_to_columns GWS[32,1,1] #4:    AVG=1121306922 ms
OpenCLTimestamps/[end]Conv2d_2_depthwise/depthwise+Conv2d_2_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc GWS[64,56,56] #6:    AVG=1121306926 ms
OpenCLTimestamps/[end]Conv2d_2_pointwise/Conv2D+Conv2d_2_pointwise/BatchNorm/gemm_mm_floating_point_f32_bifrost GWS[32,14,56] LWS[4,1,1] #8:    AVG=1121306929 ms
Another alternative is to use Armnn's ExecuteNetwork to run a tfllite model and use the -e option which will make the tool output the time consumed by each kernel
For more information about ExecuteNetwork see:
https://github.com/ARM-software/armnn/tree/branches/armnn_21_02/tests/ExecuteNetwork
Hope this helps.

From: Shraddhaa1 @.>
Sent: 14 June 2021 19:33
To: ARM-software/ComputeLibrary @.>
Cc: Subscribed @.***>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
I had contacted you a few weeks ago regarding the Execution of Inference
Workloads on Hikey970 with layer splitting. Could you please help me know
if there is any function specific in ARM-CL which would let me calculate
inference time taken by each layer of a neural network?
Thank you.
Sincerely,
Shraddha Dahal
On Fri, Apr 9, 2021 at 12:06 PM Shraddha Dahal @.***>
wrote:

Hello Sir,
Thank you for the reply. That really helped.
Have a good day.
Sincerely,
Shraddha Dahal
On Fri, Mar 26, 2021 at 5:44 AM Pablo Marquez Tello <
@.***> wrote:

Closed #882 https://github.com/ARM-software/ComputeLibrary/issues/882.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ARM-software/ComputeLibrary/issues/882#event-4513101747,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ARRDMC2KQYG5FVP7OF54Z53TFRQQVANCNFSM4ZHEPDJQ
.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-860902526, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGNXISSHV4V7LSD3ZI2TQSTTSZDQXANCNFSM4ZHEPDJQ.
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

Hello Sir,
Thank you for the response. I tried the first method you mentioned in the
previous email.
In the makefile.arm file, I added benchmark_examples:=1 such as:
BUILD:=native
NEON:=1
OPENCL:=1
ARCH:=arm64-v8a
all: release
CFLAGS:=-std=c++14
benchmark_examples:=1
debug:
scons -j8 -Q arch=$(ARCH) build=$(BUILD) neon=$(NEON) opencl=$(OPENCL)
build_dir=debug debug=1 extra_cxx_flags=$(CFLAGS)
release:
scons -j8 -Q arch=$(ARCH) build=$(BUILD) neon=$(NEON) opencl=$(OPENCL)
build_dir=release debug=0 extra_cxx_flags=$(CFLAGS)
sched:
g++ -o build/release/examples/graph_temp_scheduler2.o -c
-Wno-deprecated-declarations -Wall -DARCH_ARM -Wextra -Wno-unused-parameter
-pedantic -Wdisabled-optimization -Wformat=2 -Winit-self
-Wstrict-overflow=2 -Wswitch-default -fpermissive -std=gnu++11 -Wno-vla
-Woverloaded-virtual -Wctor-dtor-privacy -Wsign-promo -Weffc++
-Wno-format-nonliteral -Wno-overlength-strings -Wno-strict-overflow
-Wlogical-op -Wnoexcept -Wstrict-null-sentinel -Wno-implicit-fallthrough
-march=armv8-a -Wno-ignored-attributes -Werror -O3 -ftree-vectorize
-std=c++14 -D_GLIBCXX_USE_NANOSLEEP -DARM_COMPUTE_CPP_SCHEDULER=1
-DARM_COMPUTE_AARCH64_V8A -DNO_DOT_IN_TOOLCHAIN -DEMBEDDED_KERNELS
-Iinclude -I. -I. examples/graph_temp_scheduler2.cpp
g++ -o build/release/examples/graph_temp_scheduler2
-Wl,--allow-shlib-undefined build/release/examples/graph_temp_scheduler2.o
build/release/utils/Utils.o build/release/utils/GraphUtils.o
build/release/utils/CommonGraphOptions.o -Lbuild/release -L. -lpthread
-larm_compute_graph -larm_compute -larm_compute_core
After that, I gave commands:
$make all
$sudo LD_LIBRARY_PATH=/home/shunya/ComputeLibrary1/build/release
./build/release/examples/graph_mobilenet --instruments=OPENCL_TIMESTAMPS_MS
--example_args='--target=CL'
but, I could not see the outputs with time taken by each kernel. The path
to library in the above command has libarm_compute.so,
libarm_compute_core.so, libarm_compute_graph.so,
libarm_compute_core-static.a, libarm_compute_graph-static.a and
libarm_compute-static.a. Could you please help me know where I am going
wrong?
Have a good day.
Sincerely,
Shraddha Dahal
On Thu, Jun 17, 2021 at 7:10 AM developer-compute @.***>
wrote:

Hi Shraddha,
If you build ACL with the option benchmark_examples=1 you can then
run the network and use the instruments to see how much time each kernel
is taking:
@.***_hikey_dev:~/tmp/pabtel01#
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:./acl_neon+cl_release/
./benchmark_graph_mobilenet --instruments=OPENCL_TIMESTAMPS_MS
--example_args='--target=CL'
This will produce output like below:
OpenCLTimestamps/Now OpenCL: AVG=1121307039 ms
OpenCLTimestamps/Now Wall clock: AVG=1623931494169279 us
OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/gemm_mm_floating_point_f32_bifrost
GWS[8,3136,1] LWS[4,1,1] #2: AVG=1121306920 ms
OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/im2col3x3_nhwc
GWS[2,12544,1] #1: AVG=1121306919 ms
OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/reshape_to_columns
GWS[3,3,3] #0: AVG=1121306918 ms
OpenCLTimestamps/[end]Conv2d_10_depthwise/depthwise+Conv2d_10_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1
GWS[512,7,7] #36: AVG=1121306986 ms
OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t
GWS[128,64,1] #40: AVG=1121306995 ms
OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt
GWS[128,40,1] #39: AVG=1121306989 ms
OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t
GWS[128,128,1] #38: AVG=1121306988 ms
OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/reshape_to_columns
GWS[512,1,1] #37: AVG=1121306988 ms
OpenCLTimestamps/[end]Conv2d_11_depthwise/depthwise+Conv2d_11_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1
GWS[512,7,7] #41: AVG=1121306995 ms
OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t
GWS[128,64,1] #45: AVG=1121307003 ms
OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt
GWS[128,40,1] #44: AVG=1121306998 ms
OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t
GWS[128,128,1] #43: AVG=1121306997 ms
OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/reshape_to_columns
GWS[512,1,1] #42: AVG=1121306997 ms
OpenCLTimestamps/[end]Conv2d_12_depthwise/depthwise+Conv2d_12_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc
GWS[512,7,7] #46: AVG=1121307004 ms
OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t
GWS[256,16,1] #50: AVG=1121307012 ms
OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt
GWS[128,10,1] #49: AVG=1121307008 ms
OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t
GWS[256,128,1] #48: AVG=1121307008 ms
OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/reshape_to_columns
GWS[512,1,1] #47: AVG=1121307007 ms
OpenCLTimestamps/[end]Conv2d_13_depthwise/depthwise+Conv2d_13_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1
GWS[1024,4,4] #51: AVG=1121307012 ms
OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t
GWS[256,16,1] #55: AVG=1121307025 ms
OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt
GWS[256,10,1] #54: AVG=1121307018 ms
OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t
GWS[256,256,1] #53: AVG=1121307018 ms
OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/reshape_to_columns
GWS[1024,1,1] #52: AVG=1121307016 ms
OpenCLTimestamps/[end]Conv2d_1_depthwise/depthwise+Conv2d_1_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1
GWS[32,56,56] #3: AVG=1121306921 ms
OpenCLTimestamps/[end]Conv2d_1_pointwise/Conv2D+Conv2d_1_pointwise/BatchNorm/gemm_mm_floating_point_f32_bifrost
GWS[16,28,112] LWS[4,1,1] #5: AVG=1121306924 ms
OpenCLTimestamps/[end]Conv2d_1_pointwise/Conv2D+Conv2d_1_pointwise/BatchNorm/reshape_to_columns
GWS[32,1,1] #4: AVG=1121306922 ms
OpenCLTimestamps/[end]Conv2d_2_depthwise/depthwise+Conv2d_2_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc
GWS[64,56,56] #6: AVG=1121306926 ms
OpenCLTimestamps/[end]Conv2d_2_pointwise/Conv2D+Conv2d_2_pointwise/BatchNorm/gemm_mm_floating_point_f32_bifrost
GWS[32,14,56] LWS[4,1,1] #8: AVG=1121306929 ms
Another alternative is to use Armnn's ExecuteNetwork to run a tfllite
model and use the -e option which will make the tool output the time
consumed by each kernel
For more information about ExecuteNetwork see:
https://github.com/ARM-software/armnn/tree/branches/armnn_21_02/tests/ExecuteNetwork
Hope this helps.

From: Shraddhaa1 @.>
Sent: 14 June 2021 19:33
To: ARM-software/ComputeLibrary @.>
Cc: Subscribed @.***>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
I had contacted you a few weeks ago regarding the Execution of Inference
Workloads on Hikey970 with layer splitting. Could you please help me know
if there is any function specific in ARM-CL which would let me calculate
inference time taken by each layer of a neural network?
Thank you.
Sincerely,
Shraddha Dahal
On Fri, Apr 9, 2021 at 12:06 PM Shraddha Dahal @.***>
wrote:

Hello Sir,
Thank you for the reply. That really helped.
Have a good day.
Sincerely,
Shraddha Dahal
On Fri, Mar 26, 2021 at 5:44 AM Pablo Marquez Tello <
@.***> wrote:

Closed #882 <https://github.com/ARM-software/ComputeLibrary/issues/882
.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<
https://github.com/ARM-software/ComputeLibrary/issues/882#event-4513101747
,
or unsubscribe
<
https://github.com/notifications/unsubscribe-auth/ARRDMC2KQYG5FVP7OF54Z53TFRQQVANCNFSM4ZHEPDJQ

.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub<
https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-860902526>,
or unsubscribe<
https://github.com/notifications/unsubscribe-auth/AGNXISSHV4V7LSD3ZI2TQSTTSZDQXANCNFSM4ZHEPDJQ

.

IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy the
information in any medium. Thank you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-863185227,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ARRDMC6MGOTC7JHJOXE7NFTTTHQ2XANCNFSM4ZHEPDJQ
.

Hi,
Please try running benchmark_graph_mobilenet instead of graph_mobilenet.
Hope this helps.

From: Shraddhaa1 @.>
Sent: 21 June 2021 21:46
To: ARM-software/ComputeLibrary @.>
Cc: developer-compute @.>; Comment @.>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
Thank you for the response. I tried the first method you mentioned in the
previous email.
In the makefile.arm file, I added benchmark_examples:=1 such as:
BUILD:=native
NEON:=1
OPENCL:=1
ARCH:=arm64-v8a
all: release
CFLAGS:=-std=c++14
benchmark_examples:=1
debug:
scons -j8 -Q arch=$(ARCH) build=$(BUILD) neon=$(NEON) opencl=$(OPENCL)
build_dir=debug debug=1 extra_cxx_flags=$(CFLAGS)
release:
scons -j8 -Q arch=$(ARCH) build=$(BUILD) neon=$(NEON) opencl=$(OPENCL)
build_dir=release debug=0 extra_cxx_flags=$(CFLAGS)
sched:
g++ -o build/release/examples/graph_temp_scheduler2.o -c
-Wno-deprecated-declarations -Wall -DARCH_ARM -Wextra -Wno-unused-parameter
-pedantic -Wdisabled-optimization -Wformat=2 -Winit-self
-Wstrict-overflow=2 -Wswitch-default -fpermissive -std=gnu++11 -Wno-vla
-Woverloaded-virtual -Wctor-dtor-privacy -Wsign-promo -Weffc++
-Wno-format-nonliteral -Wno-overlength-strings -Wno-strict-overflow
-Wlogical-op -Wnoexcept -Wstrict-null-sentinel -Wno-implicit-fallthrough
-march=armv8-a -Wno-ignored-attributes -Werror -O3 -ftree-vectorize
-std=c++14 -D_GLIBCXX_USE_NANOSLEEP -DARM_COMPUTE_CPP_SCHEDULER=1
-DARM_COMPUTE_AARCH64_V8A -DNO_DOT_IN_TOOLCHAIN -DEMBEDDED_KERNELS
-Iinclude -I. -I. examples/graph_temp_scheduler2.cpp
g++ -o build/release/examples/graph_temp_scheduler2
-Wl,--allow-shlib-undefined build/release/examples/graph_temp_scheduler2.o
build/release/utils/Utils.o build/release/utils/GraphUtils.o
build/release/utils/CommonGraphOptions.o -Lbuild/release -L. -lpthread
-larm_compute_graph -larm_compute -larm_compute_core
After that, I gave commands:
$make all
$sudo LD_LIBRARY_PATH=/home/shunya/ComputeLibrary1/build/release
./build/release/examples/graph_mobilenet --instruments=OPENCL_TIMESTAMPS_MS
--example_args='--target=CL'
but, I could not see the outputs with time taken by each kernel. The path
to library in the above command has libarm_compute.so,
libarm_compute_core.so, libarm_compute_graph.so,
libarm_compute_core-static.a, libarm_compute_graph-static.a and
libarm_compute-static.a. Could you please help me know where I am going
wrong?
Have a good day.
Sincerely,
Shraddha Dahal
On Thu, Jun 17, 2021 at 7:10 AM developer-compute @.***>
wrote:

Hi Shraddha,
If you build ACL with the option benchmark_examples=1 you can then
run the network and use the instruments to see how much time each kernel
is taking:
@.***_hikey_dev:~/tmp/pabtel01#
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:./acl_neon+cl_release/
./benchmark_graph_mobilenet --instruments=OPENCL_TIMESTAMPS_MS
--example_args='--target=CL'
This will produce output like below:
OpenCLTimestamps/Now OpenCL: AVG=1121307039 ms
OpenCLTimestamps/Now Wall clock: AVG=1623931494169279 us
OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/gemm_mm_floating_point_f32_bifrost
GWS[8,3136,1] LWS[4,1,1] #2: AVG=1121306920 ms
OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/im2col3x3_nhwc
GWS[2,12544,1] #1: AVG=1121306919 ms
OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/reshape_to_columns
GWS[3,3,3] #0: AVG=1121306918 ms
OpenCLTimestamps/[end]Conv2d_10_depthwise/depthwise+Conv2d_10_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1
GWS[512,7,7] #36: AVG=1121306986 ms
OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t
GWS[128,64,1] #40: AVG=1121306995 ms
OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt
GWS[128,40,1] #39: AVG=1121306989 ms
OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t
GWS[128,128,1] #38: AVG=1121306988 ms
OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/reshape_to_columns
GWS[512,1,1] #37: AVG=1121306988 ms
OpenCLTimestamps/[end]Conv2d_11_depthwise/depthwise+Conv2d_11_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1
GWS[512,7,7] #41: AVG=1121306995 ms
OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t
GWS[128,64,1] #45: AVG=1121307003 ms
OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt
GWS[128,40,1] #44: AVG=1121306998 ms
OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t
GWS[128,128,1] #43: AVG=1121306997 ms
OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/reshape_to_columns
GWS[512,1,1] #42: AVG=1121306997 ms
OpenCLTimestamps/[end]Conv2d_12_depthwise/depthwise+Conv2d_12_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc
GWS[512,7,7] #46: AVG=1121307004 ms
OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t
GWS[256,16,1] #50: AVG=1121307012 ms
OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt
GWS[128,10,1] #49: AVG=1121307008 ms
OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t
GWS[256,128,1] #48: AVG=1121307008 ms
OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/reshape_to_columns
GWS[512,1,1] #47: AVG=1121307007 ms
OpenCLTimestamps/[end]Conv2d_13_depthwise/depthwise+Conv2d_13_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1
GWS[1024,4,4] #51: AVG=1121307012 ms
OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t
GWS[256,16,1] #55: AVG=1121307025 ms
OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt
GWS[256,10,1] #54: AVG=1121307018 ms
OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t
GWS[256,256,1] #53: AVG=1121307018 ms
OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/reshape_to_columns
GWS[1024,1,1] #52: AVG=1121307016 ms
OpenCLTimestamps/[end]Conv2d_1_depthwise/depthwise+Conv2d_1_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1
GWS[32,56,56] #3: AVG=1121306921 ms
OpenCLTimestamps/[end]Conv2d_1_pointwise/Conv2D+Conv2d_1_pointwise/BatchNorm/gemm_mm_floating_point_f32_bifrost
GWS[16,28,112] LWS[4,1,1] #5: AVG=1121306924 ms
OpenCLTimestamps/[end]Conv2d_1_pointwise/Conv2D+Conv2d_1_pointwise/BatchNorm/reshape_to_columns
GWS[32,1,1] #4: AVG=1121306922 ms
OpenCLTimestamps/[end]Conv2d_2_depthwise/depthwise+Conv2d_2_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc
GWS[64,56,56] #6: AVG=1121306926 ms
OpenCLTimestamps/[end]Conv2d_2_pointwise/Conv2D+Conv2d_2_pointwise/BatchNorm/gemm_mm_floating_point_f32_bifrost
GWS[32,14,56] LWS[4,1,1] #8: AVG=1121306929 ms
Another alternative is to use Armnn's ExecuteNetwork to run a tfllite
model and use the -e option which will make the tool output the time
consumed by each kernel
For more information about ExecuteNetwork see:
https://github.com/ARM-software/armnn/tree/branches/armnn_21_02/tests/ExecuteNetwork
Hope this helps.

From: Shraddhaa1 @.>
Sent: 14 June 2021 19:33
To: ARM-software/ComputeLibrary @.>
Cc: Subscribed @.***>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
I had contacted you a few weeks ago regarding the Execution of Inference
Workloads on Hikey970 with layer splitting. Could you please help me know
if there is any function specific in ARM-CL which would let me calculate
inference time taken by each layer of a neural network?
Thank you.
Sincerely,
Shraddha Dahal
On Fri, Apr 9, 2021 at 12:06 PM Shraddha Dahal @.***>
wrote:

Hello Sir,
Thank you for the reply. That really helped.
Have a good day.
Sincerely,
Shraddha Dahal
On Fri, Mar 26, 2021 at 5:44 AM Pablo Marquez Tello <
@.***> wrote:

Closed #882 <https://github.com/ARM-software/ComputeLibrary/issues/882
.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<
https://github.com/ARM-software/ComputeLibrary/issues/882#event-4513101747
,
or unsubscribe
<
https://github.com/notifications/unsubscribe-auth/ARRDMC2KQYG5FVP7OF54Z53TFRQQVANCNFSM4ZHEPDJQ

.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub<
https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-860902526>,
or unsubscribe<
https://github.com/notifications/unsubscribe-auth/AGNXISSHV4V7LSD3ZI2TQSTTSZDQXANCNFSM4ZHEPDJQ

.

IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy the
information in any medium. Thank you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-863185227,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ARRDMC6MGOTC7JHJOXE7NFTTTHQ2XANCNFSM4ZHEPDJQ
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-865332541, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGNXISU4USITBC3ODKTNV4LTT6QIRANCNFSM4ZHEPDJQ.
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

Hello Sir,
Thank you for the reply. The file benchmark_graph_mobilenet, is not
generated anywhere inside the Compute Library. Could you please help me
know if the way I used to build ARM-CL with benchmarks_examples=1 correct?
Have a good day.
Sincerely,
Shraddha Dahal
On Tue, Jun 22, 2021 at 2:20 AM developer-compute @.***>
wrote:

Hi,
Please try running benchmark_graph_mobilenet instead of graph_mobilenet.
Hope this helps.

From: Shraddhaa1 @.>
Sent: 21 June 2021 21:46
To: ARM-software/ComputeLibrary @.>
Cc: developer-compute @.>; Comment @.>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
Thank you for the response. I tried the first method you mentioned in the
previous email.
In the makefile.arm file, I added benchmark_examples:=1 such as:
BUILD:=native
NEON:=1
OPENCL:=1
ARCH:=arm64-v8a
all: release
CFLAGS:=-std=c++14
benchmark_examples:=1
debug:
scons -j8 -Q arch=$(ARCH) build=$(BUILD) neon=$(NEON) opencl=$(OPENCL)
build_dir=debug debug=1 extra_cxx_flags=$(CFLAGS)
release:
scons -j8 -Q arch=$(ARCH) build=$(BUILD) neon=$(NEON) opencl=$(OPENCL)
build_dir=release debug=0 extra_cxx_flags=$(CFLAGS)
sched:
g++ -o build/release/examples/graph_temp_scheduler2.o -c
-Wno-deprecated-declarations -Wall -DARCH_ARM -Wextra -Wno-unused-parameter
-pedantic -Wdisabled-optimization -Wformat=2 -Winit-self
-Wstrict-overflow=2 -Wswitch-default -fpermissive -std=gnu++11 -Wno-vla
-Woverloaded-virtual -Wctor-dtor-privacy -Wsign-promo -Weffc++
-Wno-format-nonliteral -Wno-overlength-strings -Wno-strict-overflow
-Wlogical-op -Wnoexcept -Wstrict-null-sentinel -Wno-implicit-fallthrough
-march=armv8-a -Wno-ignored-attributes -Werror -O3 -ftree-vectorize
-std=c++14 -D_GLIBCXX_USE_NANOSLEEP -DARM_COMPUTE_CPP_SCHEDULER=1
-DARM_COMPUTE_AARCH64_V8A -DNO_DOT_IN_TOOLCHAIN -DEMBEDDED_KERNELS
-Iinclude -I. -I. examples/graph_temp_scheduler2.cpp
g++ -o build/release/examples/graph_temp_scheduler2
-Wl,--allow-shlib-undefined build/release/examples/graph_temp_scheduler2.o
build/release/utils/Utils.o build/release/utils/GraphUtils.o
build/release/utils/CommonGraphOptions.o -Lbuild/release -L. -lpthread
-larm_compute_graph -larm_compute -larm_compute_core
After that, I gave commands:
$make all
$sudo LD_LIBRARY_PATH=/home/shunya/ComputeLibrary1/build/release
./build/release/examples/graph_mobilenet --instruments=OPENCL_TIMESTAMPS_MS
--example_args='--target=CL'
but, I could not see the outputs with time taken by each kernel. The path
to library in the above command has libarm_compute.so,
libarm_compute_core.so, libarm_compute_graph.so,
libarm_compute_core-static.a, libarm_compute_graph-static.a and
libarm_compute-static.a. Could you please help me know where I am going
wrong?
Have a good day.
Sincerely,
Shraddha Dahal
On Thu, Jun 17, 2021 at 7:10 AM developer-compute @.***>
wrote:

Hi Shraddha,
If you build ACL with the option benchmark_examples=1 you can then
run the network and use the instruments to see how much time each kernel
is taking:
@.***_hikey_dev:~/tmp/pabtel01#
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:./acl_neon+cl_release/
./benchmark_graph_mobilenet --instruments=OPENCL_TIMESTAMPS_MS
--example_args='--target=CL'
This will produce output like below:
OpenCLTimestamps/Now OpenCL: AVG=1121307039 ms
OpenCLTimestamps/Now Wall clock: AVG=1623931494169279 us

OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/gemm_mm_floating_point_f32_bifrost

GWS[8,3136,1] LWS[4,1,1] #2: AVG=1121306920 ms
OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/im2col3x3_nhwc
GWS[2,12544,1] #1: AVG=1121306919 ms
OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/reshape_to_columns
GWS[3,3,3] #0: AVG=1121306918 ms

OpenCLTimestamps/[end]Conv2d_10_depthwise/depthwise+Conv2d_10_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[512,7,7] #36: AVG=1121306986 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[128,64,1] #40: AVG=1121306995 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[128,40,1] #39: AVG=1121306989 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[128,128,1] #38: AVG=1121306988 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/reshape_to_columns

GWS[512,1,1] #37: AVG=1121306988 ms

OpenCLTimestamps/[end]Conv2d_11_depthwise/depthwise+Conv2d_11_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[512,7,7] #41: AVG=1121306995 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[128,64,1] #45: AVG=1121307003 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[128,40,1] #44: AVG=1121306998 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[128,128,1] #43: AVG=1121306997 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/reshape_to_columns

GWS[512,1,1] #42: AVG=1121306997 ms

OpenCLTimestamps/[end]Conv2d_12_depthwise/depthwise+Conv2d_12_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc

GWS[512,7,7] #46: AVG=1121307004 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[256,16,1] #50: AVG=1121307012 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[128,10,1] #49: AVG=1121307008 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[256,128,1] #48: AVG=1121307008 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/reshape_to_columns

GWS[512,1,1] #47: AVG=1121307007 ms

OpenCLTimestamps/[end]Conv2d_13_depthwise/depthwise+Conv2d_13_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[1024,4,4] #51: AVG=1121307012 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[256,16,1] #55: AVG=1121307025 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[256,10,1] #54: AVG=1121307018 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[256,256,1] #53: AVG=1121307018 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/reshape_to_columns

GWS[1024,1,1] #52: AVG=1121307016 ms

OpenCLTimestamps/[end]Conv2d_1_depthwise/depthwise+Conv2d_1_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[32,56,56] #3: AVG=1121306921 ms

OpenCLTimestamps/[end]Conv2d_1_pointwise/Conv2D+Conv2d_1_pointwise/BatchNorm/gemm_mm_floating_point_f32_bifrost

GWS[16,28,112] LWS[4,1,1] #5: AVG=1121306924 ms

OpenCLTimestamps/[end]Conv2d_1_pointwise/Conv2D+Conv2d_1_pointwise/BatchNorm/reshape_to_columns

GWS[32,1,1] #4: AVG=1121306922 ms

OpenCLTimestamps/[end]Conv2d_2_depthwise/depthwise+Conv2d_2_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc

GWS[64,56,56] #6: AVG=1121306926 ms

OpenCLTimestamps/[end]Conv2d_2_pointwise/Conv2D+Conv2d_2_pointwise/BatchNorm/gemm_mm_floating_point_f32_bifrost

GWS[32,14,56] LWS[4,1,1] #8: AVG=1121306929 ms
Another alternative is to use Armnn's ExecuteNetwork to run a tfllite
model and use the -e option which will make the tool output the time
consumed by each kernel
For more information about ExecuteNetwork see:

https://github.com/ARM-software/armnn/tree/branches/armnn_21_02/tests/ExecuteNetwork

Hope this helps.

From: Shraddhaa1 @.>
Sent: 14 June 2021 19:33
To: ARM-software/ComputeLibrary @.>
Cc: Subscribed @.***>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
I had contacted you a few weeks ago regarding the Execution of Inference
Workloads on Hikey970 with layer splitting. Could you please help me know
if there is any function specific in ARM-CL which would let me calculate
inference time taken by each layer of a neural network?
Thank you.
Sincerely,
Shraddha Dahal
On Fri, Apr 9, 2021 at 12:06 PM Shraddha Dahal @.***>
wrote:

Hello Sir,
Thank you for the reply. That really helped.
Have a good day.
Sincerely,
Shraddha Dahal
On Fri, Mar 26, 2021 at 5:44 AM Pablo Marquez Tello <
@.***> wrote:

Closed #882 <
https://github.com/ARM-software/ComputeLibrary/issues/882
.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<

https://github.com/ARM-software/ComputeLibrary/issues/882#event-4513101747

,

or unsubscribe
<

https://github.com/notifications/unsubscribe-auth/ARRDMC2KQYG5FVP7OF54Z53TFRQQVANCNFSM4ZHEPDJQ

.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub<

https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-860902526

,
or unsubscribe<

https://github.com/notifications/unsubscribe-auth/AGNXISSHV4V7LSD3ZI2TQSTTSZDQXANCNFSM4ZHEPDJQ

.

IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy
the
information in any medium. Thank you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<
https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-863185227
,
or unsubscribe
<
https://github.com/notifications/unsubscribe-auth/ARRDMC6MGOTC7JHJOXE7NFTTTHQ2XANCNFSM4ZHEPDJQ
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub<
https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-865332541>,
or unsubscribe<
https://github.com/notifications/unsubscribe-auth/AGNXISU4USITBC3ODKTNV4LTT6QIRANCNFSM4ZHEPDJQ

.

IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy the
information in any medium. Thank you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-865664696,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ARRDMC7EFF42VJLI3ECM4LDTUA2TTANCNFSM4ZHEPDJQ
.

Hi,
The benchmark examples are only available in the development repo: https://review.mlplatform.org/admin/repos/ml%2FComputeLibrary
Try: git clone "https://review.mlplatform.org/ml/ComputeLibrary"
Then build with benchmark_examples=1
Hope this helps

From: Shraddhaa1 @.>
Sent: 22 June 2021 18:31
To: ARM-software/ComputeLibrary @.>
Cc: developer-compute @.>; Comment @.>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
Thank you for the reply. The file benchmark_graph_mobilenet, is not
generated anywhere inside the Compute Library. Could you please help me
know if the way I used to build ARM-CL with benchmarks_examples=1 correct?
Have a good day.
Sincerely,
Shraddha Dahal
On Tue, Jun 22, 2021 at 2:20 AM developer-compute @.***>
wrote:

Hi,
Please try running benchmark_graph_mobilenet instead of graph_mobilenet.
Hope this helps.

From: Shraddhaa1 @.>
Sent: 21 June 2021 21:46
To: ARM-software/ComputeLibrary @.>
Cc: developer-compute @.>; Comment @.>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
Thank you for the response. I tried the first method you mentioned in the
previous email.
In the makefile.arm file, I added benchmark_examples:=1 such as:
BUILD:=native
NEON:=1
OPENCL:=1
ARCH:=arm64-v8a
all: release
CFLAGS:=-std=c++14
benchmark_examples:=1
debug:
scons -j8 -Q arch=$(ARCH) build=$(BUILD) neon=$(NEON) opencl=$(OPENCL)
build_dir=debug debug=1 extra_cxx_flags=$(CFLAGS)
release:
scons -j8 -Q arch=$(ARCH) build=$(BUILD) neon=$(NEON) opencl=$(OPENCL)
build_dir=release debug=0 extra_cxx_flags=$(CFLAGS)
sched:
g++ -o build/release/examples/graph_temp_scheduler2.o -c
-Wno-deprecated-declarations -Wall -DARCH_ARM -Wextra -Wno-unused-parameter
-pedantic -Wdisabled-optimization -Wformat=2 -Winit-self
-Wstrict-overflow=2 -Wswitch-default -fpermissive -std=gnu++11 -Wno-vla
-Woverloaded-virtual -Wctor-dtor-privacy -Wsign-promo -Weffc++
-Wno-format-nonliteral -Wno-overlength-strings -Wno-strict-overflow
-Wlogical-op -Wnoexcept -Wstrict-null-sentinel -Wno-implicit-fallthrough
-march=armv8-a -Wno-ignored-attributes -Werror -O3 -ftree-vectorize
-std=c++14 -D_GLIBCXX_USE_NANOSLEEP -DARM_COMPUTE_CPP_SCHEDULER=1
-DARM_COMPUTE_AARCH64_V8A -DNO_DOT_IN_TOOLCHAIN -DEMBEDDED_KERNELS
-Iinclude -I. -I. examples/graph_temp_scheduler2.cpp
g++ -o build/release/examples/graph_temp_scheduler2
-Wl,--allow-shlib-undefined build/release/examples/graph_temp_scheduler2.o
build/release/utils/Utils.o build/release/utils/GraphUtils.o
build/release/utils/CommonGraphOptions.o -Lbuild/release -L. -lpthread
-larm_compute_graph -larm_compute -larm_compute_core
After that, I gave commands:
$make all
$sudo LD_LIBRARY_PATH=/home/shunya/ComputeLibrary1/build/release
./build/release/examples/graph_mobilenet --instruments=OPENCL_TIMESTAMPS_MS
--example_args='--target=CL'
but, I could not see the outputs with time taken by each kernel. The path
to library in the above command has libarm_compute.so,
libarm_compute_core.so, libarm_compute_graph.so,
libarm_compute_core-static.a, libarm_compute_graph-static.a and
libarm_compute-static.a. Could you please help me know where I am going
wrong?
Have a good day.
Sincerely,
Shraddha Dahal
On Thu, Jun 17, 2021 at 7:10 AM developer-compute @.***>
wrote:

Hi Shraddha,
If you build ACL with the option benchmark_examples=1 you can then
run the network and use the instruments to see how much time each kernel
is taking:
@.***_hikey_dev:~/tmp/pabtel01#
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:./acl_neon+cl_release/
./benchmark_graph_mobilenet --instruments=OPENCL_TIMESTAMPS_MS
--example_args='--target=CL'
This will produce output like below:
OpenCLTimestamps/Now OpenCL: AVG=1121307039 ms
OpenCLTimestamps/Now Wall clock: AVG=1623931494169279 us

OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/gemm_mm_floating_point_f32_bifrost

GWS[8,3136,1] LWS[4,1,1] #2: AVG=1121306920 ms
OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/im2col3x3_nhwc
GWS[2,12544,1] #1: AVG=1121306919 ms
OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/reshape_to_columns
GWS[3,3,3] #0: AVG=1121306918 ms

OpenCLTimestamps/[end]Conv2d_10_depthwise/depthwise+Conv2d_10_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[512,7,7] #36: AVG=1121306986 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[128,64,1] #40: AVG=1121306995 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[128,40,1] #39: AVG=1121306989 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[128,128,1] #38: AVG=1121306988 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/reshape_to_columns

GWS[512,1,1] #37: AVG=1121306988 ms

OpenCLTimestamps/[end]Conv2d_11_depthwise/depthwise+Conv2d_11_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[512,7,7] #41: AVG=1121306995 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[128,64,1] #45: AVG=1121307003 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[128,40,1] #44: AVG=1121306998 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[128,128,1] #43: AVG=1121306997 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/reshape_to_columns

GWS[512,1,1] #42: AVG=1121306997 ms

OpenCLTimestamps/[end]Conv2d_12_depthwise/depthwise+Conv2d_12_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc

GWS[512,7,7] #46: AVG=1121307004 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[256,16,1] #50: AVG=1121307012 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[128,10,1] #49: AVG=1121307008 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[256,128,1] #48: AVG=1121307008 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/reshape_to_columns

GWS[512,1,1] #47: AVG=1121307007 ms

OpenCLTimestamps/[end]Conv2d_13_depthwise/depthwise+Conv2d_13_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[1024,4,4] #51: AVG=1121307012 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[256,16,1] #55: AVG=1121307025 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[256,10,1] #54: AVG=1121307018 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[256,256,1] #53: AVG=1121307018 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/reshape_to_columns

GWS[1024,1,1] #52: AVG=1121307016 ms

OpenCLTimestamps/[end]Conv2d_1_depthwise/depthwise+Conv2d_1_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[32,56,56] #3: AVG=1121306921 ms

OpenCLTimestamps/[end]Conv2d_1_pointwise/Conv2D+Conv2d_1_pointwise/BatchNorm/gemm_mm_floating_point_f32_bifrost

GWS[16,28,112] LWS[4,1,1] #5: AVG=1121306924 ms

OpenCLTimestamps/[end]Conv2d_1_pointwise/Conv2D+Conv2d_1_pointwise/BatchNorm/reshape_to_columns

GWS[32,1,1] #4: AVG=1121306922 ms

OpenCLTimestamps/[end]Conv2d_2_depthwise/depthwise+Conv2d_2_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc

GWS[64,56,56] #6: AVG=1121306926 ms

OpenCLTimestamps/[end]Conv2d_2_pointwise/Conv2D+Conv2d_2_pointwise/BatchNorm/gemm_mm_floating_point_f32_bifrost

GWS[32,14,56] LWS[4,1,1] #8: AVG=1121306929 ms
Another alternative is to use Armnn's ExecuteNetwork to run a tfllite
model and use the -e option which will make the tool output the time
consumed by each kernel
For more information about ExecuteNetwork see:

https://github.com/ARM-software/armnn/tree/branches/armnn_21_02/tests/ExecuteNetwork

Hope this helps.

From: Shraddhaa1 @.>
Sent: 14 June 2021 19:33
To: ARM-software/ComputeLibrary @.>
Cc: Subscribed @.***>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
I had contacted you a few weeks ago regarding the Execution of Inference
Workloads on Hikey970 with layer splitting. Could you please help me know
if there is any function specific in ARM-CL which would let me calculate
inference time taken by each layer of a neural network?
Thank you.
Sincerely,
Shraddha Dahal
On Fri, Apr 9, 2021 at 12:06 PM Shraddha Dahal @.***>
wrote:

Hello Sir,
Thank you for the reply. That really helped.
Have a good day.
Sincerely,
Shraddha Dahal
On Fri, Mar 26, 2021 at 5:44 AM Pablo Marquez Tello <
@.***> wrote:

Closed #882 <
https://github.com/ARM-software/ComputeLibrary/issues/882
.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<

https://github.com/ARM-software/ComputeLibrary/issues/882#event-4513101747

,

or unsubscribe
<

https://github.com/notifications/unsubscribe-auth/ARRDMC2KQYG5FVP7OF54Z53TFRQQVANCNFSM4ZHEPDJQ

.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub<

https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-860902526

,
or unsubscribe<

https://github.com/notifications/unsubscribe-auth/AGNXISSHV4V7LSD3ZI2TQSTTSZDQXANCNFSM4ZHEPDJQ

.

IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy
the
information in any medium. Thank you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<
https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-863185227
,
or unsubscribe
<
https://github.com/notifications/unsubscribe-auth/ARRDMC6MGOTC7JHJOXE7NFTTTHQ2XANCNFSM4ZHEPDJQ
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub<
https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-865332541>,
or unsubscribe<
https://github.com/notifications/unsubscribe-auth/AGNXISU4USITBC3ODKTNV4LTT6QIRANCNFSM4ZHEPDJQ

.

IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy the
information in any medium. Thank you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-865664696,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ARRDMC7EFF42VJLI3ECM4LDTUA2TTANCNFSM4ZHEPDJQ
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-866187809, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGNXISUBGWRVL4IE2NWXNG3TUDCIJANCNFSM4ZHEPDJQ.
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

Hello Sir,
Thank you for the reply. I can now observe time taken by each kernel of the
neural networks. Could you please help me know if there has been any update
on how specific layers of a neural network can be specified to either CPU
or GPU? With the repo:
https://github.com/adityagupta1089/ComputeLibrary.git
I could mix the use of CPU and GPU, but I am trying to specify layers to
CPU or GPU. Could you please help me understand how it can be obtained?
Have a good evening.
Sincerely,
Shraddha Dahal
On Tue, Jun 22, 2021 at 3:07 PM developer-compute @.***>
wrote:

Hi,
The benchmark examples are only available in the development repo:
https://review.mlplatform.org/admin/repos/ml%2FComputeLibrary
Try: git clone "https://review.mlplatform.org/ml/ComputeLibrary"
Then build with benchmark_examples=1
Hope this helps

From: Shraddhaa1 @.>
Sent: 22 June 2021 18:31
To: ARM-software/ComputeLibrary @.>
Cc: developer-compute @.>; Comment @.>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
Thank you for the reply. The file benchmark_graph_mobilenet, is not
generated anywhere inside the Compute Library. Could you please help me
know if the way I used to build ARM-CL with benchmarks_examples=1 correct?
Have a good day.
Sincerely,
Shraddha Dahal
On Tue, Jun 22, 2021 at 2:20 AM developer-compute @.***>
wrote:

Hi,
Please try running benchmark_graph_mobilenet instead of graph_mobilenet.
Hope this helps.

From: Shraddhaa1 @.>
Sent: 21 June 2021 21:46
To: ARM-software/ComputeLibrary @.>
Cc: developer-compute @.>; Comment @.>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
Thank you for the response. I tried the first method you mentioned in the
previous email.
In the makefile.arm file, I added benchmark_examples:=1 such as:
BUILD:=native
NEON:=1
OPENCL:=1
ARCH:=arm64-v8a
all: release
CFLAGS:=-std=c++14
benchmark_examples:=1
debug:
scons -j8 -Q arch=$(ARCH) build=$(BUILD) neon=$(NEON) opencl=$(OPENCL)
build_dir=debug debug=1 extra_cxx_flags=$(CFLAGS)
release:
scons -j8 -Q arch=$(ARCH) build=$(BUILD) neon=$(NEON) opencl=$(OPENCL)
build_dir=release debug=0 extra_cxx_flags=$(CFLAGS)
sched:
g++ -o build/release/examples/graph_temp_scheduler2.o -c
-Wno-deprecated-declarations -Wall -DARCH_ARM -Wextra
-Wno-unused-parameter
-pedantic -Wdisabled-optimization -Wformat=2 -Winit-self
-Wstrict-overflow=2 -Wswitch-default -fpermissive -std=gnu++11 -Wno-vla
-Woverloaded-virtual -Wctor-dtor-privacy -Wsign-promo -Weffc++
-Wno-format-nonliteral -Wno-overlength-strings -Wno-strict-overflow
-Wlogical-op -Wnoexcept -Wstrict-null-sentinel -Wno-implicit-fallthrough
-march=armv8-a -Wno-ignored-attributes -Werror -O3 -ftree-vectorize
-std=c++14 -D_GLIBCXX_USE_NANOSLEEP -DARM_COMPUTE_CPP_SCHEDULER=1
-DARM_COMPUTE_AARCH64_V8A -DNO_DOT_IN_TOOLCHAIN -DEMBEDDED_KERNELS
-Iinclude -I. -I. examples/graph_temp_scheduler2.cpp
g++ -o build/release/examples/graph_temp_scheduler2
-Wl,--allow-shlib-undefined
build/release/examples/graph_temp_scheduler2.o
build/release/utils/Utils.o build/release/utils/GraphUtils.o
build/release/utils/CommonGraphOptions.o -Lbuild/release -L. -lpthread
-larm_compute_graph -larm_compute -larm_compute_core
After that, I gave commands:
$make all
$sudo LD_LIBRARY_PATH=/home/shunya/ComputeLibrary1/build/release
./build/release/examples/graph_mobilenet
--instruments=OPENCL_TIMESTAMPS_MS
--example_args='--target=CL'
but, I could not see the outputs with time taken by each kernel. The path
to library in the above command has libarm_compute.so,
libarm_compute_core.so, libarm_compute_graph.so,
libarm_compute_core-static.a, libarm_compute_graph-static.a and
libarm_compute-static.a. Could you please help me know where I am going
wrong?
Have a good day.
Sincerely,
Shraddha Dahal
On Thu, Jun 17, 2021 at 7:10 AM developer-compute @.***>
wrote:

Hi Shraddha,
If you build ACL with the option benchmark_examples=1 you can then
run the network and use the instruments to see how much time each
kernel
is taking:
@.***_hikey_dev:~/tmp/pabtel01#
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:./acl_neon+cl_release/
./benchmark_graph_mobilenet --instruments=OPENCL_TIMESTAMPS_MS
--example_args='--target=CL'
This will produce output like below:
OpenCLTimestamps/Now OpenCL: AVG=1121307039 ms
OpenCLTimestamps/Now Wall clock: AVG=1623931494169279 us

OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/gemm_mm_floating_point_f32_bifrost

GWS[8,3136,1] LWS[4,1,1] #2: AVG=1121306920 ms
OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/im2col3x3_nhwc
GWS[2,12544,1] #1: AVG=1121306919 ms
OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/reshape_to_columns
GWS[3,3,3] #0: AVG=1121306918 ms

OpenCLTimestamps/[end]Conv2d_10_depthwise/depthwise+Conv2d_10_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[512,7,7] #36: AVG=1121306986 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[128,64,1] #40: AVG=1121306995 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[128,40,1] #39: AVG=1121306989 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[128,128,1] #38: AVG=1121306988 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/reshape_to_columns

GWS[512,1,1] #37: AVG=1121306988 ms

OpenCLTimestamps/[end]Conv2d_11_depthwise/depthwise+Conv2d_11_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[512,7,7] #41: AVG=1121306995 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[128,64,1] #45: AVG=1121307003 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[128,40,1] #44: AVG=1121306998 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[128,128,1] #43: AVG=1121306997 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/reshape_to_columns

GWS[512,1,1] #42: AVG=1121306997 ms

OpenCLTimestamps/[end]Conv2d_12_depthwise/depthwise+Conv2d_12_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc

GWS[512,7,7] #46: AVG=1121307004 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[256,16,1] #50: AVG=1121307012 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[128,10,1] #49: AVG=1121307008 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[256,128,1] #48: AVG=1121307008 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/reshape_to_columns

GWS[512,1,1] #47: AVG=1121307007 ms

OpenCLTimestamps/[end]Conv2d_13_depthwise/depthwise+Conv2d_13_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[1024,4,4] #51: AVG=1121307012 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[256,16,1] #55: AVG=1121307025 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[256,10,1] #54: AVG=1121307018 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[256,256,1] #53: AVG=1121307018 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/reshape_to_columns

GWS[1024,1,1] #52: AVG=1121307016 ms

OpenCLTimestamps/[end]Conv2d_1_depthwise/depthwise+Conv2d_1_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[32,56,56] #3: AVG=1121306921 ms

OpenCLTimestamps/[end]Conv2d_1_pointwise/Conv2D+Conv2d_1_pointwise/BatchNorm/gemm_mm_floating_point_f32_bifrost

GWS[16,28,112] LWS[4,1,1] #5: AVG=1121306924 ms

OpenCLTimestamps/[end]Conv2d_1_pointwise/Conv2D+Conv2d_1_pointwise/BatchNorm/reshape_to_columns

GWS[32,1,1] #4: AVG=1121306922 ms

OpenCLTimestamps/[end]Conv2d_2_depthwise/depthwise+Conv2d_2_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc

GWS[64,56,56] #6: AVG=1121306926 ms

OpenCLTimestamps/[end]Conv2d_2_pointwise/Conv2D+Conv2d_2_pointwise/BatchNorm/gemm_mm_floating_point_f32_bifrost

GWS[32,14,56] LWS[4,1,1] #8: AVG=1121306929 ms
Another alternative is to use Armnn's ExecuteNetwork to run a tfllite
model and use the -e option which will make the tool output the time
consumed by each kernel
For more information about ExecuteNetwork see:

https://github.com/ARM-software/armnn/tree/branches/armnn_21_02/tests/ExecuteNetwork

Hope this helps.

From: Shraddhaa1 @.>
Sent: 14 June 2021 19:33
To: ARM-software/ComputeLibrary @.>
Cc: Subscribed @.***>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
I had contacted you a few weeks ago regarding the Execution of
Inference
Workloads on Hikey970 with layer splitting. Could you please help me
know
if there is any function specific in ARM-CL which would let me
calculate
inference time taken by each layer of a neural network?
Thank you.
Sincerely,
Shraddha Dahal
On Fri, Apr 9, 2021 at 12:06 PM Shraddha Dahal @.***>
wrote:

Hello Sir,
Thank you for the reply. That really helped.
Have a good day.
Sincerely,
Shraddha Dahal
On Fri, Mar 26, 2021 at 5:44 AM Pablo Marquez Tello <
@.***> wrote:

Closed #882 <
https://github.com/ARM-software/ComputeLibrary/issues/882
.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<

https://github.com/ARM-software/ComputeLibrary/issues/882#event-4513101747

,

or unsubscribe
<

https://github.com/notifications/unsubscribe-auth/ARRDMC2KQYG5FVP7OF54Z53TFRQQVANCNFSM4ZHEPDJQ

.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub<

https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-860902526

,
or unsubscribe<

https://github.com/notifications/unsubscribe-auth/AGNXISSHV4V7LSD3ZI2TQSTTSZDQXANCNFSM4ZHEPDJQ

.

IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy
the
information in any medium. Thank you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<

https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-863185227

,
or unsubscribe
<

https://github.com/notifications/unsubscribe-auth/ARRDMC6MGOTC7JHJOXE7NFTTTHQ2XANCNFSM4ZHEPDJQ

.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub<

https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-865332541

,
or unsubscribe<

https://github.com/notifications/unsubscribe-auth/AGNXISU4USITBC3ODKTNV4LTT6QIRANCNFSM4ZHEPDJQ

.

IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy
the
information in any medium. Thank you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<
https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-865664696
,
or unsubscribe
<
https://github.com/notifications/unsubscribe-auth/ARRDMC7EFF42VJLI3ECM4LDTUA2TTANCNFSM4ZHEPDJQ
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub<
https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-866187809>,
or unsubscribe<
https://github.com/notifications/unsubscribe-auth/AGNXISUBGWRVL4IE2NWXNG3TUDCIJANCNFSM4ZHEPDJQ

.

IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy the
information in any medium. Thank you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-866297436,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ARRDMC6ZEHGM54QQBSQEDCLTUDURVANCNFSM4ZHEPDJQ
.

Hi,
The graph api in ACL does not support that level of granularity, you must run all the workloads on the backend specified with the --target option. You cannot have some workloads on the CPU and others on the GPU.
Hope this helps.

From: Shraddhaa1 @.>
Sent: 23 June 2021 03:46
To: ARM-software/ComputeLibrary @.>
Cc: developer-compute @.>; Comment @.>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
Thank you for the reply. I can now observe time taken by each kernel of the
neural networks. Could you please help me know if there has been any update
on how specific layers of a neural network can be specified to either CPU
or GPU? With the repo:
https://github.com/adityagupta1089/ComputeLibrary.git
I could mix the use of CPU and GPU, but I am trying to specify layers to
CPU or GPU. Could you please help me understand how it can be obtained?
Have a good evening.
Sincerely,
Shraddha Dahal
On Tue, Jun 22, 2021 at 3:07 PM developer-compute @.***>
wrote:

Hi,
The benchmark examples are only available in the development repo:
https://review.mlplatform.org/admin/repos/ml%2FComputeLibrary
Try: git clone "https://review.mlplatform.org/ml/ComputeLibrary"
Then build with benchmark_examples=1
Hope this helps

From: Shraddhaa1 @.>
Sent: 22 June 2021 18:31
To: ARM-software/ComputeLibrary @.>
Cc: developer-compute @.>; Comment @.>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
Thank you for the reply. The file benchmark_graph_mobilenet, is not
generated anywhere inside the Compute Library. Could you please help me
know if the way I used to build ARM-CL with benchmarks_examples=1 correct?
Have a good day.
Sincerely,
Shraddha Dahal
On Tue, Jun 22, 2021 at 2:20 AM developer-compute @.***>
wrote:

Hi,
Please try running benchmark_graph_mobilenet instead of graph_mobilenet.
Hope this helps.

From: Shraddhaa1 @.>
Sent: 21 June 2021 21:46
To: ARM-software/ComputeLibrary @.>
Cc: developer-compute @.>; Comment @.>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
Thank you for the response. I tried the first method you mentioned in the
previous email.
In the makefile.arm file, I added benchmark_examples:=1 such as:
BUILD:=native
NEON:=1
OPENCL:=1
ARCH:=arm64-v8a
all: release
CFLAGS:=-std=c++14
benchmark_examples:=1
debug:
scons -j8 -Q arch=$(ARCH) build=$(BUILD) neon=$(NEON) opencl=$(OPENCL)
build_dir=debug debug=1 extra_cxx_flags=$(CFLAGS)
release:
scons -j8 -Q arch=$(ARCH) build=$(BUILD) neon=$(NEON) opencl=$(OPENCL)
build_dir=release debug=0 extra_cxx_flags=$(CFLAGS)
sched:
g++ -o build/release/examples/graph_temp_scheduler2.o -c
-Wno-deprecated-declarations -Wall -DARCH_ARM -Wextra
-Wno-unused-parameter
-pedantic -Wdisabled-optimization -Wformat=2 -Winit-self
-Wstrict-overflow=2 -Wswitch-default -fpermissive -std=gnu++11 -Wno-vla
-Woverloaded-virtual -Wctor-dtor-privacy -Wsign-promo -Weffc++
-Wno-format-nonliteral -Wno-overlength-strings -Wno-strict-overflow
-Wlogical-op -Wnoexcept -Wstrict-null-sentinel -Wno-implicit-fallthrough
-march=armv8-a -Wno-ignored-attributes -Werror -O3 -ftree-vectorize
-std=c++14 -D_GLIBCXX_USE_NANOSLEEP -DARM_COMPUTE_CPP_SCHEDULER=1
-DARM_COMPUTE_AARCH64_V8A -DNO_DOT_IN_TOOLCHAIN -DEMBEDDED_KERNELS
-Iinclude -I. -I. examples/graph_temp_scheduler2.cpp
g++ -o build/release/examples/graph_temp_scheduler2
-Wl,--allow-shlib-undefined
build/release/examples/graph_temp_scheduler2.o
build/release/utils/Utils.o build/release/utils/GraphUtils.o
build/release/utils/CommonGraphOptions.o -Lbuild/release -L. -lpthread
-larm_compute_graph -larm_compute -larm_compute_core
After that, I gave commands:
$make all
$sudo LD_LIBRARY_PATH=/home/shunya/ComputeLibrary1/build/release
./build/release/examples/graph_mobilenet
--instruments=OPENCL_TIMESTAMPS_MS
--example_args='--target=CL'
but, I could not see the outputs with time taken by each kernel. The path
to library in the above command has libarm_compute.so,
libarm_compute_core.so, libarm_compute_graph.so,
libarm_compute_core-static.a, libarm_compute_graph-static.a and
libarm_compute-static.a. Could you please help me know where I am going
wrong?
Have a good day.
Sincerely,
Shraddha Dahal
On Thu, Jun 17, 2021 at 7:10 AM developer-compute @.***>
wrote:

Hi Shraddha,
If you build ACL with the option benchmark_examples=1 you can then
run the network and use the instruments to see how much time each
kernel
is taking:
@.***_hikey_dev:~/tmp/pabtel01#
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:./acl_neon+cl_release/
./benchmark_graph_mobilenet --instruments=OPENCL_TIMESTAMPS_MS
--example_args='--target=CL'
This will produce output like below:
OpenCLTimestamps/Now OpenCL: AVG=1121307039 ms
OpenCLTimestamps/Now Wall clock: AVG=1623931494169279 us

OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/gemm_mm_floating_point_f32_bifrost

GWS[8,3136,1] LWS[4,1,1] #2: AVG=1121306920 ms
OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/im2col3x3_nhwc
GWS[2,12544,1] #1: AVG=1121306919 ms
OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/reshape_to_columns
GWS[3,3,3] #0: AVG=1121306918 ms

OpenCLTimestamps/[end]Conv2d_10_depthwise/depthwise+Conv2d_10_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[512,7,7] #36: AVG=1121306986 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[128,64,1] #40: AVG=1121306995 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[128,40,1] #39: AVG=1121306989 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[128,128,1] #38: AVG=1121306988 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/reshape_to_columns

GWS[512,1,1] #37: AVG=1121306988 ms

OpenCLTimestamps/[end]Conv2d_11_depthwise/depthwise+Conv2d_11_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[512,7,7] #41: AVG=1121306995 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[128,64,1] #45: AVG=1121307003 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[128,40,1] #44: AVG=1121306998 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[128,128,1] #43: AVG=1121306997 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/reshape_to_columns

GWS[512,1,1] #42: AVG=1121306997 ms

OpenCLTimestamps/[end]Conv2d_12_depthwise/depthwise+Conv2d_12_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc

GWS[512,7,7] #46: AVG=1121307004 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[256,16,1] #50: AVG=1121307012 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[128,10,1] #49: AVG=1121307008 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[256,128,1] #48: AVG=1121307008 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/reshape_to_columns

GWS[512,1,1] #47: AVG=1121307007 ms

OpenCLTimestamps/[end]Conv2d_13_depthwise/depthwise+Conv2d_13_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[1024,4,4] #51: AVG=1121307012 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[256,16,1] #55: AVG=1121307025 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[256,10,1] #54: AVG=1121307018 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[256,256,1] #53: AVG=1121307018 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/reshape_to_columns

GWS[1024,1,1] #52: AVG=1121307016 ms

OpenCLTimestamps/[end]Conv2d_1_depthwise/depthwise+Conv2d_1_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[32,56,56] #3: AVG=1121306921 ms

OpenCLTimestamps/[end]Conv2d_1_pointwise/Conv2D+Conv2d_1_pointwise/BatchNorm/gemm_mm_floating_point_f32_bifrost

GWS[16,28,112] LWS[4,1,1] #5: AVG=1121306924 ms

OpenCLTimestamps/[end]Conv2d_1_pointwise/Conv2D+Conv2d_1_pointwise/BatchNorm/reshape_to_columns

GWS[32,1,1] #4: AVG=1121306922 ms

OpenCLTimestamps/[end]Conv2d_2_depthwise/depthwise+Conv2d_2_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc

GWS[64,56,56] #6: AVG=1121306926 ms

OpenCLTimestamps/[end]Conv2d_2_pointwise/Conv2D+Conv2d_2_pointwise/BatchNorm/gemm_mm_floating_point_f32_bifrost

GWS[32,14,56] LWS[4,1,1] #8: AVG=1121306929 ms
Another alternative is to use Armnn's ExecuteNetwork to run a tfllite
model and use the -e option which will make the tool output the time
consumed by each kernel
For more information about ExecuteNetwork see:

https://github.com/ARM-software/armnn/tree/branches/armnn_21_02/tests/ExecuteNetwork

Hope this helps.

From: Shraddhaa1 @.>
Sent: 14 June 2021 19:33
To: ARM-software/ComputeLibrary @.>
Cc: Subscribed @.***>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
I had contacted you a few weeks ago regarding the Execution of
Inference
Workloads on Hikey970 with layer splitting. Could you please help me
know
if there is any function specific in ARM-CL which would let me
calculate
inference time taken by each layer of a neural network?
Thank you.
Sincerely,
Shraddha Dahal
On Fri, Apr 9, 2021 at 12:06 PM Shraddha Dahal @.***>
wrote:

Hello Sir,
Thank you for the reply. That really helped.
Have a good day.
Sincerely,
Shraddha Dahal
On Fri, Mar 26, 2021 at 5:44 AM Pablo Marquez Tello <
@.***> wrote:

Closed #882 <
https://github.com/ARM-software/ComputeLibrary/issues/882
.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<

https://github.com/ARM-software/ComputeLibrary/issues/882#event-4513101747

,

or unsubscribe
<

https://github.com/notifications/unsubscribe-auth/ARRDMC2KQYG5FVP7OF54Z53TFRQQVANCNFSM4ZHEPDJQ

.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub<

https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-860902526

,
or unsubscribe<

https://github.com/notifications/unsubscribe-auth/AGNXISSHV4V7LSD3ZI2TQSTTSZDQXANCNFSM4ZHEPDJQ

.

IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy
the
information in any medium. Thank you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<

https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-863185227

,
or unsubscribe
<

https://github.com/notifications/unsubscribe-auth/ARRDMC6MGOTC7JHJOXE7NFTTTHQ2XANCNFSM4ZHEPDJQ

.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub<

https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-865332541

,
or unsubscribe<

https://github.com/notifications/unsubscribe-auth/AGNXISU4USITBC3ODKTNV4LTT6QIRANCNFSM4ZHEPDJQ

.

IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy
the
information in any medium. Thank you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<
https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-865664696
,
or unsubscribe
<
https://github.com/notifications/unsubscribe-auth/ARRDMC7EFF42VJLI3ECM4LDTUA2TTANCNFSM4ZHEPDJQ
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub<
https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-866187809>,
or unsubscribe<
https://github.com/notifications/unsubscribe-auth/AGNXISUBGWRVL4IE2NWXNG3TUDCIJANCNFSM4ZHEPDJQ

.

IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy the
information in any medium. Thank you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-866297436,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ARRDMC6ZEHGM54QQBSQEDCLTUDURVANCNFSM4ZHEPDJQ
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-866479229, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGNXISRK34AZAZJCC7G3GLDTUFDKBANCNFSM4ZHEPDJQ.
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

Hello Sir,
Thank you for the response. It really helped.
Have a good day.
Sincerely,
Shraddha Dahal
On Sat, Jun 26, 2021 at 10:52 AM developer-compute @.***>
wrote:

Hi,
The graph api in ACL does not support that level of granularity, you must
run all the workloads on the backend specified with the --target option.
You cannot have some workloads on the CPU and others on the GPU.
Hope this helps.

From: Shraddhaa1 @.>
Sent: 23 June 2021 03:46
To: ARM-software/ComputeLibrary @.>
Cc: developer-compute @.>; Comment @.>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
Thank you for the reply. I can now observe time taken by each kernel of the
neural networks. Could you please help me know if there has been any update
on how specific layers of a neural network can be specified to either CPU
or GPU? With the repo:
https://github.com/adityagupta1089/ComputeLibrary.git
I could mix the use of CPU and GPU, but I am trying to specify layers to
CPU or GPU. Could you please help me understand how it can be obtained?
Have a good evening.
Sincerely,
Shraddha Dahal
On Tue, Jun 22, 2021 at 3:07 PM developer-compute @.***>
wrote:

Hi,
The benchmark examples are only available in the development repo:
https://review.mlplatform.org/admin/repos/ml%2FComputeLibrary
Try: git clone "https://review.mlplatform.org/ml/ComputeLibrary"
Then build with benchmark_examples=1
Hope this helps

From: Shraddhaa1 @.>
Sent: 22 June 2021 18:31
To: ARM-software/ComputeLibrary @.>
Cc: developer-compute @.>; Comment @.>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
Thank you for the reply. The file benchmark_graph_mobilenet, is not
generated anywhere inside the Compute Library. Could you please help me
know if the way I used to build ARM-CL with benchmarks_examples=1
correct?
Have a good day.
Sincerely,
Shraddha Dahal
On Tue, Jun 22, 2021 at 2:20 AM developer-compute @.***>
wrote:

Hi,
Please try running benchmark_graph_mobilenet instead of
graph_mobilenet.
Hope this helps.

From: Shraddhaa1 @.>
Sent: 21 June 2021 21:46
To: ARM-software/ComputeLibrary @.>
Cc: developer-compute @.>; Comment @.>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
Thank you for the response. I tried the first method you mentioned in
the
previous email.
In the makefile.arm file, I added benchmark_examples:=1 such as:
BUILD:=native
NEON:=1
OPENCL:=1
ARCH:=arm64-v8a
all: release
CFLAGS:=-std=c++14
benchmark_examples:=1
debug:
scons -j8 -Q arch=$(ARCH) build=$(BUILD) neon=$(NEON) opencl=$(OPENCL)
build_dir=debug debug=1 extra_cxx_flags=$(CFLAGS)
release:
scons -j8 -Q arch=$(ARCH) build=$(BUILD) neon=$(NEON) opencl=$(OPENCL)
build_dir=release debug=0 extra_cxx_flags=$(CFLAGS)
sched:
g++ -o build/release/examples/graph_temp_scheduler2.o -c
-Wno-deprecated-declarations -Wall -DARCH_ARM -Wextra
-Wno-unused-parameter
-pedantic -Wdisabled-optimization -Wformat=2 -Winit-self
-Wstrict-overflow=2 -Wswitch-default -fpermissive -std=gnu++11 -Wno-vla
-Woverloaded-virtual -Wctor-dtor-privacy -Wsign-promo -Weffc++
-Wno-format-nonliteral -Wno-overlength-strings -Wno-strict-overflow
-Wlogical-op -Wnoexcept -Wstrict-null-sentinel
-Wno-implicit-fallthrough
-march=armv8-a -Wno-ignored-attributes -Werror -O3 -ftree-vectorize
-std=c++14 -D_GLIBCXX_USE_NANOSLEEP -DARM_COMPUTE_CPP_SCHEDULER=1
-DARM_COMPUTE_AARCH64_V8A -DNO_DOT_IN_TOOLCHAIN -DEMBEDDED_KERNELS
-Iinclude -I. -I. examples/graph_temp_scheduler2.cpp
g++ -o build/release/examples/graph_temp_scheduler2
-Wl,--allow-shlib-undefined
build/release/examples/graph_temp_scheduler2.o
build/release/utils/Utils.o build/release/utils/GraphUtils.o
build/release/utils/CommonGraphOptions.o -Lbuild/release -L. -lpthread
-larm_compute_graph -larm_compute -larm_compute_core
After that, I gave commands:
$make all
$sudo LD_LIBRARY_PATH=/home/shunya/ComputeLibrary1/build/release
./build/release/examples/graph_mobilenet
--instruments=OPENCL_TIMESTAMPS_MS
--example_args='--target=CL'
but, I could not see the outputs with time taken by each kernel. The
path
to library in the above command has libarm_compute.so,
libarm_compute_core.so, libarm_compute_graph.so,
libarm_compute_core-static.a, libarm_compute_graph-static.a and
libarm_compute-static.a. Could you please help me know where I am going
wrong?
Have a good day.
Sincerely,
Shraddha Dahal
On Thu, Jun 17, 2021 at 7:10 AM developer-compute @.***>
wrote:

Hi Shraddha,
If you build ACL with the option benchmark_examples=1 you can then
run the network and use the instruments to see how much time each
kernel
is taking:
@.***_hikey_dev:~/tmp/pabtel01#
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:./acl_neon+cl_release/
./benchmark_graph_mobilenet --instruments=OPENCL_TIMESTAMPS_MS
--example_args='--target=CL'
This will produce output like below:
OpenCLTimestamps/Now OpenCL: AVG=1121307039 ms
OpenCLTimestamps/Now Wall clock: AVG=1623931494169279 us

OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/gemm_mm_floating_point_f32_bifrost

GWS[8,3136,1] LWS[4,1,1] #2: AVG=1121306920 ms
OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/im2col3x3_nhwc
GWS[2,12544,1] #1: AVG=1121306919 ms
OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/reshape_to_columns
GWS[3,3,3] #0: AVG=1121306918 ms

OpenCLTimestamps/[end]Conv2d_10_depthwise/depthwise+Conv2d_10_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[512,7,7] #36: AVG=1121306986 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[128,64,1] #40: AVG=1121306995 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[128,40,1] #39: AVG=1121306989 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[128,128,1] #38: AVG=1121306988 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/reshape_to_columns

GWS[512,1,1] #37: AVG=1121306988 ms

OpenCLTimestamps/[end]Conv2d_11_depthwise/depthwise+Conv2d_11_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[512,7,7] #41: AVG=1121306995 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[128,64,1] #45: AVG=1121307003 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[128,40,1] #44: AVG=1121306998 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[128,128,1] #43: AVG=1121306997 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/reshape_to_columns

GWS[512,1,1] #42: AVG=1121306997 ms

OpenCLTimestamps/[end]Conv2d_12_depthwise/depthwise+Conv2d_12_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc

GWS[512,7,7] #46: AVG=1121307004 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[256,16,1] #50: AVG=1121307012 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[128,10,1] #49: AVG=1121307008 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[256,128,1] #48: AVG=1121307008 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/reshape_to_columns

GWS[512,1,1] #47: AVG=1121307007 ms

OpenCLTimestamps/[end]Conv2d_13_depthwise/depthwise+Conv2d_13_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[1024,4,4] #51: AVG=1121307012 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[256,16,1] #55: AVG=1121307025 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[256,10,1] #54: AVG=1121307018 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[256,256,1] #53: AVG=1121307018 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/reshape_to_columns

GWS[1024,1,1] #52: AVG=1121307016 ms

OpenCLTimestamps/[end]Conv2d_1_depthwise/depthwise+Conv2d_1_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[32,56,56] #3: AVG=1121306921 ms

OpenCLTimestamps/[end]Conv2d_1_pointwise/Conv2D+Conv2d_1_pointwise/BatchNorm/gemm_mm_floating_point_f32_bifrost

GWS[16,28,112] LWS[4,1,1] #5: AVG=1121306924 ms

OpenCLTimestamps/[end]Conv2d_1_pointwise/Conv2D+Conv2d_1_pointwise/BatchNorm/reshape_to_columns

GWS[32,1,1] #4: AVG=1121306922 ms

OpenCLTimestamps/[end]Conv2d_2_depthwise/depthwise+Conv2d_2_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc

GWS[64,56,56] #6: AVG=1121306926 ms

OpenCLTimestamps/[end]Conv2d_2_pointwise/Conv2D+Conv2d_2_pointwise/BatchNorm/gemm_mm_floating_point_f32_bifrost

GWS[32,14,56] LWS[4,1,1] #8: AVG=1121306929 ms
Another alternative is to use Armnn's ExecuteNetwork to run a tfllite
model and use the -e option which will make the tool output the time
consumed by each kernel
For more information about ExecuteNetwork see:

https://github.com/ARM-software/armnn/tree/branches/armnn_21_02/tests/ExecuteNetwork

Hope this helps.

From: Shraddhaa1 @.>
Sent: 14 June 2021 19:33
To: ARM-software/ComputeLibrary @.>
Cc: Subscribed @.***>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
I had contacted you a few weeks ago regarding the Execution of
Inference
Workloads on Hikey970 with layer splitting. Could you please help me
know
if there is any function specific in ARM-CL which would let me
calculate
inference time taken by each layer of a neural network?
Thank you.
Sincerely,
Shraddha Dahal
On Fri, Apr 9, 2021 at 12:06 PM Shraddha Dahal @.***>
wrote:

Hello Sir,
Thank you for the reply. That really helped.
Have a good day.
Sincerely,
Shraddha Dahal
On Fri, Mar 26, 2021 at 5:44 AM Pablo Marquez Tello <
@.***> wrote:

Closed #882 <
https://github.com/ARM-software/ComputeLibrary/issues/882
.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<

https://github.com/ARM-software/ComputeLibrary/issues/882#event-4513101747

,

or unsubscribe
<

https://github.com/notifications/unsubscribe-auth/ARRDMC2KQYG5FVP7OF54Z53TFRQQVANCNFSM4ZHEPDJQ

.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub<

https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-860902526

,
or unsubscribe<

https://github.com/notifications/unsubscribe-auth/AGNXISSHV4V7LSD3ZI2TQSTTSZDQXANCNFSM4ZHEPDJQ

.

IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose
the
contents to any other person, use it for any purpose, or store or
copy
the
information in any medium. Thank you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<

https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-863185227

,
or unsubscribe
<

https://github.com/notifications/unsubscribe-auth/ARRDMC6MGOTC7JHJOXE7NFTTTHQ2XANCNFSM4ZHEPDJQ

.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub<

https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-865332541

,
or unsubscribe<

https://github.com/notifications/unsubscribe-auth/AGNXISU4USITBC3ODKTNV4LTT6QIRANCNFSM4ZHEPDJQ

.

IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy
the
information in any medium. Thank you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<

https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-865664696

,
or unsubscribe
<

https://github.com/notifications/unsubscribe-auth/ARRDMC7EFF42VJLI3ECM4LDTUA2TTANCNFSM4ZHEPDJQ

.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub<

https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-866187809

,
or unsubscribe<

https://github.com/notifications/unsubscribe-auth/AGNXISUBGWRVL4IE2NWXNG3TUDCIJANCNFSM4ZHEPDJQ

.

IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy
the
information in any medium. Thank you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<
https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-866297436
,
or unsubscribe
<
https://github.com/notifications/unsubscribe-auth/ARRDMC6ZEHGM54QQBSQEDCLTUDURVANCNFSM4ZHEPDJQ
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub<
https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-866479229>,
or unsubscribe<
https://github.com/notifications/unsubscribe-auth/AGNXISRK34AZAZJCC7G3GLDTUFDKBANCNFSM4ZHEPDJQ

.

IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy the
information in any medium. Thank you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-869021541,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ARRDMC3JFBHEPA7T7A67TWTTUXZVRANCNFSM4ZHEPDJQ
.

Hello Sir,
I was working on obtaining time taken by each kernel with target CL for the
available networks under development repo, as you suggested in previous
emails. Could you please help me know if there are any such instruments
available, which would help me know the time taken by each kernel when
assigning benchmark networks under CPU? I am currently using the command
mentioned below for target CL:
sudo LD_LIBRARY_PATH=/home/shunya/ComputeLibrary/build
./build/tests/benchmark_graph_mobilenet_v2
--instruments=OPENCL_TIMESTAMPS_MS --example_args='--target=CL'
Could you please help me know how I can obtain similar information with
target NEON?
Have a good day.
Sincerely,
Shraddha Dahal
On Thu, Jul 1, 2021 at 2:34 PM Shraddha Dahal @.***>
wrote:

Hello Sir,
Thank you for the response. It really helped.
Have a good day.
Sincerely,
Shraddha Dahal
On Sat, Jun 26, 2021 at 10:52 AM developer-compute <
@.***> wrote:

Hi,
The graph api in ACL does not support that level of granularity, you must
run all the workloads on the backend specified with the --target option.
You cannot have some workloads on the CPU and others on the GPU.
Hope this helps.

From: Shraddhaa1 @.>
Sent: 23 June 2021 03:46
To: ARM-software/ComputeLibrary @.>
Cc: developer-compute @.>; Comment @.>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
Thank you for the reply. I can now observe time taken by each kernel of
the
neural networks. Could you please help me know if there has been any
update
on how specific layers of a neural network can be specified to either CPU
or GPU? With the repo:
https://github.com/adityagupta1089/ComputeLibrary.git
I could mix the use of CPU and GPU, but I am trying to specify layers to
CPU or GPU. Could you please help me understand how it can be obtained?
Have a good evening.
Sincerely,
Shraddha Dahal
On Tue, Jun 22, 2021 at 3:07 PM developer-compute @.***>
wrote:

Hi,
The benchmark examples are only available in the development repo:
https://review.mlplatform.org/admin/repos/ml%2FComputeLibrary
Try: git clone "https://review.mlplatform.org/ml/ComputeLibrary"
Then build with benchmark_examples=1
Hope this helps

From: Shraddhaa1 @.>
Sent: 22 June 2021 18:31
To: ARM-software/ComputeLibrary @.>
Cc: developer-compute @.>; Comment @.>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
Thank you for the reply. The file benchmark_graph_mobilenet, is not
generated anywhere inside the Compute Library. Could you please help me
know if the way I used to build ARM-CL with benchmarks_examples=1
correct?
Have a good day.
Sincerely,
Shraddha Dahal
On Tue, Jun 22, 2021 at 2:20 AM developer-compute @.***>
wrote:

Hi,
Please try running benchmark_graph_mobilenet instead of
graph_mobilenet.
Hope this helps.

From: Shraddhaa1 @.>
Sent: 21 June 2021 21:46
To: ARM-software/ComputeLibrary @.>
Cc: developer-compute @.>; Comment @.>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
Thank you for the response. I tried the first method you mentioned in
the
previous email.
In the makefile.arm file, I added benchmark_examples:=1 such as:
BUILD:=native
NEON:=1
OPENCL:=1
ARCH:=arm64-v8a
all: release
CFLAGS:=-std=c++14
benchmark_examples:=1
debug:
scons -j8 -Q arch=$(ARCH) build=$(BUILD) neon=$(NEON) opencl=$(OPENCL)
build_dir=debug debug=1 extra_cxx_flags=$(CFLAGS)
release:
scons -j8 -Q arch=$(ARCH) build=$(BUILD) neon=$(NEON) opencl=$(OPENCL)
build_dir=release debug=0 extra_cxx_flags=$(CFLAGS)
sched:
g++ -o build/release/examples/graph_temp_scheduler2.o -c
-Wno-deprecated-declarations -Wall -DARCH_ARM -Wextra
-Wno-unused-parameter
-pedantic -Wdisabled-optimization -Wformat=2 -Winit-self
-Wstrict-overflow=2 -Wswitch-default -fpermissive -std=gnu++11
-Wno-vla
-Woverloaded-virtual -Wctor-dtor-privacy -Wsign-promo -Weffc++
-Wno-format-nonliteral -Wno-overlength-strings -Wno-strict-overflow
-Wlogical-op -Wnoexcept -Wstrict-null-sentinel
-Wno-implicit-fallthrough
-march=armv8-a -Wno-ignored-attributes -Werror -O3 -ftree-vectorize
-std=c++14 -D_GLIBCXX_USE_NANOSLEEP -DARM_COMPUTE_CPP_SCHEDULER=1
-DARM_COMPUTE_AARCH64_V8A -DNO_DOT_IN_TOOLCHAIN -DEMBEDDED_KERNELS
-Iinclude -I. -I. examples/graph_temp_scheduler2.cpp
g++ -o build/release/examples/graph_temp_scheduler2
-Wl,--allow-shlib-undefined
build/release/examples/graph_temp_scheduler2.o
build/release/utils/Utils.o build/release/utils/GraphUtils.o
build/release/utils/CommonGraphOptions.o -Lbuild/release -L. -lpthread
-larm_compute_graph -larm_compute -larm_compute_core
After that, I gave commands:
$make all
$sudo LD_LIBRARY_PATH=/home/shunya/ComputeLibrary1/build/release
./build/release/examples/graph_mobilenet
--instruments=OPENCL_TIMESTAMPS_MS
--example_args='--target=CL'
but, I could not see the outputs with time taken by each kernel. The
path
to library in the above command has libarm_compute.so,
libarm_compute_core.so, libarm_compute_graph.so,
libarm_compute_core-static.a, libarm_compute_graph-static.a and
libarm_compute-static.a. Could you please help me know where I am
going
wrong?
Have a good day.
Sincerely,
Shraddha Dahal
On Thu, Jun 17, 2021 at 7:10 AM developer-compute @.***>
wrote:

Hi Shraddha,
If you build ACL with the option benchmark_examples=1 you can then
run the network and use the instruments to see how much time each
kernel
is taking:
@.***_hikey_dev:~/tmp/pabtel01#
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:./acl_neon+cl_release/
./benchmark_graph_mobilenet --instruments=OPENCL_TIMESTAMPS_MS
--example_args='--target=CL'
This will produce output like below:
OpenCLTimestamps/Now OpenCL: AVG=1121307039 ms
OpenCLTimestamps/Now Wall clock: AVG=1623931494169279 us

OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/gemm_mm_floating_point_f32_bifrost

GWS[8,3136,1] LWS[4,1,1] #2: AVG=1121306920 ms
OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/im2col3x3_nhwc
GWS[2,12544,1] #1: AVG=1121306919 ms
OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/reshape_to_columns
GWS[3,3,3] #0: AVG=1121306918 ms

OpenCLTimestamps/[end]Conv2d_10_depthwise/depthwise+Conv2d_10_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[512,7,7] #36: AVG=1121306986 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[128,64,1] #40: AVG=1121306995 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[128,40,1] #39: AVG=1121306989 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[128,128,1] #38: AVG=1121306988 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/reshape_to_columns

GWS[512,1,1] #37: AVG=1121306988 ms

OpenCLTimestamps/[end]Conv2d_11_depthwise/depthwise+Conv2d_11_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[512,7,7] #41: AVG=1121306995 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[128,64,1] #45: AVG=1121307003 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[128,40,1] #44: AVG=1121306998 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[128,128,1] #43: AVG=1121306997 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/reshape_to_columns

GWS[512,1,1] #42: AVG=1121306997 ms

OpenCLTimestamps/[end]Conv2d_12_depthwise/depthwise+Conv2d_12_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc

GWS[512,7,7] #46: AVG=1121307004 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[256,16,1] #50: AVG=1121307012 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[128,10,1] #49: AVG=1121307008 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[256,128,1] #48: AVG=1121307008 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/reshape_to_columns

GWS[512,1,1] #47: AVG=1121307007 ms

OpenCLTimestamps/[end]Conv2d_13_depthwise/depthwise+Conv2d_13_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[1024,4,4] #51: AVG=1121307012 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[256,16,1] #55: AVG=1121307025 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[256,10,1] #54: AVG=1121307018 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[256,256,1] #53: AVG=1121307018 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/reshape_to_columns

GWS[1024,1,1] #52: AVG=1121307016 ms

OpenCLTimestamps/[end]Conv2d_1_depthwise/depthwise+Conv2d_1_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[32,56,56] #3: AVG=1121306921 ms

OpenCLTimestamps/[end]Conv2d_1_pointwise/Conv2D+Conv2d_1_pointwise/BatchNorm/gemm_mm_floating_point_f32_bifrost

GWS[16,28,112] LWS[4,1,1] #5: AVG=1121306924 ms

OpenCLTimestamps/[end]Conv2d_1_pointwise/Conv2D+Conv2d_1_pointwise/BatchNorm/reshape_to_columns

GWS[32,1,1] #4: AVG=1121306922 ms

OpenCLTimestamps/[end]Conv2d_2_depthwise/depthwise+Conv2d_2_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc

GWS[64,56,56] #6: AVG=1121306926 ms

OpenCLTimestamps/[end]Conv2d_2_pointwise/Conv2D+Conv2d_2_pointwise/BatchNorm/gemm_mm_floating_point_f32_bifrost

GWS[32,14,56] LWS[4,1,1] #8: AVG=1121306929 ms
Another alternative is to use Armnn's ExecuteNetwork to run a
tfllite
model and use the -e option which will make the tool output the time
consumed by each kernel
For more information about ExecuteNetwork see:

https://github.com/ARM-software/armnn/tree/branches/armnn_21_02/tests/ExecuteNetwork

Hope this helps.

From: Shraddhaa1 @.>
Sent: 14 June 2021 19:33
To: ARM-software/ComputeLibrary @.>
Cc: Subscribed @.***>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
I had contacted you a few weeks ago regarding the Execution of
Inference
Workloads on Hikey970 with layer splitting. Could you please help me
know
if there is any function specific in ARM-CL which would let me
calculate
inference time taken by each layer of a neural network?
Thank you.
Sincerely,
Shraddha Dahal
On Fri, Apr 9, 2021 at 12:06 PM Shraddha Dahal @.***>
wrote:

Hello Sir,
Thank you for the reply. That really helped.
Have a good day.
Sincerely,
Shraddha Dahal
On Fri, Mar 26, 2021 at 5:44 AM Pablo Marquez Tello <
@.***> wrote:

Closed #882 <
https://github.com/ARM-software/ComputeLibrary/issues/882
.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<

https://github.com/ARM-software/ComputeLibrary/issues/882#event-4513101747

,

or unsubscribe
<

https://github.com/notifications/unsubscribe-auth/ARRDMC2KQYG5FVP7OF54Z53TFRQQVANCNFSM4ZHEPDJQ

.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub<

https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-860902526

,
or unsubscribe<

https://github.com/notifications/unsubscribe-auth/AGNXISSHV4V7LSD3ZI2TQSTTSZDQXANCNFSM4ZHEPDJQ

.

IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose
the
contents to any other person, use it for any purpose, or store or
copy
the
information in any medium. Thank you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<

https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-863185227

,
or unsubscribe
<

https://github.com/notifications/unsubscribe-auth/ARRDMC6MGOTC7JHJOXE7NFTTTHQ2XANCNFSM4ZHEPDJQ

.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub<

https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-865332541

,
or unsubscribe<

https://github.com/notifications/unsubscribe-auth/AGNXISU4USITBC3ODKTNV4LTT6QIRANCNFSM4ZHEPDJQ

.

IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose
the
contents to any other person, use it for any purpose, or store or copy
the
information in any medium. Thank you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<

https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-865664696

,
or unsubscribe
<

https://github.com/notifications/unsubscribe-auth/ARRDMC7EFF42VJLI3ECM4LDTUA2TTANCNFSM4ZHEPDJQ

.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub<

https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-866187809

,
or unsubscribe<

https://github.com/notifications/unsubscribe-auth/AGNXISUBGWRVL4IE2NWXNG3TUDCIJANCNFSM4ZHEPDJQ

.

IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy
the
information in any medium. Thank you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<
https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-866297436
,
or unsubscribe
<
https://github.com/notifications/unsubscribe-auth/ARRDMC6ZEHGM54QQBSQEDCLTUDURVANCNFSM4ZHEPDJQ
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub<
https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-866479229>,
or unsubscribe<
https://github.com/notifications/unsubscribe-auth/AGNXISRK34AZAZJCC7G3GLDTUFDKBANCNFSM4ZHEPDJQ

.

IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy the
information in any medium. Thank you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-869021541,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ARRDMC3JFBHEPA7T7A67TWTTUXZVRANCNFSM4ZHEPDJQ
.

From: Shraddhaa1 @.>
Sent: 09 July 2021 20:39
To: ARM-software/ComputeLibrary @.>
Cc: developer-compute @.>; Comment @.>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
I was working on obtaining time taken by each kernel with target CL for the
available networks under development repo, as you suggested in previous
emails. Could you please help me know if there are any such instruments
available, which would help me know the time taken by each kernel when
assigning benchmark networks under CPU? I am currently using the command
mentioned below for target CL:
sudo LD_LIBRARY_PATH=/home/shunya/ComputeLibrary/build
./build/tests/benchmark_graph_mobilenet_v2
--instruments=OPENCL_TIMESTAMPS_MS --example_args='--target=CL'
Could you please help me know how I can obtain similar information with
target NEON?
Have a good day.
Sincerely,
Shraddha Dahal
On Thu, Jul 1, 2021 at 2:34 PM Shraddha Dahal @.***>
wrote:

Hello Sir,
Thank you for the response. It really helped.
Have a good day.
Sincerely,
Shraddha Dahal
On Sat, Jun 26, 2021 at 10:52 AM developer-compute <
@.***> wrote:

Hi,
The graph api in ACL does not support that level of granularity, you must
run all the workloads on the backend specified with the --target option.
You cannot have some workloads on the CPU and others on the GPU.
Hope this helps.

From: Shraddhaa1 @.>
Sent: 23 June 2021 03:46
To: ARM-software/ComputeLibrary @.>
Cc: developer-compute @.>; Comment @.>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
Thank you for the reply. I can now observe time taken by each kernel of
the
neural networks. Could you please help me know if there has been any
update
on how specific layers of a neural network can be specified to either CPU
or GPU? With the repo:
https://github.com/adityagupta1089/ComputeLibrary.git
I could mix the use of CPU and GPU, but I am trying to specify layers to
CPU or GPU. Could you please help me understand how it can be obtained?
Have a good evening.
Sincerely,
Shraddha Dahal
On Tue, Jun 22, 2021 at 3:07 PM developer-compute @.***>
wrote:

Hi,
The benchmark examples are only available in the development repo:
https://review.mlplatform.org/admin/repos/ml%2FComputeLibrary
Try: git clone "https://review.mlplatform.org/ml/ComputeLibrary"
Then build with benchmark_examples=1
Hope this helps

From: Shraddhaa1 @.>
Sent: 22 June 2021 18:31
To: ARM-software/ComputeLibrary @.>
Cc: developer-compute @.>; Comment @.>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
Thank you for the reply. The file benchmark_graph_mobilenet, is not
generated anywhere inside the Compute Library. Could you please help me
know if the way I used to build ARM-CL with benchmarks_examples=1
correct?
Have a good day.
Sincerely,
Shraddha Dahal
On Tue, Jun 22, 2021 at 2:20 AM developer-compute @.***>
wrote:

Hi,
Please try running benchmark_graph_mobilenet instead of
graph_mobilenet.
Hope this helps.

From: Shraddhaa1 @.>
Sent: 21 June 2021 21:46
To: ARM-software/ComputeLibrary @.>
Cc: developer-compute @.>; Comment @.>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
Thank you for the response. I tried the first method you mentioned in
the
previous email.
In the makefile.arm file, I added benchmark_examples:=1 such as:
BUILD:=native
NEON:=1
OPENCL:=1
ARCH:=arm64-v8a
all: release
CFLAGS:=-std=c++14
benchmark_examples:=1
debug:
scons -j8 -Q arch=$(ARCH) build=$(BUILD) neon=$(NEON) opencl=$(OPENCL)
build_dir=debug debug=1 extra_cxx_flags=$(CFLAGS)
release:
scons -j8 -Q arch=$(ARCH) build=$(BUILD) neon=$(NEON) opencl=$(OPENCL)
build_dir=release debug=0 extra_cxx_flags=$(CFLAGS)
sched:
g++ -o build/release/examples/graph_temp_scheduler2.o -c
-Wno-deprecated-declarations -Wall -DARCH_ARM -Wextra
-Wno-unused-parameter
-pedantic -Wdisabled-optimization -Wformat=2 -Winit-self
-Wstrict-overflow=2 -Wswitch-default -fpermissive -std=gnu++11
-Wno-vla
-Woverloaded-virtual -Wctor-dtor-privacy -Wsign-promo -Weffc++
-Wno-format-nonliteral -Wno-overlength-strings -Wno-strict-overflow
-Wlogical-op -Wnoexcept -Wstrict-null-sentinel
-Wno-implicit-fallthrough
-march=armv8-a -Wno-ignored-attributes -Werror -O3 -ftree-vectorize
-std=c++14 -D_GLIBCXX_USE_NANOSLEEP -DARM_COMPUTE_CPP_SCHEDULER=1
-DARM_COMPUTE_AARCH64_V8A -DNO_DOT_IN_TOOLCHAIN -DEMBEDDED_KERNELS
-Iinclude -I. -I. examples/graph_temp_scheduler2.cpp
g++ -o build/release/examples/graph_temp_scheduler2
-Wl,--allow-shlib-undefined
build/release/examples/graph_temp_scheduler2.o
build/release/utils/Utils.o build/release/utils/GraphUtils.o
build/release/utils/CommonGraphOptions.o -Lbuild/release -L. -lpthread
-larm_compute_graph -larm_compute -larm_compute_core
After that, I gave commands:
$make all
$sudo LD_LIBRARY_PATH=/home/shunya/ComputeLibrary1/build/release
./build/release/examples/graph_mobilenet
--instruments=OPENCL_TIMESTAMPS_MS
--example_args='--target=CL'
but, I could not see the outputs with time taken by each kernel. The
path
to library in the above command has libarm_compute.so,
libarm_compute_core.so, libarm_compute_graph.so,
libarm_compute_core-static.a, libarm_compute_graph-static.a and
libarm_compute-static.a. Could you please help me know where I am
going
wrong?
Have a good day.
Sincerely,
Shraddha Dahal
On Thu, Jun 17, 2021 at 7:10 AM developer-compute @.***>
wrote:

Hi Shraddha,
If you build ACL with the option benchmark_examples=1 you can then
run the network and use the instruments to see how much time each
kernel
is taking:
@.***_hikey_dev:~/tmp/pabtel01#
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:./acl_neon+cl_release/
./benchmark_graph_mobilenet --instruments=OPENCL_TIMESTAMPS_MS
--example_args='--target=CL'
This will produce output like below:
OpenCLTimestamps/Now OpenCL: AVG=1121307039 ms
OpenCLTimestamps/Now Wall clock: AVG=1623931494169279 us

OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/gemm_mm_floating_point_f32_bifrost

GWS[8,3136,1] LWS[4,1,1] #2: AVG=1121306920 ms
OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/im2col3x3_nhwc
GWS[2,12544,1] #1: AVG=1121306919 ms
OpenCLTimestamps/[end]Conv2d_0+Conv2d_0/BatchNorm/reshape_to_columns
GWS[3,3,3] #0: AVG=1121306918 ms

OpenCLTimestamps/[end]Conv2d_10_depthwise/depthwise+Conv2d_10_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[512,7,7] #36: AVG=1121306986 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[128,64,1] #40: AVG=1121306995 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[128,40,1] #39: AVG=1121306989 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[128,128,1] #38: AVG=1121306988 ms

OpenCLTimestamps/[end]Conv2d_10_pointwise/Conv2D+Conv2d_10_pointwise/BatchNorm/reshape_to_columns

GWS[512,1,1] #37: AVG=1121306988 ms

OpenCLTimestamps/[end]Conv2d_11_depthwise/depthwise+Conv2d_11_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[512,7,7] #41: AVG=1121306995 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[128,64,1] #45: AVG=1121307003 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[128,40,1] #44: AVG=1121306998 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[128,128,1] #43: AVG=1121306997 ms

OpenCLTimestamps/[end]Conv2d_11_pointwise/Conv2D+Conv2d_11_pointwise/BatchNorm/reshape_to_columns

GWS[512,1,1] #42: AVG=1121306997 ms

OpenCLTimestamps/[end]Conv2d_12_depthwise/depthwise+Conv2d_12_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc

GWS[512,7,7] #46: AVG=1121307004 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[256,16,1] #50: AVG=1121307012 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[128,10,1] #49: AVG=1121307008 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[256,128,1] #48: AVG=1121307008 ms

OpenCLTimestamps/[end]Conv2d_12_pointwise/Conv2D+Conv2d_12_pointwise/BatchNorm/reshape_to_columns

GWS[512,1,1] #47: AVG=1121307007 ms

OpenCLTimestamps/[end]Conv2d_13_depthwise/depthwise+Conv2d_13_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[1024,4,4] #51: AVG=1121307012 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_mm_reshaped_lhs_nt_rhs_t

GWS[256,16,1] #55: AVG=1121307025 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_reshape_lhs_matrix_nt

GWS[256,10,1] #54: AVG=1121307018 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/gemm_reshape_rhs_matrix_t

GWS[256,256,1] #53: AVG=1121307018 ms

OpenCLTimestamps/[end]Conv2d_13_pointwise/Conv2D+Conv2d_13_pointwise/BatchNorm/reshape_to_columns

GWS[1024,1,1] #52: AVG=1121307016 ms

OpenCLTimestamps/[end]Conv2d_1_depthwise/depthwise+Conv2d_1_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc_stride1

GWS[32,56,56] #3: AVG=1121306921 ms

OpenCLTimestamps/[end]Conv2d_1_pointwise/Conv2D+Conv2d_1_pointwise/BatchNorm/gemm_mm_floating_point_f32_bifrost

GWS[16,28,112] LWS[4,1,1] #5: AVG=1121306924 ms

OpenCLTimestamps/[end]Conv2d_1_pointwise/Conv2D+Conv2d_1_pointwise/BatchNorm/reshape_to_columns

GWS[32,1,1] #4: AVG=1121306922 ms

OpenCLTimestamps/[end]Conv2d_2_depthwise/depthwise+Conv2d_2_depthwise/BatchNorm/depthwise_convolution_3x3_nhwc

GWS[64,56,56] #6: AVG=1121306926 ms

OpenCLTimestamps/[end]Conv2d_2_pointwise/Conv2D+Conv2d_2_pointwise/BatchNorm/gemm_mm_floating_point_f32_bifrost

GWS[32,14,56] LWS[4,1,1] #8: AVG=1121306929 ms
Another alternative is to use Armnn's ExecuteNetwork to run a
tfllite
model and use the -e option which will make the tool output the time
consumed by each kernel
For more information about ExecuteNetwork see:

https://github.com/ARM-software/armnn/tree/branches/armnn_21_02/tests/ExecuteNetwork

Hope this helps.

From: Shraddhaa1 @.>
Sent: 14 June 2021 19:33
To: ARM-software/ComputeLibrary @.>
Cc: Subscribed @.***>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
I had contacted you a few weeks ago regarding the Execution of
Inference
Workloads on Hikey970 with layer splitting. Could you please help me
know
if there is any function specific in ARM-CL which would let me
calculate
inference time taken by each layer of a neural network?
Thank you.
Sincerely,
Shraddha Dahal
On Fri, Apr 9, 2021 at 12:06 PM Shraddha Dahal @.***>
wrote:

Hello Sir,
Thank you for the reply. That really helped.
Have a good day.
Sincerely,
Shraddha Dahal
On Fri, Mar 26, 2021 at 5:44 AM Pablo Marquez Tello <
@.***> wrote:

Closed #882 <
https://github.com/ARM-software/ComputeLibrary/issues/882
.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<

https://github.com/ARM-software/ComputeLibrary/issues/882#event-4513101747

,

or unsubscribe
<

https://github.com/notifications/unsubscribe-auth/ARRDMC2KQYG5FVP7OF54Z53TFRQQVANCNFSM4ZHEPDJQ

.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub<

https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-860902526

,
or unsubscribe<

https://github.com/notifications/unsubscribe-auth/AGNXISSHV4V7LSD3ZI2TQSTTSZDQXANCNFSM4ZHEPDJQ

.

IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose
the
contents to any other person, use it for any purpose, or store or
copy
the
information in any medium. Thank you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<

https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-863185227

,
or unsubscribe
<

https://github.com/notifications/unsubscribe-auth/ARRDMC6MGOTC7JHJOXE7NFTTTHQ2XANCNFSM4ZHEPDJQ

.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub<

https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-865332541

,
or unsubscribe<

https://github.com/notifications/unsubscribe-auth/AGNXISU4USITBC3ODKTNV4LTT6QIRANCNFSM4ZHEPDJQ

.

IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose
the
contents to any other person, use it for any purpose, or store or copy
the
information in any medium. Thank you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<

https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-865664696

,
or unsubscribe
<

https://github.com/notifications/unsubscribe-auth/ARRDMC7EFF42VJLI3ECM4LDTUA2TTANCNFSM4ZHEPDJQ

.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub<

https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-866187809

,
or unsubscribe<

https://github.com/notifications/unsubscribe-auth/AGNXISUBGWRVL4IE2NWXNG3TUDCIJANCNFSM4ZHEPDJQ

.

IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy
the
information in any medium. Thank you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<
https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-866297436
,
or unsubscribe
<
https://github.com/notifications/unsubscribe-auth/ARRDMC6ZEHGM54QQBSQEDCLTUDURVANCNFSM4ZHEPDJQ
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub<
https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-866479229>,
or unsubscribe<
https://github.com/notifications/unsubscribe-auth/AGNXISRK34AZAZJCC7G3GLDTUFDKBANCNFSM4ZHEPDJQ

.

IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy the
information in any medium. Thank you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-869021541,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ARRDMC3JFBHEPA7T7A67TWTTUXZVRANCNFSM4ZHEPDJQ
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/ARM-software/ComputeLibrary/issues/882#issuecomment-877414890, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGNXISSW55NW4TS74O3QJXDTW5GATANCNFSM4ZHEPDJQ.
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

Hello Sir,
Thank you for the response. It worked. I am also currently working on
accessing performance monitoring counters such as cache misses, IPC and
memory bandwidth. I was trying to check how memory intensive each neural
network is.  I was using perf tool for CPU profiling but the above
mentioned parameters are not supported in HiKey970 board. This can be
observed from the image attached below:
[image: perf_result.png]
The parameters were not on the perf list:
[image: perf_list.png]
Could you please help me know if there are tools which would help me obtain
these parameters when running a neural network?
Thanks again.
Have a good day.

From: Shraddhaa1 @.>
Sent: 09 July 2021 20:39
To: ARM-software/ComputeLibrary @.>
Cc: developer-compute @.>; Comment @.>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
I was working on obtaining time taken by each kernel with target CL for the
available networks under development repo, as you suggested in previous
emails. Could you please help me know if there are any such instruments
available, which would help me know the time taken by each kernel when
assigning benchmark networks under CPU? I am currently using the command
mentioned below for target CL:
sudo LD_LIBRARY_PATH=/home/shunya/ComputeLibrary/build
./build/tests/benchmark_graph_mobilenet_v2
--instruments=OPENCL_TIMESTAMPS_MS --example_args='--target=CL'
Could you please help me know how I can obtain similar information with
target NEON?
Have a good day.
Sincerely,
Shraddha Dahal
On Thu, Jul 1, 2021 at 2:34 PM Shraddha Dahal @.***>
wrote:

Hello Sir,
Thank you for the response. It really helped.
Have a good day.
Sincerely,
Shraddha Dahal
On Sat, Jun 26, 2021 at 10:52 AM developer-compute <
@.***> wrote:

Hi,
The graph api in ACL does not support that level of granularity, you
must
run all the workloads on the backend specified with the --target option.
You cannot have some workloads on the CPU and others on the GPU.
Hope this helps.

From: Shraddhaa1 @.>
Sent: 23 June 2021 03:46
To: ARM-software/ComputeLibrary @.>
Cc: developer-compute @.>; Comment @.>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
Thank you for the reply. I can now observe time taken by each kernel of
the
neural networks. Could you please help me know if there has been any
update
on how specific layers of a neural network can be specified to either
CPU
or GPU? With the repo:
https://github.com/adityagupta1089/ComputeLibrary.git
I could mix the use of CPU and GPU, but I am trying to specify layers to
CPU or GPU. Could you please help me understand how it can be obtained?
Have a good evening.
Sincerely,
Shraddha Dahal
On Tue, Jun 22, 2021 at 3:07 PM developer-compute @.***>
wrote:

Hi,
The benchmark examples are only available in the development repo:
https://review.mlplatform.org/admin/repos/ml%2FComputeLibrary
Try: git clone "https://review.mlplatform.org/ml/ComputeLibrary"
Then build with benchmark_examples=1
Hope this helps

From: Shraddhaa1 @.>
Sent: 22 June 2021 18:31
To: ARM-software/ComputeLibrary @.>
Cc: developer-compute @.>; Comment @.>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
Thank you for the reply. The file benchmark_graph_mobilenet, is not
generated anywhere inside the Compute Library. Could you please help
me
know if the way I used to build ARM-CL with benchmarks_examples=1
correct?
Have a good day.
Sincerely,
Shraddha Dahal
On Tue, Jun 22, 2021 at 2:20 AM developer-compute @.***>
wrote:

Hi,
Please try running benchmark_graph_mobilenet instead of
graph_mobilenet.
Hope this helps.

From: Shraddhaa1 @.>
Sent: 21 June 2021 21:46
To: ARM-software/ComputeLibrary @.>
Cc: developer-compute @.>; Comment @.>
Subject: Re: [ARM-software/ComputeLibrary] Execution of Inference
Workloads on Hikey970 with layer splitting (#882)
Hello Sir,
Thank you for the response. I tried the first method you mentioned
in
the
previous email.
In the makefile.arm file, I added benchmark_examples:=1 such as:
BUILD:=native
NEON:=1
OPENCL:=1
ARCH:=arm64-v8a
all: release
CFLAGS:=-std=c++14
benchmark_examples:=1
debug:
scons -j8 -Q arch=$(ARCH) build=$(BUILD) neon=$(NEON)
opencl=$(OPENCL)
build_dir=debug debug=1 extra_cxx_flags=$(CFLAGS)
release:
scons -j8 -Q arch=$(ARCH) build=$(BUILD) neon=$(NEON)
opencl=$(OPENCL)
build_dir=release debug=0 extra_cxx_flags=$(CFLAGS)
sched:
g++ -o build/release/examples/graph_temp_scheduler2.o -c
-Wno-deprecated-declarations -Wall -DARCH_ARM -Wextra
-Wno-unused-parameter
-pedantic -Wdisabled-optimization -Wformat=2 -Winit-self
-Wstrict-overflow=2 -Wswitch-default -fpermissive -std=gnu++11
-Wno-vla
-Woverloaded-virtual -Wctor-dtor-privacy -Wsign-promo -Weffc++
-Wno-format-nonliteral -Wno-overlength-strings -Wno-strict-overflow
-Wlogical-op -Wnoexcept -Wstrict-null-sentinel
-Wno-implicit-fallthrough
-march=armv8-a -Wno-ignored-attributes -Werror -O3 -ftree-vectorize
-std=c++14 -D_GLIBCXX_USE_NANOSLEEP -DARM_COMPUTE_CPP_SCHEDULER=1