How to use AWS SDK C++ XRay in a AWS Lambda Layer implemented in C++ called by a Lambda function in Python?
Asked Answered
B

0

21

My team implemented a pipeline with Computer Vision (OpenCV) and a DNN in Tensorflow and Keras using C++. This pipeline is an AWS Layer used by an AWS Lambda Function implemented in Python and this Layer is invoked through Boost library.

As needed, it was also created a Docker container with all the requirements (OpenCV, Boost, Python 3.7, Serverless, etc.) to build the pipeline code (using CMake) and deploy, as well. Everything was working fine, the pipeline and the Lambda function were built and deployed with success.

Now, my challenge: To improve the performance of the pipeline, I would like to measure the duration of several steps of this pipeline and I was trying to do that using aws-sdk-cpp-xray.

Before I started coding, I added and built the aws-sdk-cpp in the same Docker container that I was using to build and deploy the pipeline (as a Layer) and the Lambda functions (Below is the piece I added in the Dockerfile):

# Build AWS SDK with BUILD_ONLY XRay
RUN mkdir /tmp/${AWS_SDK_CPP}/build
WORKDIR /tmp/${AWS_SDK_CPP}/build

RUN cmake -D CMAKE_BUILD_TYPE=RELEASE \
        -D BUILD_ONLY="xray" \
        -D TARGET_ARCH=LINUX \
        -D CMAKE_INSTALL_PREFIX=/usr/local \
        -D ENABLE_TESTING=OFF \
        -D SIMPLE_INSTALL=OFF \
        -D BUILD_SHARED_LIBS=ON \
        -D BUILD_DEPS=ON \
        ..


RUN make -j $(nproc) && make install

Then, I put the references of aws-sdk-cpp-xray in the CMakeList.txt of my C++ project (the pipeline) like below:

cmake_minimum_required(VERSION 3.4.1)
set( CMAKE_CXX_STANDARD 11 )

find_package(AWSSDK REQUIRED COMPONENTS xray)
add_definitions(-DUSE_IMPORT_EXPORT)

SET(GCC_COVERAGE_COMPILE_FLAGS "-rdynamic -O3 -ffunction-sections -fdata-sections")
add_definitions(${GCC_COVERAGE_COMPILE_FLAGS})

find_package(OpenCV REQUIRED)

set( LIB_FOLDER /lib/src )
link_libraries(lib ${OpenCV_LIBS} ${AWSSDK_LINK_LIBRARIES})
target_link_libraries(lib ${OpenCV_LIBS})

include_directories(${LIB_FOLDER}/include)

file(GLOB lib_src
    "${LIB_FOLDER}/**/**.cpp"
)
include_directories(${CMAKE_INSTALL_PREFIX})

file(GLOB aws_sdk_src
    "${CMAKE_INSTALL_PREFIX}/**/**/**.*"
)
add_library(lib SHARED ${lib_src} ${aws_sdk_src})

Finally, I wrote several codes using the aws-sdk-cpp-xray inside the pipeline project, but without success on coding (due to missing samples) and even with the Lambda Layer execution (when testing the X-Ray code).

Now, when I execute the Lambda function that uses this Layer (the pipeline), I'm getting the error below:

Runtime.ImportModuleError: Unable to import module 'functions/myfunction': /opt/lib/lib.so: undefined symbol: _ZTVN3Aws35AmazonSerializableWebServiceRequestE

  • myfunction is the Lambda Function in Python.
  • lib.so is the Layer (pipeline) in C++.

So... Looks like the execution issue is caused by the aws-sdk-cpp installing in the Dockerfile. Or maybe it's caused by the entries in the CMakeList.txt pipeline project file... Or even, it's missing to add some *.so, *.a or *.so* files into the Layer package. But, due to the lack of documentation/manual and nothing found about the errors above, I'm asking for help with some example about use of AWS X-Ray using C++ or even alternatives to implement this measurement.

Ps.: I already asked for examples in GitHub AWS SDK home, as well. Please, at least, can you vote there?

Thank you!

Benisch answered 26/2, 2020 at 17:54 Comment(1)
Looks like a horrible architecture... Suggestions: 1) Rename your dynmaic lib to something meaningful, so you make sure that lib.so contains your aws layer stuff and not something from a different build (like aws-sdk-cpp-xray build). 2) your lib.so contains "name mangled" functions, thus its a C++, not C .so. This complicates the import of functions from your .so. Check with nm -gD lib.so if your function is in your soAideaidedecamp

© 2022 - 2024 — McMap. All rights reserved.