Login/Register
Segmentation Fault in Aerospike Python Client
Asked Answered
Solved pythonsegmentation-faultaerospike
B

1

9

I am using Aerospike Python CLient to put and get some key-value pair:

aerospike_dca_client.put(key,bin)
...
...
bls_key_a = (AEROSPIKE_NAMESPACE,SET_NAME,bls_key)
hp_bid_str_a = aerospike_dca_client.get(bls_key_a)[3];
#Terminates at the line above

However after the put, my program terminates with SEGFAULT during Get command. Following is the stacktrace I have got from GDB:

Program received signal SIGSEGV, Segmentation fault.
as_shm_node_get (cluster=0x353bd90508, ns=0x7fffffffc851 "test", digest=0x7fffffffc8e1 "/\231!\221h\223\240\021KX\377\357O\346u\214q\356\267#\177", write=false, replica=AS_POLICY_REPLICA_MASTER)
    at src/main/aerospike/as_shm_cluster.c:431
    431 src/main/aerospike/as_shm_cluster.c: No such file or directory.
        in src/main/aerospike/as_shm_cluster.c
        Missing separate debuginfos, use: debuginfo-install blas-3.2.1-4.el6.x86_64
        (gdb) bt
#0  as_shm_node_get (cluster=0x353bd90508, ns=0x7fffffffc851 "test", digest=0x7fffffffc8e1 "/\231!\221h\223\240\021KX\377\357O\346u\214q\356\267#\177", write=false, replica=AS_POLICY_REPLICA_MASTER)
            at src/main/aerospike/as_shm_cluster.c:431
#1  0x00007fffdf219ce5 in as_node_get (cluster=0x353bd90508, err=0x7fffffffc920, cn=0x7fffffffc7c0, command=0x7fffffffc740 "\002\003", command_len=82, timeout_ms=1000, retry=1,
                parse_results_fn=0x7fffdf21a97b <as_command_parse_result>, parse_results_data=0x7fffffffc7e0) at src/include/aerospike/as_cluster.h:562
#2  as_command_execute (cluster=0x353bd90508, err=0x7fffffffc920, cn=0x7fffffffc7c0, command=0x7fffffffc740 "\002\003", command_len=82, timeout_ms=1000, retry=1,
                    parse_results_fn=0x7fffdf21a97b <as_command_parse_result>, parse_results_data=0x7fffffffc7e0) at src/main/aerospike/as_command.c:435
#3  0x00007fffdf211a67 in aerospike_key_get (as=0x17dead0, err=0x7fffffffc920, policy=0x17dfbb0, key=<value optimized out>, rec=0x7fffffffc848) at src/main/aerospike/aerospike_key.c:114
#4  0x00007fffdf1e4295 in AerospikeClient_Get_Invoke (self=0x720d30, py_key=0x7fffb0b05730, py_policy=<value optimized out>) at src/main/client/get.c:96
#5  0x00007fffdf1e44a0 in AerospikeClient_Get (self=0x720d30, args=<value optimized out>, kwds=<value optimized out>) at src/main/client/get.c:174
#6  0x00007ffff7d12f24 in call_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4033
#7  PyEval_EvalFrameEx (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:2679
#8  0x00007ffff7d1399e in fast_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4119
#9  call_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4054
#10 PyEval_EvalFrameEx (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:2679
#11 0x00007ffff7d14a9e in PyEval_EvalCodeEx (co=0x7ffff7b0ee30, globals=<value optimized out>, locals=<value optimized out>, args=<value optimized out>, argcount=1, kws=0x7ffff7bc6068, kwcount=0,
                        defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:3265
#12 0x00007ffff7c927c8 in function_call (func=0x7fffdf1936e0, arg=0x7fffb0af9450, kw=0x7fffde2dfd70) at Objects/funcobject.c:526
#13 0x00007ffff7c631a3 in PyObject_Call (func=0x7fffdf1936e0, arg=<value optimized out>, kw=<value optimized out>) at Objects/abstract.c:2529
#14 0x00007ffff7d11746 in ext_do_call (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4346
#15 PyEval_EvalFrameEx (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:2718
#16 0x00007ffff7d14a9e in PyEval_EvalCodeEx (co=0x7fffe041f6b0, globals=<value optimized out>, locals=<value optimized out>, args=<value optimized out>, argcount=1, kws=0x1e142b0, kwcount=0, defs=0x0,
                            defcount=0, closure=0x7fffde39bbe0) at Python/ceval.c:3265
#17 0x00007ffff7d12c52 in fast_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4129
#18 call_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4054
#19 PyEval_EvalFrameEx (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:2679
#20 0x00007ffff7d14a9e in PyEval_EvalCodeEx (co=0x7fffe042b1b0, globals=<value optimized out>, locals=<value optimized out>, args=<value optimized out>, argcount=4, kws=0x7fffcbc517e0, kwcount=0,
                                defs=0x7fffdffa8428, defcount=1, closure=0x0) at Python/ceval.c:3265
#21 0x00007ffff7d12c52 in fast_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4129
#22 call_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4054
#23 PyEval_EvalFrameEx (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:2679
#24 0x00007ffff7d1399e in fast_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4119
#25 call_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4054
#26 PyEval_EvalFrameEx (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:2679
#27 0x00007ffff7d14a9e in PyEval_EvalCodeEx (co=0x7fffe041f730, globals=<value optimized out>, locals=<value optimized out>, args=<value optimized out>, argcount=2, kws=0x1d6c958, kwcount=0, defs=0x0,
                                    defcount=0, closure=0x0) at Python/ceval.c:3265
#28 0x00007ffff7d12c52 in fast_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4129
#29 call_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4054
#30 PyEval_EvalFrameEx (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:2679
#31 0x00007ffff7d14a9e in PyEval_EvalCodeEx (co=0x7ffff1ac6030, globals=<value optimized out>, locals=<value optimized out>, args=<value optimized out>, argcount=1, kws=0x7fffde5b85b8, kwcount=1,
                                        defs=0x7fffdf18bfa8, defcount=1, closure=0x0) at Python/ceval.c:3265
#32 0x00007ffff7d12c52 in fast_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4129
#33 call_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4054
#34 PyEval_EvalFrameEx (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:2679
#35 0x00007ffff7d14a9e in PyEval_EvalCodeEx (co=0x7ffff1ac6130, globals=<value optimized out>, locals=<value optimized out>, args=<value optimized out>, argcount=0, kws=0x662d48, kwcount=1,
                                            defs=0x7fffdf18bfe8, defcount=1, closure=0x0) at Python/ceval.c:3265
#36 0x00007ffff7d12c52 in fast_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4129
#37 call_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:4054
#38 PyEval_EvalFrameEx (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:2679
#39 0x00007ffff7d14a9e in PyEval_EvalCodeEx (co=0x7ffff1ac6230, globals=<value optimized out>, locals=<value optimized out>, args=<value optimized out>, argcount=0, kws=0x0, kwcount=0, defs=0x0,
                                                defcount=0, closure=0x0) at Python/ceval.c:3265
                                                ---Type <return> to continue, or q <return> to quit---
#40 0x00007ffff7d14bb2 in PyEval_EvalCode (co=<value optimized out>, globals=<value optimized out>, locals=<value optimized out>) at Python/ceval.c:673
#41 0x00007ffff7d34850 in run_mod (fp=0x65f7b0, filename=<value optimized out>, start=<value optimized out>, globals=0x7ffff7b9b168, locals=0x7ffff7b9b168, closeit=1, flags=0x7fffffffe0e0)
                                                    at Python/pythonrun.c:1377
#42 PyRun_FileExFlags (fp=0x65f7b0, filename=<value optimized out>, start=<value optimized out>, globals=0x7ffff7b9b168, locals=0x7ffff7b9b168, closeit=1, flags=0x7fffffffe0e0) at Python/pythonrun.c:1363
#43 0x00007ffff7d34a2f in PyRun_SimpleFileExFlags (fp=0x65f7b0, filename=0x7fffffffe4d9 "bidlandscape_sf.py", closeit=1, flags=0x7fffffffe0e0) at Python/pythonrun.c:955
#44 0x00007ffff7d4a194 in Py_Main (argc=<value optimized out>, argv=<value optimized out>) at Modules/main.c:640
#45 0x000000353ba1ed5d in __libc_start_main (main=0x400710 <main>, argc=7, ubp_av=0x7fffffffe208, init=<value optimized out>, fini=<value optimized out>, rtld_fini=<value optimized out>,
                                                        stack_end=0x7fffffffe1f8) at libc-start.c:226
#46 0x0000000000400649 in _start ()

It looks like aerospike_key_get calls some subroutine in file src/main/aerospike/as_shm_cluster.c, which is absent, hence segmentation fault. I checked my $LD_LIBRARY_PATH and it does contain directory containing aerospike.so.

  1. Why this particular file is missing? Is it somewhere else other than aerospike.so?
  2. Why segmentation fault. It should be gracefully handled in Python client

Edit:

    Metadata-Version: 1.1
    Name: aerospike
    Version: 2.0.1
    Summary: Aerospike Client Library for Python
    Home-page: http://aerospike.com
    Author: Aerospike, Inc.
    Author-email: [email protected]
    License: Apache Software License
    Location: /home/hdfs/graphlab/lib/python2.7/site-packages
    Requires:
    Classifiers:
      License :: OSI Approved :: Apache Software License
      Operating System :: POSIX :: Linux
      Operating System :: MacOS :: MacOS X
      Programming Language :: Python :: 2.6
      Programming Language :: Python :: 2.7
      Programming Language :: Python :: 3.5
      Programming Language :: Python :: Implementation :: CPython
      Topic :: Database
    Files:
      ../../../aerospike
      ../../../aerospike/lua/aerospike.lua
      ../../../aerospike/lua/as.lua
      ../../../aerospike/lua/stream_ops.lua
      ../../../aerospike/usr-lua
      aerospike-2.0.1-py2.7.egg-info
      aerospike-2.0.1-py2.7.egg-info/PKG-INFO
      aerospike-2.0.1-py2.7.egg-info/SOURCES.txt
      aerospike-2.0.1-py2.7.egg-info/dependency_links.txt
      aerospike-2.0.1-py2.7.egg-info/eager_resources.txt
      aerospike-2.0.1-py2.7.egg-info/not-zip-safe
      aerospike-2.0.1-py2.7.egg-info/top_level.txt
      aerospike.so

echo $LD_LIBRARY_PATH
/home/hdfs/graphlab/lib/python2.7/site-packages/:/usr/local/lib:/home/hdfs/python-2.7.8-install/lib/:/home/hdfs/python-3.4.2-install/lib/
Brillatsavarin answered 30/3, 2016 at 7:17 Comment(1)
Once you found the resolution to your problem via the forum you should have come back here and left an answer rather then just leaving it for others to find. Also related to meta.https://mcmap.net/q/1316726/-parameterized-insert-statement-with-transaction-throwing-an-error/692942 – Robby
C
9

This has already been reported as issues 112 and 115 in the aerospike/aerospike-client-python repo on GitHub, and a posting on the discussion forum.

In your case, the 'workaround' is to use the client correctly. Do not make calls to the database after you close the connection to it explicitly.

If you need this bug to be fixed, you can either fork the project (it is open-sourced under the Apache license) and fix it yourself, or wait for the project's maintainers to fix it themselves based on the existing issues.

Carloscarlota answered 31/3, 2016 at 13:50 Comment(4)
Sorry about that, but I think there is a misunderstanding. I asked question here first, but when there is no reply I though asking on Aerospike Forums would be a better Idea. Yesterday I was going to close the question but forgot about that. I didn't asked this question yesterday. Due to edge case of timezone, it is being reported to be asked 'yesterday' – Brillatsavarin
@Mangat: the internet community has advised against undeclared cross-posting for the last 25+ years, since the days of Newsnet. It is a great source of wasted time on the internet, as someone finds that an answer they have spent time creating has been solved already on a duplicate thread. – Anabal
cross-posting is a good thing. the idea that one "shouldn't cross post" is incredibly out of date. the current paradigm is Torrent - "cross post" all information everywhere constantly at all times – Galanti
It got fixed April 18 2016, part of the Aerospike client for Python release 2.0.3. – Carloscarlota

© 2022 - 2024 — McMap. All rights reserved.