[ell-i-developers] Re: Embedding python in emulator

  • From: Pekka Nikander <pekka.nikander@xxxxxx>
  • To: ell-i-developers@xxxxxxxxxxxxx
  • Date: Wed, 15 Oct 2014 23:02:50 +0300

You are not embedding a python interpreter yourself, so you shouldn't call 
Py_Initialize.  That's been called already.  You shouldn't call Py_Finanize 
neither, as that destroys the Python interpreter there is.

You can check that with Py_IsInitialized.

My guess is that also PyEval_ThreadsInitialized should return true, meaning 
that you have the Python threads initialised.

--Pekka

On 2014–10–15, at 22:53 , Asif Sardar <engr.asif.sardar@xxxxxxxxxxxxxx> wrote:

> I am calling:
> 
> void Register::pythonCallback()
> {
> Py_Initializa();
> Py_Finalize();
> }
> 
> in Register.cpp through = operator in BSRR class
> On Oct 15, 2014 10:43 PM, "Pekka Nikander" <pekka.nikander@xxxxxx> wrote:
> My gut feeling is that you are somehow acquiring the Python context in a 
> wrong way.
> 
> It would help if you posted the relevant code from your Register.cpp.  Yes, I 
> did see it today, when I uncommented it, but I didn't really look at it from 
> this point of view, and therefore I don't remember what there was.
> 
> --Pekka
> 
> On 2014–10–15, at 22:37 , Asif Sardar <engr.asif.sardar@xxxxxxxxxxxxxx> wrote:
> 
>> Hi Ivan,
>> 
>> Thanks!
>> 
>> python 2.7.8 (32-bit manually compiled)
>> Linux asif-Satellite-C655 3.13.0-35-generic #62-Ubuntu SMP Fri Aug 15 
>> 01:58:42 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
>> 
>> Pekka assisted me solving the issue!
>> 
>> I am calling the python interpreter from Register.cpp only using ( 
>> Py_Initialize() and Py_Finalize() ), still after running the rf test case 
>> under gdb, I get following segmentation fault: Does that mean I have to call 
>> the python interpretor in a seperate thread or so from the Register.cpp 
>> class? Any guesses ... I am not calling python interpreter from the main.c 
>> or so.
>> ------------------------------------------------------------------------------------------
>> ==============================================================================
>> DigitalRead                                                                  
>>  
>> ==============================================================================
>> Read high from pin :: Test Setup: Set the pin mode to output. Test... 
>> GPIO:MODER:value=0 -> =40
>> .GPIO:MODER:value=40 -> =40
>> GPIO:BSRRL:value=0 -> =0
>> Here we call python from Register Class
>> 
>> Program received signal SIGSEGV, Segmentation fault.
>> 0x0811b581 in wait_for_thread_shutdown () at Python/pythonrun.c:1720
>> 1720     PyObject *threading = 
>> PyMapping_GetItemString(tstate->interp->modules,
>> (gdb) where
>> #0  0x0811b581 in wait_for_thread_shutdown () at Python/pythonrun.c:1720
>> #1  Py_Finalize () at Python/pythonrun.c:416
>> #2  0xf6f8923f in Register::GPIO_Callback_Python (this=0xf7130f78 
>> <GeneralPurposeInputOutput::GPIOB+84>)
>>     at 
>> /home/asif/Ell-i-Working-Directory/Ell-i-Software-Development/Runtime/stm32/tests/robot_library/../../emulator/src/Register.cpp:48
>> #3  0xf6f8a97d in Register_GPIO_BSRRL::operator= (this=0xf7130f78 
>> <GeneralPurposeInputOutput::GPIOB+84>, arg=8)
>>     at 
>> /home/asif/Ell-i-Working-Directory/Ell-i-Software-Development/Runtime/stm32/tests/robot_library/../../emulator/src/Register_GPIO_BSRRL.cpp:30
>> #4  0xf6f88b7c in digitalWrite (pin=<optimized out>, val=<optimized out>)
>>     at 
>> /home/asif/Ell-i-Working-Directory/Ell-i-Software-Development/Runtime/stm32/tests/robot_library/../../cores/arduelli/wiring_digital.h:58
>> #5  0xf76005b6 in ffi_call_SYSV ()
>>     at 
>> /home/asif/Ell-i-Working-Directory/Ell-i-Software-Testing/Python-2.7.8/Modules/_ctypes/libffi/src/x86/sysv.S:65
>> #6  0xf7600328 in ffi_call (cif=<optimized out>, cif@entry=0xffffb3e4, 
>> fn=<optimized out>, rvalue=<optimized out>, avalue=<optimized out>)
>>     at 
>> /home/asif/Ell-i-Working-Directory/Ell-i-Software-Testing/Python-2.7.8/Modules/_ctypes/libffi/src/x86/ffi.c:411
>> #7  0xf75f8c44 in _call_function_pointer (argcount=<optimized out>, 
>> resmem=<optimized out>, restype=<optimized out>, atypes=<optimized out>, 
>>     avalues=<optimized out>, pProc=<optimized out>, flags=<optimized out>)
>>     at 
>> /home/asif/Ell-i-Working-Directory/Ell-i-Software-Testing/Python-2.7.8/Modules/_ctypes/callproc.c:836
>> #8  _ctypes_callproc (pProc=pProc@entry=0xf6f88bc3 <t_digitalWrite(pin_t, 
>> uint32_t)>, argtuple=argtuple@entry=0xf7370b2c, flags=4353, 
>>     argtypes=argtypes@entry=0x0, restype=restype@entry=0xf763d3dc, 
>> checker=checker@entry=0x0)
>>     at 
>> /home/asif/Ell-i-Working-Directory/Ell-i-Software-Testing/Python-2.7.8/Modules/_ctypes/callproc.c:1183
>> #9  0xf75ef851 in PyCFuncPtr_call (self=0xf724bb8c, inargs=0xf7370b2c, 
>> kwds=0x0)
>>     at 
>> /home/asif/Ell-i-Working-Directory/Ell-i-Software-Testing/Python-2.7.8/Modules/_ctypes/_ctypes.c:3965
>> #10 0x08066b13 in PyObject_Call (func=func@entry=0xf724bb8c, 
>> arg=arg@entry=0xf7370b2c, kw=kw@entry=0x0) at Objects/abstract.c:2529
>> #11 0x080ee3af in do_call (nk=<optimized out>, na=<optimized out>, 
>> pp_stack=0xffffb548, func=0xf724bb8c) at Python/ceval.c:4251
>> #12 call_function (oparg=<optimized out>, pp_stack=0xffffb548) at 
>> Python/ceval.c:4056
>> #13 PyEval_EvalFrameEx (f=f@entry=0xf7172a04, throwflag=throwflag@entry=0) 
>> at Python/ceval.c:2679
>> #14 0x080f1467 in fast_function (nk=<optimized out>, na=<optimized out>, 
>> n=3, pp_stack=0xffffb608, func=0xf7144fb4) at Python/ceval.c:4119
>> #15 call_function (oparg=<optimized out>, pp_stack=0xffffb608) at 
>> Python/ceval.c:4054
>> #16 PyEval_EvalFrameEx (f=f@entry=0xf717419c, throwflag=throwflag@entry=0) 
>> at Python/ceval.c:2679
>> #17 0x080f318f in PyEval_EvalCodeEx (co=0xf71ad0f8, globals=0xf71afb54, 
>> locals=locals@entry=0x0, args=args@entry=0xf7370b78, argcount=2, 
>>     kws=kws@entry=0xf7d58038, kwcount=kwcount@entry=0, defs=defs@entry=0x0, 
>> defcount=defcount@entry=0, closure=0x0) at Python/ceval.c:3265
>> #18 0x08160676 in function_call (func=0xf7149454, arg=0xf7370b6c, 
>> kw=0xf7169934) at Objects/funcobject.c:526
>> #19 0x08066b13 in PyObject_Call (func=func@entry=0xf7149454, 
>> arg=arg@entry=0xf7370b6c, kw=kw@entry=0xf7169934) at Objects/abstract.c:2529
>> #20 0x080ef4fe in ext_do_call (nk=<optimized out>, na=0, flags=<optimized 
>> out>, pp_stack=0xffffb7d8, func=0xf7149454) at Python/ceval.c:4346
>> #21 PyEval_EvalFrameEx (f=f@entry=0xf716d86c, throwflag=throwflag@entry=0) 
>> at Python/ceval.c:2718
>> #22 0x080f318f in PyEval_EvalCodeEx (co=co@entry=0xf7452bf0, 
>> globals=globals@entry=0xf74c29bc, locals=locals@entry=0x0, 
>>     args=args@entry=0xf7168b4c, argcount=argcount@entry=0, kws=0xf7168b4c, 
>> kwcount=0, defs=0x0, defcount=0, closure=0xf71b68c4)
>>     at Python/ceval.c:3265
>> #23 0x080f13a5 in fast_function (nk=<optimized out>, na=0, n=<optimized 
>> out>, pp_stack=0xffffb918, func=0xf71574c4) at Python/ceval.c:4129
>> ------------------------------------------------------------------------------------------
>> 
>> 
>> On Wed, Oct 15, 2014 at 10:15 PM, Ivan Raul <supra.material@xxxxxxxxx> wrote:
>> Hi,
>> 
>> I want to help. Asif, what is the output of:
>> 
>> python --version
>> which python
>> uname -a
>> 
>> With Warm Regards, Ivan Raul
>> 
>> On Wed, Oct 15, 2014 at 8:52 PM, Jose Granados Vergara 
>> <jose.granados@xxxxxx> wrote:
>> Could you run the file command on the executable to see what it is?
>> 
>> There are some libraries that need to be installed on Ubuntu 64 bits to run 
>> 32 bits (ia32-libs, gcc-multilib, g++-multilib, binutils-multiarch, 
>> ia32-libs-dev).
>> 
>> As a last resource you can try 32 bits isolation by installing a 32 bit 
>> Ubuntu in a virtual machine.
>> 
>> -Jose
>> De: Asif Sardar [engr.asif.sardar@xxxxxxxxxxxxxx]
>> Enviado: miércoles, 15 de octubre de 2014 01:48 p.m.
>> Para: ell-i-developers@xxxxxxxxxxxxx
>> CC: Jose Granados Vergara; Lari Lehtomäki
>> Asunto: Re: [ell-i-developers] Re: Embedding python in emulator
>> 
>> Yup, cross-compiling will not work, in website the procedure was basically 
>> cross-compiling python for target platform.
>> 
>> Anyway, it seems to be 32-bit vs 64-bit platform issue. I separately 
>> compiled and linked a C++ program to embed python in it using 32-bit python. 
>> It compiled and linked with those CFLAGS and LDFLAGS, but the executable was 
>> not running reporting error: cannot execute binary file: Exec format error. 
>> I am using 64 bit OS Ubuntu and when I compile the above program using the 
>> default python I have in Ubuntu, it compiled, linked and worked. Also, the 
>> following errors seems to be processor architectural issues with binaries. I 
>> don't have enough skills with kernel-level or OS system level debugging 
>> issues, so if anybody else got any ideas then please share with us on this 
>> platform.
>> 
>> The program is also attached to this e-mail, its very simple program to test 
>> embedding python in C/C++ programs.
>> 
>> I will stick to the procedure where python calls C/C++ code in emulator and 
>> in return gets values from the emulator using callbacks.
>> 
>> On Tue, Oct 14, 2014 at 8:58 PM, Pekka Nikander <pekka.nikander@xxxxxx> 
>> wrote:
>> I don't think cross-compiling python for ARM is the right way.  You will not 
>> be able to run the result anywhere.
>> 
>> --Pekka
>> 
>> 
>> On 2014–10–14, at 20:27 , Asif Sardar <engr.asif.sardar@xxxxxxxxxxxxxx> 
>> wrote:
>> 
>>> By native I meant my own laptop e.g. using gcc-linux to compile python. I 
>>> want to use python interpretor to be called from emulator code to call some 
>>> python functions (This will help later on for multi-purposes e.g. we can 
>>> call python from C/C++ independently). Currently, the callbacks I have 
>>> implemented have to pass-on information to python by actually calling the 
>>> emulator from python itself (2-way calling). If I am able to embed python 
>>> in emulator, it will asynchronously or independently call python whenever 
>>> needed from C/C++ code. Actually, the test libraries are written in python 
>>> language in Robot Framework that is why I am interested to do these kind of 
>>> functionality to both C/C++ and python.
>>> 
>>> I will try the steps in 
>>> https://www.ailis.de/~k/archives/19-ARM-cross-compiling-howto.html#python 
>>> to cross-compile the python for ARM itself. I will share the findings here!
>>> 
>>> On Tue, Oct 14, 2014 at 8:12 PM, Jose Granados Vergara 
>>> <jose.granados@xxxxxx> wrote:
>>> Isn't the emulator native? why do you need to bind Python to the ARM 
>>> binaries?
>>> 
>>> -Jose
>>> De: Asif Sardar [engr.asif.sardar@xxxxxxxxxxxxxx]
>>> Enviado: martes, 14 de octubre de 2014 08:04 p.m.
>>> Para: Jose Granados Vergara
>>> CC: ell-i-developers@xxxxxxxxxxxxx; Lari Lehtomäki
>>> Asunto: Re: Embedding python in emulator
>>> 
>>> Thanks Jose,
>>> 
>>> On native platform, its working for me as well. I think I have to 
>>> re-compile or cross-compile 32-bit python using ARM tools e.g. using 
>>> something like CXX=arm-none-linux-gnueabi-g++ and then rebuild robot 
>>> framework.
>>> 
>>> I have to compile the emulator files using arm-tool chain and I am 
>>> including the 32-bit python in it built with native gcc toolchain.
>>> 
>>> If anyone else has some suggestions that I will really appreciate!
>>> 
>>> On Tue, Oct 14, 2014 at 7:58 PM, Jose Granados Vergara 
>>> <jose.granados@xxxxxx> wrote:
>>> This worked for me:
>>> 
>>> something.c:
>>> 
>>> #include "Python.h"
>>> 
>>> int main()
>>> {
>>>     Py_Initialize();
>>>     return 0;
>>> }
>>> 
>>> ~$ g++ -I/usr/include/python2.7 something.c -o something -lpython2.7
>>> 
>>> 
>>> -Jose
>>> De: Asif Sardar [engr.asif.sardar@xxxxxxxxxxxxxx]
>>> Enviado: martes, 14 de octubre de 2014 07:40 p.m.
>>> Para: ell-i-developers@xxxxxxxxxxxxx
>>> CC: Lari Lehtomäki; Jose Granados Vergara
>>> Asunto: Fwd: Embedding python in emulator
>>> 
>>> Hi,
>>> 
>>> Has anybody embedded python in C/C++ application. My platform is 64-bit and 
>>> python binaries I am using for Runtime/emulator is 32-bit. I have added 
>>> following flags in CFLAGS and LDFLAGS to compile and link the c/c++ files 
>>> in emulator with (#include </path/32-bit-python/include/python.h>)
>>> 
>>> CFLAGS += -I/opt/pym32/bin/include/python2.7 -fno-strict-aliasing -DNDEBUG 
>>> -g -fwrapv -O3 -Wall -Wstrict-prototypes
>>> LDFLAGS += -L/opt/pym32/lib/python2.7/config -lpthread -ldl -lutil -lm 
>>> -lpython2.7 -Xlinker -export-dynamic
>>> The files compiles but breaks when linking. Can anybody help with following 
>>> errors: 
>>> 
>>> -----------------------
>>> ./libsystem_stm32f4discovery.a(Register.o): In function 
>>> `Register::GPIO_Callback_Python(std::string, std::string, unsigned int, 
>>> std::string)':
>>> /home/asif/Ell-i-Working-Directory/Ell-i-Software-Development/Runtime/stm32/tests/robot_library/../../emulator/src/Register.cpp:41:
>>>  undefined reference to `Py_Initialize'
>>> /home/asif/Ell-i-Working-Directory/Ell-i-Software-Development/Runtime/stm32/tests/robot_library/../../emulator/src/Register.cpp:43:
>>>  undefined reference to `PyRun_SimpleStringFlags'
>>> /home/asif/Ell-i-Working-Directory/Ell-i-Software-Development/Runtime/stm32/tests/robot_library/../../emulator/src/Register.cpp:44:
>>>  undefined reference to `Py_Finalize'
>>> /home/asif/Ell-i-Working-Directory/Ell-i-Software-Development/Runtime/stm32/tests/robot_library/../../emulator/src/Register.cpp:41:
>>>  undefined reference to `Py_Initialize'
>>> /home/asif/Ell-i-Working-Directory/Ell-i-Software-Development/Runtime/stm32/tests/robot_library/../../emulator/src/Register.cpp:43:
>>>  undefined reference to `PyRun_SimpleStringFlags'
>>> /home/asif/Ell-i-Working-Directory/Ell-i-Software-Development/Runtime/stm32/tests/robot_library/../../emulator/src/Register.cpp:44:
>>>  undefined reference to `Py_Finalize'
>>> ./libsystem_stm32f4discovery.a(system_stm32f4xx.o): In function 
>>> `__exchange_and_add':
>>> /usr/include/c++/4.8/ext/atomicity.h:49: undefined reference to 
>>> `__atomic_fetch_add_4'
>>> /usr/include/c++/4.8/ext/atomicity.h:49: undefined reference to 
>>> `__atomic_fetch_add_4'
>>> /usr/include/c++/4.8/ext/atomicity.h:49: undefined reference to 
>>> `__atomic_fetch_add_4'
>>> /usr/include/c++/4.8/ext/atomicity.h:49: undefined reference to 
>>> `__atomic_fetch_add_4'
>>> -----------------------
>>> 
>>> 
>>> ---------- Forwarded message ----------
>>> From: Asif Sardar <engr.asif.sardar@xxxxxxxxxxxxxx>
>>> Date: Mon, Oct 13, 2014 at 6:17 PM
>>> Subject: Embedding python in emulator
>>> To: Pekka Nikander <pekka.nikander@xxxxxx>
>>> 
>>> 
>>> Hi Pekka,
>>> 
>>> I am trying to include python.h in emulator/inc/register.h
>>> 
>>> I have to include cflags and ldflags in Makefile system of emulator e.g.
>>> 
>>> -I/opt/include/python2.7 -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall 
>>> -Wstrict-prototypes
>>> -L/opt/lib/python2.7/config -lpthread -ldl -lutil -lm -lpython2.7 -Xlinker 
>>> -export-dynamic
>>> However, I am getting:
>>> ----------------------------------------------------
>>> g++ -m32 -march=i386 -m32 -march=i386 -Xlinker -export-dynamic -o 
>>> robot_library  robot_library.o stm32f4discovery.o libSPI.a "-L." 
>>> "-L/home/asif/Ell-i-Working-Directory/Ell-i-Software-Development/Runtime/stm32/tests/robot_library/../../variants/stm32f4discovery"
>>>  "-lsystem_stm32f4discovery" -lstdc++ 
>>> ./libsystem_stm32f4discovery.a(Register.o): In function 
>>> `Register::GPIO_Callback_Python(std::string, std::string, unsigned int, 
>>> std::string)':
>>> /home/asif/Ell-i-Working-Directory/Ell-i-Software-Development/Runtime/stm32/tests/robot_library/../../emulator/src/Register.cpp:41:
>>>  undefined reference to `Py_Initialize'
>>> /home/asif/Ell-i-Working-Directory/Ell-i-Software-Development/Runtime/stm32/tests/robot_library/../../emulator/src/Register.cpp:43:
>>>  undefined reference to `PyRun_SimpleStringFlags'
>>> /home/asif/Ell-i-Working-Directory/Ell-i-Software-Development/Runtime/stm32/tests/robot_library/../../emulator/src/Register.cpp:44:
>>>  undefined reference to `Py_Finalize'
>>> /home/asif/Ell-i-Working-Directory/Ell-i-Software-Development/Runtime/stm32/tests/robot_library/../../emulator/src/Register.cpp:41:
>>>  undefined reference to `Py_Initialize'
>>> /home/asif/Ell-i-Working-Directory/Ell-i-Software-Development/Runtime/stm32/tests/robot_library/../../emulator/src/Register.cpp:43:
>>>  undefined reference to `PyRun_SimpleStringFlags'
>>> /home/asif/Ell-i-Working-Directory/Ell-i-Software-Development/Runtime/stm32/tests/robot_library/../../emulator/src/Register.cpp:44:
>>>  undefined reference to `Py_Finalize'
>>> ./libsystem_stm32f4discovery.a(system_stm32f4xx.o): In function 
>>> `__exchange_and_add':
>>> /usr/include/c++/4.8/ext/atomicity.h:49: undefined reference to 
>>> `__atomic_fetch_add_4'
>>> /usr/include/c++/4.8/ext/atomicity.h:49: undefined reference to 
>>> `__atomic_fetch_add_4'
>>> /usr/include/c++/4.8/ext/atomicity.h:49: undefined reference to 
>>> `__atomic_fetch_add_4'
>>> /usr/include/c++/4.8/ext/atomicity.h:49: undefined reference to 
>>> `__atomic_fetch_add_4'
>>> collect2: error: ld returned 1 exit status
>>> make[1]: *** [robot_library] Error 1
>>> -----------------------------------
>>> 
>>> Can you help me in including these flags in the Makefile system and link 
>>> the python with it? It will help a lot, to call from c to python 
>>> independently.
>>> 
>>> 
>>> 
>>> -- 
>>> With Best Regards,
>>> Asif Sardar.
>>> +358 43 8265795
>>> 
>>> 
>>> 
>>> -- 
>>> With Best Regards,
>>> Asif Sardar.
>>> +358 43 8265795
>>> 
>>> 
>>> 
>>> -- 
>>> With Best Regards,
>>> Asif Sardar.
>>> +358 43 8265795
>>> 
>>> 
>>> 
>>> -- 
>>> With Best Regards,
>>> Asif Sardar.
>>> +358 43 8265795
>> 
>> 
>> 
>> 
>> -- 
>> With Best Regards,
>> Asif Sardar.
>> +358 43 8265795
>> 
>> 
>> 
>> 
>> -- 
>> With Best Regards,
>> Asif Sardar.
>> +358 43 8265795
> 

Attachment: smime.p7s
Description: S/MIME cryptographic signature

Other related posts: