ImportError when the apk launches with SDL2 bootstrap, kivy and python3crystax

[haricot]

I do not know if it actually works, but in my case it does not work with android 5.0. And I'd like to know why :-) (the apk works well with sdl2, Kivy, python 2)

...
I/python  ( 9904): Android kivy bootstrap done. __name__ is __main__
I/python  ( 9904): AND: Ran string
I/python  ( 9904): Run user program, change dir and execute entrypoint
I/python  ( 9904): main.py
I/python  ( 9904): [WARNING           ] [Config      ] Older configuration version detected (0 instead of 14)
I/python  ( 9904): [WARNING           ] [Config      ] Upgrading configuration in progress.
I/python  ( 9904): [INFO              ] [Logger      ] Record log in /data/data/org.hello.world/files/.kivy/logs/kivy_16-03-02_0.txt
I/python  ( 9904): [INFO              ] [Kivy        ] v1.9.2-dev0
I/python  ( 9904): [INFO              ] [Python      ] v3.5.0 (default, Dec 24 2015, 05:56:40) 
I/python  ( 9904): [GCC 5.3 20151204]
I/python  ( 9904):  Traceback (most recent call last):
I/python  ( 9904):    File "main.py", line 13, in <module>
I/python  ( 9904):      from kivy.app import App
I/python  ( 9904):    File "/data/data/org.hello.world/files/crystax_python/site-packages/kivy/app.py", line 319, in <module>
I/python  ( 9904):      from kivy.base import runTouchApp, stopTouchApp
I/python  ( 9904):    File "/data/data/org.hello.world/files/crystax_python/site-packages/kivy/base.py", line 30, in <module>
I/python  ( 9904):      from kivy.event import EventDispatcher
I/python  ( 9904):    File "/data/data/org.hello.world/files/crystax_python/site-packages/kivy/event.py", line 8, in <module>
I/python  ( 9904):      import kivy._event
I/python  ( 9904):  ImportError: dlopen failed: cannot locate symbol "PyErr_GivenExceptionMatches" referenced by "_event.so"...
I/python  ( 9904): Python for android ended.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[haricot]

Maybe it's a bug
python3.5 https://bugs.python.org/issue12029
python3.6 https://bugs.python.org/issue25537

Can we apply a patch python3crystax?

[inclement]

I'm not sure what's wrong here - it could maybe be the bug in python, but I know the python3 with crystax can work and the error isn't with any custom code which I haven't tested.

There's currently a bug with python3crystax and kivy that could maybe manifest this way, involving being compiled with the wrong python. Could you try building with kivy 1.9.1 (not the master branch, as is the default) - I think you should be able to use 'kivy==1.9.1' in the requirements, though you'll also need to make sure everything is rebuilt (you can do p4a clean_all first to be sure)?

[haricot]

Thank you for your answers.
I follow your instructions but I get the same result.

...
I/python  ( 9743): Android kivy bootstrap done. __name__ is __main__
I/python  ( 9743): AND: Ran string
I/python  ( 9743): Run user program, change dir and execute entrypoint
I/python  ( 9743): main.py
I/python  ( 9743): [WARNING           ] [Config      ] Older configuration version detected (0 instead of 14)
I/python  ( 9743): [WARNING           ] [Config      ] Upgrading configuration in progress.
I/python  ( 9743): [INFO              ] [Logger      ] Record log in /data/data/org.hello.world/files/.kivy/logs/kivy_16-03-04_0.txt
I/python  ( 9743): [INFO              ] [Kivy        ] v1.9.1
I/python  ( 9743): [INFO              ] [Python      ] v3.5.0 (default, Dec 24 2015, 05:56:06) 
I/python  ( 9743): [GCC 5.3 20151204]
I/python  ( 9743):  Traceback (most recent call last):
I/python  ( 9743):    File "main.py", line 1, in <module>
I/python  ( 9743):      from kivy.app import App
I/python  ( 9743):    File "/data/data/org.hello.world/files/crystax_python/site-packages/kivy/app.py", line 319, in <module>
I/python  ( 9743):      from kivy.base import runTouchApp, stopTouchApp
I/python  ( 9743):    File "/data/data/org.hello.world/files/crystax_python/site-packages/kivy/base.py", line 30, in <module>
I/python  ( 9743):      from kivy.event import EventDispatcher
I/python  ( 9743):    File "/data/data/org.hello.world/files/crystax_python/site-packages/kivy/event.py", line 8, in <module>
I/python  ( 9743):      import kivy._event
I/python  ( 9743):  ImportError: dlopen failed: cannot locate symbol "PyObject_GetAttr" referenced by "_event.so"...
I/python  ( 9743): Python for android ended.

[inclement]

Interesting that it's a different symbol it can't find. I think maybe it more generally has failed to find/load the python3 .so file. I haven't tried the build for a while, but I'll see if I can reproduce it (it could be a few days before I have time though).

If you'd like to investigate, I'd start by checking the java bootstrap code and finding a way to check if it's loading python3.5m.so correctly. There may be a log message about this already, but I don't remember the format. If it isn't loading it, that is probably the problem.

[haricot]

Yes you are right, i went a little fast, this is not the same result.
With Kivy==1.9.2
cannot locate symbol "PyErr_GivenExceptionMatches"

and with kivy==1.9.1
cannot locate symbol "PyObject_GetAttr"

I looked where the files .so point with my phone root and it seemed to be the right place but I may be wrong.

[inclement]

It's the libpython3.5m.so that's important - did you find that one? And even if it's there (in the libs/armeabi folder), the important question is probably if it's loaded by the java bootstrap.

[haricot]

I do not know if the libpython3.5m.so and finally fully charged by the bootstrap Java.

root@hlte:/data/data/org.hello.world/lib # ls
libSDL2.so
libSDL2_image.so
libSDL2_mixer.so
libSDL2_ttf.so
libcrystax.so
libmain.so
libpython3.5m.so

V/PythonUtil( 9743): Loaded everything!
V/PythonActivity( 9743): Did super onCreate
V/PythonActivity( 9743): Setting env vars for start.c and Python to use
V/PythonActivity( 9743): Access to our meta-data...
I/PythonActivity( 9743): Surface will NOT be transparent
I/python  ( 9743): Initialize Python for Android
I/python  ( 9743): Changing directory to the one provided by ANDROID_ARGUMENT
I/python  ( 9743): /data/data/org.hello.world/files
I/python  ( 9743): Preparing to initialize python
I/python  ( 9743): crystax_python exists
I/python  ( 9743): calculated paths to be...
I/python  ( 9743): /data/data/org.hello.world/files/crystax_python/stdlib.zip:/data/data/org.hello.world/files/crystax_python/modules
I/python  ( 9743): set wchar paths...
I/ActivityManager(  860): Displayed org.hello.world/org.kivy.android.PythonActivity: +2s288ms (total +44m15s972ms)
I/Timeline(  860): Timeline: Activity_windows_visible id: ActivityRecord{216ce99e u0 org.hello.world/org.kivy.android.PythonActivity t1661} time:14680474
I/python  ( 9743): Initialized python
I/python  ( 9743): AND: Init threads
I/python  ( 9743): testing python print redirection
I/python  ( 9743): Android path ['.', '/data/data/org.hello.world/files/crystax_python/stdlib.zip', '/data/data/org.hello.world/files/crystax_python/modules', '/data/data/org.hello.world/files/crystax_python/site-packages']
I/python  ( 9743): os.environ is environ({'LOOP_MOUNTPOINT': '/mnt/obb', 'ANDROID_ASSETS': '/system/app', 'ANDROID_BOOTLOGO': '1', 'PYTHONPATH': '/data/data/org.hello.world/files:/data/data/org.hello.world/files/lib', 'ANDROID_ROOT': '/system', 'MC_AUTH_TOKEN_PATH': '/efs', 'ANDROID_PROPERTY_WORKSPACE': '8,0', 'VIBE_PIPE_PATH': '/dev/pipes', 'ANDROID_PRIVATE': '/data/data/org.hello.world/files', 'KNOX_STORAGE': '/data/knox/ext_sdcard', 'ANDROID_DATA': '/data', 'EXTERNAL_STORAGE': '/storage/emulated/legacy', 'ASEC_MOUNTPOINT': '/mnt/asec', 'ANDROID_APP_PATH': '/data/data/org.hello.world/files', 'EMULATED_STORAGE_TARGET': '/storage/emulated', 'SECONDARY_STORAGE': '/storage/extSdCard', 'PYTHONHOME': '/data/data/org.hello.world/files', 'PATH': '/sbin:/vendor/bin:/system/sbin:/system/bin:/system/xbin', 'ANDROID_SOCKET_zygote': '9', 'EMULATED_STORAGE_SOURCE': '/mnt/shell/emulated', 'LD_PRELOAD': 'libsigchain.so:libNimsWrap.so', 'ANDROID_ARGUMENT': '/data/data/org.hello.world/files', 'PYTHON_NAME': 'python', 'ANDROID_STORAGE': '/storage', 'ANDROID_ENTRYPOINT': 'main.pyo', 'BOOTCLASSPATH': '/system/framework/core-libart.jar:/system/framework/conscrypt.jar:/system/framework/okhttp.jar:/system/framework/core-junit.jar:/system/framework/bouncycastle.jar:/system/framework/ext.jar:/system/framework/framework.jar:/system/framework/telephony-common.jar:/system/framework/voip-common.jar:/system/framework/ims-common.jar:/system/framework/mms-common.jar:/system/framework/android.policy.jar:/system/framework/apache-xml.jar:/system/framework/sec_edm.jar:/system/framework/timakeystore.jar:/system/framework/seccamera.jar:/system/framework/scrollpause.jar:/system/framework/stayrotation.jar:/system/framework/smartfaceservice.jar:/system/framework/commonimsinterface.jar:/system/framework/imsmanager.jar:/system/framework/sprengine.jar:/system/framework/smartbondingservice.jar:/system/framework/secocsp.jar:/system/framework/secEmailBC.jar:/system/framework/knoxvpnuidtag.jar:/system/framework/simageis.jar:/system/framework/qcmediaplayer.jar:/system/framework/WfdCommon.jar:/system/framework/oem-services.jar:/system/framework/org.codeaurora.Performance.jar:/system/framework/vcard.jar:/system/framework/tcmiface.jar', 'SYSTEMSERVERCLASSPATH': '/system/framework/services.jar:/system/framework/ethernet-service.jar:/system/framework/wifi-service.jar'})
I/python  ( 9743): Android kivy bootstrap done. __name__ is __main__
I/python  ( 9743): AND: Ran string
I/python  ( 9743): Run user program, change dir and execute entrypoint
I/python  ( 9743): main.py
I/python  ( 9743): [WARNING           ] [Config      ] Older configuration version detected (0 instead of 14)
I/python  ( 9743): [WARNING           ] [Config      ] Upgrading configuration in progress.
I/python  ( 9743): [INFO              ] [Logger      ] Record log in /data/data/org.hello.world/files/.kivy/logs/kivy_16-03-04_0.txt
I/python  ( 9743): [INFO              ] [Kivy        ] v1.9.1
I/python  ( 9743): [INFO              ] [Python      ] v3.5.0 (default, Dec 24 2015, 05:56:06) 
I/python  ( 9743): [GCC 5.3 20151204]
I/python  ( 9743):  Traceback (most recent call last):
I/python  ( 9743):    File "main.py", line 1, in <module>
I/python  ( 9743):      from kivy.app import App
I/python  ( 9743):    File "/data/data/org.hello.world/files/crystax_python/site-packages/kivy/app.py", line 319, in <module>
I/python  ( 9743):      from kivy.base import runTouchApp, stopTouchApp
I/python  ( 9743):    File "/data/data/org.hello.world/files/crystax_python/site-packages/kivy/base.py", line 30, in <module>
I/python  ( 9743):      from kivy.event import EventDispatcher
I/python  ( 9743):    File "/data/data/org.hello.world/files/crystax_python/site-packages/kivy/event.py", line 8, in <module>
I/python  ( 9743):      import kivy._event
I/python  ( 9743):  ImportError: dlopen failed: cannot locate symbol "PyObject_GetAttr" referenced by "_event.so"...
I/python  ( 9743): Python for android ended.

[haricot]

$ readelf  -Ws _event.so| grep -i PyErr_GivenExceptionMatches
    11: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND PyErr_GivenExceptionMatches
   684: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND PyErr_GivenExceptionMatches

$ readelf  -Ws libpython3.5m.so| grep -i PyErr_GivenExceptionMatches
   507: 00047851   196 FUNC    GLOBAL DEFAULT    7 PyErr_GivenExceptionMatches

$ readelf -Ws _event.so | grep -i PyObject_GetAttr
    10: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND PyObject_GetAttr
   124: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND PyObject_GetAttrString
    32: 00003445    12 FUNC    LOCAL  DEFAULT    7 __Pyx_PyObject_GetAttrStr
   683: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND PyObject_GetAttr
   797: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND PyObject_GetAttrString

readelf -Ws libpython3.5m.so | grep -i PyObject_GetAttr
   107: 0009267d    60 FUNC    GLOBAL DEFAULT    7 PyObject_GetAttrString
   120: 000926df    26 FUNC    GLOBAL DEFAULT    7 _PyObject_GetAttrId
   309: 00092611   108 FUNC    GLOBAL DEFAULT    7 PyObject_GetAttr

Perhaps we must add a few things as below to the bootstrap sdl2 _event.so) work ?
( python-for-Android/pythonforandroid/bootstrap/sdl2/build/jni/src/start.c)
(kivy/kivy/_event.pyx ( _event.so))

#Presentation EuroPython 2012: C++ APIs on Python

#Exception translation
PyObject *t, *v, *tb;
PyErr_Fetch(&t, &v, &tb);

// Check if it's a ValueError
if (PyErr_GivenExceptionMatches(

    value_error_class_obj, t))
{
 throw ValueError();
}

If you'd like to investigate, I'd start by checking the java bootstrap code and finding a way to check if it's loading python3.5m.so correctly. There may be a log message about this already, but I don't remember the format. If it isn't loading it, that is probably the problem.

Have you a clues for to invest bootstrap sdl and libpython3.5m.so ?

[haricot]

it is good it works :-)
He was the llibpython3.5m.so who has not been charged

I added this to the setup.py file Kivy

      if PY3:
            e.extra_link_args += ['-lpython3.5m']

and this to the setup.py file pyjnius:

if PY3:
    extra_link_args = ['-lpython3.5m']

I/python  (31712): Initialized python
I/python  (31712): AND: Init threads
I/python  (31712): testing python print redirection
I/python  (31712): Android path ['.', '/data/data/org.hello.world/files/crystax_python/stdlib.zip', '/data/data/org.hello.world/files/crystax_python/modules', '/data/data/org.hello.world/files/crystax_python/site-packages']
I/python  (31712): os.environ is environ({'BOOTCLASSPATH': '/system/framework/core-libart.jar:/system/framework/conscrypt.jar:/system/framework/okhttp.jar:/system/framework/core-junit.jar:/system/framework/bouncycastle.jar:/system/framework/ext.jar:/system/framework/framework.jar:/system/framework/telephony-common.jar:/system/framework/voip-common.jar:/system/framework/ims-common.jar:/system/framework/mms-common.jar:/system/framework/android.policy.jar:/system/framework/apache-xml.jar:/system/framework/sec_edm.jar:/system/framework/timakeystore.jar:/system/framework/seccamera.jar:/system/framework/scrollpause.jar:/system/framework/stayrotation.jar:/system/framework/smartfaceservice.jar:/system/framework/commonimsinterface.jar:/system/framework/imsmanager.jar:/system/framework/sprengine.jar:/system/framework/smartbondingservice.jar:/system/framework/secocsp.jar:/system/framework/secEmailBC.jar:/system/framework/knoxvpnuidtag.jar:/system/framework/simageis.jar:/system/framework/qcmediaplayer.jar:/system/framework/WfdCommon.jar:/system/framework/oem-services.jar:/system/framework/org.codeaurora.Performance.jar:/system/framework/vcard.jar:/system/framework/tcmiface.jar', 'ANDROID_STORAGE': '/storage', 'ANDROID_ENTRYPOINT': 'main.pyo', 'SYSTEMSERVERCLASSPATH': '/system/framework/services.jar:/system/framework/ethernet-service.jar:/system/framework/wifi-service.jar', 'PATH': '/sbin:/vendor/bin:/system/sbin:/system/bin:/system/xbin', 'PYTHONHOME': '/data/data/org.hello.world/files', 'SECONDARY_STORAGE': '/storage/extSdCard', 'ANDROID_APP_PATH': '/data/data/org.hello.world/files', 'PYTHON_NAME': 'python', 'ANDROID_ARGUMENT': '/data/data/org.hello.world/files', 'LD_PRELOAD': 'libsigchain.so:libNimsWrap.so', 'EMULATED_STORAGE_SOURCE': '/mnt/shell/emulated', 'ANDROID_SOCKET_zygote': '9', 'EXTERNAL_STORAGE': '/storage/emulated/legacy', 'KNOX_STORAGE': '/data/knox/ext_sdcard', 'ANDROID_PRIVATE': '/data/data/org.hello.world/files', 'EMULATED_STORAGE_TARGET': '/storage/emulated', 'ASEC_MOUNTPOINT': '/mnt/asec', 'PYTHONPATH': '/data/data/org.hello.world/files:/data/data/org.hello.world/files/lib', 'ANDROID_ROOT': '/system', 'ANDROID_BOOTLOGO': '1', 'LOOP_MOUNTPOINT': '/mnt/obb', 'ANDROID_ASSETS': '/system/app', 'ANDROID_PROPERTY_WORKSPACE': '8,0', 'VIBE_PIPE_PATH': '/dev/pipes', 'MC_AUTH_TOKEN_PATH': '/efs', 'ANDROID_DATA': '/data'})
I/python  (31712): Android kivy bootstrap done. __name__ is __main__
I/python  (31712): AND: Ran string
I/python  (31712): Run user program, change dir and execute entrypoint
I/python  (31712): main.py
I/python  (31712): [WARNING           ] [Config      ] Older configuration version detected (0 instead of 14)
I/python  (31712): [WARNING           ] [Config      ] Upgrading configuration in progress.
I/python  (31712): [INFO              ] [Logger      ] Record log in /data/data/org.hello.world/files/.kivy/logs/kivy_16-03-30_0.txt
I/python  (31712): [INFO              ] [Kivy        ] v1.9.2-dev0
I/python  (31712): [INFO              ] [Python      ] v3.5.0 (default, Dec 24 2015, 05:56:40) 
I/python  (31712): [GCC 5.3 20151204]
I/python  (31712): ['/data/data/org.hello.world/files/crystax_python/site-packages/kivy']
I/python  (31712): [INFO              ] [Factory     ] 182 symbols loaded
I/python  (31712): [INFO              ] [Image       ] Providers: img_tex, img_dds, img_gif, img_sdl2 (img_pil, img_ffpyplayer ignored)
I/python  (31712): [INFO              ] [Text        ] Provider: sdl2
I/python  (31712): [INFO              ] [OSC         ] using <multiprocessing> for socket
I/python  (31712): [ERROR             ] [Input       ] AndroidJoystick is not supported by your version of linux
I/python  (31712): Traceback (most recent call last):
I/python  (31712):   File "/data/data/org.hello.world/files/crystax_python/site-packages/kivy/input/providers/androidjoystick.py", line 17, in <module>
I/python  (31712):     import android  # NOQA
I/python  (31712): ImportError: No module named 'android'
I/python  (31712): During handling of the above exception, another exception occurred:
I/python  (31712): Traceback (most recent call last):
I/python  (31712):   File "/data/data/org.hello.world/files/crystax_python/site-packages/kivy/input/providers/__init__.py", line 57, in <module>
I/python  (31712):     import kivy.input.providers.androidjoystick
I/python  (31712):   File "/data/data/org.hello.world/files/crystax_python/site-packages/kivy/input/providers/androidjoystick.py", line 20, in <module>
I/python  (31712):     raise Exception('android lib not found.')
I/python  (31712): Exception: android lib not found.
I/python  (31712): [INFO              ] [Window      ] Provider: sdl2
I/python  (31712): [INFO              ] [GL          ] OpenGL version <b'OpenGL ES 3.0 V@84.0 AU@05.00.00.002.025 (CL@)'>
I/python  (31712): [INFO              ] [GL          ] OpenGL vendor <b'Qualcomm'>
I/python  (31712): [INFO              ] [GL          ] OpenGL renderer <b'Adreno (TM) 330'>
I/python  (31712): [INFO              ] [GL          ] OpenGL parsed version: 3, 0
I/python  (31712): [INFO              ] [GL          ] Texture max size <4096>
I/python  (31712): [INFO              ] [GL          ] Texture max units <16>
I/python  (31712): [INFO              ] [Window      ] auto add sdl2 input provider
I/python  (31712): [INFO              ] [Window      ] virtual keyboard not allowed, single mode, not docked
I/python  (31712): [WARNING           ] [Base        ] Unknown <android> provider
I/python  (31712): [INFO              ] [Base        ] Start application main loop
I/python  (31712): [INFO              ] [GL          ] NPOT texture support is available

[m-mcgowan]

I'm having the same problem with my kivy app on android (built on ubuntu), so it's great this is fixed! I'm using python-for-android - can someone please spare a few minutes to tell me how I merge the fix?

[haricot]

@m-mcgowan This bug does not appear on distributions based on Archlinux.
The pull resquests that I proposed are not ideal because it must be placed in rather p4a.
(To test you simply apply the above modif to your Kivy packages and Pyjnius (.local/share/python-for -android/packages/))
I reinstalled a distribution based on Debian but for now I do not own solution and that it works without modif on Archlinux distributions confuses me a little.