Greenlet Crystax/Python3 support, fixes #1245 by AndreMiras · Pull Request #1250 · kivy/python-for-android

AndreMiras · 2018-04-01T21:36:44Z

Fixes CFLAGS, LDSHARED and LDFLAGS to point to Crystax.

I've tested it with the following main.py on Android.

import gevent

def foo():
    print('Running in foo')
    gevent.sleep(0)
    print('Explicit context switch to foo again')

def bar():
    print('Explicit context to bar')
    gevent.sleep(0)
    print('Implicit context switch back to bar')


if __name__ == '__main__':
    gevent.joinall([
        gevent.spawn(foo),
        gevent.spawn(bar),
    ])

It worked as expected, see adb logcat | grep python output:

04-01 23:48:22.448 10853 10895 I python  : Initialize Python for Android
04-01 23:48:22.449 10853 10895 I python  : Changing directory to the one provided by ANDROID_ARGUMENT
04-01 23:48:22.449 10853 10895 I python  : /data/user/0/org.test.myapp/files/app
04-01 23:48:22.451 10853 10895 I python  : Preparing to initialize python
04-01 23:48:22.455 10853 10895 I python  : crystax_python exists
04-01 23:48:22.455 10853 10895 I python  : calculated paths to be...
04-01 23:48:22.455 10853 10895 I python  : /data/user/0/org.test.myapp/files/app/crystax_python/stdlib.zip:/data/user/0/org.test.myapp/files/app/crystax_python/modules
04-01 23:48:22.459 10853 10895 I python  : set wchar paths...
04-01 23:48:22.750 10853 10895 I python  : Initialized python
04-01 23:48:22.750 10853 10895 I python  : AND: Init threads
04-01 23:48:22.750 10853 10895 I python  : testing python print redirection
04-01 23:48:22.751 10853 10895 I python  : Android path ['.', '/data/user/0/org.test.myapp/files/app/crystax_python/stdlib.zip', '/data/user/0/org.test.myapp/files/app/crystax_python/modules', '/data/user/0/org.test.myapp/files/app/crystax_python/site-packages']
04-01 23:48:22.752 10853 10895 I python  : os.environ is environ({'PYTHONHOME': '/data/user/0/org.test.myapp/files/app', 'PATH': '/sbin:/vendor/bin:/system/sbin:/system/bin:/system/xbin:/system/vendor/bin:/vendor/xbin:/system/vendor/xbin:/product/bin:/product/xbin', 
'OEM_ROOT': '/hw_oem', 'EXTERNAL_STORAGE': '/sdcard', 'ANDROID_PRIVATE': '/data/user/0/org.test.myapp/files', 'ASEC_MOUNTPOINT': '/mnt/asec', 'ANDROID_STORAGE': '', 'PYTHONPATH': '/data/user/0/org.test.myapp/files/app:/data/user/0/org.test.myapp/files/app/lib', 'ANDR
OID_DATA': '/data', 'ANDROID_ARGUMENT': '/data/user/0/org.test.myapp/files/app', 'ANDROID_APP_PATH': '/data/user/0/org.test.myapp/files/app', 'ANDROID_UNPACK': '/data/user/0/org.test.myapp/files/app', 'ANDROID_ROOT': '/system', 'SYSTEMSERVERCLASSPATH': '/system/frame
work/services.jar:/system/framework/ethernet-service.jar:/system/framework/wifi-service.jar:/system/framework/hwServices.jar:/system/framework/hwWifi-service.jar:/system/framework/hwcustServices.jar:/system/framework/hwcustwifi-service.jar', 'ANDROID_ASSETS': '/syste
m/app', 'PYTHONOPTIMIZE': '2', 'ANDROID_BOOTLOGO': '1', 'ANDROID_ENTRYPOINT': 'main.pyo', 'PYTHON_NAME': 'python', 'CUST_POLICY_DIRS': '/system/emui/base:/system/emui/oversea:/system/emui/oversea_honor:/system/emui/lite:/system/global:/vendor/etc:/product/etc:/hw_oem
:/cust/hw/normal:/cust_comm:/version/region_comm/oversea:/cust_spec', 'BOOTCLASSPATH': '/system/framework/core-oj.jar:/system/framework/core-libart.jar:/system/framework/conscrypt.jar:/system/framework/okhttp.jar:/system/framework/core-junit.jar:/system/framework/bou
ncycastle.jar:/system/framework/ext.jar:/system/framework/framework.jar:/system/framework/telephony-common.jar:/system/framework/voip-common.jar:/system/framework/ims-common.jar:/system/framework/apache-xml.jar:/system/framework/org.apache.http.legacy.boot.jar:/syste
m/framework/hwEmui.jar:/system/framework/hwTelephony-common.jar:/system/framework/hwframework.jar:/system/framework/org.simalliance.openmobileapi.jar:/system/framework/org.ifaa.android.manager.jar:/system/framework/hwaps.jar:/system/framework/hwcustEmui.jar:/system/f
ramework/hwcustTelephony-common.jar:/system/framework/hwcustframework.jar', 'ANDROID_SOCKET_zygote_secondary': '14'})
04-01 23:48:22.752 10853 10895 I python  : Android kivy bootstrap done. __name__ is __main__
04-01 23:48:22.752 10853 10895 I python  : AND: Ran string
04-01 23:48:22.752 10853 10895 I python  : Run user program, change dir and execute entrypoint
04-01 23:48:22.752 10853 10895 I python  : main.py
04-01 23:48:23.167 10853 10895 W linker  : /data/data/org.test.myapp/files/app/crystax_python/site-packages/greenlet.so: is missing DT_SONAME will use basename as a replacement: "greenlet.so"
04-01 23:48:23.173 10853 10895 W linker  : /data/data/org.test.myapp/files/app/crystax_python/site-packages/gevent/corecext.so: is missing DT_SONAME will use basename as a replacement: "corecext.so"
04-01 23:48:24.230 10853 10895 I python  : [INFO   ] [Logger      ] Record log in /data/user/0/org.test.myapp/files/app/.kivy/logs/kivy_18-04-01_4.txt
04-01 23:48:24.230 10853 10895 I python  : [INFO   ] [Kivy        ] v1.10.0
04-01 23:48:24.230 10853 10895 I python  : [INFO   ] [Python      ] v3.5.1 (default, Mar 30 2018, 13:14:12) 
04-01 23:48:24.230 10853 10895 I python  : [GCC 5.3 20151204]
04-01 23:48:24.240 10853 10895 W linker  : /data/data/org.test.myapp/files/app/crystax_python/site-packages/kivy/_clock.so: is missing DT_SONAME will use basename as a replacement: "_clock.so"

04-01 23:48:24.522 10853 10895 W linker  : /data/data/org.test.myapp/files/app/crystax_python/site-packages/kivy/_event.so: is missing DT_SONAME will use basename as a replacement: "_event.so"
04-01 23:48:24.527 10853 10895 W linker  : /data/data/org.test.myapp/files/app/crystax_python/site-packages/kivy/properties.so: is missing DT_SONAME will use basename as a replacement: "properties.so"
04-01 23:48:24.547 10853 10895 I python  : [INFO   ] [Factory     ] 194 symbols loaded
04-01 23:48:25.008 10853 10895 W linker  : /data/data/org.test.myapp/files/app/crystax_python/site-packages/kivy/graphics/instructions.so: is missing DT_SONAME will use basename as a replacement: "instructions.so"
04-01 23:48:25.011 10853 10895 W linker  : /data/data/org.test.myapp/files/app/crystax_python/site-packages/kivy/graphics/buffer.so: is missing DT_SONAME will use basename as a replacement: "buffer.so"
04-01 23:48:25.014 10853 10895 W linker  : /data/data/org.test.myapp/files/app/crystax_python/site-packages/kivy/graphics/vertex.so: is missing DT_SONAME will use basename as a replacement: "vertex.so"
04-01 23:48:25.017 10853 10895 W linker  : /data/data/org.test.myapp/files/app/crystax_python/site-packages/kivy/graphics/cgl.so: is missing DT_SONAME will use basename as a replacement: "cgl.so"
04-01 23:48:25.022 10853 10895 W linker  : /data/data/org.test.myapp/files/app/crystax_python/site-packages/kivy/graphics/vbo.so: is missing DT_SONAME will use basename as a replacement: "vbo.so"
04-01 23:48:25.025 10853 10895 W linker  : /data/data/org.test.myapp/files/app/crystax_python/site-packages/kivy/graphics/compiler.so: is missing DT_SONAME will use basename as a replacement: "compiler.so"
04-01 23:48:25.028 10853 10895 W linker  : /data/data/org.test.myapp/files/app/crystax_python/site-packages/kivy/graphics/transformation.so: is missing DT_SONAME will use basename as a replacement: "transformation.so"
04-01 23:48:25.032 10853 10895 W linker  : /data/data/org.test.myapp/files/app/crystax_python/site-packages/kivy/graphics/shader.so: is missing DT_SONAME will use basename as a replacement: "shader.so"
04-01 23:48:25.035 10853 10895 W linker  : /data/data/org.test.myapp/files/app/crystax_python/site-packages/kivy/graphics/texture.so: is missing DT_SONAME will use basename as a replacement: "texture.so"
04-01 23:48:25.041 10853 10895 W linker  : /data/data/org.test.myapp/files/app/crystax_python/site-packages/kivy/graphics/context_instructions.so: is missing DT_SONAME will use basename as a replacement: "context_instructions.so"
04-01 23:48:25.287 10853 10895 W linker  : /data/data/org.test.myapp/files/app/crystax_python/site-packages/kivy/core/image/_img_sdl2.so: is missing DT_SONAME will use basename as a replacement: "_img_sdl2.so"
04-01 23:48:25.298 10853 10895 I python  : [INFO   ] [Image       ] Providers: img_tex, img_dds, img_sdl2, img_gif (img_pil, img_ffpyplayer ignored)
04-01 23:48:25.304 10853 10895 W linker  : /data/data/org.test.myapp/files/app/crystax_python/site-packages/kivy/graphics/fbo.so: is missing DT_SONAME will use basename as a replacement: "fbo.so"
04-01 23:48:25.310 10853 10895 W linker  : /data/data/org.test.myapp/files/app/crystax_python/site-packages/kivy/graphics/context.so: is missing DT_SONAME will use basename as a replacement: "context.so"
04-01 23:48:25.321 10853 10895 W linker  : /data/data/org.test.myapp/files/app/crystax_python/site-packages/kivy/graphics/opengl.so: is missing DT_SONAME will use basename as a replacement: "opengl.so"
04-01 23:48:25.327 10853 10895 W linker  : /data/data/org.test.myapp/files/app/crystax_python/site-packages/kivy/graphics/opengl_utils.so: is missing DT_SONAME will use basename as a replacement: "opengl_utils.so"
04-01 23:48:25.352 10853 10895 W linker  : /data/data/org.test.myapp/files/app/crystax_python/site-packages/kivy/graphics/vertex_instructions.so: is missing DT_SONAME will use basename as a replacement: "vertex_instructions.so"
04-01 23:48:25.360 10853 10895 W linker  : /data/data/org.test.myapp/files/app/crystax_python/site-packages/kivy/graphics/stencil_instructions.so: is missing DT_SONAME will use basename as a replacement: "stencil_instructions.so"
04-01 23:48:25.368 10853 10895 W linker  : /data/data/org.test.myapp/files/app/crystax_python/site-packages/kivy/graphics/gl_instructions.so: is missing DT_SONAME will use basename as a replacement: "gl_instructions.so"
04-01 23:48:25.375 10853 10895 W linker  : /data/data/org.test.myapp/files/app/crystax_python/site-packages/kivy/graphics/scissor_instructions.so: is missing DT_SONAME will use basename as a replacement: "scissor_instructions.so"
04-01 23:48:25.382 10853 10895 W linker  : /data/data/org.test.myapp/files/app/crystax_python/site-packages/kivy/weakproxy.so: is missing DT_SONAME will use basename as a replacement: "weakproxy.so"
04-01 23:48:25.392 10853 10895 W linker  : /data/data/org.test.myapp/files/app/crystax_python/site-packages/kivy/core/text/text_layout.so: is missing DT_SONAME will use basename as a replacement: "text_layout.so"
04-01 23:48:25.399 10853 10895 W linker  : /data/data/org.test.myapp/files/app/crystax_python/site-packages/kivy/core/text/_text_sdl2.so: is missing DT_SONAME will use basename as a replacement: "_text_sdl2.so"
04-01 23:48:25.402 10853 10895 I python  : [INFO   ] [Text        ] Provider: sdl2
04-01 23:48:26.219 10853 10895 I python  : Running in foo
04-01 23:48:26.219 10853 10895 I python  : Explicit context to bar
04-01 23:48:26.219 10853 10895 I python  : Explicit context switch to foo again
04-01 23:48:26.219 10853 10895 I python  : Implicit context switch back to bar
04-01 23:48:26.338 10853 10895 I python  : Python for android ended.

And buildozer.spec:

requirements = python3crystax,kivy,setuptools,hostpython3crystax,gevent

Fixes CFLAGS, LDSHARED and LDFLAGS to point to Crystax.

AndreMiras · 2018-04-01T21:37:21Z

pythonforandroid/archs.py

        if self.ctx.ndk == 'crystax':
            env['LDFLAGS'] += ' -L{}/sources/crystax/libs/{} -lcrystax'.format(self.ctx.ndk_dir, self.arch)
+            env['CFLAGS'] += ' -I{}/sources/python/{}/include/python/'.format(
+                self.ctx.ndk_dir, self.ctx.python_recipe.version[0:3])


I believe other recipes should benefit from it.

This is now probably addressed in #1361

AndreMiras · 2018-04-01T21:40:56Z

pythonforandroid/recipes/greenlet/__init__.py

+    def get_recipe_env(self, arch=None, with_flags_in_cc=True):
+        env = super(GreenletRecipe, self).get_recipe_env(arch, with_flags_in_cc)
+        # sets linker to use the correct gcc (cross compiler)
+        env['LDSHARED'] = env['CC'] + ' -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions'


Otherwise the default host linker was picked up rather than the ARM cross compiler.

Also addressed in #1361

AndreMiras · 2018-04-01T21:42:52Z

pythonforandroid/recipes/greenlet/__init__.py

+        # required additional library and path for Crystax
+        if self.ctx.ndk == 'crystax':
+            env['LDFLAGS'] += ' -L{}'.format(os.path.join(self.ctx.bootstrap.build_dir, 'libs', arch.arch))
+            env['LDFLAGS'] += ' -lpython{}m'.format(self.ctx.python_recipe.version[0:3])


So that we don't get run time cannot locate symbol errors.

ghost · 2018-09-06T20:24:22Z

Edit: unimportant confused comment 😄

Greenlet Crystax/Python3 support, fixes kivy#1245

95c3d20

Fixes CFLAGS, LDSHARED and LDFLAGS to point to Crystax.

AndreMiras commented Apr 1, 2018

View reviewed changes

This was referenced Apr 1, 2018

Python3 + greenlet install issue #1245

Closed

Python3 + greenlet install issue AndreMiras/EtherollApp#25

Closed

AndreMiras closed this Oct 6, 2018

AndreMiras deleted the feature/ticket1245_greenlet_crystax branch October 6, 2018 17:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Greenlet Crystax/Python3 support, fixes #1245#1250

Greenlet Crystax/Python3 support, fixes #1245#1250
AndreMiras wants to merge 1 commit intokivy:masterfrom
AndreMiras:feature/ticket1245_greenlet_crystax

AndreMiras commented Apr 1, 2018 •

edited

Loading

Uh oh!

AndreMiras Apr 1, 2018

Uh oh!

AndreMiras Sep 11, 2018

Uh oh!

AndreMiras Apr 1, 2018

Uh oh!

AndreMiras Sep 11, 2018

Uh oh!

AndreMiras Apr 1, 2018

Uh oh!

ghost commented Sep 6, 2018 •

edited by ghost

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

AndreMiras commented Apr 1, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AndreMiras Apr 1, 2018

Choose a reason for hiding this comment

Uh oh!

AndreMiras Sep 11, 2018

Choose a reason for hiding this comment

Uh oh!

AndreMiras Apr 1, 2018

Choose a reason for hiding this comment

Uh oh!

AndreMiras Sep 11, 2018

Choose a reason for hiding this comment

Uh oh!

AndreMiras Apr 1, 2018

Choose a reason for hiding this comment

Uh oh!

ghost commented Sep 6, 2018 • edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

AndreMiras commented Apr 1, 2018 •

edited

Loading

ghost commented Sep 6, 2018 •

edited by ghost

Loading