*.so is too small to be an ELF executable

[trivedigaurav]

Running a packaged kivy app on Android causes: "ImportError: dlopen failed: "/data/data/org.test.sms/files/lib/python2.7/site-packages/kivy/properties.so" is too small to be an ELF executable.

Run Log (using logcat): https://gist.github.com/trivedigaurav/74197a06807c4abdb485
Build Log (using P4A): https://gist.github.com/trivedigaurav/8203502e17bd1cf52e31

Related:

• Groups Discussion: https://groups.google.com/d/topic/kivy-users/jvnpObU-Ukc/discussion
• (Possibly) Related Bug: properties.so is not a valid ELF object #77

[trivedigaurav]

Since I was able to build APKs successfully before (it has recently stopped working), I had the helloworld example of an APK (built on my same system) that worked fine. I have uploaded them in case it helps.

Working fine - http://ge.tt/1CwVwah1/v/0
ELF Problem - http://ge.tt/8lmHwah1/v/0

And here's the folder diff output between these two packages: https://gist.github.com/trivedigaurav/2e657dc5f995be6a5729

[trivedigaurav]

I tried to build a distribution using the Ubuntu VM provided in the downloads section on the kivy website as well. I get a very similar error there as well:

I/python  (18650):    File "/home/kivy/code/test/hello/main.py", line 1, in <module>
I/python  (18650):    File "/home/kivy/android/python-for-android/build/python-install/lib/python2.7/site-packages/kivy/app.py", line 321, in <module>
I/python  (18650):    File "/home/kivy/android/python-for-android/build/python-install/lib/python2.7/site-packages/kivy/base.py", line 29, in <module>
I/python  (18650):    File "/home/kivy/android/python-for-android/build/python-install/lib/python2.7/site-packages/kivy/event.py", line 8, in <module>
I/python  (18650):  ImportError: dlopen failed: "/data/data/org.hello.world/files/lib/python2.7/site-packages/kivy/_event.so" has unexpected e_machine: 3
I/python  (18650): Python for android ended.
I/ActivityManager(  815): Process org.hello.world:python (pid 18650) (adj 0) has died.

This confirms that this is not a mac specific issue.

Also, here's the log while creating a distribution using disitribute.sh: http://pastebin.com/LgfZPE1s; It looks like that it may be due an error in the kivy code.

[trivedigaurav]

I am to confirm that I see these errors when I use the local copy of the Kivy code by setting $P4A_kivy_DIR variable. The APKs run just fine when using the default kivy version in P4A (by not setting the variable). This probably suggests this issue may be moved to kivy/kivy.

Thanks to @brousch for helping me confirm these.

[brousch]

I am able to replicate this problem by using the P4A_kivy_DIR environment variable. When using a local kivy clone and that environment setting, the APK is built, but crashes on Android with the "too small to be an ELF error". Additionally, the APKs generated using a local Kivy clone and barebones Hello World program are about 5.6MB. APKs created using the normal python-for-android downloaded Kivy end up around 6.3MB.

My best guess is that python-for-android is doing something different when downloading and extracting Kivy compared to when it's using a local clone.

Here is an APK generated using P4A_kivy_DIR with a local Kivy clone with the 1.8.0 branch checked out: https://www.dropbox.com/s/zugkjaydvtk9t4e/HelloWorld-1.0-debug-local.apk

Here is an APK generated using the normal python-for-android Kivy download of the stable branch: https://www.dropbox.com/s/uc6dgxiekmbvmj3/HelloWorld-1.0-debug-normal.apk

[kived]

FYI - this usually happens for me when there is a C extension conflict. C module names (NOT including namespace!) must be unique. So, for example, if you have package1.module and package2.module as C extensions, this conflict will happen (both are called module, containing functions init_module() and generate module.so). This is because all .so files are packed into a single libpymodules.so. The .so files in the Python folders are fakes (0 bytes) to trick the interpreter into loading modules which are already loaded via libpymodules.so.

Things get even trickier with pure C (non-Cython) extensions, like cdecimal. Here we have to worry about ALL function name collisions, because nothing is guaranteed to have a unique name, whereas Cython generates names like __pyx_gb_4kivy_6_event_2generator(). The module init function will still be init_event() for kivy._event whether it's pure C or Cython.

FWIW we always use P4A_kivy_DIR, as we have our own modifications to Kivy.

[brousch]

I can now reliably reproduce and fix this bug using plain python-for-android and buildozer.

Buildozer:

1. Set the P4A_kivy_DIR to a local clone of Kivy
2. Run buildozer android debug - this will produce a working APK
3. Delete the app's .buildozer dir
4. Run buildozer android debug - this will produce an APK with the small ELF problem
5. Delete the Kivy clone and the app's .buildozer dir
6. Run buildozer android debug - this will produce a working APK

python-for-android

1. Set the P4A_kivy_DIR to a local clone of Kivy
2. Run ./distribute.sh -m "kivy" - this will produce a working distribution
3. Delete the build and dist dirs
4. Run ./distribute.sh -m "kivy" - this will produce a non-working distribution
5. Delete the Kivy clone and the build and dist dirs
6. Run ./distribute.sh -m "kivy" - this will produce a working distribution

[brousch]

I've narrowed the problem further. You only need to delete the dist directory to trigger it. In buildozer, that's .buildozer/android/platform/python-for-android/dist and in python-for-android that's dist.

[trivedigaurav]

@brousch I am able to reproduce this issue (ie. create a distribution that makes an apk with ELF errors) with the following steps using python-for-android:

1. set p4a variable in ~/.bashrc:

   export P4A_kivy_DIR = /home/code/kivy

2. go to P4A directory

3. clean builds
   rm -rf build/

4. also remove the previous dist (you may skip this step if you'd be creating another distribution with a new name using distribute.sh -d option)

   rm -rf dist/default

5. now create the distribution

   distribute.sh -m "kivy"

6. Run build.sh from the new dist

In order to create, a working distribution you need to follow all of the above steps but without the P4A variable set. Remember to source bashrc again or open a new terminal.

My guess is that we may have introduced the problem mentioned by @kived between the stable version of kivy and now.

[trivedigaurav]

@akshayaurora has further narrowed this issue. You can work around this issue by deleting the build/ directory from your P4A_kivy_DIR before you create a distribution. Make sure you clean your P4A directory (builds and dists) as well before you do this the first time.

[inclement]

The problem seems to definitely be that the object files in an existing build directory are not packed properly into libpymodules.so - you can see by comparison of apk sizes, a working apk is about 0.7MB larger, and unpacking reveals this difference is all in libpymodules.so. Checking that with objdump, sure enough the not working one doesn't include everything from kivy.

I still don't know why (looking into it still), but adding that here in case it helps anyone.

[inclement]

I think I tracked this down to being related to the most recent change in the kivy recipe, replacing 'cython' with '$CYTHON' which is really 'cython -t'. Does anyone else find that reverting that change makes things work again?

I asked tito about it (copied below to not forget...), and he gave directions to investigate further so I'll keep going - the correct fix is probably not to revert the above change (it should work) but to fix something else that causes the problem.

00:25 <+tito> what symbol are missing?
00:27 <+tito> that should give an hint about what's failing
00:28 <+tito> and the issue is now either on the strip, biglink, or deduplicate
00:28 -!- jwinterm [~quassel@cpe-76-184-247-51.tx.res.rr.com] has quit [Ping timeout: 252 seconds]
00:28 <+tito> strip -> in recipe/XX.sh usually, done manually
00:28 <+tito> biglink -> the python script
00:28 <+tito> deduplicate -> the tiny sh function (uber uber hackish)
00:29 <+tito> i bet deduplicating twice, maybe.

Of these, I think biglink is fine - the kivy files are already missing from the argument directory passed to it. I pasted the files it sees at http://inclem.net/files/kivy/biglink_sofiles.txt .

[inclement]

I did a pr for a fix, but it needs some discussion and confirmation about what's really going on - I wrote my semi-speculation there, but I don't follow all the technical details and it might be the wrong way to change things.