HOSTPYTHON Fails to compile module

[j-santander]

Using latest python-for-android, downloaded from github on Ubuntu 15.04
I had problems creating a distribution including Beautiful Soup 4: when the modules were compiled at the run_distribute() function some of the files (in particular bs4/builder/_htmlparser.py) failed to compile with

SyntaxError: ("(unicode error) \N escapes not supported (can't load unicodedata module)", ('../lib/python2.7/site-packages/bs4/builder/_htmlparser.py', 77, None, 'data = u"\N{REPLACEMENT CHARACTER}"\n'))

I tracked this down to the line of run_distribute()

debug "Copy python distribution"
$HOSTPYTHON -OO -m compileall $BUILD_PATH/python-install

Where $HOSTPYTHON is defined HOSTPYTHON="$BUILD_PATH/python-install/bin/python.host"

Now, this was very surprising as build/hostpython/Python-2.7.2/hostpython and $BUILD_PATH/python-install/bin/python.host were the same:

$ cksum build/hostpython/Python-2.7.2/hostpython 
1098726172 11063768 build/hostpython/Python-2.7.2/hostpython
$ cksum dist/test/python-install/bin/python.host 
1098726172 11063768 dist/test/python-install/bin/python.host

but calling build/hostpython/Python-2.7.2/hostpython -OO -m compileall $BUILD_PATH/python-install worked without failure.

In fact, printing the sys.path for both of them: (doing some text replacements as env variables)

echo -e "import sys\nprint sys.path" | '${BUILD_PATH}/python-install/bin/python.host 
['', '${BUILD_PATH}/python-install/lib/python27.zip', '${BUILD_PATH}/python-install/lib/python2.7', '${BUILD_PATH}/python-install/lib/python2.7/plat-linux3', '${BUILD_PATH}/python-install/lib/python2.7/lib-tk', '${BUILD_PATH}/python-install/lib/python2.7/lib-old', '${BUILD_PATH}/python-install/lib/python2.7/lib-dynload', '${HOME}/.local/lib/python2.7/site-packages', '${BUILD_PATH}/python-install/lib/python2.7/site-packages', '${BUILD_PATH}/python-install/lib/python2.7/site-packages/PIL']

and

echo -e "import sys\nprint sys.path" | ${BUILD_hostpython}/hostpython 
['', '/usr/local/lib/python27.zip', '${BUILD_hostpython}/Lib', '${BUILD_hostpython}/Lib/plat-linux3', '${BUILD_hostpython}/Lib/lib-tk', '${BUILD_hostpython}/Lib/lib-old', '${BUILD_hostpython}/build/lib.linux-x86_64-2.7', '${HOME}/.local/lib/python2.7/site-packages', '/usr/local/lib/python2.7/site-packages']

My guess is that python.host being placed in directories of the ARM python is failing to locate platform-dependent modules (I guess unicode support is one?), causing compiling the modules to fail.

Not 100% sure how to solve this. So far I could work-around it by:

1. Doing the compilation manually again using hostpython instead.
2. Copying manually the resulting files to private directory.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[inclement]

I added an option to the SDL2 bootstrap a while ago, --try-system-python-compile, as a workaround for this. I didn't see this issue at the time, but it should be an okay workaround. You need python2.7 installed and in your PATH for it to work.