Use python3's venv instead of virtualenv by opacam · Pull Request #2152 · kivy/python-for-android

opacam · 2020-04-24T21:12:26Z

Here:

we remove one p4a dependency virtualenv
move from pytoml to toml, because pytoml is deprecated
remove unneeded code related with virtualenv

Notes:

close Fix a problem with the way we were calling virtualenv #2145

Because `pytoml` is deprecated See also: https://pypi.org/project/pytoml/

Because when running local tests for `test_pythonpackages` and we have some virtual envs created with tox the function also copies the hidden tox folder causing error on our tests

So this way we will get hostpython's _ssl module build, and we will be able to use hostpython's pip without security errors.

AndreMiras

Well done, thanks!

AndreMiras · 2020-04-24T21:31:53Z

We probably also want to ditch virtualenv<20 from install_reqs on setup.py.
But I think this looks good enough for a merge

AndreMiras · 2020-04-24T22:32:45Z

I'm not so sure yet why the macOS build is failing on Travis only with this error.
https://travis-ci.org/github/kivy/python-for-android/jobs/679209347

/Applications/Xcode-11.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libpython3.8.a(pymath.o) has no symbols
gcc     -Wl,-stack_size,1000000  -framework CoreFoundation -o python.exe Programs/python.o libpython3.8.a -ldl   -framework CoreFoundation    
gcc     -Wl,-stack_size,1000000  -framework CoreFoundation -o Programs/_testembed Programs/_testembed.o libpython3.8.a -ldl   -framework CoreFoundation    
./python.exe -E -S -m sysconfig --generate-posix-vars ;\
	if test $? -ne 0 ; then \
		echo "generate-posix-vars failed" ; \
		rm -f ./pybuilddir.txt ; \
		exit 1 ; \
	fi
 CC='gcc' LDSHARED='gcc -bundle -undefined dynamic_lookup    ' OPT='-DNDEBUG -g -fwrapv -O3 -Wall' 	_TCLTK_INCLUDES='' _TCLTK_LIBS='' 	./python.exe -E /Users/travis/.python-for-android/build/other_builds/hostpython3/desktop/hostpython3/setup.py  build
running build
running build_ext
error: [Errno 2] No such file or directory: '/usr/local/include/python3.8/pyconfig.h'
make[1]: *** [sharedmods] Error 1
  STDERR:
make: *** [testapps/armeabi-v7a] Error 1
The command "make testapps/armeabi-v7a PYTHON_WITH_VERSION=python3" exited with 2.

because it seems that hostpython's build scripts gets confused on about locating the `pyconfig.h` file **Note:** This only happens to macOS platform

opacam · 2020-04-25T15:41:32Z

We probably also want to ditch virtualenv<20 from install_reqs on setup.py...

This is already done at commit fdd66d3 😉

I'm not so sure yet why the macOS build is failing on Travis only with this error...

Ok, I've been looking into a solution to this problem with macOs and it's clearly related with virtualenv/venv. When trying to run p4a inside a virtualenv/venv the hostpython's build scripts gets confused on about to locating the pyconfig.h file. So the solution is to not use virtualenv or venv when trying to build from a macOs...:thinking:...of course is a partial solution (it's more like a shortcut than a fix), but will solve the problem for now...probably this should be addressed at CPython.

Note: kivy-ios seems to has the same problem, kivy/kivy-ios#401

AndreMiras

Nice well done, I'm good for a merge

AndreMiras · 2020-04-25T16:25:41Z

tests/test_pythonpackage_basic.py

+            subprocess.check_output([
+                os.path.join(test_dir, "venv", "bin", "pip"),
+                "install", "-U", "toml"
+            ])


This is a bit out of scope of this PR, but I'm thinking why don't we pip install -e then?

Well I suppose that you are speaking of pip install -e . right?

In this case...why should we? we don't need all p4a dependencies for this test, and the venv will be destroyed after is passed. Also, considering that we are speaking of one of the slowest tests of p4a, IMHO, better to try to save some time 😉

makes sense 👍

inclement

Awesome clean up, thanks!

This PR removes the `virtualenv` dependency from `setup.py`, as it looks redundant. To be fair, I am not fully certain why setup.py was declaring `virtualenv` as needed, I think it's a historical remnant of python 2 days. The min required python version is atm 3.6: https://github.com/kivy/buildozer/blob/9503f5fd153f3f08e9160799fc04b23faf6aabeb/setup.py#L16 `venv` is part of the stdlib since python 3.3, and "is now recommended for creating virtual environments" since 3.5: https://docs.python.org/3/library/venv.html Note that `venv` is a subset of `virtualenv`, but buildozer AFAICT is only using that subset. Similar change in p4a: kivy/python-for-android#2152

opacam added 4 commits April 24, 2020 20:23

👽 Migrate to toml and apply isort to pythonpackage.py

16d38ad

Because `pytoml` is deprecated See also: https://pypi.org/project/pytoml/

🐛 Avoid copying .tox folder when extracting metadata from package

d6bc038

Because when running local tests for `test_pythonpackages` and we have some virtual envs created with tox the function also copies the hidden tox folder causing error on our tests

🔥 Use python3's venv instead of virtualenv

fdd66d3

💚 Add new dependency to docker libssl-dev

e520735

So this way we will get hostpython's _ssl module build, and we will be able to use hostpython's pip without security errors.

AndreMiras previously approved these changes Apr 24, 2020

View reviewed changes

AndreMiras mentioned this pull request Apr 25, 2020

Fix a problem with the way we were calling virtualenv #2145

Closed

opacam added 2 commits April 25, 2020 16:36

📝 Update outdated inline comment

ab2e942

🍎 Don't run p4a inside virtualenv/venv

9b83258

because it seems that hostpython's build scripts gets confused on about locating the `pyconfig.h` file **Note:** This only happens to macOS platform

opacam dismissed AndreMiras’s stale review via 9b83258 April 25, 2020 14:54

AndreMiras approved these changes Apr 25, 2020

View reviewed changes

inclement approved these changes Apr 25, 2020

View reviewed changes

inclement merged commit 6781179 into kivy:develop Apr 25, 2020

opacam deleted the feature-update-deps branch April 26, 2020 08:35

SomberNight mentioned this pull request Oct 14, 2022

dependencies: rm virtualenv, already using python3 -m venv kivy/buildozer#1515

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use python3's venv instead of virtualenv#2152

Use python3's venv instead of virtualenv#2152
inclement merged 6 commits intokivy:developfrom
opacam:feature-update-deps

opacam commented Apr 24, 2020

Uh oh!

AndreMiras left a comment

Uh oh!

AndreMiras commented Apr 24, 2020

Uh oh!

AndreMiras commented Apr 24, 2020

Uh oh!

opacam commented Apr 25, 2020

Uh oh!

AndreMiras left a comment

Uh oh!

AndreMiras Apr 25, 2020

Uh oh!

opacam Apr 25, 2020

Uh oh!

AndreMiras Apr 25, 2020

Uh oh!

inclement left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

opacam commented Apr 24, 2020

Uh oh!

AndreMiras left a comment

Choose a reason for hiding this comment

Uh oh!

AndreMiras commented Apr 24, 2020

Uh oh!

AndreMiras commented Apr 24, 2020

Uh oh!

opacam commented Apr 25, 2020

Uh oh!

AndreMiras left a comment

Choose a reason for hiding this comment

Uh oh!

AndreMiras Apr 25, 2020

Choose a reason for hiding this comment

Uh oh!

opacam Apr 25, 2020

Choose a reason for hiding this comment

Uh oh!

AndreMiras Apr 25, 2020

Choose a reason for hiding this comment

Uh oh!

inclement left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants