bpo-1294959 - better support for systems with /usr/lib64 by matejcik · Pull Request #3698 · python/cpython

matejcik · 2017-09-22T16:45:18Z

This introduces a new configure and sysconfig variable platsubdir, which is the name of subdirectory of prefix that holds platform-specific libraries. On many Linux distributions, there is separation between /usr/lib and /usr/lib64, as opposed to prefix vs exec_prefix, which is what Python is accustomed to.

This should improve the situation, and platsubdir will be used to differentiate library subdirectories below prefix.

Based on last comments in the bug report http://bugs.python.org/issue1294959 , I have renamed "platlibdir" to "platsubdir" -- IMHO it's more important to note that this is platform-specific than that it is a library directory. (otherwise we'd end up with platlibsubdir and that's horrible ;) )

https://bugs.python.org/issue1294959

matejcik · 2017-09-22T16:46:58Z

didn't add a NEWS entry because I don't know how to do it ... is there a script that generates the file names for the NEWS.d structure? Or can I just drop a file called "bpo-1294959.txt" there?

stratakis · 2017-09-22T16:56:41Z

@matejcik You will have to use blurb for the NEWS entry: https://pypi.python.org/pypi/blurb

hroncok · 2017-09-23T17:05:56Z

In Fedora, we patch Python to support lib64 as follows:

https://src.fedoraproject.org/rpms/python3/blob/master/f/00102-lib64.patch (only applied on 64bit builds)
https://src.fedoraproject.org/rpms/python3/blob/master/f/00205-make-libpl-respect-lib64.patch (always applied)

This is only a note, I haven't looked at this PR more closely just yet, I'll try to compare the patches as soon as I can, to see if this PR would enable us to drop both of them.

hroncok · 2017-09-24T10:35:19Z

Apparently, replacing those 2 patches with this one builds and tests fine. https://koji.fedoraproject.org/koji/taskinfo?taskID=22047740

matejcik · 2017-10-11T13:00:35Z

news entry added

hroncok · 2017-11-28T12:33:03Z

What can we do to make this happen?

encukou · 2018-05-15T21:03:37Z

I am a core dev now and I can commit to pushing this through.
Please rebase and re-check on Fedora; I can then do the sanity check on Ubuntu that Matthias requested on the issue.
If this ever again becomes stuck on the CPython side, ping me directly.

grimreaper

General comment: I wonder if it would make sense to make use of ax_set_default_paths_system.m4 as well

grimreaper · 2018-05-16T07:49:02Z

configure.ac

+# platsubdir must be defined before LIBPL definition
+AC_MSG_CHECKING(for custom platsubdir)
+AC_ARG_WITH(custom-platsubdir,
+    [AS_HELP_STRING([--with-custom-platsubdir=<libdirname>],


instead of --with-custom... just use --with. Also this is more commonly called --with-libdir

the whole point of the PR is that this is distinct from (how Python uses) libdir

okay, I've finally had a chance to re-read the original bug and this code. I'm not the biggest fan of introducing Yet Another variable and would prefer to fix how libdir gets used, but that seems like a bigger and more invasive change. Lets at least remove the custom portion of the name.

While I likely won't get to this for some time, it might be worth a more general conversation about what tunables we offer in configure, what they mean, and how they get used.

grimreaper · 2018-05-16T07:51:53Z

Lib/test/test_sysconfig.py

                global_path = global_path.replace(sys.base_prefix, sys.prefix)
                base = base.replace(sys.base_prefix, sys.prefix)
+
+            if platsubdir != "lib":


I don't like how we're special casing "lib" here. I'd prefer to always take the same code path and use some special token that gets replaced.

this is the same thing the code did before though.
i suppose we could introduce puresubdir as the other option and prefill it to lib somewhere, is that what you're suggesting?

hm, no, i misread which part of the code you're commenting. sorry.

grimreaper · 2018-05-16T07:52:12Z

Misc/NEWS.d/next/Build/2017-10-11-14-52-36.bpo-1294959.Ukwa9V.rst

@@ -0,0 +1,3 @@
+On 64bit Linux systems that use ``$prefix/lib64``, this is now the default


This is not specific to Linux.

vstinner · 2019-01-31T11:48:33Z

@matejcik: I'm now interested to review your PR, but first I would like to see conflicts resolved. Would you like to fix conflicts? Or do you prefer that I create a new PR based on your PR?

This introduces a new configure and sysconfig variable `platsubdir`, which is the name of subdirectory of $prefix that holds platform-specific libraries. On many Linux distributions, there is separation between /usr/lib and /usr/lib64, which was causing some problems due to the way Python manages its install paths. Now, platsubdir is used to select the appropriate lib directory.

news entry news entry

matejcik · 2019-01-31T15:13:28Z

@vstinner thanks, I fixed the basic conflicts.

Unfortunately, I see that the code (esp. in Modules/getpath.c) has drifted significantly, and this is no longer part of my day job. So I probably won't devote much time in getting this where it needs to go. Maybe @hroncok would be interested in taking over, or perhaps @mcepl ?

hroncok · 2019-01-31T15:18:21Z

Not me. I can help @vstinner get this done and tested in Fedora, but that's it, sorry.

mcepl · 2019-01-31T15:20:50Z

@matejcik @hroncok I hopefully will get to this later, but currently I struggle with fixing many packages failing on 3.7.*.

stratakis · 2019-01-31T17:04:37Z

I can start working on it, should be having some more time soon.

stratakis · 2019-02-04T15:13:08Z

Opened #11755 in order to move this forward.

csabella · 2019-05-23T23:03:57Z

Thank you to everyone for your work on this. I'm going to close this in favor of #11755 since that (at the time) was a rebase over master. If this one ends up being the better pull request to move the project forward, then this can be reopened.

vstinner · 2020-02-12T09:16:57Z

PR #11755 is an updated version of this PR. (EDIT: oops, I missed @stratakis's comment.)

the-knights-who-say-ni added the CLA signed label Sep 22, 2017

bedevere-bot added the awaiting review label Sep 22, 2017

grimreaper suggested changes May 16, 2018

View reviewed changes

bedevere-bot added awaiting core review and removed awaiting review labels May 16, 2018

matejcik added 2 commits January 31, 2019 16:06

acknowledge myself

2a6e255

news entry news entry

matejcik force-pushed the build-64bit branch from f305dea to 2a6e255 Compare January 31, 2019 15:07

stratakis mentioned this pull request Feb 4, 2019

[WIP] bpo-1294959: Better support for systems with /usr/lib64 #11755

Closed

brettcannon added the type-feature A feature request or enhancement label Mar 29, 2019

csabella closed this May 23, 2019

vstinner mentioned this pull request Feb 11, 2020

bpo-1294959: Add --with-platlibdir to configure #18381

Merged

		@@ -0,0 +1,3 @@
		On 64bit Linux systems that use ``$prefix/lib64``, this is now the default

Uh oh!

Conversation

matejcik commented Sep 22, 2017 • edited by bedevere-bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

matejcik commented Sep 22, 2017

Uh oh!

stratakis commented Sep 22, 2017

Uh oh!

hroncok commented Sep 23, 2017

Uh oh!

hroncok commented Sep 24, 2017

Uh oh!

matejcik commented Oct 11, 2017

Uh oh!

hroncok commented Nov 28, 2017

Uh oh!

encukou commented May 15, 2018

Uh oh!

grimreaper left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vstinner commented Jan 31, 2019

Uh oh!

matejcik commented Jan 31, 2019

Uh oh!

hroncok commented Jan 31, 2019

Uh oh!

mcepl commented Jan 31, 2019

Uh oh!

stratakis commented Jan 31, 2019

Uh oh!

stratakis commented Feb 4, 2019

Uh oh!

csabella commented May 23, 2019

Uh oh!

vstinner commented Feb 12, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants

matejcik commented Sep 22, 2017 •

edited by bedevere-bot

Loading

vstinner commented Feb 12, 2020 •

edited

Loading