[Distutils] What is the syntax for passing conditional non-extra dependencies in setuptools?

Discussion:

Daniel Holth

2014-03-27 03:29:31 UTC

How do I specify a conditional (marker-guarded) non-extra dependency
in setuptools? The syntax for a conditional extra dependency is
currently:

extras_require = {
"ssl:sys_platform=='win32'": "wincertstore==0.2",
"certs": "certifi==1.0.1",
},

PJ Eby

2014-03-27 18:39:46 UTC

Permalink

Post by Daniel Holth
How do I specify a conditional (marker-guarded) non-extra dependency
in setuptools? The syntax for a conditional extra dependency is
extras_require = {
"ssl:sys_platform=='win32'": "wincertstore==0.2",
"certs": "certifi==1.0.1",
},

I only implemented support via extras, and the feature wasn't officially
supported (still isn't, I don't think) because the PEP specifying the
syntax wasn't fully baked yet. I figured that if *only* setuptools itself
used it, then if the syntax changed only setuptools would break... but fix
itself at the same time.

The same cannot be said for any other package, so use at your own risk. Or
better yet, don't use it. ;-)

(At least, not until it's a documented feature w/a PEP-approved syntax.)

_______________________________________________

Post by Daniel Holth
https://mail.python.org/mailman/listinfo/distutils-sig

Nick Coghlan

2014-03-27 21:16:56 UTC

Permalink

Post by PJ Eby

I only implemented support via extras, and the feature wasn't officially

supported (still isn't, I don't think) because the PEP specifying the
syntax wasn't fully baked yet. I figured that if *only* setuptools itself
used it, then if the syntax changed only setuptools would break... but fix
itself at the same time.

Post by PJ Eby
The same cannot be said for any other package, so use at your own risk.

Or better yet, don't use it. ;-)

Post by PJ Eby
(At least, not until it's a documented feature w/a PEP-approved syntax.)

Various things got in the way of finalising at least PEP 440, but as far as
I am aware, it, 426 and 459 are basically done aside from updating the JSON
schema specs and redoing the PyPI compatibility analysis.

The other thing they need is for people to read through from scratch,
looking for inconsistencies and things that seem out of place, and
reporting those as issues at
https://bitbucket.org/pypa/pypi-metadata-formats

Cheers,
Nick.

Post by PJ Eby

Post by Daniel Holth
_______________________________________________
https://mail.python.org/mailman/listinfo/distutils-sig

_______________________________________________
https://mail.python.org/mailman/listinfo/distutils-sig

Donald Stufft

2014-03-27 21:18:16 UTC

Permalink

I only implemented support via extras, and the feature wasn't officially supported (still isn't, I don't think) because the PEP specifying the syntax wasn't fully baked yet. I figured that if *only* setuptools itself used it, then if the syntax changed only setuptools would break... but fix itself at the same time.
The same cannot be said for any other package, so use at your own risk. Or better yet, don't use it. ;-)
(At least, not until it's a documented feature w/a PEP-approved syntax.)

Various things got in the way of finalising at least PEP 440, but as far as I am aware, it, 426 and 459 are basically done aside from updating the JSON schema specs and redoing the PyPI compatibility analysis.
The other thing they need is for people to read through from scratch, looking for inconsistencies and things that seem out of place, and reporting those as issues at https://bitbucket.org/pypa/pypi-metadata-formats

I can take a read through everything again this weekend and take a close look.

Cheers,
Nick.

Post by Daniel Holth
_______________________________________________
https://mail.python.org/mailman/listinfo/distutils-sig

_______________________________________________
https://mail.python.org/mailman/listinfo/distutils-sig

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

Nick Coghlan

2014-03-27 21:27:28 UTC

Permalink

Post by Nick Coghlan

Post by PJ Eby

I only implemented support via extras, and the feature wasn't

officially supported (still isn't, I don't think) because the PEP
specifying the syntax wasn't fully baked yet. I figured that if *only*
setuptools itself used it, then if the syntax changed only setuptools would
break... but fix itself at the same time.

Post by Nick Coghlan

Post by PJ Eby
The same cannot be said for any other package, so use at your own

risk. Or better yet, don't use it. ;-)

Post by Nick Coghlan

Post by PJ Eby
(At least, not until it's a documented feature w/a PEP-approved syntax.)

Various things got in the way of finalising at least PEP 440, but as far

as I am aware, it, 426 and 459 are basically done aside from updating the
JSON schema specs and redoing the PyPI compatibility analysis.

Actually, there are a some other existing issues I need to address as well
(I especially want to do the SPDX one):
https://bitbucket.org/pypa/pypi-metadata-formats/issues?status=new&status=open&component=Metadata%202.x

Cheers,
Nick.

Post by Nick Coghlan
The other thing they need is for people to read through from scratch,

looking for inconsistencies and things that seem out of place, and
reporting those as issues at
https://bitbucket.org/pypa/pypi-metadata-formats

Post by Nick Coghlan
Cheers,
Nick.

Post by PJ Eby

Post by Daniel Holth
_______________________________________________
https://mail.python.org/mailman/listinfo/distutils-sig

_______________________________________________
https://mail.python.org/mailman/listinfo/distutils-sig

Daniel Holth

2014-04-07 19:09:05 UTC

Permalink

I think setuptools' :-separated environment markers mechanism is
great, and shouldn't need to be ratified anywhere. I also think the
actual environment markers specification is stable, although I noticed
the IPython wheel for example is using the deprecated .-separated
rather than _-separated identifiers.

These kinds of conditionals are obviously an important feature for
wheel and replace a common if/else statement inside setup.py, but can
only be specified in wheels by placing a copy of the requirements in
setup.cfg. It would be better if bdist_wheel could also just translate
the setuptools requirements.txt to its own format.

I assume extras_require = { ":sys_platform='win32'" :
"wincertstore==0.2" } would be likely to specify a non-extra (the
extra named '' or None?) dependency?

Post by PJ Eby

I only implemented support via extras, and the feature wasn't officially
supported (still isn't, I don't think) because the PEP specifying the syntax
wasn't fully baked yet. I figured that if *only* setuptools itself used it,
then if the syntax changed only setuptools would break... but fix itself at
the same time.
The same cannot be said for any other package, so use at your own risk. Or
better yet, don't use it. ;-)
(At least, not until it's a documented feature w/a PEP-approved syntax.)

Post by Daniel Holth
_______________________________________________
https://mail.python.org/mailman/listinfo/distutils-sig

Daniel Holth

2014-04-07 20:03:01 UTC

Permalink

OK, it does in fact work that way; empty-string-before-colon specifies
a default requirement with a marker.

Parses out to the following _dep_map with 'None' representing 'not an
extra dependency':

{None: [Requirement.parse('coding'), Requirement.parse('stuff')],
'quux': [Requirement.parse('more-stuff')]}

requirements.txt:

coding

[:sys_platform=='linux2']
stuff

[quux:sys_platform=='linux2']
more_stuff

Post by Daniel Holth
I think setuptools' :-separated environment markers mechanism is
great, and shouldn't need to be ratified anywhere. I also think the
actual environment markers specification is stable, although I noticed
the IPython wheel for example is using the deprecated .-separated
rather than _-separated identifiers.
These kinds of conditionals are obviously an important feature for
wheel and replace a common if/else statement inside setup.py, but can
only be specified in wheels by placing a copy of the requirements in
setup.cfg. It would be better if bdist_wheel could also just translate
the setuptools requirements.txt to its own format.
"wincertstore==0.2" } would be likely to specify a non-extra (the
extra named '' or None?) dependency?

Post by PJ Eby

I only implemented support via extras, and the feature wasn't officially
supported (still isn't, I don't think) because the PEP specifying the syntax
wasn't fully baked yet. I figured that if *only* setuptools itself used it,
then if the syntax changed only setuptools would break... but fix itself at
the same time.
The same cannot be said for any other package, so use at your own risk. Or
better yet, don't use it. ;-)
(At least, not until it's a documented feature w/a PEP-approved syntax.)

Post by Daniel Holth
_______________________________________________
https://mail.python.org/mailman/listinfo/distutils-sig

PJ Eby

2014-04-07 23:00:23 UTC

Permalink

Post by Daniel Holth
OK, it does in fact work that way; empty-string-before-colon specifies
a default requirement with a marker.
Parses out to the following _dep_map with 'None' representing 'not an
{None: [Requirement.parse('coding'), Requirement.parse('stuff')],
'quux': [Requirement.parse('more-stuff')]}
coding
[:sys_platform=='linux2']
stuff
[quux:sys_platform=='linux2']
more_stuff

Huh. Either I did that intentionally and forgot about it, or it just fell
out as a brilliant accidental side effect of the orthogonal design of the
parsing functions. Either way, I feel like I did something smart there.
;-)

Nick Coghlan

2014-04-08 03:40:35 UTC

Permalink

I think the one open question was whether or not to tolerate those dots
from the previous iteration of the syntax where applicable.

Cheers,
Nick.

Daniel Holth

2014-04-08 17:34:10 UTC

Permalink

I think the one open question was whether or not to tolerate those dots from
the previous iteration of the syntax where applicable.

I'd lean towards saying we are stuck with them for now in the
.dist-info directories or requirements.txt, but setuptools could
choose to forbid you from passing new dots through setup.py. The only
problem the dots caused was forgetting whether you should use dots or
underscores, and now it's easy to remember that the answer is always
underscore.