Tests

[jayvdb]

It would be nice to have some tests for this.

I'd like to use this as part of rpm packaging, forcing all scripts in all built packages to be fast, as they definitely shouldn't need to runtime checks that the setuptools pkg_resources thing is doing. Any checks required can be done at rpm build time.

But to rely on this in all rpm spec files I would need to ensure this bit cant silently break, so need some smoke tests.

An easy-ish way to do some testing is to use tox, and then the test scripts inspect the result of the installed versions in the tox venv.

If you have other plans for testing, I am all ears.

[ninjaaron]

Thanks for opening this issue. Tests are definitely needed. I'll get to thinking about out it. I'm on vacation at the moment, but I will be back to the normal routine in a week and should be able to devote some time to it. Unfortunately, I'm not as experienced or savvy as I'd like to be with testing and CI, but I will try to get up to speed as quickly as possible.

I might make an alternative suggestion, however: fastentrypoints is really only useful when you're building packages the old way, with setuptools directly. If you build your packages as wheels, which is now considered best practice, fastentrypoints is unnecessary. (In fact, the generated script from fastentrypoints was designed to be identical to what is produced by wheels.)

So, I agree that tests are needed, but binary package maintainers should really use wheels instead. I use fastentrypoints primarily for development. I frequently install the packages I maintain with the -e flag, meaning the scripts will run directly out of the source directory so I am always dog-fooding the latest version. This can't be done with wheels, at least not yet, so fastentrypoints still has utility in that situation.

[jayvdb]

RPMs are usually built using setup.py, and tested using setup.py test. See https://build.opensuse.org/project/show/devel:languages:python for a small collection of python packages with .spec files in each. Fedora has the same, as does Debian.

[ninjaaron]

That explains a lot! I am very much aware that most Python packages on Ubuntu use setup.py, but I wasn't sure why. Linux distros should try to use wheels for new packages, but I realize that's easier said than done. I'll get to working on tests when my vacation is over. Thanks again for the info and the issue.

[jayvdb]

One of the problems with wheels is that they still do not produce reproducible builds. i.e. each pip install foo.whl produces a different set of installed files (the cache files). pypa/pip#5648

The testing inside tests inside tox example I was thinking about earlier is https://github.com/pglass/tox-pip-version/tree/master/tests
(found it because I need to write tests for a new tox plugin, and I think that approach should be quite economical)

[jayvdb]

I have finished creating my own example of using tox to test something which requires isolated environments, with command python -c 'assert ...' checking the state within the isolated environment.

https://github.com/jayvdb/tox-backticks

This approach is not so good for verifying what happens when installation fails, as the tox error handling takes over. But it is good for quickly establishing some minimal acceptance criteria to prove it works correctly when used correctly.

[ninjaaron]

Alright, I'm finally starting to look at this, @jayvdb. Sorry for the long delay. To be honest, I'm not exactly sure what I should be testing. This package is so trivial.

Maybe a dummy package and a shell script that installs it with fastentrypoints to make sure they haven't changed the distutils API on me? Any thought about that? I can work on getting that going. Any support or advice you can give about getting started with the automation part would be appreciated.

[jayvdb]

Ya. Maybe read the generated executable and check its contents are what were expected, i.e. not the garbage setuptools creates.

[ninjaaron]

Probably way too late for this to be relevant anymore, but I added a test script. XD

[jayvdb]

Thanks a bundle. I'll attempt to get it packaged and tested today.

[jayvdb]

I linked to this issue at https://discuss.python.org/t/build-system-and-undeclared-dependency-on-it-during-runtime/2455/12 , where there seems to be some fresh energy to tackle the old problem of/with pkg_resources.

Packaged at https://build.opensuse.org/package/show/home:jayvdb:py-fep/python-fastentrypoints - the .spec is a bit messy atm, but that will be fixed after a PRs and a release.

I've set up a build project with ~70 prominent Python packages at https://build.opensuse.org/project/show/home:jayvdb:py-fep , where fastep is used before the build phase, and... boom ... syntax errors caused by fastep

[ninjaaron]

fastep was just something I added as a convenience feature so I could quickly add fastentrypoints to several of my projects quickly without having to type the same thing over and over. I've considered deprecating its use because it really was never intended to work on complex setup.py files that do anything more than call the setup function, and it seems more like a liability than a feature now that people are actually using the project. Certainly, making fastep part of an automated build step indiscriminately for scores of packages, as you've discovered, is well beyond the scale I had in mind when I wrote it. Beyond the fastep command itself, fastentrypoints is not the right fit for every project (I don't even use it in all of my own projects). Its use should really be evaluated on a per-project basis. Ideally, it's a decision that would be made by project maintainers upstream, but I realize that's not always going to be the case.

I can probably do some fixes that will allow fastep to not break setup scripts than import __future__, which seems to be the most common problem, but I don't think there is a theoretical version of fastep that can be universally guaranteed to work.

[jayvdb]

Lets continue the discussion about fastep on the #18