Based on sampleproject
I've created a fork here.
git clone https://github.com/alexiswl/sampleproject
If our end goal is to have a binary with a simple goal, we need to have a main, so
let's update our simple.py to catch main
import sys
def add_one(number):
return number + 1
if __name__ == "__main__":
print(add_one(int(sys.argv[1])))
/bin/bash -c "$(curl -fsSL https://static.pantsbuild.org/setup/one_step_setup.sh)"
pants.toml
We set allow the pyoxidizer experimentl package, don't generate the setup.py and
use the --release arg for pyoxidizer building
[GLOBAL]
pants_version = "2.14.0"
backend_packages = [
"pants.backend.experimental.python.packaging.pyoxidizer",
"pants.backend.python"
]
[setup-py-generation]
generate_setup_default = false
[pyoxidizer]
args = [ "--release" ]
BUILD
# Copyright 2020 Pants project contributors.
# Licensed under the Apache License, Version 2.0 (see LICENSE).
# This target sets the metadata for all the Python non-test files in this directory.
resource(
name="sampleproject_res",
source="pyproject.toml"
)
# Create pex binary
pex_binary(
name="pex_binary",
entry_point="sample.simple",
dependencies=[":sampleproject_res"]
)
# Required for pyox binary
python_distribution(
name="sample_dist",
dependencies=[
":sampleproject_res"
],
provides=python_artifact(
name="sampleproject",
version="2.0.0"
)
)
# Build pyoxidizer binary
pyoxidizer_binary(
name="pyox_binary",
entry_point="sample.simple",
dependencies=[":sample_dist"]
)
./pants \
--level=debug \
--no-pantsd \
--no-local-cache \
package ":sample_dist"
Which creates the output files:
But if we look inside the wheel, it doesn't seem to have picked up any of the
requirements from the setup.py (the pyproject.toml should read setup.py - should it not?)
$ unzip dist/backend_shim-0.0.0-py3-none-any.whl
Archive: dist/backend_shim-0.0.0-py3-none-any.whl
inflating: backend_shim.py
inflating: backend_shim-0.0.0.dist-info/METADATA
inflating: backend_shim-0.0.0.dist-info/WHEEL
inflating: backend_shim-0.0.0.dist-info/top_level.txt
inflating: backend_shim-0.0.0.dist-info/RECORD
$ cat backend_shim-0.0.0.dist-info/METADATA
Metadata-Version: 2.1
Name: backend-shim
Version: 0.0.0
$ cat backend_shim-0.0.0.dist-info/top_level.txt
backend_shim
$ cat backend_shim-0.0.0.dist-info/RECORD
backend_shim.py,sha256=tMfCMQd30OulboVgGHYa2RHcClFY9pD9ERynjzDVAuE,614
backend_shim-0.0.0.dist-info/METADATA,sha256=giO3YLXz2rXq0eKG983qo0Gk0MvEpB9W1hrQ2EDvmYM,57
backend_shim-0.0.0.dist-info/WHEEL,sha256=2wepM1nk4DS4eFpYrW1TTqPcoGNfHhhO_i5m4cOimbo,92
backend_shim-0.0.0.dist-info/top_level.txt,sha256=EKWQsiPSspK-bGKZTmzpG0it55Yl6-Y032orzB86crY,13
backend_shim-0.0.0.dist-info/RECORD,,
One way to mitigate this appears to be specifying a second build file inside the src/sample directory
via the 'tailor' goal and then adding this file to the distribution.
./pants \
--level=debug \
--no-pantsd \
--no-local-cache \
tailor src/::
This generates a BUILD file under src/sample with just 'python_sources()'.
Let's update the name attribute
src/sample/BUILD
python_sources(
name="lib"
)
Let's now add this BUILD to the dependencies for the python distribution in the top level BUILD
BUILD.python_distrubtion
# Required for pyox binary
python_distribution(
name="sample_dist",
dependencies=[
":sampleproject_res",
"src/sample:lib"
],
provides=python_artifact(
name="sampleproject",
version="2.0.0"
)
)
And re-run the pants package :sample_dist command above.
This outputs two files to the dist directory:
These files DO have the sample/simple.py and `sample/init.py .
$ unzip dist/sample-0.0.0-py3-none-any.whl
Archive: dist/sample-0.0.0-py3-none-any.whl
inflating: sample/__init__.py
inflating: sample/simple.py
inflating: sample-0.0.0.dist-info/METADATA
inflating: sample-0.0.0.dist-info/WHEEL
inflating: sample-0.0.0.dist-info/top_level.txt
inflating: sample-0.0.0.dist-info/RECORD
However, this doesn't have any of the requirements in the METADATA or WHEEL or RECORD like above.
If we update pyproject.toml and set the project section we can set the dependencies in here instead.
It seems odd that pyproject.toml doesn't read these from setup.py.
Let's copy over the dependencies in setup.py into pyproject.toml under the project section.
We can also use this as an opportunity to name the package.
[build-system]
...
[project]
name = "sampleproject"
version = "2.0.0"
dependencies = [
"peppercorn"
]
Let's try rebuild the distribution now.
./pants \
--level=debug \
--no-pantsd \
--no-local-cache \
package :sample_dist
Gives us the following outputs
Let's have a peek in!
$ cat sampleproject-2.0.0.dist-info/METADATA
Metadata-Version: 2.1
Name: sampleproject
Version: 2.0.0
Requires-Dist: peppercorn
Hurrah! So the peppercorn dependency now exists in the METADATA!
./pants \
--level=debug \
--no-pantsd \
--no-local-cache \
package :pyox_binary
And launch!
time dist/pyox_binary/x86_64-unknown-linux-gnu/release/install/pyox_binary 23434
Wow
23435
real 0m0.034s
user 0m0.034s
sys 0m0.001s
I know it's a very simple addition... but that is fast! I'm very excited to see how this scales up with more complex projects!
Anyway, back to the bits that don't quite make sense
Why do we need the src/sample/BUILD file? We should be able to specify
packages under the setuptools package-dir right?
In theory, setuptools SHOULD be able to find the packages under src/.
Even if it can't search correctly, we should be able to explicitly add the package directory under
[build-system]
...
[project]
...
[tool.setuptools]
package-dir = {"" = "src"}
[tool.setuptools.packages.find]
where = [ "src" ]
include = [ "sample" ]
namespaces = false
However, if we retry with src/sample:lib removed from the python_distribution in BUILD, this yields the following error -
17:41:37.86 [ERROR] 1 Exception encountered:
ProcessExecutionFailure: Process 'Run setuptools.build_meta for //:sample_dist' failed with exit code 1.
stdout:
running bdist_wheel
running build
installing to build/bdist.linux-x86_64/wheel
running install
running install_egg_info
stderr:
/home/alexiswl/.cache/pants/named_caches/pex_root/venvs/s/022089bc/venv/lib/python3.9/site-packages/setuptools/config/pyprojecttoml.py:108: _BetaConfiguration: Support for `[tool.setuptools]` in `pyproject.toml` is still *beta*.
warnings.warn(msg, _BetaConfiguration)
error: error in 'egg_base' option: 'src' does not exist or is not a directory
Use `--keep-sandboxes=on_failure` to preserve the process chroot for inspection.
If pyproject.toml won't read setup.cfg, nor setup.py, but
also doesn't support [tool.setuptools], then what is
the best way to package up a project like this?
It doesn't seem right to require a BUILD file inside every package.
Nevertheless, it's a small compromise for the speed pyoxidizer appears to be showing.
Hope this tutorial is helpful to others as well!