If you use CPython Wrote an extension , Then pack it to wheel In the release , How to operate ？ You search the Internet , No matter in English or Chinese , All the answers are half understood . According to the official documents , You may be able to finish one soon wheel package , But with the real wheel The bag is thousands of miles away . Here we mainly consider two issues ：1. Package structure ,2. How to package dependent libraries .
Because it involves C/C++ Code , Then the best learning resource is opencv-python Source code . Ideally , The bag should be able to pass pip Command to install on any platform .
that opencv-python How to compile for different platforms wheel Of ？ Through the source code can be found , It uses scikit-build, adopt CMake To compile the C/C++ Code . We can run it directly GitHub Upper Example project To experience .
The project is very simple ,setup.py There is only one package name written in ：
from skbuild import setup
setup(
name="hello-cpp",
version="1.2.3",
description="a minimal example package (cpp version)",
author='The scikit-build team',
license="MIT",
packages=['hello'],
python_requires=">=3.7",
)
Leave the rest to CMakeLists.txt To complete ：
cmake_minimum_required(VERSION 3.4...3.22)
project(hello)
find_package(PythonExtensions REQUIRED)
add_library(_hello MODULE hello/_hello.cxx)
python_extension_module(_hello)
install(TARGETS _hello LIBRARY DESTINATION hello)
pyproject.toml The compilation environment is configured in .
[build-system]
requires = [
"setuptools>=42",
"scikit-build>=0.13",
"cmake>=3.18",
"ninja",
]
build-backend = "setuptools.build_meta"
[tool.cibuildwheel]
manylinux-x86_64-image = "manylinux2014"
manylinux-aarch64-image = "manylinux2014"
skip = ["pp*", "*-win32", "*-manylinux_i686", "*-musllinux_*"]
[tool.cibuildwheel.windows]
archs = ["AMD64"]
[tool.cibuildwheel.linux]
repair-wheel-command = "auditwheel repair -w {
dest_dir} {
wheel} --plat manylinux2014_x86_64"
archs = ["x86_64"]
[tool.cibuildwheel.macos]
archs = ["x86_64"]
repair-wheel-command = [
"delocate-listdeps {wheel}",
"delocate-wheel --require-archs {delocate_archs} -w {dest_dir} {wheel}",
]
We run pip wheel . You can get one *.whl package .
Now open this package with the decompression software to have a look . We see that the root directory contains two folders , One is *dist-info, One is what you define package. stay package You can find the compiled library files and __init__.py file . That's right wheel Package structure . If you use it directly according to the official tutorial python setup.py bdist_wheel Pack one C Extension engineering , You'll find that , The compiled library is packaged in the root directory . What's the problem ？ Open the installation directory after installation Lib\site-packages, You will find that the library is copied directly to this directory , Environment pollution .
When your C/C++ The code also depends on other libraries , Then we need to consider the link and packaging of the Library .Windows It's very simple , All libraries can be placed in the same directory , but Linux and macOS You need to set the relative path . Be careful , The setting method is different . The method is as follows ：
if(CMAKE_HOST_UNIX)
if(CMAKE_HOST_APPLE)
SET(CMAKE_CXX_FLAGS "-std=c++11 -O3 -Wl,-rpath,@loader_path")
SET(CMAKE_INSTALL_RPATH "@loader_path")
else()
SET(CMAKE_CXX_FLAGS "-std=c++11 -O3 -Wl,-rpath=$ORIGIN")
SET(CMAKE_INSTALL_RPATH "$ORIGIN")
endif()
SET(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE)
endif()
Other links , Installation is omitted here , Look directly at the source code later .
No, then CMake Can you achieve the same packaging effect ？ use setuptools It's totally achievable .
We are setup.py There are several things to do in ：1. Configure compile parameters .2. Copy dependent Library .
First judge the system and CPU framework , Then add the corresponding compilation parameters .
dbr_lib_dir = ''
dbr_include = ''
dbr_lib_name = 'DynamsoftBarcodeReader'
if sys.platform == "linux" or sys.platform == "linux2":
# linux
if platform.uname()[4] == 'AMD64' or platform.uname()[4] == 'x86_64':
dbr_lib_dir = 'lib/linux'
elif platform.uname()[4] == 'aarch64':
dbr_lib_dir = 'lib/aarch64'
else:
dbr_lib_dir = 'lib/arm32'
elif sys.platform == "darwin":
# OS X
dbr_lib_dir = 'lib/macos'
pass
elif sys.platform == "win32":
# Windows
dbr_lib_name = 'DBRx64'
dbr_lib_dir = 'lib/win'
if sys.platform == "linux" or sys.platform == "linux2":
ext_args = dict(
library_dirs = [dbr_lib_dir],
extra_compile_args = ['-std=c++11'],
extra_link_args = ["-Wl,-rpath=$ORIGIN"],
libraries = [dbr_lib_name],
include_dirs=['include']
)
elif sys.platform == "darwin":
ext_args = dict(
library_dirs = [dbr_lib_dir],
extra_compile_args = ['-std=c++11'],
extra_link_args = ["-Wl,-rpath,@loader_path"],
libraries = [dbr_lib_name],
include_dirs=['include']
)
if sys.platform == "linux" or sys.platform == "linux2" or sys.platform == "darwin":
module_barcodeQrSDK = Extension('barcodeQrSDK', ['src/barcodeQrSDK.cpp'], **ext_args)
else:
module_barcodeQrSDK = Extension('barcodeQrSDK',
sources = ['src/barcodeQrSDK.cpp'],
include_dirs=['include'], library_dirs=[dbr_lib_dir], libraries=[dbr_lib_name])
We set up custom functions to implement python setup.py build:
def copylibs(src, dst):
if os.path.isdir(src):
filelist = os.listdir(src)
for file in filelist:
libpath = os.path.join(src, file)
shutil.copy2(libpath, dst)
else:
shutil.copy2(src, dst)
class CustomBuildExt(build_ext.build_ext):
def run(self):
build_ext.build_ext.run(self)
dst = os.path.join(self.build_lib, "barcodeQrSDK")
copylibs(dbr_lib_dir, dst)
filelist = os.listdir(self.build_lib)
for file in filelist:
filePath = os.path.join(self.build_lib, file)
if not os.path.isdir(file):
copylibs(filePath, dst)
# delete file for wheel package
os.remove(filePath)
setup (name = 'barcode-qr-code-sdk',
...
cmdclass={
'build_ext': CustomBuildExt,},
)
When executed build_ext.build_ext.run(self) When , Triggered the default compilation . At this time, we will generate what we need Python library . Then copy all the libraries we use to the output directory . When executing the packaging command ,build The command is triggered first , Then all the files in the output directory will be packaged into wheel in . In this way, and
scikit-build Same effect .
Linux Made bag , It also needs to go through auditwheel Of repair Command regenerates a support manylinux Of wheel package .
auditwheel repair *.whl --plat manylinux2014_x86_64"
If you don't do this , Generated Linux Packages cannot be uploaded pypi Of .
Now in GitHub Action Create an automated compilation, packaging and publishing process ：
name: Build and upload to PyPI
on: [push, pull_request]
jobs:
build_wheels:
name: Build wheels on ${
{
matrix.os }}
runs-on: ${
{
matrix.os }}
strategy:
matrix:
os: [ubuntu-20.04, windows-2019, macos-10.15]
steps:
- uses: actions/[email protected]
- name: Build wheels
uses: pypa/[email protected]
- uses: actions/upload-[email protected]
with:
path: ./wheelhouse/*.whl
build_sdist:
name: Build source distribution
runs-on: ubuntu-latest
steps:
- uses: actions/[email protected]
- name: Build sdist
run: pipx run build --sdist
- uses: actions/upload-[email protected]
with:
path: dist/*.tar.gz
upload_pypi:
needs: [build_wheels, build_sdist]
runs-on: ubuntu-latest
# upload to PyPI on every tag starting with 'v'
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
# alternatively, to publish when a GitHub Release is created, use the following rule:
# if: github.event_name == 'release' && github.event.action == 'published'
steps:
- uses: actions/download-[email protected]
with:
name: artifact
path: dist
- uses: pypa/gh-action-pypi-[email protected]
with:
user: __token__
password: ${
{
secrets.pypi_password }}
skip_existing: true
All kinds of mainstream python The corresponding package is made .
https://github.com/yushulx/python-barcode-qrcode-sdk
