Python For so long, there has not been a de facto standard project management and construction tool , So as to cause Python The structure and construction methods of the project are diverse . This may reflect Python Free will .
Unlike Java After the initial manual construction , To semi automated Ant, Until then Maven It's basically the de facto standard . meanwhile Maven Also accepted other Gradle(Android Project main push ), SBT( Mainly Scala project ), Ant+Ivy, Buildr And so on , But it's hard to shake Maven In the Jianghu , And the others almost follow Maven Table of contents layout .
go back to Python, Produced pip, pipenv, conda Package management tools like that , However, there is no agreement on the directory layout of the project .
A lot about building continues the tradition Makefile The way , Plus setup.py and build.py Install and build with program code . About project catalog layout , There are project templates , Then make a tool to apply the project template .
Let's take a look at the use of four tools
$ pip install cookiecutter $ cookiecutter gh:audreyr/cookiecutter-pypackage # With github Upper audreyr/cookiecutter-pypackage As a template , Answer a bunch of questions to generate a Python project ...... project_name [Python Boilerplate]: sample ......
Finally by cookiecutter The generated project template looks like the following ：
$ tree sample sample ├── AUTHORS.rst ├── CONTRIBUTING.rst ├── HISTORY.rst ├── LICENSE ├── MANIFEST.in ├── Makefile ├── README.rst ├── docs │ ├── Makefile │ ├── authors.rst │ ├── conf.py │ ├── contributing.rst │ ├── history.rst │ ├── index.rst │ ├── installation.rst │ ├── make.bat │ ├── readme.rst │ └── usage.rst ├── requirements_dev.txt ├── sample │ ├── __init__.py │ ├── cli.py │ └── sample.py ├── setup.cfg ├── setup.py ├── tests │ ├── __init__.py │ └── test_sample.py └── tox.ini 3 directories, 26 files
This is probably the main framework of the popular directory structure , The main elements are ：
$ tree sample sample ├── Makefile ├── README.rst ├── docs │ └── index.rst ├── requirements.txt ├── sample │ ├── __init__.py │ └── sample.py ├── setup.cfg ├── setup.py └── tests ├── __init__.py └── test_sample.py
project sample Duplicate in directory sample Put... In the directory Python Source file ,tests In the directory is the test file , Add one docs Put the directory into the document ,README.rst, Others used to build setup, setup.cfg and Makefile file .
This is actually a very classic Python Project structure , The next build uses make The command , Input make You will see the definition in Makefile Instruction in file
$ make clean remove all build, test, coverage and Python artifacts clean-build remove build artifacts clean-pyc remove Python file artifacts clean-test remove test and coverage artifacts lint check style test run tests quickly with the default Python test-all run tests on every Python version with tox coverage check code coverage quickly with the default Python docs generate Sphinx HTML documentation, including API docs servedocs compile the docs watching for changes release package and upload a release dist builds source and wheel package install install the package to the active Python's site-packages
To use the above build process , The corresponding package needs to be installed , Such as tox, wheel, coverage, sphinx, flake8, They all pass through pip To install . Then you can make test, make coverage, make docs,make dist etc. . among make docs Can generate a very beautiful Web file .
PyScaffold seeing the name of a thing one thinks of its function , It is used to create Python Tools for project scaffolding , Installation and use ：
$ pip install pyscaffold $ putup sample
This creates a Python project , The directory structure is the same as before cookiecutter The selected template is similar , It just puts the source file in src Catalog , Instead of sample Catalog .
$ tree sample sample ├── AUTHORS.rst ├── CHANGELOG.rst ├── CONTRIBUTING.rst ├── LICENSE.txt ├── README.rst ├── docs │ ├── Makefile │ ├── _static │ ├── authors.rst │ ├── changelog.rst │ ├── conf.py │ ├── contributing.rst │ ├── index.rst │ ├── license.rst │ ├── readme.rst │ └── requirements.txt ├── pyproject.toml ├── setup.cfg ├── setup.py ├── src │ └── sample │ ├── __init__.py │ └── skeleton.py ├── tests │ ├── conftest.py │ └── test_skeleton.py └── tox.ini
The construction of the whole project will use tox This tool .tox Is an automated test and build tool , It can be created during the build process Python A virtual environment , This allows a clean environment for testing and building .
tox -av Can show the definition in tox.ini All the tasks in ：
$ tox -av default environments: default -> Invoke pytest to run automated tests additional environments: build -> Build the package in isolation according to PEP517, see https://github.com/pypa/build clean -> Remove old distribution files and temporary build artifacts (./build and ./dist) docs -> Invoke sphinx-build to build the docs doctests -> Invoke sphinx-build to run doctests linkcheck -> Check for broken links in the documentation publish -> Publish the package you have been developing to a package index server. By default, it uses testpypi. If you really want to publish your package to be publicly accessible in PyPI, use the `-- --repository pypi` option.
To execute which command, use tox -e build, tox -e docs etc.
In my experience tox In the course of the order , Every step seems to be slow , It should take some time to create a virtual machine .
It's best to look at another build tool PyBuilder, The directory structure it creates is very close to Maven, Let's take a look
$ pip install pybuilder $ mkdir sample && cd sample # The project directory needs to be created manually $ pyb --start-project # After answering some questions, create the required directories and files
After that, look at its directory structure ：
$ tree sample . ├── build.py ├── docs ├── pyproject.toml ├── setup.py └── src ├── main │ ├── python │ └── scripts └── unittest └── python
The build process is still done with pyb command , You can use pyb -h view help ,pyb -t List all tasks , PyBuilder The task of is added as a plug-in , The plug-in is configured in build.py In file .
$ pyb -t sample Tasks found for project "sample": analyze - Execute analysis plugins. depends on tasks: prepare run_unit_tests clean - Cleans the generated output. compile_sources - Compiles source files that need compilation. depends on tasks: prepare coverage - <no description available> depends on tasks: verify install - Installs the published project. depends on tasks: package publish(optional) package - Packages the application. Package a python application. depends on tasks: compile_sources run_unit_tests(optional) prepare - Prepares the project for building. Creates target VEnvs print_module_path - Print the module path. print_scripts_path - Print the script path. publish - Publishes the project. depends on tasks: package verify(optional) coverage(optional) run_integration_tests - Runs integration tests on the packaged application. depends on tasks: package run_unit_tests - Runs all unit tests. Runs unit tests based on Python's unittest module depends on tasks: compile_sources upload - Upload a project to PyPi. verify - Verifies the project and possibly integration tests. depends on tasks: run_integration_tests(optional) $ pyb run_unit_tests sample
PyBuilder Also create a virtual environment before building or testing , from 0.12.9 The version can be started through the parameter --no-venvs Skip the step of creating a virtual environment . Used --no-venvs Words Python The code will be running pyb The current Python Execution in the environment , The required dependencies will be installed manually .
Project dependencies should also be defined in build.py In file
@init
def set_properties(project):
project.depends_on('boto3', '>=1.18.52')
project.build_depends_on('mock') Then in execution pyb The above dependencies will be installed when creating a virtual environment , And run the test and build .
the last one Poetry, It feels like a more mature , Project activity is also higher Python structure , It has more powerful trust management function , use poetry add boto3 You can add dependencies ,poetry show --tree Show dependency tree . See how to install and create a project
$ pip install poetry $ poetry new sample
It creates projects that are simpler than the above
$ tree sample sample ├── README.rst ├── pyproject.toml ├── sample │ └── __init__.py └── tests ├── __init__.py └── test_sample.py
If poetry new close --src Parameters , Then the source file directory sample It will be placed in src Under the table of contents , namely sample/src/sample. poetry init Will generate... In the current directory pyproject.toml file , The generation of directories, etc. needs to be completed manually .
It doesn't focus on document generation , Check of code specification , No code coverage . Its project configuration is more centralized , All in pyproject.toml In file ,toml What is it? ？ It is a configuration file format Tom's Obvious, Minimal Language (https://github.com/toml-lang/toml).
pyproject.toml Some similar NodeJS Of package.json file , such as poetry add, poetry install Command line
# Go to pyproject.toml Add a pair of boto3 And install (add Also from local or git To install dependencies ), poetry add boto3 # Will be in accordance with pyproject.toml The file defines the corresponding dependencies of the installation to the current Python In a virtual environment # For example <test-venv>/lib/python3.9/site-packages Directory , After the module is installed, the test case can also be used poetry install
Other major
1. poetry build # Build installable *.whl and tar.gz file 2. poetry shell # Will be defined in pyproject.toml Create and use virtual environments based on dependencies in files 3. poetry run pytest # Run using pytest Test cases for , Such as tests/test_sample.py 4. poetry run python -m unittest tests/sample_tests.py # function unittest The test case 5. poetry export --without-hashes --output requirements.txt # export requirements.txt file , --dev Export contains dev Dependence , Or use poetry export --without-hashes > requirements.txt
poetry run Can execute any system command , But it will execute in the virtual environment it wants . So you can imagine meeting ,poetry To generate documents or coverage for your project, you must use poetry run ... Command to support sphinx, coverage or flake8.
stay sample Catalog ( And pyproject.toml Document level ) Create file in my_module.py, The content is
def main():
print('hello poetry') And then in pyproject.toml Write in
[tool.poetry.scripts] my-script="sample.my_module:main"
Re execution
$ poetry run my-script
Will be output "hello poetry".
Through the understanding of the above four tools , The complexity of the project structure is determined by cookiecutter-pyproject -> PyScaffold -> PyBuilder -> Poetry Lower in turn , The difficulty of using is roughly the same order .
