Building a Cookiecutter Template for Multiple Python Package Managers

In my previous post, I described how you can use Python and Makefiles to quickly get new projects started. In this post I explore how you can use cookiecutters more advanced Jinja templating techniques to make a cookiecutter that can be used for multiple package managers. Specifically, we’ll be extending the existing template to build another that can be used with either Poetry or Pipenv.

Our New Cookiecutter Template

As a reminder, Cookiecutter allows us to template project and directory content and structure using Jinja2 logic.

To begin, lets take a look at the new project structure. In this we have both our original Pipfile (for Pipenv based projects) and a pyproject.toml (for Poetry).

cookiecutter-python-github
├── cookiecutter.json
├── {{cookiecutter.project_slug}}
│   ├── app
│   │   ├── app.py
│   │   └── __init__.py
│   ├── .github
│   │   └── workflows
│   │       └── pull_request.yml
│   ├── LICENSE
│   ├── Makefile
│   ├── Pipfile
│   ├── pyproject.toml
│   ├── README.md
│   └── tests
│       ├── __init__.py
│       └── test_app.py
├── hooks
│   └── post_gen_project.sh
└── README.md

The new pyproject.toml has a few new variables - namely those for version, email, name and license. You can see the same packages are included as those in the Pipfile:

[tool.poetry]
name = "{{cookiecutter.project_name}}"
version = "{{ cookiecutter.version }}"
description = ""
readme = "README.md"
authors = ["{{cookiecutter.full_name}} <{{cookiecutter.email}}>"]
license = "{{ cookiecutter.license }}"

[tool.poetry.dependencies]
python = "^{{cookiecutter.python_version}}"

[tool.poetry.dev-dependencies]
pytest = "^5.2"
black = "20.8b1"
flake8 = "^3.8.4"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

The cookiecutter.json includes these variables as well as a couple of others for managing directory/package names.

{
        "project_name": "someproject",
        "project_short_description": "A project to",
        "project_slug": "{{ cookiecutter.project_name.lower().replace(' ', '_') }}",
        "package_slug": "{{ cookiecutter.project_name.lower().replace(' ', '_').replace('-', '_') }}",
        "python_version": "3.8.6",
        "package_manager": ["pipenv", "poetry"],
        "version": "0.1.0",
        "full_name": "Ian Wootten",
        "email": "hi@niftydigits.com",
        "license": ["MIT", "BSD-3-Clause", "GPL-3.0-only", "Apache-2.0"]
}

You can see that I’m now using python functions with the cookiecutter variables to default the additional naming variables to something that makes sense for our project.

Using Multiple Choice Variables

The package_manager and license options in the new cookiecutter.json both make use of a multiple choice variable. When we run the cookiecutter we’ll be presented with options in the terminal like so:

Select license:
1 - MIT
2 - BSD-3-Clause
3 - GPL-3.0-only
4 - Apache-2.0
Choose from 1, 2, 3, 4 [1]

The license renders the text of the license we choose to the LICENSE file by checking the actual value in a jinja decorator.

{% if cookiecutter.license == 'MIT' -%}
MIT License

Copyright (c) {% now 'local', '%Y' %}, {{ cookiecutter.full_name }}

Permission is hereby granted, free of charge, to any person obtaining a copy
...
{% elif cookiecutter.license == 'BSD-3-Clause' %}

BSD 3-Clause "New" or "Revised" License

Copyright (c) {% now 'local', '%Y' %}, {{ cookiecutter.full_name }}
All rights reserved.
...

Supporting Multiple Package Managers

Much like the license, we now specify a multiple choice variable to allow a different package manager to be chosen. It currently supports both pipenv and poetry.

Select package_manager:
1 - pipenv
2 - poetry
Choose from 1, 2 [1]: 

To account for our new package manager options, the Makefile is updated to use these cookiecutter variables. We use a Jinja if condition for rendering the appropriate package managers install command. Once rendered, the Makefile will only show the output for the manager we have chosen.

init:
        git init
        git add -A
        git commit -m "Initial commit"
        git branch -M main
        gh repo create
        @echo "push with: git push -u origin main"
install:
        {%- if cookiecutter.package_manager == "poetry" %}
        poetry install
        {%- elif cookiecutter.package_manager == "pipenv" %}
        pipenv install --dev
        {%- endif %}
lint:
        {{ cookiecutter.package_manager }} run flake8
        {{ cookiecutter.package_manager }} run black --check .
test: lint
        {{ cookiecutter.package_manager }} run py.test
format:
        {{ cookiecutter.package_manager }} run black .

Although we’ve updated our Makefile, the package manager we want to use also needs to be installed when run as part of a github workflow. We update the pull_request workflow as follows:

name: CI
on: pull_request

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-python@v2
        with:
          python-version: {{cookiecutter.python_version}}
{%- if cookiecutter.package_manager == "poetry" %}
      - name: Install Poetry
        run: |
          curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -
          echo "$HOME/.poetry/bin" >> $GITHUB_PATH
{%- elif cookiecutter.package_manager == "pipenv" %}
      - name: Install Pipenv
        run: pip install pipenv==2020.11.15
{%- endif %}
      - name: Install venv
        run: make install
      - name: Test
        run: make test

This means that only one of the commands will actually be rendered to the file when the project is generated.

Using Render Hooks for Clean Up

As the template includes both a pyproject.toml and Pipfile, we need to remove the package manager file we’re not making use of once our choices have been made. Fortunately, cookiecutter is able to handle this too, using pre or post generate hooks.

At the same level as our {{cookiecutter.project_name}} file, we define a hooks folder with a post_gen_project.sh script within it which will be run after cookiecutter generates the project. We could write it in Python, but this was simpler as a bash script. It looks like the following:

#!/bin/bash

rm {{ "Pipfile" if cookiecutter.package_manager == "poetry" else "pyproject.toml" }}

As you can see, it’s possible to use jinja templates and cookiecutter variables within the hook itself. We therefore define a hook to remove the package file we won’t be using once the project is generated to make sure we don’t have any files we don’t want. This is then run immediately after our project folder is created by cookiecutter, removing the unneccessary file.

Conclusion

This template has become a lot more versatile since my original Pipenv based one. I started this with an idea to convert the original template to one which created poetry projects (which is also available on github), but extending it to encompass both together was pretty simple. I’ve successfully automated away even more of the pain when starting a new project. If I wanted to, I can even extend this further to support additional package managers.

The entire Python/Pipenv template is up on github and you can use it yourself with cookiecutter by calling:

cookiecutter https://github.com/iwootten/cookiecutter-python-github