Publishing your Python project to the PyPI repository

Sometimes we might want to play with python scripts that are usefull for us when ran locally. But othertimes the script you have been playing with starts to be more and more important and involved in different projects. In such cases it is an interessting option to upload the python script to an internet repository so it can easily be accessed by you, your team members or other people that might need the same functionalities your script offers.

When considering publishing a python script, you should not look further than the Python Package Index. It offers the possibility to easily find, install and publish Python packages in an easy and straightforward way.

Recently I had to create a small script to be able to connect to a Java based server we have running locally for one of our projects. While writing and testing the script, we decided that it could be beneficial to publish the python script to PyPI so other people who work on parallel project inside Cylab could also benefit from it.

This tutorial will go over the different steps that you need to overcome to publish your project to PyPI.

Packaging your Project

For this tutorial we will use a simple project named "python_project_pkg". When first you create your python project it is good to have the following structure:

/packaging_tutorial
  /python_project_pkg
    __init__.py
  /tests
    __init__.py

You can also edit the __init__.py file to add the name of your project:

name = "python_project_pkg"

Now you can start writing your python scripts and add them to your project.

When you are ready to try and test to publish your project, you will need couple of extra files that prepare the distribution:

/packaging_tutorial
  /python_project_pkg
    __init__.py
  /tests
    __init__.py
  setup.py
  LICENSE
  README.md

The setup.py script

One of the most important files for the distribution is the setup.py script, which will hold information about the project and how setuptools is supposed to build and package it.

import setuptools

with open("README.md", "r") as fh:
    long_description = fh.read()

setuptools.setup(
    name="python_project_name",
    version="0.0.1",
    author="Example Author",
    author_email="author@example.com",
    description="A small example package",
    long_description=long_description,
    long_description_content_type="text/markdown",
    url="https://gitlab.com/pypa/sampleproject",
    packages=setuptools.find_packages(exclude=("tests")),
    classifiers=[
        "Programming Language :: Python :: 3",
        "License :: OSI Approved :: MIT License",
        "Operating System :: OS Independent",
    ],
)

In the setup.py script we specified that the description of the package will be loaded from the README.md file and that it uses "text/markdown". We also have to give the project a unique name so it doesn't conflict with other packages uploaded by other people. Setup() takes several arguments, we are not using all of them during this tutorial.

• name - the distribution name of your package.
• version - the package version. For more information on versioning, you can check PEP 440
• author and author_email - used to identify the author of the package
• description - one line description summary of the package
• long_description and long_description_content_type - a long description of the project, in this case loaded from the README.md file, we stipulate the content type of the file too.
• url - the URL of the homepage of the project. For many project this might be a GitHub, GitLab, BitBucket, or a simular code hosting service.
• packages - list of all Python import packages that should be included in the distribution package. Instead of going through the trouble to list all the needed packages, we can use find_packages() to automatically do that. We are also following good programming ettiquete and we have written unit tests for our project. We don't want to distribute the tests for this specific project, so we can use the exclude= parameter to exclude a specific folder from the distribution package.
• classifiers - provides some additional metadata about your package to PyPI, such as the licensing, the targeted operating system and which version of Python the project uses.

Creating a LICENSE

Every package uploaded to the Python Package Index must include a license. This will inform users that might use your package how it can be used. There are multiple different licenses available and you can check https://choosealicense.com/ for more information.

In this tutorial we will use the MIT license:

Copyright (c) 2018 The Python Packaging Authority

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

Creating a README.md

The README.md file uses Markdown. We can create a simple example for our tutorial project:

# My Python Project

This is a simple python project example. You can use [Github-flavored Markdown](https://guides.github.com/features/mastering-markdown/)
to write your content.

Generating distribution archives

Now we come into the most important test- preparing your project for distribution by generating an archive that can be published. To do that we need the latest version of setuptools and wheel installed:

$ python3 -m pip install --user --upgrade setuptools wheel

The --user parameter is used so you don't have to run pip with sudo and it will install the packages for the current user. If you haven't installed the packages before, you can drop the --update parameter as there wont be anything to upgrade.

Now we can run the setup.py script to generate the archives:

python3 setup.py sdist bdist_wheel

This command will show you the output in the console which at the end:

dist/
  python_project_name-0.0.1-py3-none-any.whl
  python_project_name-0.0.1.tar.gz

The tar.gzfile is a source archive whereas the .whl file is a built distribution.

Uploading your distribution archive

Before officially uploading your distribution archive to PyPI, you can deploy it to Test PyPI. The Test PyPI offers the possibility to upload a distribution archive and test if the setup of the project package went without problems. For Test PyPI as for PyPI you will need to register a new account at https://test.pypi.org/account/register/.

Once that is done, you can install twine to facilitate the upload of the distribution package.

$ python3 -m pip install --user --upgrade twine

After that you can run Twine and upload the archives found in the dist folder:

$ python3 -m twine upload --repository-url https://test.pypi.org/legacy/ dist/*

You will be prompted for your account username and password for Test PyPI. After they are supplied, the upload will start and you will see something like:

Enter your username: [your username]
Enter your password:
Uploading python_project_name-0.0.1-py3-none-any.whl
100%|█████████████████████| 4.65k/4.65k [00:01<00:00, 2.88kB/s]
Uploading python_project_name-0.0.1.tar.gz
100%|█████████████████████| 4.25k/4.25k [00:01<00:00, 3.05kB/s]

When the upload is finished you can check your package on the Test PyPI website, at for example, https://test.pypi.org/project/epython_project_name

To upload your package to the official PyPI website, follow the same steps but ommit the --repository-url https://test.pypi.org/legacy/ parameter.

Installing your newly uploaded package

Now that your python package has been uploaded to PyPI you can download it from anywhere using pip:

python3 -m pip install --index-url https://test.pypi.org/simple/ --no-deps python_project_name

After the installation is finished you can test it directly from the console:

$ python

Import it and display the package's name:

>>> import python_project_pkg
>>> python_project_pkg.name
'python_project_pkg'

Now you are ready to share your python projects with the world!

This blog post is licensed under CC BY-SA 4.0