mypy_boto3

Auto-generated documentation index.

PyPI - mypy-boto3 PyPI - Python Version Docs Coverage

Type annotations for boto3 compatible with mypy, VSCode, PyCharm and other tools.

Based on boto3_type_annotations by @alliefitter.

Full mypy-boto3 project documentation can be found in Modules

Installation

# install `boto3` type annotations
# for ec2, s3, rds, lambda, sqs, dynamo and cloudformation
# Consumes ~7 MB of space
python -m pip install 'boto3-stubs[essential]'

# or install annotations for services you use
python -m pip install 'boto3-stubs[acm,apigateway]'

Build services index manually

This package generates a few source files depending on services that you installed. Generation is done by a post-install script, so as long as you use pip, pipfile or poetry everything should be done automatically.

However, if you use any other way or notice that services stubs do not work, you can build services index manually.

# Use this command when you add or remove service packages
python -m mypy_boto3

If you generate requirements.txt from Pipfile.lock, you also should build index manually because package installation order is not the same.

# Add boto3-stubs with services
pipenv install 'boto3-stubs[s3,ec2]'

# Generate requirements.txt file
pipenv lock --requirements > requirements.txt

...

# Install pacakges to your new environment
pip intall requirements.txt

# Generate index
python -m mypy_boto3

How to uninstall

Some files are generated by service post-install scripts, so pip does not fully remove packages. To properly uninstall boto3-stubs, use these commands:

# remove generated files and cache
python -m mypy_boto3 --clean

# remove boto3-stubs
python -m pip uninstall -y boto3-stubs

# remove mypy-boto3 and submodules
python -m pip freeze | grep mypy-boto3 | xargs python -m pip uninstall -y

Alternatively you can just remove mypy_boto3 folder from site-packages after uninstall.

rm -r `python -m site --user-site`/mypy_boto3

Usage

import boto3

from mypy_boto3 import s3

# you need explicit type annotatins only if your IDE do not support
# function overloads (e.g. VSCode). For PyCharm anf mypy you do not need
# to set types explicitly
client: s3.S3Client = boto3.client("s3")

# IDE autocomplete suggests function name and arguments here
client.create_bucket(Bucket="bucket")

# (mypy) error: Missing positional argument "Key" in call to "get_object" of "S3Client"
client.get_object(Bucket="bucket")

# (mypy) error: Argument "Key" to "get_object" of "S3Client" has incompatible type "None"; expected "str"
client.get_object(Bucket="bucket", Key=None)

resource: s3.S3ServiceResource = boto3.Session(region_name="us-west-1").resource("s3")

# IDE autocomplete suggests function name and arguments here
bucket = resource.Bucket("bucket")

# (mypy) error: Unexpected keyword argument "key" for "upload_file" of "Bucket"; did you mean "Key"?
bucket.upload_file(Filename="my.txt", key="my-txt")

# waiters and paginators are annotated as well
waiter: s3.BucketExistsWaiter = client.get_waiter("bucket_exists")
paginator: s3.ListMultipartUploadsPaginator = client.get_paginator(
    "list_multipart_uploads"
)

Setup your IDE

VSCode

PyCharm

Official mypy plugin does not work for some reason for me. If you know how to setup it correctly, please hep me to update this section.

Other IDEs

You need explicit type annotations for code auto-complete, but mypy works even without them.

Explicit type annotations

Automatic type discovery is too stressful for PyCharm and does not work in VSCode. So implicit type annotations support has been removed as it is not useful.

To get full advantage of boto3-stubs, you can set types explicitly.

import boto3

from mypy_boto3 import ec2

# covered by boto3-stubs, no explicit type required
session = boto3.session.Session(region_name="us-west-1")

# by default it is Any, but we explicitly set it to EC2Client
# to make method auto-complete work
ec2_client: ec2.EC2Client = boto3.client("ec2", region_name="us-west-1")

# same for resource
ec2_resource: ec2.EC2ServiceResource = session.resource("ec2")

# PyCharm does not need explicit type annotations here, but VSCode does
bundle_task_complete_waiter: ec2.BundleTaskCompleteWaiter = ec2_client.get_waiter("bundle_task_complete")
describe_volumes_paginator: ec2.DescribeVolumesPaginator = ec2_client.get_paginator("describe_volumes")

# ec2_client, ec2_resource, bundle_task_complete_waiter and describe_volumes_paginator
# now have correct type so IDE autocomplete for methods, arguments and return types
# works as expected. You do not need to specify types explicitly further

Pylint compatibility

It is totally safe to use TYPE_CHECKING flag in order to avoid boto3-stubs dependency in production. However, there is an issue in pylint that it complains about undefined variables. To fix it, set all types to object in non-TYPE_CHECKING mode.

import boto3
from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from mypy_boto3.ec2 import EC2Client, EC2ServiceResource
    from mypy_boto3.ec2.waiters import BundleTaskCompleteWaiter
    from mypy_boto3.ec2.paginators import DescribeVolumesPaginator
else:
    EC2Client = object
    EC2ServiceResource = object
    BundleTaskCompleteWaiter = object
    DescribeVolumesPaginator = object

...

How to build

Locally

mypy-boto3 is built for the latest version of boto3. If you need type annotations for another version of boto3, you can use mypy-boto3-builder.

# Install preferred version of `boto3`
python -m pip install boto3==1.10.18 botocore==1.13.18

# Install `mypy-boto3-builder`
python -m pip install mypy-boto3-builder

# Build all packages
# You can specify required services explicitly like
# ./scripts/build.sh -s ec2 s3
./scripts/build.sh

# Install custom `mypy-boto3` packages
./scripts/install.sh

With Docker image

docker pull docker.pkg.github.com/vemel/mypy_boto3/mypy_boto3_builder:latest
docker tag docker.pkg.github.com/vemel/mypy_boto3/mypy_boto3_builder:latest mypy_boto3_builder

mkdir output

# generate stubs for all services
docker run -v `pwd`/output:/output -ti mypy_boto3_builder

# generate stubs for s3 service
docker run -v `pwd`/output:/output -ti mypy_boto3_builder -s s3

# generate stubs for a specific boto3 version
docker run -e BOTO3_VERSION=1.10.18 BOTOCORE_VERSION=1.13.18 -v `pwd`/output:/output -ti mypy_boto3_builder

What's new

Implemented features

Latest changes

Full changelog can be found in Releases.

Versioning

mypy_boto3_builder version is not related to boto3 version and follows Semantic Versioning.

Generated packages use format <boto3_version>.<build>, e.g. for boto3 1.10.40, mypy_boto3 versions are is 1.10.40.0 and 1.10.40.1.

Thank you

Sub-modules

Examples

You can install any sub-modules using pip

# pip install boto3-stubs[<submodule_name>,...]

# install `boto3` type annotations
# for ec2, s3, rds, lambda, sqs, dynamo and cloudformation
python -m pip install 'boto3-stubs[essential]'

# install ec2, s3 and sqs type annotations
python -m pip install 'boto3-stubs[s3,ec2,sqs]'

List of all sub-modules