Skip to content
Snippets Groups Projects
Commit 360c82ce authored by nimrod's avatar nimrod
Browse files

Migrate to GitLab.

- Replace tox tests with pre-commit hooks.
- Replace Travis CI with GitLab CI.
- Address issues from checks.
- Format with Black.
- Release with GitLab CI.
- Update documentation.
parent 56687605
No related branches found
No related tags found
No related merge requests found
Pipeline #266 failed
---
stages:
- test
- release
pre-commit:
stage: test
image: adarnimrod/ci-images:pre-commit
variables: &variables
XDG_CACHE_HOME: "$CI_PROJECT_DIR/.cache"
script:
- pre-commit run -a
cache: &cache
paths:
- .cache/
upload:
stage: release
image: python:3.6
before_script:
- pip install twine
script:
- mv "$pypirc" $HOME/.pypirc
- python setup.py bdist_wheel
- twine upload dist/*
variables: *variables
cache: *cache
rules:
- if: $CI_COMMIT_TAG
artifacts:
paths:
- dist/*.whl
release:
stage: release
image: registry.gitlab.com/gitlab-org/release-cli:latest
script:
- !!str true
rules:
- if: $CI_COMMIT_TAG
release:
name: Release $CI_COMMIT_TAG
tag_name: $CI_COMMIT_TAG
ref: $CI_COMMIT_TAG
description: Release $CI_COMMIT_TAG
- repo: git://github.com/pre-commit/pre-commit-hooks # vim:ff=unix ts=2 sw=2 ai expandtab
sha: v0.7.1 ---
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v3.4.0
hooks: hooks:
- id: check-added-large-files - id: check-executables-have-shebangs
- id: check-json
- id: check-xml
- id: check-yaml
- id: check-merge-conflict - id: check-merge-conflict
- id: trailing-whitespace
- repo: https://github.com/ambv/black
rev: 20.8b1
hooks:
- id: black
args:
- |
--line-length=79
- repo: https://github.com/Lucas-C/pre-commit-hooks-markup
rev: v1.0.1
hooks:
- id: rst-linter
- repo: https://github.com/myint/rstcheck.git
rev: master
hooks:
- id: rstcheck
- repo: https://github.com/adrienverge/yamllint
rev: v1.25.0
hooks:
- id: yamllint
- repo: https://github.com/amperser/proselint/
rev: 0.10.2
hooks:
- id: proselint
types: [plain-text]
exclude: LICENSE
- repo: https://github.com/PyCQA/prospector
rev: 1.3.1
hooks:
- id: prospector
args:
- |-
--max-line-length=79
- |-
--with-tool=pyroma
- |-
--with-tool=bandit
- |-
--without-tool=pep257
- |-
--doc-warnings
- |-
--test-warnings
- |-
--full-pep8
- |-
--strictness=high
additional_dependencies:
- bandit
- pyroma
- repo: https://gitlab.com/pycqa/flake8.git
rev: 3.8.4
hooks:
- id: flake8 - id: flake8
- id: check-symlinks args:
- |-
--doctests
additional_dependencies:
- flake8-bugbear
- repo: https://github.com/Yelp/detect-secrets
rev: v0.14.3
hooks:
- id: detect-secrets
- repo: https://github.com/mgedmin/check-manifest
rev: '0.45'
hooks:
- id: check-manifest
---
language: python
python: ["2.7", "3.3", "3.4", "3.5", "3.6"]
dist: trusty
sudo: false
cache:
- pip
matrix:
include:
- python: "3.5"
env: TOXENV=docs
- python: "2.7"
env: TOXENV=pre-commit
- python: "3.5"
env: TOXENV=pre-commit
- python: "3.5"
env: TOXENV=bandit
install:
- pip install tox-travis | cat
script:
- tox
notifications:
email: false
recursive-include eb-prune *.py recursive-include eb-prune *.py
exclude .pre-commit-config.yaml exclude .pre-commit-config.yaml
exclude .travis.yml exclude .gitlab-ci.yml
include *.rst include *.rst
include *.txt include *.txt
include VERSION include VERSION
eb-prune eb-prune
######## ########
.. image:: https://travis-ci.org/adarnimrod/eb-prune.svg?branch=master .. image:: https://git.shore.co.il/nimrod/eb-prune/badges/master/pipeline.svg
:target: https://travis-ci.org/adarnimrod/eb-prune :target: https://git.shore.co.il/nimrod/eb-prune/-/commits/master
A CLI tool to prune old versions of Elastic Beanstalk. A CLI tool to prune old versions of Elastic Beanstalk.
...@@ -42,16 +42,16 @@ file). ...@@ -42,16 +42,16 @@ file).
Testing Testing
------- -------
Tests require Python 2.7, Python 3.3 or later and Tox and are run by running Various linters are configured with `pre-commit <https://pre-commit.com/>`_.
:code:`tox`. Also, Travis CI is used to test on multiple Python versions for Those are run in CI in `GitLab
every push. <https://git.shore.co.il/nimrod/eb-prune/-/pipelines>`_. To run locally, install
pre-commit and run it.
Release Release
------- -------
Releases require Python 2.7 or Python 3.3 or later and Tox. To release a new Update the version in :code:`VERSION`, commit, tag and push. GitLab will do the
version bump the version in the :code:`VERSION` file and run :code:`tox -e rest.
release`.
Author Author
------ ------
...@@ -59,10 +59,4 @@ Author ...@@ -59,10 +59,4 @@ Author
Nimrod Adar, `contact me <nimrod@shore.co.il>`_ or visit my `website Nimrod Adar, `contact me <nimrod@shore.co.il>`_ or visit my `website
<https://www.shore.co.il/>`_. Patches are welcome via `git send-email <https://www.shore.co.il/>`_. Patches are welcome via `git send-email
<http://git-scm.com/book/en/v2/Git-Commands-Email>`_. The repository is located <http://git-scm.com/book/en/v2/Git-Commands-Email>`_. The repository is located
at: https://www.shore.co.il/git/. at: https://git.shore.co.il/explore.
TODO
----
- Release to PyPI on tagged commits from Travis CI.
- Add tests using moto.
#!/usr/bin/env python #!/usr/bin/env python
'''Prune older versions of an application in Elastic Beanstalk.''' """Prune older versions of an application in Elastic Beanstalk."""
from __future__ import (absolute_import, division, print_function, from __future__ import (
unicode_literals) absolute_import,
division,
print_function,
unicode_literals,
)
from argparse import ArgumentParser from argparse import ArgumentParser
from botocore import session from botocore import session # pylint: disable=import-error
def prune(versions_to_keep, dry_run): def prune(versions_to_keep, dry_run):
if dry_run: if dry_run:
print('DRY RUN! NOTHING WILL BE REMOVED.') print("DRY RUN! NOTHING WILL BE REMOVED.")
print('Pruning Elastic Beanstalk versions.') print("Pruning Elastic Beanstalk versions.")
aws_session = session.get_session() aws_session = session.get_session()
beanstalk_client = aws_session.create_client('elasticbeanstalk') beanstalk_client = aws_session.create_client("elasticbeanstalk")
response = beanstalk_client.describe_application_versions() response = beanstalk_client.describe_application_versions()
if response['ResponseMetadata']['HTTPStatusCode'] != 200: if response["ResponseMetadata"]["HTTPStatusCode"] != 200:
raise RuntimeError('Failed to describe application versions.') raise RuntimeError("Failed to describe application versions.")
# Get all EB versions. # Get all EB versions.
versions = response['ApplicationVersions'] versions = response["ApplicationVersions"]
response = beanstalk_client.describe_environments() response = beanstalk_client.describe_environments()
if response['ResponseMetadata']['HTTPStatusCode'] != 200: if response["ResponseMetadata"]["HTTPStatusCode"] != 200:
raise RuntimeError('Failed to describe environments.') raise RuntimeError("Failed to describe environments.")
# Remove the currently in-use versions from the list. # Remove the currently in-use versions from the list.
active_versions = [env['VersionLabel'] for env in response['Environments']] active_versions = [env["VersionLabel"] for env in response["Environments"]]
previous_versions = filter( previous_versions = filter(
lambda x: (not x['VersionLabel'] in active_versions) and lambda x: (not x["VersionLabel"] in active_versions)
x['Status'] == 'UNPROCESSED', versions) and x["Status"] == "UNPROCESSED",
versions,
)
# Remove the newest versions from the list. # Remove the newest versions from the list.
old_versions = sorted( old_versions = sorted(
previous_versions, previous_versions, key=lambda x: x.get("DateCreated")
key=lambda x: x.get('DateCreated'))[:-versions_to_keep] )[:-versions_to_keep]
for version in old_versions: for version in old_versions:
if not dry_run: if not dry_run:
response = beanstalk_client.delete_application_version( response = beanstalk_client.delete_application_version(
ApplicationName=version['ApplicationName'], ApplicationName=version["ApplicationName"],
VersionLabel=version['VersionLabel'], VersionLabel=version["VersionLabel"],
DeleteSourceBundle=True) DeleteSourceBundle=True,
if response['ResponseMetadata']['HTTPStatusCode'] != 200: )
raise RuntimeError('Failed to delete version {0}.'.format( if response["ResponseMetadata"]["HTTPStatusCode"] != 200:
version['VersionLabel'])) raise RuntimeError(
print('Deleted version {0} of {1}.'.format(version['VersionLabel'], "Failed to delete version {0}.".format(
version['ApplicationName'])) version["VersionLabel"]
print('Deleted {0} versions.'.format(len(old_versions))) )
)
print(
"Deleted version {0} of {1}.".format(
version["VersionLabel"], version["ApplicationName"]
)
)
print("Deleted {0} versions.".format(len(old_versions)))
def main(): def main():
parser = ArgumentParser() parser = ArgumentParser()
parser.add_argument('versions_to_keep', parser.add_argument(
help='The number of versions to keep.', "versions_to_keep", help="The number of versions to keep.", type=int
type=int) )
parser.add_argument('-d', parser.add_argument(
'--dry-run', "-d",
help='Dry run, do not delete versions.', "--dry-run",
action='store_true') help="Dry run, do not delete versions.",
action="store_true",
)
args = parser.parse_args() args = parser.parse_args()
prune(args.versions_to_keep, args.dry_run) prune(args.versions_to_keep, args.dry_run)
if __name__ == '__main__': if __name__ == "__main__":
main() main()
...@@ -2,28 +2,33 @@ ...@@ -2,28 +2,33 @@
from setuptools import setup, find_packages from setuptools import setup, find_packages
setup( setup(
name='eb-prune', name="eb-prune",
version=open('VERSION', 'r').read(), version=open("VERSION", "r").read(),
description='Pruning of Elastic Beanstalk versions.', description="Pruning of Elastic Beanstalk versions.",
long_description=open('README.rst', 'r').read(), long_description=open("README.rst", "r").read(),
url='https://www.shore.co.il/git/eb-prune', url="https://www.shore.co.il/git/eb-prune",
author='Nimrod Adar', author="Nimrod Adar",
author_email='nimrod@shore.co.il', author_email="nimrod@shore.co.il",
license='MIT', license="License :: OSI Approved :: MIT License",
classifiers=[ classifiers=[
'Development Status :: 4 - Beta', "Development Status :: 4 - Beta",
'Intended Audience :: Developers', "Intended Audience :: Developers",
'Programming Language :: Python :: 3', "Programming Language :: Python :: 3",
'Programming Language :: Python :: 2', "Programming Language :: Python :: 3.6",
'Intended Audience :: System Administrators', "Programming Language :: Python :: 3.7",
'Topic :: Utilities', "Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 2",
"Intended Audience :: System Administrators",
"Topic :: Utilities",
], ],
keywords='beanstalk AWS', keywords="beanstalk AWS",
packages=find_packages(), packages=find_packages(),
install_requires=['botocore'], install_requires=["botocore"],
extras_require={ extras_require={
'dev': ['tox'], }, "dev": ["tox"],
},
entry_points={ entry_points={
'console_scripts': [ "console_scripts": ["eb-prune=eb_prune:main"],
'eb-prune=eb_prune:main'], }, },
) )
[tox]
envlist = py{2,3}
[travis]
python =
2.7: py2
3.3: py3
3.4: py3
3.5: py3
3.6: py3
[testenv]
basepython =
py2: python2
py3: python3
deps =
check-manifest
flake8
commands =
check-manifest --ignore tox.ini,tests*
flake8 .
eb-prune --help
[testenv:docs]
basepython = python
deps = readme_renderer
commands = python setup.py check -m -r -s
[testenv:release]
basepython = python
whitelist_externals =
sh
deps =
twine
wheel
commands =
sh -c 'git tag "$(cat VERSION)" && git push --tags'
python setup.py bdist_wheel
twine upload --skip-existing dist/*.whl
[testenv:bandit]
basepython = python
deps = bandit
commands = bandit --recursive ./ --exclude .tox/,build/,dist/,eb-prune.egg-info
[testenv:pre-commit]
basepython = python
deps = pre-commit
commands = pre-commit run --all-files
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment