🗃️pypi_package_validator.py

This tool handles removing deleted packages from the data set CSV

Description

pypi_package_validator.py enumerates the data set CSV and performs a HEAD request on each PyPI package to determine if the package was deleted. If the package no longer exists, then the record is removed from the data set. This also helps to reduce the file size of the data set.

Dependencies

pypi_package_validator.py requires the following dependencies:

• argparse

  • pip install argparse

Usage

Parameter --file, -f

• type : str

• path to data set CSV file

Parameter --output, -o

• type : str

• New data set CSV file name

Parameter --tmpfile, -tmp

• type : bool

• Use this flag if progress was stopped and you need to resume progress

Example 1

pypi_package_validator.py -f "pypi_info_main_db.csv"

• validates the given data set CSV file

Example 2

pypi_package_validator.py -f 'pypi_info_main_db.csv' -tmp

• validates the given data set CSV file

• uses the -tmp flag to resume progress from the last validated package