python/README.md

Awesome Python

A curated list of awesome Python frameworks, libraries and software. Inspired by awesome-php.

• Awesome Python
  • Environment Management
  • Package Management
  • Files
  • Date and Time
  • Text Processing
  • Natural Language Processing
  • Command-line Tools
  • Documentation
  • Imagery
  • Audio
  • Video
  • Geolocation
  • HTTP
  • Database
  • Database Drivers
  • ORM
  • Web Frameworks
  • CMS
  • RESTful API
  • Authentication and OAuth
  • Template Engine
  • Queue
  • Search
  • Asset Management
  • Caching
  • Email
  • Internationalization
  • URL Manipulation
  • HTML Manipulation
  • Web Crawling
  • Web Content Extracting
  • Forms
  • Anti-spam
  • Tagging
  • Processes and Threads
  • Networking
  • Admin Panels
  • DevOps Tools
  • GUI
  • Game Development
  • Logging
  • Testing
  • Code Analysis and Linter
  • Debugging Tools
  • Science and Data Aanalysis
  • Machine Learning
  • MapReduce
  • Third-party APIs
  • Algorithms and Design Patterns
  • Miscellaneous
  • Foreign Function Interface
  • High Performance
  • Editor Plugins
• Resources
  • Websites
  • Weekly
  • Twitter

Environment Management

Libraries for Python version and environment management.

• pyenv - Simple Python version management.
• virtualenv - A tool to create isolated Python environments.
• virtualenvwrapper - A set of extensions to virtualenv

Package Management

Libraries for package and dependency management.

• pip / Python Package Index - The package and dependency manager.
• wheel - The new standard of python distribution and are intended to replace eggs.
• buildout - Python-based build system for creating, assembling and deploying applications from multiple parts, some of which may be non-Python-based.

Files

Libraries for file manipulation and MIME type detection.

• mimetypes - (Python standard library) Map filenames to MIME types.
• imghdr - (Python standard library) Determine the type of an image.
• python-magic - A Python interface to the libmagic file type identification library.
• path.py - A module wrapper for os.path.
• watchdog - API and shell utilities to monitor file system events.
• Unipath - An object-oriented approach to file/directory operations.
• pathlib - An cross-platform, object-oriented path library (included in Python 3.4)

Date and Time

Libraries for working with dates and times.

• arrow - Better dates & times for Python.
• dateutil - Extensions to the standard Python datetime module.
• delorean - A library for clearing up the inconvenient truths that arise dealing with datetimes in Python.
• when.py - Providing user-friendly functions to help perform common date and time actions.
• moment - A Python library for dealing with dates/times. Inspired by Moment.js.

Text Processing

Libraries for parsing and manipulating texts.

• General
  • difflib - (Python standard library) Helpers for computing deltas.
  • Levenshtein - Fast computation of Levenshtein distance and string similarity.
  • fuzzywuzzy - Fuzzy String Matching.
  • esmre - Regular expression accelerator.
  • shortuuid - A generator library for concise, unambiguous and URL-safe UUIDs.
  • python-slugify - A Python slugify library that handles unicode.
  • unicode-slugify - A slugifier that generates unicode slugs. Developed by Mozilla.
  • unidecode - ASCII transliterations of Unicode text.
  • chardet - Python 2/3 compatible character encoding detector.
  • xpinyin - A library to translate chinese hanzi (漢字) to pinyin (拼音).
  • pangu.py - Spacing texts.
• Specific Format
  • tablib - A module for Tabular Datasets in XLS, CSV, JSON, YAML.
  • python-docx - Reads, queries and modifies Microsoft Word 2007/2008 docx files.
  • xlwt / xlrd - Packages is for writing and reading data and formatting information from Excel files.
  • XlsxWriter - A Python module for creating Excel .xlsx files.
  • PDFMiner - A tool for extracting information from PDF documents.
  • PyYAML - YAML implementations for Python.
• Parser
  • phonenumbers - Library for parsing, formatting, storing and validating international phone numbers.
  • python-user-agents - Browser user agent parser.
  • sqlparse - A non-validating SQL parser.
  • Pygments - A generic syntax highlighter.
  • python-nameparser - A simple Python module for parsing human names into their individual components.

Natural Language Processing

Libraries for working with human languages.

• NLTK - A leading platform for building Python programs to work with human language data.
• Pattern - A web mining module for the Python programming language. It has tools for natural language processing, machine learning, among others.
• TextBlob - Providing a consistent API for diving into common natural language processing (NLP) tasks. Stands on the giant shoulders of NLTK and Pattern, and plays nicely with both.
• jieba - Chinese Words Segementation Utilities.
• snownlp - A library for processing Chinese text.
• loso - Another Chinese segmentation library.

Command-line Tools

Libraries for building command line utilities.

• click - A package for creating beautiful command line interfaces in a composable way.
• clint - Python Command-line Application Tools.
• docopt - Pythonic command line arguments parser.
• colorama - Cross-platform colored terminal text.

Documentation

Libraries for generating project documentation.

• Sphinx - Python Documentation generator.
• reStructuredText - Markup Syntax and Parser Component of Docutils.
• Python-Markdown - A Python implementation of John Gruber’s Markdown.
• MkDocs - Markdown friendly documentation generator.

Imagery

Libraries for manipulating images.

• pillow - Pillow is the friendly PIL fork. PIL is the Python Imaging Library.
• thumbor - A smart imaging service. It enables on-demand crop, resizing and flipping of images.
• imgSeek - A project for searching a collection of images using visual similarity.
• python-qrcode - A pure Python QR Code generator.
• pygram - Instagram-like image filters.
• Quads - Computer art based on quadtrees.
• nude.py - Nudity detection.
• wand - Python bindings for MagickWand, C API for ImageMagick.

Audio

Libraries for manipulating audio.

• django-elastic-transcoder - Django + AWS Elastic Transcoder.
• beets - A music library manager and MusicBrainz tagger.
• pyechonest - Python client for the Echo Nest API.
• dejavu - Audio fingerprinting and recognition.
• pydub - Manipulate audio with a simple and easy high level interface.
• audioread - Cross-library (GStreamer + Core Audio + MAD + FFmpeg) audio decoding.
• mutagen - A Python module to handle audio metadata.
• tinytag - A library for reading music meta data of MP3, OGG, FLAC and Wave files.

Video

Libraries for manipulating video and GIFs.

• moviepy - A module for script-based movie editing with many formats, including animated GIFs.
• shorten.tv - Video summarization.
• youtube-dl - A small command-line program to download videos from YouTube.
• you-get - A YouTube/Youku/Niconico video downloader written in Python 3.
• coursera - Script for downloading Coursera.org videos and naming them.

Geolocation

Libraries for geocoding addresses and working with latitudes and longitudes.

• GeoDjango - A world-class geographic web framework.
• geopy - Python Geocoding Toolbox.
• pygeoip - Pure Python GeoIP API.
• GeoIP - Python API for MaxMind GeoIP Legacy Database.
• geojson - Python bindings and utlities for GeoJSON.

HTTP

Libraries for working with HTTP.

• requests - HTTP Requests for Humans™.
• httpie - A command line HTTP client, a user-friendly cURL replacement.

Database

Databases implemented in Python.

• ZODB - A native object database for Python. A key-value and object graph database.

Database Drivers

Libraties for connecting and operating databases

• Relational Databases
  • mysql-python - The MySQL database connector for Python.
  • psycopg2 - The most popular PostgreSQL adapter for the Python.
• NoSQL Databases
  • PyMongo - The official Python client for MongoDB.
  • redis-py - The Redis Python Client.

ORM

Libraries that implement Object-Relational Mapping or datamapping techniques.

• Relational Databases
  • Django Models - A part of Django.
  • SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper.
  • peewee - A small, expressive ORM.
  • PonyORM - ORM that provides a generator-oriented interface to SQL.
• NoSQL Databases
  • MongoEngine - A Python Object-Document-Mapper for working with MongoDB.
  • redisco - A Python Library for Simple Models and Containers Persisted in Redis.

Web Frameworks

Web development frameworks.

• Django - The most popular web framework in Python.
• Flask - A microframework for Python.
• Bottle - A fast, simple and lightweight WSGI micro web-framework.
• Pyramid - A small, fast, down-to-earth, open source Python web framework.
• web2py - A full stack web framework and platform focused in the ease of use.
• TurboGears - The Web Framework that scales with you. Starts as a microframework and scales up to a fullstack solution.
• Grok - A framework built on the existing Zope 3 libraries, offers a lot of building blocks for web development.
• CherryPy - A Minimalist Python Web Framework, HTTP/1.1-compliant and WSGI thread-pooled.
• web.py - A web framework for Python that is as simple as it is powerful.
• Zope Toolkit - A set of libraries intended for reuse by projects to develop web applications or web frameworks.
• Bluebream - An open-source web application server, framework and library, created by the Zope community and formerly known as Zope 3.

CMS

Content Management Systems

• Mezzanine - A powerful, consistent, and flexible content management platform.
• Wagtail - A Django content management system.
• django-oscar - An open-source ecommerce framework for Django.
• Quokka CMS - Flexible, extensible, small CMS powered by Flask and MongoDB.
• Opps CMS - A Django-based CMS for magazines, newspapers websites and portals with high-traffic.
• Plone - Content Management System built on top of the open source application server Zope and the accompanying Content Management Framework.
• django-cms - An Open source enterprise content management system based on the django framework.

RESTful API

Libraries for developing RESTful APIs.

• cornice - A REST framework for Pyramid
• django-rest-framework - A powerful and flexible toolkit that makes it easy to build Web APIs.
• django-tastypie - Creating delicious APIs for Django apps.
• flask-api - An implementation of the same web browsable APIs that django-rest-framework provides.
• flask-restful - An extension for Flask that adds support for quickly building REST APIs.
• falcon - A high-performance Python framework for building cloud APIs and web app backends.
• eve - REST API framework powered by Flask, MongoDB and good intentions.
• sandman - Automated REST APIs for existing database-driven systems.

Authentication and OAuth

Libraries for implementing authentications schemes.

• python-social-auth - An easy-to-setup social authentication mechanism.
• django-oauth-toolkit - OAuth2 goodies for the Djangonauts.
• django-oauth2-provider - Providing OAuth2 access to Django app.
• python-oauth2 - A fully tested, abstract interface to creating OAuth clients and servers.
• rauth - A Python library for OAuth 1.0/a, 2.0, and Ofly.

Template Engine

Libraries and tools for templating and lexing.

• Jinja2 - A modern and designer friendly templating language.
• Genshi - Python templating toolkit for generation of web-aware output.
• Mako - Hyperfast and lightweight templating for the Python platform.
• Chameleon - Chameleon is an HTML/XML template engine for Python. Modeled after ZPT, optimized for speed.
• Spitfire - A very fast Python template compiler.

Queue

Libraries for working with event and task queues.

• celery - An asynchronous task queue/job queue based on distributed message passing.
• huey - Little multi-threaded task queue.
• mrq - Mr. Queue - A distributed worker task queue in Python using Redis & gevent.
• rq - Simple job queues for Python.

Search

Libraries and software for indexing and performing search queries on data.

• django-haystack - Modular search for Django.
• elasticsearch-py - The official low-level Python client for Elasticsearch.
• solrpy - A Python client for solr.
• Whoosh - A fast, pure Python search engine library.

Asset Management

Tools for managing, compressing and minifying website assets.

• django-compressor - Compresses linked and inline javascript or CSS into a single cached file.
• webassets - Bundles, optimizes, and manages unique cache-busting URLs for static resources.
• fanstatic - Packages, optimizes, and serves static file dependencies as Python packages.
• fileconveyor - Monitors changes, processes, and transports assets to CDNs and file storage systems.
• django-storages - A collection of custom storage backends for Django.

Caching

Libraries for caching data.

• Beaker Caching & Sessions - Beaker is a library for caching and sessions for use with web applications and stand-alone Python scripts and applications.
• HermesCache - Python caching library with tag-based invalidation and dogpile effect prevention.

Email

Libraries for sending and parsing email.

• inbox.py - Python SMTP Server for Humans.
• lamson - Pythonic SMTP Application Server.
• imbox - Python IMAP for Humans.
• flanker - A email address and Mime parsing library.
• marrow.mailer - High-performance extensible mail delivery framework.
• django-celery-ses - Django email backend with AWS SES and Celery.

Internationalization

Libraries for woking with i18n.

• Babel - An internationalization library for Python.

URL Manipulation

Libraries for parsing URLs.

• furl - A small Python library that makes manipulating URLs simple.
• purl - A simple, immutable URL class with a clean API for interrogation and manipulation.
• pyshorteners - A pure python URL shortening lib.

HTML Manipulation

Libraries for working with HTML and XML.

• BeautifulSoup - Providing Pythonic idioms for iterating, searching, and modifying HTML or XML.
• lxml - A very fast, easy-to-use and versatile library for handling HTML and XML.
• html5lib - A standards-compliant library for parsing and serializing HTML documents and fragments.
• pyquery - A jQuery-like library for parsing HTML.
• cssutils - A CSS library for Python.
• MarkupSafe - Implements a XML/HTML/XHTML Markup safe string for Python.
• bleach - A whitelist-based HTML sanitization and text linkification library.
• xmltodict - Working with XML feel like you are working with JSON.
• xhtml2pdf - HTML/CSS to PDF converter.

Web Crawling

Libraries for scraping websites.

• Scrapy - A fast high-level screen scraping and web crawling framework.
• portia - Visual scraping for Scrapy.
• feedparser - Universal feed parser.
• RoboBrowser - A simple, Pythonic library for browsing the web without a standalone web browser.

Web Content Extracting

Libraries for extracting web contents.

• newspaper - News extraction, article extraction and content curation in Python.
• html2text - Convert HTML to Markdown-formatted text.
• python-goose - HTML Content/Article Extractor.
• lassie - Web Content Retrieval for Humans.
• micawber - A small library for extracting rich content from URLs.
• sumy - A module for automatic summarization of text documents and HTML pages.
• Haul - An Extensible Image Crawler.
• python-readability - Fast Python port of arc90's readability tool.

Forms

Libraries for working with forms.

• django-crispy-forms - A Django app which lets you create beautiful forms in a very elegant and DRY way.
• WTForms - A flexible forms validation and rendering library.

Anti-spam

Libraries for fighting spam.

• Stopspam - Intelligent spam detection for Python.
• django-simple-spam-blocker - Simple spam blocker for Django.
• django-simple-captcha - A simple and highly customizable Django app to add captcha images to any Django form.

Tagging

Libraries for tagging items.

• django-taggit - Simple tagging for Django.

Processes and Threads

Libraries for woking with processes or threads

• envoy - Python Subprocesses for Humans™.
• sh - A full-fledged subprocess replacement for Python.

Networking

Libraries for network programming.

• gevent - A coroutine-based Python networking library that uses greenlet.
• Twisted - An event-driven networking engine.
• Tornado - A Web framework and asynchronous networking library.
• pulsar - Event-driven concurrent framework for Python.
• diesel - Greenlet-based event I/O Framework for Python.
• pyzmq - A Python wrapper for the 0MQ message library.

Admin Panels

Libraries for administrative interfaces.

• Ajenti - The admin panel your servers deserve.
• django-xadmin - Drop-in replacement of Django admin comes with lots of goodies.
• flask-admin - Simple and extensible administrative interface framework for Flask.
• flower - Real-time monitor and web admin for Celery.

DevOps Tools

Software and libraries for DevOps

• OpenStack - Open source software for building private and public clouds.
• Ansible - An IT automation tool.
• SaltStack - Infrastructure automation and management system.
• Fabric - Tool for streamlining the use of SSH for application deployment or systems administration tasks.
• Fabtools - Tools for writing awesome Fabric files.
• cuisine - Chef-like functionality for Fabric.
• gunicorn - A WSGI HTTP Server for UNIX, fast clients and sleepy applications.
• Supervisor - A Process Control System.
• psutil - A cross-platform process and system utilities module.

GUI

Libraries for working with graphical user interface applications.

• PySide - Python bindings for the Qt cross-platform application and UI framework.
• wxPython - A blending of the wxWidgets C++ class library with the Python.
• kivy - A library for creating NUI applications, running on Windows, Linux, Mac OS X, Android and iOS.
• curses - Built-in wrapper for ncurses used to create terminal GUI applications.
• urwid - A library for creating terminal GUI applications with strong support for widgets, events, rich colors, etc.
• pyglet - A cross-platform windowing and multimedia library for Python.
• Tkinter - Tkinter is Python's de-facto standard GUI package.

Game Development

Awesome game development libraries.

• Pygame - Pygame is a set of Python modules designed for writing games.
• Cocos2d - cocos2d is a framework for building 2D games, demos, and other graphical/interactive applications. It is based on pyglet.
• PySDL2 - A ctypes based wrapper for the SDL2 library.
• Panda3D - 3D game engine developed by Disney and maintained by Carnegie Mellon's Entertainment Technology Center. Written in C++, completely wrapped in Python.
• PyOgre - Python bindings for the Ogre 3D render engine, can be used for games, simulations, anything 3D.
• PyOpenGL - Python ctypes bindings for OpenGL and it's related APIs.

Logging

Libraries for generating and working with log files.

• logging - (Python standard library) Logging facility for Python.
• Sentry - A realtime logging and aggregation server.
• Raven - The Python client for Sentry.

Testing

Libraries for testing codebases and generating test data.

• Testing Frameworks
  • unittest - (Python standard library) Unit testing framework.
  • nose - nose extends unittest.
  • pytest - A mature full-featured Python testing tool.
  • mamba - The definitive testing tool for Python. Born under the banner of Behavior Driven Development (BDD).
• Mock
  • mock - A Python Mocking and Patching Library for Testing.
  • responses - A utility library for mocking out the requests Python library.
  • doublex - Powerful test doubles framework for Python.
• Fake Data
  • faker - A Python package that generates fake data.
  • model_mommy - Creating random fixtures for testing in Django.
• Code Coverage
  • coverage - Code coverage measurement.
• Load Testing
  • locust - Scalable user load testing tool written in Python.
• Error Handler
  • FuckIt.py - FuckIt.py uses state-of-the-art technology to make sure your Python code runs whether it has any right to or not.

Code Analysis and Linter

Libraries and tools for analysing, parsing and manipulation codebases.

• pysonar2 - A type inferencer and indexer for Python.
• Flake8 - The modular source code checker: pep8, pyflakes and co.
• Pylint - A source code analyzer.

Debugging Tools

Libraries for dubugging and developing

• pdb - (Python standard library) The Python Debugger.
• ipdb - IPython-enabled pdb.
• winpdb - A Platform Independent Python Debugger with GUI.
• pyringe - Debugger capable of attaching to and injecting code into Python processes.
• django-debug-toolbar - Display various debug information about the current request/response.
• django-devserver - A drop-in replacement for Django's runserver.
• memory_profiler - Monitor Memory usage of Python code.

Science and Data Analysis

Libraries for scientific computing and data analyzing.

• SciPy - A Python-based ecosystem of open-source software for mathematics, science, and engineering.
• NumPy - A fundamental package for scientific computing with Python.
• Numba - Python JIT (just in time) complier to LLVM aimed at scientific Python by the developers of Cython and NumPy.
• NetworkX - A high-productivity software for complex networks.
• Pandas - A library providing high-performance, easy-to-use data structures and data analysis tools.
• PyMC - Markov Chain Monte Carlo sampling toolkit.
• zipline - A Pythonic algorithmic trading library.
• PyDy - Short for Python Dynamics, used to assist with workflow in the modeling of dynamic motion based around NumPy, SciPy, IPython, and matplotlib.
• SymPy - A Python library for symbolic mathematics.
• statsmodels - Statistical modeling and econometrics in Python.

Data Visualization

Libraries for visualizing data

• matplotlib - A Python 2D plotting library.
• bokeh - Interactive Web Plotting for Python.
• vincent - A Python to Vega translator.
• d3py - A plottling library for Python, based on D3.js.
• ggplot - Same API as ggplot2 for R.
• Kartograph.py - Rendering beautiful SVG maps in Python.

Machine Learning

Libraries for Machine Learning

• scikit-learn - A Python module for machine learning built on top of SciPy.
• pattern - Web mining module for Python.
• NuPIC - Numenta Platform for Intelligent Computing.
• Pylearn2 - A Machine Learning library based on Theano.
• hebel - GPU-Accelerated Deep Learning Library in Python.
• gensim - Topic Modelling for Humans.
• PyBrain - Another Python Machine Learning Library.
• Crab - A flexible, fast recommender engine.
• python-recsys - A Python library for implementing a Recommender System.

MapReduce

Framworks and libraries for MapReduce.

• PySpark - The Spark Python API.
• dpark - Python clone of Spark, a MapReduce alike framework in Python.
• luigi - A module that helps you build complex pipelines of batch jobs.
• mrjob - Run MapReduce jobs on Hadoop or Amazon Web Services.
• dumbo - Python module that allows one to easily write and run Hadoop programs.

Third-party APIs

Libraries for accessing third party APIs.

• apache-libcloud - One Python library for all clouds.
• boto - Python interface to Amazon Web Services.
• s3cmd - A command line tool for managing Amazon S3 and CloudFront.
• twython - A Python wrapper for the Twitter API.
• soundcloud-python - A Python wrapper around the Soundcloud API.
• google-api-python-client - Google APIs Client Library for Python.
• facebook-sdk - Facebook Platform Python SDK.
• Wikipedia - A Pythonic wrapper for the Wikipedia API.
• python-instagram - A Python Client for Instagram API.
• gmail - A Pythonic interface for Gmail.

Algorithms and Design Patterns

Collections of algorithms and design patterns.

• python-patterns - A collection of design patterns in Python.
• algorithms - module of algorithms for Python.

Miscellaneous

Useful libraries or tools that don't fit in the categories above.

• IPython - IPython provides a rich toolkit to help you make the most out of using Python interactively.

Foreign Function Interface

Libraries for providing foreign function interface.

• cffi - Foreign Function Interface for Python calling C code.

High Performance

Libraries for making Python faster.

• Cython - Optimizing Static Complier for Python. Uses type mixins to compile Python into C or C++ modules resulting in large performance gains.
• PyPy - An implmentation of Python in Python. The interpreter uses black magic to make Python very fast without having to add in additional type information.
• Stackless Python - An enhanced version of the Python.
• Pyston - A Python implementation built using LLVM and modern JIT techniques with the goal of achieving good performance.

Editor Plugins

• Vim
  • TODO
• Emacs
  • TODO
• Sublime Text
  • SublimeJEDI - A Sublime Text plugin to the awesome autocomplete library Jedi.
  • Anaconda - Anaconda turns your Sublime Text 3 in a full featured Python development IDE.
  • Djaneiro - Django support for Sublime Text.

Resources

Where to discover new Python libraries.

Websites

• r/Python - News about Python.
• Python 3 Wall of Superpowers - Too many popular Python packages don't support Python 3.
• Trending Python repositories on GitHub today - Good place to find new Python libraries.
• Django Packages - A directory of reusable apps, sites, tools, and more for Django projects.

Weekly

• Pycoder's Weekly - A free weekly newsletter, on Fridays, for those interested in Python development and various topics around Python.
• Python Weekly - A free weekly newsletter featuring curated news, articles, new releases, jobs etc related to Python.

Twitter

• @pypi
• @getpy
• @planetpython
• @pycoders
• @PythonWeekly

Contributing

Your contributions are always welcome!