dhaivat home

Documenting this so I can read it in future.

Part 1 : build custom pips

Basic Example: say you are working on a python app called nerf, with layout like

% tree nerf 
    nerf
    ├── MANIFEST.in
    ├── nerf
    │   ├── __init__.py
    │   └── nerfwork.py
    ├── README.txt
    └── setup.py
1 directory, 4 files

files critical to the mission of rolling a custom pip are setup.py and README.txt

from setuptools import setup

setup(
    name='nerf',
    version='0.0.1',
    author='Nerf Dev',
    author_email='nerf@eat.me',
    packages=['nerf'],
    scripts=[],
    url='http://nerftastic.com/nerf',
    license='LICENSE.txt',
    description='Demo nerf app',
    long_description=open('README.txt').read(),
    install_requires=[
        "mock==0.8.0",
        "uWSGI==1.1.2"
    ],
)

If you are planning to add datafiles (config files, static files etc), it can be done with package_data={ } for me that did not work.
I ended up using a MANIFEST.in file with something like this

include configs/uwsgi.ini

With manifest.in in play I had to extend the setup.py to copy the config file to the right place.

Now setup.py looks something like this..

from setuptools import setup
from setuptools.dist import Distribution
from setuptools.command.install import install as _install
import os

class NerfInstall(_install):
    def run(self):
        uwsgi_conf_path = os.path.join(os.environ.get('VIRTUAL_ENV'),
                'etc/uwsgi/')
        os.system('mkdir -p %s ' % (uwsgi_conf_path))
        os.system('cp %s %s' % ('configs/uwsgi.ini', uwsgi_conf_path))
        _install.run(self)

class NerfDist(Distribution):
    def __init__(self, *attrs):
        Distribution.__init__(self, *attrs)
        self.cmdclass['install'] = NerfInstall


setup(
    name='nerf',
    version='0.0.1',
    author='Nerf Dev',
    author_email='nerf@eat.me',
    packages=['nerf'],
    package_data={ "nerf" : { "*.ini" } ],
    scripts=[],
    url='http://nerftastic.com/nerf',
    license='LICENSE.txt',
    description='Demo nerf app',
    long_description=open('README.txt').read(),
    install_requires=[
        "mock==0.8.0",
        "uWSGI==1.1.2"
    ],
    distclass=NerfDist,
)

To build the package run following command from toplevel nerf dir

python setup.py sdist

check dist folder for the built package, poke around to see if it looks ok.

Part 2 : Setup private pypi server and upload+install packages

Following commands will,

setup a virtualenv
install required packages
start pypi server

## setup virtual env
virtualenv pypi-site
source pypi-site/bin/activate

## install packages
pip install django gunicorn djangopypi2

## setup djangopypi2
manage-pypi-site syncdb
manage-pypi-site collectstatic
manage-pypi-site loaddata initial

## run the server
gunicorn_django djangopypi2.website.settings

Once the server is setup, head to http://localhost:8000/ look around if things look ok, update/create your ~/.pypirc, notice that I’ve left pypi server credentials to dummy credentials because I don’t want my private code accidentally uploaded to public server

[distutils]
index-servers =
    local
    pypi

[pypi]
username:user
password:secret

[local]
username:admin
password:admin
repository:http://localhost:8000/pypi/

To upload the package to the pypi server run, it will register the package with PyPI server, build it and upload it.

%python setup.py register -r local sdist upload -r local
.... skipped ...
Registering nerf to http://localhost:8000/pypi/
Server response (200): OK
running sdist
Creating tar archive
... skipped ...
removing 'nerf-0.0.0' (and everything under it)
running upload
Submitting dist/nerf-0.0.0.tar.gz to http://localhost:8000/pypi/
Server response (200): OK

Almost there….

Last part – install the package you just built, the package is on a non standard pypi server, to include it in your install/search path, modify ~/.pip/pip.conf file to look like below, and run pip install nerf

% cat ~/.pip/pip.conf
[global]
extra-index-url = http://localhost:8000/simple
% pip install nerf
... usual pip output..

DjangoPyPI
DjangoPyPI2 Installation

comments powered by Disqus
Fork me on GitHub