Skip to content

Pip modules

If additional pip modules required:

python.pip 'module1:1.0', 'module2:1.0'

or

python {
    pip 'module1:1.0'
    pip 'module2:1.0'
}

Module format is: name:version (will mean name==version in pip notion). Non-strict version definition is not allowed (for obvious reasons). Dependencies are installed in declaration order. If duplicate declaration specified then only the latest declaration will be used:

python.pip 'module1:2.0', 'module2:1.0', 'module1:1.0'

Will install version 1.0 of module1 because it was the latest declaration. "Module overrides" works for all declaration types (see below): the latest declared module version always wins.

Dependencies are installed with pipInstall task which is called before any declared PythonTask.

By default, dependencies are installed inside project specific virtualenv (project specific copy of python environment, configured with python.envPath).

Behaviour matrix for possible scope and installVirtualenv configurations:

scope installVirtualenv Behaviour default
GLOBAL ignored packages installed in global scope (pip install name)
USER ignored packages installed in user scope (pip install name --user)
VIRTUALENV_OR_USER true if virtualenv not installed, install it in user scope; create project specific virtualenv and use it default
VIRTUALENV_OR_USER false when virtualenv is not installed install packages in user scope (same as USER); when virtualenv installed create project specific virtualenv and use it
VIRTUALENV true if virtualenv not installed, install it in user scope; create project specific virtualenv and use it
VIRTUALENV false throw error when virtualenv not installed

Note that VIRTUALENV + true and VIRTUALENV_OR_USER + true behaviours are the same. Different scope name here describes behavior for unexpected installVirtualenv=false change (to fail or fallback to user scope).

USER and GLOBAL scopes will ignore local (virtual)environment, even if project-specific environment was created before, with these options global python will be used instead.

Pip module extra features

You can declare modules with extra features in module name to install special version of module (with enabled features):

python.pip 'requests[socks,security]:2.18.4'

Important

It is impossible to track if this "variation" of module is installed, so plugin performs up-to-date check for such modules by name only (for example, if 'requests==2.18.4' is already installed). For most cases, this is suitable behaviour because, by default, modules are installed in virtualenv and so you will always have correct module installed. For other cases, you can disable up-to-date checks (delegate all dependencies logic to pip): python.alwaysInstallModules = true

VCS pip modules

You can declare vcs modules: modules installed directly from version control (e.g. git, svn). Format:

vcs+protocol://repo_url/@vcsVersion#egg=pkg-pkgVersion
  • @vcsVersion part is required: prefer using commit version or tag for reproducible builds
  • -pkgVersion is installed module version. Required to be able to compare declared plugin with installed version.

For example:

python.pip 'git+https://github.com/ictxiangxin/boson/@b52727f7170acbedc5a1b4e1df03972bd9bb85e3#egg=boson-0.9'

Declares module boson version 0.9, installed from git commit b52727f7170acbedc5a1b4e1df03972bd9bb85e3 (it may be tag name or branch, but prefer not using branch names).

pipInstall will be considered up-to-date if boson==0.9 is already installed. Note that declared module version is completely free: you can set any version (0.10, 1.2, etc.), it is not checked and used only for up-to-date validation.

Warning

Module version part assumed to follow the last dash, so if you specify version like somethinf-12.0-alpha.1 it would be parsed incorrectly (as package somethinf-12.0 version alpha.1)! Don't use dashes in version!

Vcs module installation is: source checkout and module build (using setup.py). You may need to specify subdirectory as &subdirectory=pkg_dir (see docs)

To avoid installation problems, package version is not used for actual installation (in spite of the fact that its official convention, it doesn't work in some cases). For example, module above will be installed as (without -0.9):

pip install git+https://github.com/ictxiangxin/boson/@b52727f7170acbedc5a1b4e1df03972bd9bb85e3#egg=boson

All pip supported vcs could be used: git, svn, hg, bzr

If up-to-date logic, implemented by pipInstall task, does not suit your needs, you can always disable it with python.alwaysInstallModules = true (pip always called). But this will be slower.

Note

Since pip 20, compiled vcs module is cached (before it was build on each execution), but it is possible to disable cache (for all modules) with python.usePipCache=false configuration (applies --no-cache-dir pip flag)

Extra pip repositories

To add additional pip repositories (probably self-hosted):

python {
    extraIndexUrls = ["http://extra-url.com", "http://extra-url.com"]
}

or with shortcut method (shortcut may be used multiple times):

python {
    extraIndexUrls "http://extra-url.com", "http://extra-url2.com" 
}

Extra urls will be applied as --extra-index-url flag for pip commands supporting it: install, download, list and wheel. By default, it only affects pipInstall and pipList tasks. Applied for all BasePipTask, so if you have custom pip tasks, it would be affected too.

In case of ssl problems (stale or self-signed certificated), mark domains as trusted:

python {
    trustedHosts = ["extra-url.com"]
}

or

python {
    trustedHosts "extra-url.com"
}

Applied as --trusted-host option only for pipInstall (because pip install is the only command supporting this option).

Note

If, for some reason, you don't want to specify it for all pip tasks, you can configure exact task, for example: pipInstall.extraIndexUrls = ["http://extra-url.com", "http://extra-url2.com"]

Extra pip install options

It is impossible to support directly all possible pip install options usages directly with api (safe way), so there is a direct configuration for an additional options. For example:

pipInstall.options('--upgrade-strategy', 'only-if-needed')

Shortcut method above may be called multiple times:

pipInstall.options('--a', 'value')
pipInstall.options('--b', 'value')

Or you can use property directly:

pipInstall.options = ['--a', 'value', '--b', 'value']

Scope

Pip dependencies could be installed per project, for current user (~/) or globally.

Default behaviour:

  • if virtualenv module installed (or automatically installed): manage pip dependencies per project (env .gradle/python created)
  • if no virtualenv - use user scope (--user pip flag): pip modules are installed only for current user (this avoid permission problems on linux)

To change defaults:

python.scope = VIRTUALENV
  • GLOBAL - install modules globally (this may not work on linux due to permissions)
  • USER - use --user flag to install for current user only
  • VIRTUALENV_OR_USER - default
  • VIRTUALENV - use virtualenv (if module not installed - error thrown)

Note

Values may be declared without quotes because it's an enum which values are declared as project ext properties (ext.USER==ru.vyarus.gradle.plugin.python.PythonExtension.Scope.USER).

Complete behaviour matrix see above.

Check modules updates

To quick check if new versions are available for the registered pip modules use pipUpdates task:

:pipUpdates
The following modules could be updated:

    package            version latest type 
    ------------------ ------- ------ -----
    click              6.6     6.7    wheel

Note that it will not show versions for transitive modules, only for modules specified directly in python.pip.

To see all available updates (without filtering):

pipUpdates.all = true

Note

If you see an error like

TypeError: '>' not supported between instances of 'Version' and 'SetuptoolsVersion'

then update pip:

pip install -U pip