Native Parallel Client
API documentation for the ssh2-python
(libssh2
) based parallel client.
- class pssh.clients.native.parallel.ParallelSSHClient(hosts, user=None, password=None, port=22, pkey=None, num_retries=3, timeout=None, pool_size=100, allow_agent=True, host_config=None, retry_delay=5, proxy_host=None, proxy_port=None, proxy_user=None, proxy_password=None, proxy_pkey=None, forward_ssh_agent=False, keepalive_seconds=60, identity_auth=True, ipv6_only=False)
ssh2-python based parallel client.
- Parameters:
hosts (list(str)) – Hosts to connect to
user (str) – (Optional) User to login as. Defaults to logged in user
password (str) – (Optional) Password to use for login. Defaults to no password
port (int) – (Optional) Port number to use for SSH connection. Defaults to 22.
pkey (str or bytes) – Private key file path or private key data to use. Paths must be str type and either absolute path or relative to user home directory like
~/<path>
. Bytes type input is used as private key data for authentication.num_retries (int) – (Optional) Number of connection and authentication attempts before the client gives up. Defaults to 3.
retry_delay (int or float) – Number of seconds to wait between retries. Defaults to
pssh.constants.RETRY_DELAY
(Optional) Global timeout setting in seconds for all remote operations including all SSH client operations DNS, opening connections, reading output from remote servers, et al.
For concurrent functions this is a cumulative timeout setting for all concurrent operations. These functions, eg
run_command
andjoin
, also allow timeout to be set just for those functions if not set globally via this option.For per host operations like
list(host_out.stdout)
for reading output it is applied per host if set. Host output read timeout can also be set separately viarun_command(<..>, read_timeout=<seconds>)
Defaults to OS default - usually 60 seconds.pool_size (int) – (Optional) Greenlet pool size. Controls concurrency, on how many hosts to execute tasks in parallel. Defaults to 100. Overhead in event loop will determine how high this can be set to, see scaling guide lines in project’s readme.
host_config (list(
pssh.config.HostConfig
)) – (Optional) Per-host configuration for cases where not all hosts use the same configuration.allow_agent (bool) – (Optional) set to False to disable connecting to the system’s SSH agent.
identity_auth (bool) – (Optional) set to False to disable attempting to authenticate with default identity files from pssh.clients.base_ssh_client.BaseSSHClient.IDENTITIES
proxy_host (str) – (Optional) SSH host to tunnel connection through so that SSH clients connect to host via client -> proxy_host -> host
proxy_port (int) – (Optional) SSH port to use to login to proxy host if set. Defaults to 22.
proxy_user (str) – (Optional) User to login to
proxy_host
as. Defaults to logged in user.proxy_password (str) – (Optional) Password to login to
proxy_host
with. Defaults to no password.proxy_pkey (str or bytes) – (Optional) Private key file to be used for authentication with
proxy_host
. Defaults to available keys from SSHAgent and user’s SSH identities. Bytes type input is used as private key data for authentication.forward_ssh_agent (bool) – (Optional) Turn on SSH agent forwarding. Currently unused.
ipv6_only (bool) – Choose IPv6 addresses only if multiple are available for the host(s) or raise NoIPv6AddressFoundError otherwise. Note this will disable connecting to an IPv4 address if an IP address is provided instead.
- Raises:
pssh.exceptions.PKeyFileError
on errors finding provided private key.
- copy_file(local_file, remote_file, recurse=False, copy_args=None)
Copy local file to remote file in parallel via SFTP.
This function returns a list of greenlets which can be join-ed on to wait for completion.
gevent.joinall()
function may be used to join on all greenlets and will also raise exceptions from them if called withraise_error=True
- default is False.Alternatively call .get() on each greenlet to raise any exceptions from it.
Exceptions listed here are raised when either
gevent.joinall(<greenlets>, raise_error=True)
or.get()
on each greenlet are called, not this function itself.- Parameters:
local_file (str) – Local filepath to copy to remote host
remote_file (str) – Remote filepath on remote host to copy file to
recurse (bool) – Whether or not to descend into directories recursively.
copy_args (tuple or list) – (Optional) format local_file and remote_file strings with per-host arguments in
copy_args
.copy_args
length must equal length of host list -pssh.exceptions.HostArgumentError
is raised otherwise
- Return type:
list(
gevent.Greenlet
) of greenlets for remote copy commands- Raises:
ValueError
when a directory is supplied to local_file and recurse is not set- Raises:
pssh.exceptions.HostArgumentError
on number of per-host copy arguments not equal to number of hosts- Raises:
pssh.exceptions.SFTPError
on SFTP initialisation errors- Raises:
pssh.exceptions.SFTPIOError
on I/O errors writing via SFTP- Raises:
OSError
on local OS errors like permission denied
Note
Remote directories in
remote_file
that do not exist will be created as long as permissions allow.
- copy_remote_file(remote_file, local_file, recurse=False, suffix_separator='_', copy_args=None, encoding='utf-8')
Copy remote file(s) in parallel via SFTP as <local_file><suffix_separator><host>
With a
local_file
value ofmyfile
and default separator_
the resulting filename will bemyfile_myhost
for the file from hostmyhost
.This function, like
ParallelSSHClient.copy_file()
, returns a list of greenlets which can be join-ed on to wait for completion.gevent.joinall()
function may be used to join on all greenlets and will also raise exceptions if called withraise_error=True
- default is False.Alternatively call .get on each greenlet to raise any exceptions from it.
Exceptions listed here are raised when either
gevent.joinall(<greenlets>, raise_error=True)
is called or.get
is called on each greenlet, not this function itself.- Parameters:
remote_file (str) – remote filepath to copy to local host
local_file (str) – local filepath on local host to copy file to
recurse (bool) – whether or not to recurse
suffix_separator (str) – (Optional) Separator string between filename and host, defaults to
_
. For example, for alocal_file
value ofmyfile
and default separator the resulting filename will bemyfile_myhost
for the file from hostmyhost
.suffix_separator
has no meaning ifcopy_args
is providedcopy_args (tuple or list) – (Optional) format remote_file and local_file strings with per-host arguments in
copy_args
.copy_args
length must equal length of host list -pssh.exceptions.HostArgumentError
is raised otherwiseencoding (str) – Encoding to use for file paths.
- Return type:
list(
gevent.Greenlet
) of greenlets for remote copy commands- Raises:
ValueError
when a directory is supplied to local_file and recurse is not set- Raises:
pssh.exceptions.HostArgumentError
on number of per-host copy arguments not equal to number of hosts- Raises:
pssh.exceptions.SFTPError
on SFTP initialisation errors- Raises:
pssh.exceptions.SFTPIOError
on I/O errors reading from SFTP- Raises:
OSError
on local OS errors like permission denied
Note
Local directories in local_file that do not exist will be created as long as permissions allow.
Note
File names will be de-duplicated by appending the hostname to the filepath separated by
suffix_separator
.
- run_command(command, sudo=False, user=None, stop_on_errors=True, use_pty=False, host_args=None, shell=None, encoding='utf-8', read_timeout=None)
Run command on all hosts in parallel, honoring self.pool_size, and return output.
This function will block until all commands have been received by remote servers and then return immediately.
More explicitly, function will return after connection and authentication establishment in the case of on new connections and after execute commands have been accepted by successfully established SSH channels.
Any connection and/or authentication exceptions will be raised here and need catching unless
run_command
is called withstop_on_errors=False
in which case exceptions are added to individual host output instead.- Parameters:
command (str) – Command to run
sudo (bool) – (Optional) Run with sudo. Defaults to False
user (str) – (Optional) User to run command as. Requires sudo access for that user from the logged in user account.
stop_on_errors (bool) – (Optional) Raise exception on errors running command. Defaults to True. With stop_on_errors set to False, exceptions are instead added to output of run_command. See example usage below.
shell (str) – (Optional) Override shell to use to run command with. Defaults to login user’s defined shell. Use the shell’s command syntax, eg shell=’bash -c’ or shell=’zsh -c’.
use_pty (bool) – (Optional) Enable/Disable use of pseudo terminal emulation. Defaults to
False
host_args (tuple or list) – (Optional) Format command string with per-host arguments in
host_args
.host_args
length must equal length of host list -pssh.exceptions.HostArgumentError
is raised otherwiseencoding (str) – Encoding to use for command string and output. Must be valid Python codec
read_timeout (float) – (Optional) Timeout in seconds for reading from stdout or stderr. Reading from stdout/stderr will raise
pssh.exceptions.Timeout
aftertimeout
seconds when set if remote output is not ready.
- Return type:
list(
pssh.output.HostOutput
)- Raises:
pssh.exceptions.AuthenticationError
on authentication error- Raises:
pssh.exceptions.UnknownHostError
on DNS resolution error- Raises:
pssh.exceptions.ConnectionError
on error connecting- Raises:
pssh.exceptions.HostArgumentError
on number of host arguments not equal to number of hosts- Raises:
TypeError
on not enough host arguments for cmd string format- Raises:
KeyError
on no host argument key in arguments dict for cmd string format- Raises:
pssh.exceptions.ProxyError
on errors connecting to proxy if a proxy host has been set.- Raises:
pssh.exceptions.Timeout
on timeout starting command.- Raises:
Exceptions from
ssh2.exceptions
for all other specific errors such asssh2.exceptions.SocketDisconnectError
et al.
- scp_recv(remote_file, local_file, recurse=False, copy_args=None, suffix_separator='_')
Copy remote file(s) in parallel via SCP as <local_file><suffix_separator><host> or as per
copy_args
argument.With a
local_file
value ofmyfile
and default separator_
the resulting filename will bemyfile_myhost
for the file from hostmyhost
.De-duplication behaviour is configurable by providing
copy_args
argument, see below.This function, like
ParallelSSHClient.scp_send()
, returns a list of greenlets which can be join-ed on to wait for completion.gevent.joinall()
function may be used to join on all greenlets and will also raise exceptions if called withraise_error=True
- default is False.Alternatively call .get on each greenlet to raise any exceptions from it.
Exceptions listed here are raised when either
gevent.joinall(<greenlets>, raise_error=True)
is called or.get
is called on each greenlet, not this function itself.- Parameters:
remote_file (str) – remote filepath to copy to local host
local_file (str) – local filepath on local host to copy file to
recurse (bool) – whether or not to recurse
suffix_separator (str) – (Optional) Separator string between filename and host, defaults to
_
. For example, for alocal_file
value ofmyfile
and default separator the resulting filename will bemyfile_myhost
for the file from hostmyhost
.suffix_separator
has no meaning ifcopy_args
is providedcopy_args (tuple or list) – (Optional) format remote_file and local_file strings with per-host arguments in
copy_args
.copy_args
length must equal length of host list -pssh.exceptions.HostArgumentError
is raised otherwise
- Return type:
list(
gevent.Greenlet
) of greenlets for remote copy commands.- Raises:
ValueError
when a directory is supplied to local_file and recurse is not set.- Raises:
pssh.exceptions.HostArgumentError
on number of per-host copy arguments not equal to number of hosts.- Raises:
pssh.exceptions.SCPError
on errors copying file.- Raises:
OSError
on local OS errors like permission denied.
Note
Local directories in
local_file
that do not exist will be created as long as permissions allow.Note
File names will be de-duplicated by appending the hostname to the filepath separated by
suffix_separator
or as percopy_args
argument if provided.
- scp_send(local_file, remote_file, recurse=False, copy_args=None)
Copy local file to remote file in parallel via SCP.
This function returns a list of greenlets which can be join-ed on to wait for completion.
gevent.joinall()
function may be used to join on all greenlets and will also raise exceptions from them if called withraise_error=True
- default is False.Alternatively call .get() on each greenlet to raise any exceptions from it.
Note
Creating remote directories when either
remote_file
contains directory paths orrecurse
is enabled requires SFTP support on the server as libssh2 SCP implementation lacks directory creation support.- Parameters:
local_file (str) – Local filepath to copy to remote host
remote_file (str) – Remote filepath on remote host to copy file to
copy_args (tuple or list) – (Optional) format local_file and remote_file strings with per-host arguments in
copy_args
.copy_args
length must equal length of host list -pssh.exceptions.HostArgumentError
is raised otherwiserecurse (bool) – Whether or not to descend into directories recursively.
- Return type:
list(
gevent.Greenlet
) of greenlets for remote copy commands.- Raises:
pssh.exceptions.SCPError
on errors copying file.- Raises:
OSError
on local OS errors like permission denied.