Manager

class Manager(port=7473, num_workers=-1, ipports=None, worker_port=7474, only_connect=False)[source]

Bases: ServerBase

BQSKit Runtime Manager.

A Manager is a middle node in the process hierarchy and is responsible for managing workers or other managers. The manager is part of the detached architecture. Here managers are started individually as separate processes, which in turn start their own workers. Then, if necessary, more managers can be started to manage the level-1 managers, and so on, until finally, a detached server is started and clients can connect.

__init__(port=7473, num_workers=-1, ipports=None, worker_port=7474, only_connect=False)[source]

Create a manager instance in one of two ways:

  1. Leave all options default and start a manager which spawns and

    manages as many worker processes as there are os threads. You can also specify the number of workers to spawn via the num_workers parameter. In this mode, the manager is a level-1 manager and manages workers.

  2. Specify ip and port pairs, then assume at each endpoint there

    is a listening manager and attempt to establish a connection. In this mode, the manager will not spawn any workers and just manage the specified managers.

In either case, if any problems arise during startup, no recovery is attempted and the manager terminates.

Parameters:

  • port (int) – The port this manager listens for server connections. Default can be found in the default_manager_port global variable.

  • num_workers (int) – The number of workers to spawn. If -1, then spawn as many workers as CPUs on the system. (Default: -1). Ignored if ipports is not None.

  • ipports (list[tuple[str, int]] | None) – If not None, then all the addresses and ports of running managers to connect to.

  • worker_port (int) – The port this server will listen for workers on. Default can be found in the default_worker_port global variable.

  • only_connect (bool) – If true, do not spawn workers, only connect to already spawned workers.

Attributes

upper_id_bound

The node starts with an ID range from 0 -> 2^30.

running

True while the node is running.

sel

Used to efficiently idle and wake when communication is ready.

logger

Logger used to print operational log messages.

employees

Tracks this node's employees, which are managers or workers.

conn_to_employee_dict

Used to find the employee associated with a message.

Methods

assign_tasks(tasks)

Go through the tasks and assign each one to an employee.

broadcast_cancel(addr)

Broadcast a cancel message to my employees.

connect_to_manager(ip, port, lb, ub)

Connect to a manager at the endpoint given by ip and port.

connect_to_managers(ipports)

Connect to all managers given by endpoints in ipports.

connect_to_workers([num_workers, port])

Connect to worker processes.

get_employee_responsible_for(worker_id)

Return the employee that manages worker_id.

handle_disconnect(conn)

Remove conn from the server.

handle_message(msg, direction, conn, payload)

Process the message coming from direction.

handle_result_from_below(result)

Forward the result to its destination and track the completion.

handle_shutdown()

Shutdown the manager and clean up spawned processes.

handle_system_error(error_str)

Handle an error in runtime code as opposed to client code.

handle_update(conn, task_diff)

Handle a task count update from a lower level manager or worker.

handle_waiting(conn, new_idle_count)

Record that an employee is idle with nothing to do.

is_my_worker(worker_id)

Return true if worker_id is one of my workers (recursively).

listen_once(ip, port)

Listen on ip:port for a connection and return on first one.

run()

Main loop.

schedule_tasks(tasks)

Schedule tasks between this node's employees.

send_result_down(result)

Send the result to the appropriate employee.

send_up_or_schedule_tasks(tasks)

Either send the tasks upstream or schedule them downstream.

spawn_workers([num_workers, port])

Spawn worker processes.

update_upstream_idle_workers()

Update the total number of idle workers upstream.