Maestro

relai.maestro.Maestro(client, agent_fn, goal=None, max_memory=20, name='No Name', log_to_platform=True)

Maestro automatically optimizes an AI agent to maximize its Critico score, navigating the space of configurations to intelligently improve performance on the chosen criteria.

Parameters:

Name	Type	Description	Default
client	AsyncRELAI	An instance of the AsyncRELAI client to interact with the RELAI platform.	required
agent_fn	AsyncAgent	The agent function to be optimized.	required
goal	str	Optional description of the goal of optimization. If None, increasing evaluation score will be considered as the only goal. Defaults to None.	None
max_memory	int	Control the maximum number of previous optimization history visible at each optimization step. Defaults to 20.	20
name	str	Name of the configuration optimization visualization on RELAI platform. Defaults to "No Name".	'No Name'
log_to_platform	bool	Whether to log optimization progress and results on RELAI platform. Defaults to True.	True

add_setup(simulator, critico, weight=1)

Add a new setup consisting of a simulator and a critico to Maestro.

Parameters:

Name	Type	Description	Default
simulator	AsyncSimulator	An AsyncSimulator to run the agent in the new setup.	required
critico	Critico	A Critico with evaluators for the new setup.	required
weight	float	A positive float representing the weight of this setup in comparson to others. Defaults to 1.	1

optimize_config(total_rollouts, batch_size=8, explore_radius=5, explore_factor=0.5, group_id=None, verbose=True, agent_version_uuid=None, environment_uuid=None) async

Optimize the configs (parameters) of the agent.

Parameters:

Name	Type	Description	Default
total_rollouts	int	Total number of rollouts to use for optimization.	required
batch_size	int	Base batch size to use for individual optimization steps. Defaults to 8.	8
explore_radius	int	A positive integer controlling the aggressiveness of exploration during optimization. A larger explore_radius encourages the optimizer to make more substantial changes between successive configurations. Defaults to 5.	5
explore_factor	float	A float between 0 to 1 controlling the exploration-exploitation trade-off. A higher explore_factor allocates more rollouts to discover new configs, while a lower value allocates more rollouts to ensure the discovered configs are thoroughly evaluated. Defaults to 0.5.	0.5
group_id	str	An optional group ID to associate all runs together. If not provided, a new UUID will be generated.	None
verbose	bool	If True, related information will be printed during the optimization step. Defaults to True.	True
agent_version_uuid	str	An optional agent version UUID to associate with the runs.	None
environment_uuid	str	An optional environment UUID to associate with the runs.	None

Raises:

Type	Description
ValueError	If the input parameters are not valid.

Returns:

Type	Description
dict[str, Any]	dict[str, Any]: Dictionary representation of all optimized parameters.

optimize_structure(total_rollouts, batch_size=10, description=None, code_paths=None, code=None, code_verifier=None, code_context=None, group_id=None, verbose=True, agent_version_uuid=None, environment_uuid=None) async

Propose structural changes (i.e. changes that cannot be achieved by setting parameters alone) to improve the agent.

Parameters:

Name	Type	Description	Default
total_rollouts	int	Total number of rollouts to use for optimization.	required
batch_size	int	Number of rollouts to use in each batch. Generally, a moderate batch_size (e.g. 10-20) is required and recommended. For agents with longer execution traces: Try reducing batch_size if an error is raised.	10
description	str	Text description of the current structure/workflow/... of the agent.	None
code_paths	list[str]	A list of paths corresponding to code files containing the implementation of the agent.	None
code	str	A string representing the code (python, json, etc.) of the agent. If both code_paths and code are provided, code_paths will be ignored.	None
code_verifier	Callable[[str], tuple[bool, str]]	A function that verifies the provided code and raises errors if any issues are found. It takes a string (code) as input and returns a tuple of a boolean and a string. If the boolean is True, the code is verified successfully; otherwise, the string contains the issues identified during verification. The verifier will only be applied if code or code_paths is provided, and the optimizer will try to generate code that passes the verification.	None
code_context	str	Additional context or information about the code to assist generating code that passes the verification. This is especially useful when code_verifier is provided and when the code is in json or certain domain-specific languages/formats.	None
group_id	str	An optional group ID to associate all runs together. If not provided, a new UUID will be generated.	None
verbose	bool	If True, additional information will be printed during the optimization. Defaults to True.	True
agent_version_uuid	str	An optional agent version UUID to associate with the runs.	None
environment_uuid	str	An optional environment UUID to associate with the runs.	None

Returns:

Name	Type	Description
str	str	Suggestion for structural changes to the agent.