agent_inspect.models.tools package

Submodules

agent_inspect.models.tools.analysis_models module

class agent_inspect.models.tools.analysis_models.AnalyzedSubgoalValidation(subgoal_validation, data_sample_id, base_error, agent_run_id=None)[source]

Bases: object

Represents the error analysis result for a single subgoal validation within a data sample.

Parameters:

  • subgoal_validation (SubGoalValidationResult)

  • data_sample_id (int)

  • base_error (str | None)

  • agent_run_id (int | None)

agent_run_id: Optional[int] = None

Unique identifier for the agent run associated with this data sample.

base_error: Optional[str]

A description of the identified error in the subgoal validation.

data_sample_id: int

The unique identifier of the data sample that this result’s subgoal validation originates from.

subgoal_validation: SubGoalValidationResult

The subgoal validation result being analyzed.

class agent_inspect.models.tools.analysis_models.ErrorAnalysisDataSample(data_sample_id, subgoal_validations, agent_run_id=None)[source]

Bases: object

Represents a single data sample with its associated subgoal validations.

Parameters:
agent_run_id: Optional[int] = None

Unique identifier for the agent run associated with this data sample.

data_sample_id: int

Unique identifier for the data sample. Final results of error analysis will reference this id.

subgoal_validations: List[SubGoalValidationResult]

A list of ordered subgoal validation results to perform error analysis on.

class agent_inspect.models.tools.analysis_models.ErrorAnalysisResult(analyzed_validations_clustered_by_errors, completed_subgoal_validations)[source]

Bases: object

Represents the overall error analysis result for a set of data samples.

Parameters:
analyzed_validations_clustered_by_errors: Dict[str, List[AnalyzedSubgoalValidation]]

A mapping from generalized errors to lists of analyzed subgoal validations that exhibit those errors.

completed_subgoal_validations: List[AnalyzedSubgoalValidation]

A list of analyzed subgoal validations consisting of the subgoal validations that were marked completed, and thus have no associated errors.

class agent_inspect.models.tools.analysis_models.StatisticAnalysisResult(data_sample_id, subgoal_validations, judge_expectation=None, judge_std=None, agent_run_id=None)[source]

Bases: object

Represents the statistical analysis result for a single data sample.

Parameters:

  • data_sample_id (int)

  • subgoal_validations (List[SubGoalValidationResult])

  • judge_expectation (float | None)

  • judge_std (float | None)

  • agent_run_id (int | None)

agent_run_id: Optional[int] = None

Unique identifier for the agent run associated with this data sample.

data_sample_id: int

The unique identifier of the data sample that this statistic analysis result corresponds to.

judge_expectation: Optional[float] = None

The computed expectation (mean) of judge scores across all subgoals in one data sample.

judge_std: Optional[float] = None

The computed standard deviation of judge scores across all subgoals in one data sample.

subgoal_validations: List[SubGoalValidationResult]

A list of subgoal validation results to perform error analysis on.

Module contents

class agent_inspect.models.tools.AnalyzedSubgoalValidation(subgoal_validation, data_sample_id, base_error, agent_run_id=None)[source]

Bases: object

Represents the error analysis result for a single subgoal validation within a data sample.

Parameters:

  • subgoal_validation (SubGoalValidationResult)

  • data_sample_id (int)

  • base_error (str | None)

  • agent_run_id (int | None)

agent_run_id: Optional[int] = None

Unique identifier for the agent run associated with this data sample.

base_error: Optional[str]

A description of the identified error in the subgoal validation.

data_sample_id: int

The unique identifier of the data sample that this result’s subgoal validation originates from.

subgoal_validation: SubGoalValidationResult

The subgoal validation result being analyzed.

class agent_inspect.models.tools.ErrorAnalysisDataSample(data_sample_id, subgoal_validations, agent_run_id=None)[source]

Bases: object

Represents a single data sample with its associated subgoal validations.

Parameters:
agent_run_id: Optional[int] = None

Unique identifier for the agent run associated with this data sample.

data_sample_id: int

Unique identifier for the data sample. Final results of error analysis will reference this id.

subgoal_validations: List[SubGoalValidationResult]

A list of ordered subgoal validation results to perform error analysis on.

class agent_inspect.models.tools.ErrorAnalysisResult(analyzed_validations_clustered_by_errors, completed_subgoal_validations)[source]

Bases: object

Represents the overall error analysis result for a set of data samples.

Parameters:
analyzed_validations_clustered_by_errors: Dict[str, List[AnalyzedSubgoalValidation]]

A mapping from generalized errors to lists of analyzed subgoal validations that exhibit those errors.

completed_subgoal_validations: List[AnalyzedSubgoalValidation]

A list of analyzed subgoal validations consisting of the subgoal validations that were marked completed, and thus have no associated errors.

class agent_inspect.models.tools.StatisticAnalysisResult(data_sample_id, subgoal_validations, judge_expectation=None, judge_std=None, agent_run_id=None)[source]

Bases: object

Represents the statistical analysis result for a single data sample.

Parameters:

  • data_sample_id (int)

  • subgoal_validations (List[SubGoalValidationResult])

  • judge_expectation (float | None)

  • judge_std (float | None)

  • agent_run_id (int | None)

agent_run_id: Optional[int] = None

Unique identifier for the agent run associated with this data sample.

data_sample_id: int

The unique identifier of the data sample that this statistic analysis result corresponds to.

judge_expectation: Optional[float] = None

The computed expectation (mean) of judge scores across all subgoals in one data sample.

judge_std: Optional[float] = None

The computed standard deviation of judge scores across all subgoals in one data sample.

subgoal_validations: List[SubGoalValidationResult]

A list of subgoal validation results to perform error analysis on.