torchdrivesim.simulator

Attributes

logger

Classes

CollisionMetric	Method used to calculate collisions between agents.
TorchDriveConfig	Top-level configuration for a TorchDriveSim simulator.
SpawnController	Handles spawning and despawning of NPCs.
NPCController	Base class for non-playable character controllers. It leaves the state unchanged on each step.
CompoundNPCController	Combines multiple NPC controllers by assigning each agent to one of the controllers.
Simulator	param road_mesh: a mesh indicating the driveable area

Module Contents

torchdrivesim.simulator.logger = None

class torchdrivesim.simulator.CollisionMetric

Bases: enum.Enum

Method used to calculate collisions between agents.

iou = 'iou'

discs = 'discs'

nograd = 'nograd'

nograd_pytorch3d = 'nograd-pytorch3d'

class torchdrivesim.simulator.TorchDriveConfig

Top-level configuration for a TorchDriveSim simulator.

renderer: torchdrivesim.rendering.RendererConfig

single_agent_rendering: bool = False

collision_metric: CollisionMetric

offroad_threshold: float = 0.5

left_handed_coordinates: bool = False

wrong_way_angle_threshold: float

lanelet_inclusion_tolerance: float = 1.0

waypoint_removal_threshold: float = 2.0

class torchdrivesim.simulator.SpawnController(exit_boundary: torch.Tensor | None = None, spawn_states: torch.Tensor | None = None, spawn_masks: torch.Tensor | None = None)

Handles spawning and despawning of NPCs. If exit_boundary is provided, NPCs will be despawned if they are outside the boundary. If spawn_states and spawn_masks are provided, NPCs will be spawned at the specified states and masks if they’re not already present.

Parameters:

• exit_boundary – Bx2xN tensor, where N is the number of vertices of the polygon.

• spawn_states – BxAxTx4 tensor, where A is the number of NPCs, T is the number of timesteps.

• spawn_masks – BxAxT boolean tensor, where A is the number of NPCs, T is the number of timesteps.

exit_boundary = None

spawn_states = None

spawn_masks = None

time = 0

spawn_despawn_npcs(simulator: Simulator) → None

to(device)

copy()

extend(n, in_place=True)

select_batch_elements(idx, in_place=True)

class torchdrivesim.simulator.NPCController(npc_size: torch.Tensor, npc_state: torch.Tensor, npc_present_mask: torch.Tensor | None = None, npc_types: torch.Tensor | None = None, agent_type_names: List[str] | None = None, spawn_controller: SpawnController | None = None)

Base class for non-playable character controllers. It leaves the state unchanged on each step.

npc_size

npc_state

npc_present_mask = None

npc_types = None

agent_type_names = None

spawn_controller = None

get_npc_state()

get_npc_size()

get_npc_types()

get_npc_present_mask()

spawn_despawn_npcs(simulator: Simulator) → None

advance_npcs(simulator: Simulator) → None

to(device)

copy()

extend(n, in_place=True)

select_batch_elements(idx, in_place=True)

class torchdrivesim.simulator.CompoundNPCController(controllers: List[NPCController], controller_indices: torch.Tensor)

Bases: NPCController

Combines multiple NPC controllers by assigning each agent to one of the controllers.

Parameters:

• controllers – List of NPCController objects

• controller_indices – BxA tensor of indices into the controllers list

controllers

controller_indices

gather_npc_states()

advance_npcs(simulator: Simulator) → None

to(device)

copy()

extend(n, in_place=True)

select_batch_elements(idx, in_place=True)

class torchdrivesim.simulator.Simulator(road_mesh: torchdrivesim.mesh.BirdviewMesh, kinematic_model: torchdrivesim.kinematic.KinematicModel, agent_size: torch.Tensor, initial_present_mask: torch.Tensor, cfg: TorchDriveConfig, renderer: torchdrivesim.rendering.BirdviewRenderer | None = None, lanelet_map: List[torchdrivesim.lanelet2.LaneletMap | None] | None = None, recenter_offset: torch.Tensor | None = None, birdview_mesh_generator: torchdrivesim.mesh.BirdviewRGBMeshGenerator | None = None, internal_time: int = 0, traffic_controls: Dict[str, torchdrivesim.traffic_controls.BaseTrafficControl] | None = None, waypoint_goals: torchdrivesim.goals.WaypointGoal | None = None, agent_types: torch.Tensor | None = None, agent_type_names: List[str] | None = None, npc_controller: NPCController | None = None, agent_lr: torch.Tensor | None = None, lane_features: torchdrivesim.lanelet2.LaneFeatures | None = None, observation_noise_model: torchdrivesim.observation_noise.ObservationNoise | None = None, action_model_extras: Dict[str, Any] | None = None)

Parameters:

• road_mesh – a mesh indicating the driveable area

• kinematic_model – determines the action space, constraints, and the initial state of all agents

• agent_size – a functor of Bx2 tensors indicating agent length and width

• initial_present_mask – a functor of BxA tensors indicating which agents are initially present and not padding

• cfg – holds various configuration options

• renderer – specify if using a non-standard renderer or static meshes beyond the road mesh (default from config)

• lanelet_map – provide the map to compute orientation losses, one map per batch element where available

• recenter_offset – if the coordinate system from lanelet_map was shifted, this value will be used to shift it back

• internal_time – initial value for step counter

• traffic_controls – applicable traffic controls by type

• waypoint_goals – waypoints for each agent

• agent_types – a tensor of BxA long tensors indicating the agent type index for each agent

• agent_type_names – a list of agent type names to index into

road_mesh

lanelet_map = None

recenter_offset = None

kinematic_model

agent_size

present_mask

action_model_extras = None

_agent_types = None

_batch_size

agent_type = None

agent_lr = None

lane_features = None

npc_controller = None

cfg: TorchDriveConfig

traffic_controls = None

waypoint_goals = None

warned_no_lanelet = False

internal_time = 0

property agent_types: List[str] | None

List of agent types used by this simulator, or None if only one type used.

property action_size: int

Defines the size of the action space for each agent type.

property batch_size: int

to(device) → typing_extensions.Self

Modifies the simulator in-place, putting all tensors on the device provided.

copy() → typing_extensions.Self

Duplicates this simulator, allowing for independent subsequent execution. The copy is relatively shallow, in that the tensors are the same objects but dictionaries referring to them are shallowly copied.

extend(n: int, in_place: bool = True) → typing_extensions.Self

Multiplies the first batch dimension by the given number. Like in pytorch3d, this is equivalent to introducing extra batch dimension on the right and then flattening.

select_batch_elements(idx, in_place=True) → typing_extensions.Self

Picks selected elements of the batch. The input is a tensor of indices into the batch dimension.

__getitem__(item: torch.Tensor) → typing_extensions.Self

Allows indexing syntax. item should be an iterable collection of indices.

property agent_count: int

How many agents of each type are there in the simulation. This counts the available slots, not taking present masks into consideration.

property npc_count: int

How many non-playable characters are there in the simulation.

validate_agent_types()

validate_tensor_shapes()

get_action_model_extras() → Dict[str, Any]

Returns any extra information to be passed to the action model

get_world_center() → torch.Tensor

Returns a Bx2 tensor with the coordinates of the map center.

get_state() → torch.Tensor

Returns a functor of BxAxSt tensors representing current agent states.

get_waypoints(count: int = 1) → torch.Tensor

Returns a functor of BxAxcount*Mx2 tensors representing current agent waypoints.

get_waypoints_state() → torch.Tensor

Returns a functor of BxAx1 tensors representing current agent waypoints state.

get_waypoints_mask(count: int = 1) → torch.Tensor

Returns a functor of BxAxcount*M boolean tensors representing current agent waypoints present mask.

compute_wrong_way() → torch.Tensor

Wrong-way metric for each agent, based on the inner product between the agent and lane direction. See torchdrivesim.infractions.lanelet_orientation_loss for details.

Returns:

a functor of BxA tensors

get_agent_size() → torch.Tensor

Returns a functor of BxAx2 tensors representing agent length and width.

get_agent_type() → torch.Tensor

Returns a functor of BxA long tensors containing agent type indexes relative to the list containing all agent types

as returned by Simulator.agent_types.

get_agent_type_names() → List[str]

Returns a list of all agent types used in the simulation.

get_agent_lr() → torch.Tensor

Returns a functor of BxA long tensors containing the rear offset

get_present_mask() → torch.Tensor

Returns a functor of BxA boolean tensors indicating which agents are currently present in the simulation.

get_noisy_state() → torch.Tensor

Returns a functor of BxAx(A+Npc)xSt tensors representing current agent states.

get_noisy_agent_size() → torch.Tensor

Returns a functor of BxAx(A+Npc)x2 tensors representing agent length and width.

get_noisy_present_mask() → torch.Tensor

Returns a functor of BxAx(A+Npc) boolean tensors indicating which agents are currently present in the simulation.

get_npc_state() → torch.Tensor

Returns a functor of BxNpcxSt tensors representing current non-playable character states.

get_npc_size() → torch.Tensor

Returns a functor of BxNpcx2 tensors representing non-playable character length and width.

get_npc_present_mask() → torch.Tensor

Returns a functor of BxNpc boolean tensors indicating which non-playable characters are currently present in the simulation.

get_npc_types() → torch.Tensor

Returns a functor of BxNpc long tensors containing non-playable character type indexes relative to the list containing all agent types

as returned by Simulator.agent_types.

get_all_agent_state() → torch.Tensor

Returns a functor of Bx(A+Npc)x4 tensors, where the last dimension contains the following information: x, y, psi, v.

get_all_agent_size() → torch.Tensor

Returns a functor of Bx(A+Npc)x2 tensors, where the last dimension contains the following information: length, width.

get_all_agent_present_mask() → torch.Tensor

Returns a functor of Bx(A+Npc) boolean tensors, indicating which agents are currently present in the simulation.

get_all_agent_type() → torch.Tensor

Returns a functor of Bx(A+Npc) long tensors, indicating the agent type index for each agent.

get_all_agents_absolute() → torch.Tensor

Returns a functor of Bx(A+Npc)x6 tensors, where the last dimension contains the following information: x, y, psi, length, width, present. Typically used to implement non-visual observation modalities.

get_noisy_all_agents_absolute() → torch.Tensor

Returns a functor of BxAx(A+Npc)x6 tensors, where the last dimension contains the following information: x, y, psi, length, width, present. Typically used to implement non-visual observation modalities.

get_all_agents_relative(exclude_self: bool = True) → torch.Tensor

Returns a functor of BxAx(A+Npc)x6 tensors, specifying for each of A agents the relative position about the other agents. ‘All’ is the number of all agents in the simulation, including hidden ones, across all agent types. If exclude_self is set, for each agent in A, that agent itself is removed from All. The final dimension has the same meaning as in get_all_agents_absolute, except now the positions and orientations are relative to the specified agent.

get_noisy_all_agents_relative(exclude_self: bool = True) → torch.Tensor

Returns a functor of BxAx(A+Npc)x6 tensors, specifying for each of A agents the relative position about the other agents. ‘All’ is the number of all agents in the simulation, including hidden ones, across all agent types. If exclude_self is set, for each agent in A, that agent itself is removed from All. The final dimension has the same meaning as in get_noisy_all_agents_absolute, except now the positions and orientations are relative to the specified agent.

get_traffic_controls() → Dict[str, torchdrivesim.traffic_controls.BaseTrafficControl]

Produces all traffic controls existing in the simulation, grouped by type.

get_noisy_lane_features() → torchdrivesim.lanelet2.LaneFeatures

get_noisy_road_mesh()

get_noisy_background_mesh() → torchdrivesim.lanelet2.LaneFeatures

get_noisy_traffic_controls() → Dict[str, torchdrivesim.traffic_controls.BaseTrafficControl]

step(agent_action: torch.Tensor) → None

Runs the simulation for one step with given agent actions. Input is a functor of BxAxAc tensors, where Ac is determined by the kinematic model.

set_state(agent_state: torch.Tensor, mask: torch.Tensor | None = None) → None

Arbitrarily set the state of the agents, without advancing the simulation. The change is effective immediately, without waiting for the next step.

Parameters:

• agent_state – a functor of BxAx4 tensors with agent states

• mask – a functor of BxA boolean tensors, deciding which agent states to update; all by default

update_present_mask(present_mask: torch.Tensor) → None

Sets the present mask of agents to the provided value.

Parameters:

present_mask – a functor of BxA boolean tensors

fit_action(future_state: torch.Tensor, current_state: torch.Tensor | None = None) → torch.Tensor

Computes an action that would (aproximately) produce the desired state.

Parameters:

• future_state – a functor of BxAx4 tensors defining the desired state

• current_state – if different from the current simulation state, in the same format as future state

Returns:

a functor of BxAxAc tensors

render(camera_xy: torch.Tensor, camera_psi: torch.Tensor, res: torchdrivesim.utils.Resolution | None = None, rendering_mask: torch.Tensor | None = None, fov: float | None = None, waypoints: torch.Tensor | None = None, waypoints_rendering_mask: torch.Tensor | None = None, custom_agent_colors: torch.Tensor | None = None, noisy_perception: bool = False) → torch.Tensor

Renders the world from bird’s eye view using cameras in given positions.

Parameters:

• camera_xy – BxNx2 tensor of x-y positions for N cameras

• camera_psi – BxNx1 tensor of orientations for N cameras

• res – desired image resolution (only square resolutions are supported; by default use value from config)

• rendering_mask – functor of BxNxAll tensors, indicating which agents should be rendered each camera

• fov – the field of view of the resulting image in meters (by default use value from config)

• waypoints – BxNxMx2 tensor of M waypoints per camera (x,y)

• waypoints_rendering_mask – BxNxM tensor of M waypoint masks per camera, indicating which waypoints should be rendered

• custom_agent_colors – BxNxAllx3 RGB tensor defining the color of each agent to each camera

Returns:

BxNxCxHxW tensor of resulting RGB images for each camera

render_egocentric(ego_rotate: bool = True, res: torchdrivesim.utils.Resolution | None = None, fov: float | None = None, visibility_matrix: torch.Tensor | None = None, custom_agent_colors: torch.Tensor | None = None, n_subsequent_waypoints: int = 1, noisy_perception: bool = False) → torch.Tensor

Renders the world using cameras placed on each agent.

Parameters:

• ego_rotate – whether to orient the cameras such that the ego agent faces up in the image

• res – desired image resolution (only square resolutions are supported; by default use value from config)

• fov – the field of view of the resulting image in meters (by default use value from config)

• visibility_matrix – a BxAxAll boolean tensor indicating which agents can see each other

• custom_agent_colors – a BxAxAllx3 RGB tensor specifying what colors agent see each other as

• n_subsequent_waypoints – the number of subsequent waypoints to render

Returns:

a functor of BxAxCxHxW tensors of resulting RGB images for each agent.

compute_offroad() → torch.Tensor

Offroad metric for each agent, defined as the distance to the road mesh. See torchdrivesim.infractions.offroad_infraction_loss for details.

Returns:

a functor of BxA tensors

compute_traffic_lights_violations() → torch.Tensor

Boolean value indicating whether each agent is committing a traffic light violation. See torchdrivesim.infractions.traffic_controls.TrafficLightControl.compute_violations for details.

Returns:

a functor of BxA tensors

_compute_collision_of_single_agent(box: torch.Tensor, remove_self_overlap: torch.Tensor | None = None, agent_types: List[str] | None = None) → torch.Tensor

Computes the collision metric for an agent specified as a bounding box. Includes collisions with all agents in the simulation, including the ones not exposed through the interface of this class. Used with discs and iou metrics.

Parameters:

• box – Bx5 tensor, with the last dimension being (x,y,length,width,psi).

• remove_self_overlap – B boolean tensor, where if the input agent is present in the simulation, set this to subtract self-overlap. By default it is assumed that self overlapping exists and will be removed.

• agent_types – An optional list of specific agent types for computing collisions with. By default all available agent types will be used.

Returns:

a tensor with a single dimension of B elements

_compute_collision_of_multi_agents(mask: torch.Tensor | None = None) → torch.Tensor

Computes the collision metric for selected (default all) agents in the simulation. Includes collisions with all agents in the simulation, including the ones not exposed through the interface of this class. Used with nograd and nograd-pytorch3d metrics.

Parameters:

mask – a functor of BxA boolean tensors, indicating for which agents to compute the loss (by default use present mask)

Returns:

a functor of BxA tensors

compute_collision(agent_types: List[str] | None = None) → torch.Tensor

Compute the collision metric for agents exposed through the interface of this class. Includes collisions with agents not exposed through the interface. Collisions are defined as overlap of agents’ bounding boxes, with details determined by the specific method chosen in the config.

Parameters:

agent_types – An optional list of specific agent types for computing collisions with. Not supported by the collision metrics nograd and nograd-pytorch3d.

Returns:

a BxA tensor