torchdrivesim.rendering¶

Renderers used to visualize the state of the environments. Currently three backends are supported (opencv, pytorch3d and nvdiffrast), along with a dummy renderer generating black images.

Submodules¶

Attributes¶

logger

Classes¶

`RendererConfig`	Determines behavior of the renderer.
`DummyRendererConfig`	For DummyRenderer.
`BirdviewRenderer`	A renderer producing simple 2D birdview images based on static background meshes and rectangular agents.
`DummyRenderer`	Produces a black image of the required size. Mostly used for debugging and benchmarking.
`CV2RendererConfig`	Determines behavior of the renderer.
`CV2Renderer`	Renderer based on OpenCV. Slow, but easy to install. Renders on CPU.
`Pytorch3DRendererConfig`	Configuration of pytorch3d-based renderer.
`Pytorch3DRenderer`	Renderer based on pytorch3d, using an orthographic projection and a trivial shader.
`NvdiffrastRendererConfig`	Configuration of nvdiffrast-based renderer.
`NvdiffrastRenderer`	Similar to PyTorch3DRenderer, and producing indistinguishable images, but sometimes faster.

Functions¶

renderer_from_config(→ base.BirdviewRenderer)

Construct the selected renderer from config, by default using Pytorch3DRenderer.

Package Contents¶

class torchdrivesim.rendering.RendererConfig[source]¶

Determines behavior of the renderer. Subclasses determine renderer class used.

backend: str = 'default'¶

render_agent_direction: bool = True¶

left_handed_coordinates: bool = False¶

highlight_ego_vehicle: bool = False¶

shift_mesh_by_camera_before_rendering: bool = True¶

device: str | None = None¶

class torchdrivesim.rendering.DummyRendererConfig[source]¶

Bases: RendererConfig

For DummyRenderer.

backend: str = 'dummy'¶

class torchdrivesim.rendering.BirdviewRenderer(cfg: RendererConfig, color_map: Dict[str, Tuple[int, int, int]] | None = None, rendering_levels: Dict[str, float] | None = None, res: torchdrivesim.utils.Resolution = Resolution(64, 64), fov: float = 35)[source]¶

Bases: abc.ABC

A renderer producing simple 2D birdview images based on static background meshes and rectangular agents. Currently only square resolutions are supported. The renderer always operates in batch mode, with a single batch dimension on the left.

Parameters:

cfg – configuration object, usually subclassed
color_map – a dictionary of RGB tuples in 0-255 range specifying colors of different rendered elements
res – default resolution

cfg: RendererConfig¶

res¶

scale = 0.05714285714285714¶

color_map = None¶

rendering_levels = None¶

copy()[source]¶

get_color(element_type: str) → Tuple[int, int, int][source]¶

render_frame(rgb_mesh: torchdrivesim.mesh.RGBMesh, camera_xy: torch.Tensor, camera_sc: torch.Tensor, res: torchdrivesim.utils.Resolution | None = None, fov: float | None = None)[source]¶

Renders a given rgb mesh according the camera position and rotation.

Parameters:

rgb_mesh – the given rgb mesh to render which should be already expanded to match the number of cameras.
camera_xy – BxNcx2 tensor of camera positions, by default one camera placed on each agent
camera_sc – BxNcx2 tensor of camera orientations (sine and cosine), by default matching agent orientations
res – resolution HxW of the resulting image, currently only square resolutions are supported
fov – Field of view in meters

Returns:

tensor image of float RGB values in [0,255] range with shape shape (B*Nc)xCxHxW

abstract render_rgb_mesh(mesh: torchdrivesim.mesh.RGBMesh, res: torchdrivesim.utils.Resolution, cameras: Cameras) → torch.Tensor[source]¶: Renders a given mesh, producing BxHxWxC tensor image of float RGB values in [0,255] range.

construct_cameras(xy: torch.Tensor, sc: torch.Tensor, scale: float | None = None) → Cameras[source]¶: Create PyTorch3D cameras object for given positions and orientations. Input tensor dimensions should be Bx2.

class torchdrivesim.rendering.DummyRenderer(cfg: RendererConfig, color_map: Dict[str, Tuple[int, int, int]] | None = None, rendering_levels: Dict[str, float] | None = None, res: torchdrivesim.utils.Resolution = Resolution(64, 64), fov: float = 35)[source]¶

Bases: BirdviewRenderer

Produces a black image of the required size. Mostly used for debugging and benchmarking.

render_rgb_mesh(mesh: torchdrivesim.mesh.RGBMesh, res: torchdrivesim.utils.Resolution, cameras: Cameras) → torch.Tensor[source]¶: Renders a given mesh, producing BxHxWxC tensor image of float RGB values in [0,255] range.

class torchdrivesim.rendering.CV2RendererConfig[source]¶

Bases: torchdrivesim.rendering.RendererConfig

Determines behavior of the renderer. Subclasses determine renderer class used.

backend: str = 'cv2'¶

trim_mesh_before_rendering: bool = True¶

class torchdrivesim.rendering.CV2Renderer(cfg: CV2RendererConfig, *args, **kwargs)[source]¶

Bases: torchdrivesim.rendering.BirdviewRenderer

Renderer based on OpenCV. Slow, but easy to install. Renders on CPU.

cfg: CV2RendererConfig¶

render_rgb_mesh(mesh: torchdrivesim.mesh.RGBMesh, res: torchdrivesim.utils.Resolution, cameras: torchdrivesim.rendering.base.Cameras) → torch.Tensor[source]¶: Renders a given mesh, producing BxHxWxC tensor image of float RGB values in [0,255] range.

class torchdrivesim.rendering.Pytorch3DRendererConfig[source]¶

Bases: torchdrivesim.rendering.base.RendererConfig

Configuration of pytorch3d-based renderer.

backend: str = 'pytorch3d'¶

differentiable_rendering: RenderingBlend¶

class torchdrivesim.rendering.Pytorch3DRenderer(cfg: Pytorch3DRendererConfig, *args, **kwargs)[source]¶

Bases: torchdrivesim.rendering.base.BirdviewRenderer

Renderer based on pytorch3d, using an orthographic projection and a trivial shader. Works on both GPU and CPU, but CPU is very slow.

cfg: Pytorch3DRendererConfig¶

renderer¶

render_rgb_mesh(mesh: torchdrivesim.mesh.RGBMesh, res: torchdrivesim.utils.Resolution, cameras: torchdrivesim.rendering.base.Cameras) → torch.Tensor[source]¶: Renders a given mesh, producing BxHxWxC tensor image of float RGB values in [0,255] range.

classmethod make_renderer(res, blend, background_color)[source]¶

class torchdrivesim.rendering.NvdiffrastRendererConfig[source]¶

Bases: torchdrivesim.rendering.base.RendererConfig

Configuration of nvdiffrast-based renderer.

backend: str = 'nvdiffrast'¶

antialias: bool = False¶

opengl: bool = False¶

max_minibatch_size: int | None = None¶

class torchdrivesim.rendering.NvdiffrastRenderer(cfg: NvdiffrastRendererConfig, *args, **kwargs)[source]¶

Bases: torchdrivesim.rendering.base.BirdviewRenderer

Similar to PyTorch3DRenderer, and producing indistinguishable images, but sometimes faster. Note that nvdiffrast requires separate installation and is subject to its own license terms.

cfg: NvdiffrastRendererConfig¶

glctx = None¶

render_rgb_mesh(mesh: torchdrivesim.mesh.RGBMesh, res: torchdrivesim.utils.Resolution, cameras: torchdrivesim.rendering.base.Cameras) → torch.Tensor[source]¶: Renders a given mesh, producing BxHxWxC tensor image of float RGB values in [0,255] range.

torchdrivesim.rendering.logger = None[source]¶

torchdrivesim.rendering.renderer_from_config(cfg: base.RendererConfig, *args, **kwargs) → base.BirdviewRenderer[source]¶: Construct the selected renderer from config, by default using Pytorch3DRenderer. Additional arguments are passed to the constructor.