torchdrivesim.rendering ======================= .. py:module:: torchdrivesim.rendering .. autoapi-nested-parse:: Renderers used to visualize the state of the environments. Currently three backends are supported (opencv, pytorch3d and nvdiffrast), along with a dummy renderer generating black images. Submodules ---------- .. toctree:: :maxdepth: 1 /autoapi/torchdrivesim/rendering/base/index /autoapi/torchdrivesim/rendering/cv2/index /autoapi/torchdrivesim/rendering/nvdiffrast/index /autoapi/torchdrivesim/rendering/pytorch3d/index Attributes ---------- .. autoapisummary:: torchdrivesim.rendering.logger Classes ------- .. autoapisummary:: torchdrivesim.rendering.RendererConfig torchdrivesim.rendering.DummyRendererConfig torchdrivesim.rendering.BirdviewRenderer torchdrivesim.rendering.DummyRenderer torchdrivesim.rendering.CV2RendererConfig torchdrivesim.rendering.CV2Renderer torchdrivesim.rendering.Pytorch3DRendererConfig torchdrivesim.rendering.Pytorch3DRenderer torchdrivesim.rendering.NvdiffrastRendererConfig torchdrivesim.rendering.NvdiffrastRenderer Functions --------- .. autoapisummary:: torchdrivesim.rendering.renderer_from_config Package Contents ---------------- .. py:class:: RendererConfig Determines behavior of the renderer. Subclasses determine renderer class used. .. py:attribute:: backend :type: str :value: 'default' .. py:attribute:: render_agent_direction :type: bool :value: True .. py:attribute:: left_handed_coordinates :type: bool :value: False .. py:attribute:: highlight_ego_vehicle :type: bool :value: False .. py:attribute:: shift_mesh_by_camera_before_rendering :type: bool :value: True .. py:attribute:: device :type: Optional[str] :value: None .. py:class:: DummyRendererConfig Bases: :py:obj:`RendererConfig` For DummyRenderer. .. py:attribute:: backend :type: str :value: 'dummy' .. py:class:: BirdviewRenderer(cfg: RendererConfig, color_map: Optional[Dict[str, Tuple[int, int, int]]] = None, rendering_levels: Optional[Dict[str, float]] = None, res: torchdrivesim.utils.Resolution = Resolution(64, 64), fov: float = 35) Bases: :py:obj:`abc.ABC` A renderer producing simple 2D birdview images based on static background meshes and rectangular agents. Currently only square resolutions are supported. The renderer always operates in batch mode, with a single batch dimension on the left. :param cfg: configuration object, usually subclassed :param color_map: a dictionary of RGB tuples in 0-255 range specifying colors of different rendered elements :param res: default resolution .. py:attribute:: cfg :type: RendererConfig .. py:attribute:: res .. py:attribute:: scale :value: 0.05714285714285714 .. py:attribute:: color_map :value: None .. py:attribute:: rendering_levels :value: None .. py:method:: copy() .. py:method:: get_color(element_type: str) -> Tuple[int, int, int] .. py:method:: render_frame(rgb_mesh: torchdrivesim.mesh.RGBMesh, camera_xy: torch.Tensor, camera_sc: torch.Tensor, res: Optional[torchdrivesim.utils.Resolution] = None, fov: Optional[float] = None) Renders a given rgb mesh according the camera position and rotation. :param rgb_mesh: the given rgb mesh to render which should be already expanded to match the number of cameras. :param camera_xy: BxNcx2 tensor of camera positions, by default one camera placed on each agent :param camera_sc: BxNcx2 tensor of camera orientations (sine and cosine), by default matching agent orientations :param res: resolution HxW of the resulting image, currently only square resolutions are supported :param fov: Field of view in meters :returns: tensor image of float RGB values in [0,255] range with shape shape (B*Nc)xCxHxW .. py:method:: render_rgb_mesh(mesh: torchdrivesim.mesh.RGBMesh, res: torchdrivesim.utils.Resolution, cameras: Cameras) -> torch.Tensor :abstractmethod: Renders a given mesh, producing BxHxWxC tensor image of float RGB values in [0,255] range. .. py:method:: construct_cameras(xy: torch.Tensor, sc: torch.Tensor, scale: Optional[float] = None) -> Cameras Create PyTorch3D cameras object for given positions and orientations. Input tensor dimensions should be Bx2. .. py:class:: DummyRenderer(cfg: RendererConfig, color_map: Optional[Dict[str, Tuple[int, int, int]]] = None, rendering_levels: Optional[Dict[str, float]] = None, res: torchdrivesim.utils.Resolution = Resolution(64, 64), fov: float = 35) Bases: :py:obj:`BirdviewRenderer` Produces a black image of the required size. Mostly used for debugging and benchmarking. .. py:method:: render_rgb_mesh(mesh: torchdrivesim.mesh.RGBMesh, res: torchdrivesim.utils.Resolution, cameras: Cameras) -> torch.Tensor Renders a given mesh, producing BxHxWxC tensor image of float RGB values in [0,255] range. .. py:class:: CV2RendererConfig Bases: :py:obj:`torchdrivesim.rendering.RendererConfig` Determines behavior of the renderer. Subclasses determine renderer class used. .. py:attribute:: backend :type: str :value: 'cv2' .. py:attribute:: trim_mesh_before_rendering :type: bool :value: True .. py:class:: CV2Renderer(cfg: CV2RendererConfig, *args, **kwargs) Bases: :py:obj:`torchdrivesim.rendering.BirdviewRenderer` Renderer based on OpenCV. Slow, but easy to install. Renders on CPU. .. py:attribute:: cfg :type: CV2RendererConfig .. py:method:: render_rgb_mesh(mesh: torchdrivesim.mesh.RGBMesh, res: torchdrivesim.utils.Resolution, cameras: torchdrivesim.rendering.base.Cameras) -> torch.Tensor Renders a given mesh, producing BxHxWxC tensor image of float RGB values in [0,255] range. .. py:class:: Pytorch3DRendererConfig Bases: :py:obj:`torchdrivesim.rendering.base.RendererConfig` Configuration of pytorch3d-based renderer. .. py:attribute:: backend :type: str :value: 'pytorch3d' .. py:attribute:: differentiable_rendering :type: RenderingBlend .. py:class:: Pytorch3DRenderer(cfg: Pytorch3DRendererConfig, *args, **kwargs) Bases: :py:obj:`torchdrivesim.rendering.base.BirdviewRenderer` Renderer based on pytorch3d, using an orthographic projection and a trivial shader. Works on both GPU and CPU, but CPU is very slow. .. py:attribute:: cfg :type: Pytorch3DRendererConfig .. py:attribute:: renderer .. py:method:: render_rgb_mesh(mesh: torchdrivesim.mesh.RGBMesh, res: torchdrivesim.utils.Resolution, cameras: torchdrivesim.rendering.base.Cameras) -> torch.Tensor Renders a given mesh, producing BxHxWxC tensor image of float RGB values in [0,255] range. .. py:method:: make_renderer(res, blend, background_color) :classmethod: .. py:class:: NvdiffrastRendererConfig Bases: :py:obj:`torchdrivesim.rendering.base.RendererConfig` Configuration of nvdiffrast-based renderer. .. py:attribute:: backend :type: str :value: 'nvdiffrast' .. py:attribute:: antialias :type: bool :value: False .. py:attribute:: opengl :type: bool :value: False .. py:attribute:: max_minibatch_size :type: Optional[int] :value: None .. py:class:: NvdiffrastRenderer(cfg: NvdiffrastRendererConfig, *args, **kwargs) Bases: :py:obj:`torchdrivesim.rendering.base.BirdviewRenderer` Similar to PyTorch3DRenderer, and producing indistinguishable images, but sometimes faster. Note that nvdiffrast requires separate installation and is subject to its own license terms. .. py:attribute:: cfg :type: NvdiffrastRendererConfig .. py:attribute:: glctx :value: None .. py:method:: render_rgb_mesh(mesh: torchdrivesim.mesh.RGBMesh, res: torchdrivesim.utils.Resolution, cameras: torchdrivesim.rendering.base.Cameras) -> torch.Tensor Renders a given mesh, producing BxHxWxC tensor image of float RGB values in [0,255] range. .. py:data:: logger :value: None .. py:function:: renderer_from_config(cfg: base.RendererConfig, *args, **kwargs) -> base.BirdviewRenderer Construct the selected renderer from config, by default using Pytorch3DRenderer. Additional arguments are passed to the constructor.