Skip to content

0.24.0 - Light mode, streaming video and data model improvements

Choose a tag to compare

View all tags
@rerun-bot rerun-bot released this 17 Jul 17:36
· 1049 commits to main since this release
0.24.0
8353026

Rerun is an easy-to-use database and visualization toolbox for multimodal and temporal data.
Try it live at https://rerun.io/viewer.

✨ Overview & highlights

💽 Tagged components

Starting with 0.22 Rerun began storing additional meta-information for each component,
identifying its field name (e.g. vertex_colors) and its archetype (e.g. Mesh3D).
We finally concluded this effort and made all parts of the Viewer and API aware of this.

While this is mostly about under-the-hood changes, this has a lot of surface level benefits:

  • You can now log the same component type multiple times on a single entity path.
    For example, logging multiple archetypes which use the Color component onto the same entity was possible before, but the colors would have overwritten each other:
rr.log("path", rr.GeoPoints(lat_lon=[some_coordinate], colors=[0xFF0000FF]))
rr.log("path", rr.Points2D(positions=[0.0, 0.0], colors=[0x0000FFFF]))
rr.log("path", rr.Mesh3D(vertex_positions=vertices, vertex_colors=[0x00FF00FF]))
  • The UI now groups by archetype everywhere and will show the same field names you use in the logging SDKs
  • Better blueprint defaults: component defaults are now per archetype field and not per type. Making them a lot more useful.
  • No more indicator components! These showed previously up in the UI and needed special handling for some advanced use cases.

🎥 Video streams

Rerun previously supported video only in the form of MP4 files,
so it was not possible to stream live encoded video to Rerun.
The new VideoStream archetype remedies that!

If you already have encoded video frames, it is just as easy as logging images now:

rr.set_time("time", duration=float(packet.pts * packet.time_base))
rr.log("video_stream", rr.VideoStream(codec=rr.VideoCodec.H264, sample=bytes(packet)))

For now, we only handle H.264, but support for more codecs is on the roadmap.
Learn more on the updated video reference page.

😎 Light mode

Rerun finally has a light mode.

The theme defaults to your OS's setting, but can be adjusted in the settings menu.

🔀 Multi sink

Previously, the SDK's recording stream could only send to one sink at a time.
We now expose the ability to set multiple sinks at once, so you can simultaneously
stream to both the Viewer and a file.

Here's what that looks like in Python (but the API is available in C++ & Rust as well!)

rr.set_sinks(
    # Connect to a local viewer using the default URL
    rr.GrpcSink(),
    # Write data to a `data.rrd` file in the current directory
    rr.FileSink("data.rrd"),
)

🤖 Built-in URDF data loader

You can now log URDF files directly to Rerun using the log_file API.

👀 Other highlights in short

  • New Cylinder archetype
  • AnyValues utility for Rust
  • rerun rrd stats for retrieving statistics about rrd files
  • rerun rrd route command to manipulate .rrd files on transport level
  • Much faster & better compaction for both rerun rrd compact and automatic in-Viewer compaction
  • Viewers started with spawn use now less memory since it no longer sets up an GRPC server for message forwarding
  • View3D's eye control type & speed can now be changed from blueprint (and as such is persisted over session!) more properties will follow in the future. (thanks to @jccampagne)

🔎 Details

🪵 Log API

  • Remove deprecated Scalar, SeriesLine, SeriesPoint archetypes #9892
  • Remove untagged Component::descriptor() #10038
  • Add Cylinders3D archetype #10139
  • Provide AnyValues helpers in Rust SDK #10074
  • Rename ComponentDescriptor fields and use colons in Sorbet metadata #10245
  • Stop logging indicators and drop them during migration #10521
  • Make SeriesPoints::markers component required #10572
  • Remove indicators from codegen, rerun_py, and rerun_cpp #10581
  • Removes remnants of indicators from dataframe queries and viewer #10584
  • rerun rrd stats #10593
  • Source default batcher settings from sink #10620

🌊 C++ API

  • (CMake) download arrow inside the rerun build folder instead of general binary folder #10141 (thanks @jzubizarreta!)
  • Fix documentation wording on which arrow-cpp version to use #10235
  • Disable gRPC server history for Spawn/CLI #10314
  • Add set_sinks to C++ RecordingStream #10359

🐍 Python API

  • Add Dataset.register_batch and wrappers for task ids #9895
  • Introduce ConnectionRegistry for centralised redap client and token management #10078
  • Build in manylinux_2_28 container #10148
  • Add APIs to Dataset to query and update the associated blueprint #10156
  • Support for seconds-since-Epoch numpy arrays for constructing TimeColumn #10168 (thanks @MichaelGrupp!)
  • Fix initializing two recordings with the same recording id causing SDK hangs #10201 (thanks @AhmedMousa-ag!)
  • Improve rerun_notebook startup times #10111
  • New dataset API for just retrieving chunk_ids associated with a query #10261
  • Multi-sink / tee Python API #10158
  • Rename to tee to set_sinks #10312
  • Add support for index=None to local and remote dataframe APIs and deprecate select_static #10332
  • Disable gRPC server history for Spawn/CLI #10314
  • Deprecate serve_web and improve documentation around how to serve a web viewer #10360
  • Add partition ID to TimeUpdateEvent #10403
  • Fix set_time_ctrl not doing anything when called twice #10547
  • Notebook auto-sizing #10554
  • Add RegisterTable rpc, LanceTable message, and related proto changes #10538
  • Add open_url/close_url notebook APIs #10602
  • AnyValue Torch Performance Improvement #10647
  • Make it easier to disable the 3D line grid in Python (blueprint) #10621
  • Add to_arrow_reader() to TableEntry and DataFusionTable #10601
  • Make it possible to configure batcher explicitly via python APIs #10657

🦀 Rust API

  • Update MSRV to 1.85 #9798
  • Improve rerun crate forwarding, removing need to depend on other re_* crates for viewer customization #9825
  • Update glam to 0.30, and replace re_math with macaw #10119
  • Add RecordingStream::set_timestamp_nanos_since_epoch #10200
  • Add set_sinks to Rust RecordingStream #10336
  • Disable gRPC server history for Spawn/CLI #10314
  • Deprecate serve_web and improve documentation around how to serve a web viewer #10360
  • Experimental connection status inspection for Rust gRPC connections #10433
  • Implement From for more narrow integers and floats for Float64 #10463

🪳 Bug fixes

  • Fix non-deterministic mesh rendering order #10117
  • LeRobot Dataloader: Mark robot type as optional #10174
  • Fix some actions happening on key up instead of key down #10232
  • Fix showing too much data for generic blobs of data (arrow ListArrays) #10275
  • Make it possible to click the barchart view to select it #10257
  • Fix integer overflow in TimeSeriesView for too large timestamps #10310
  • Fix SIGBUS startup crash on some Mac systems #10334
  • Silence harmeless "Unrecognized present mode" warning #10379
  • Fix copy and cut from notebook cells #10401
  • Fix videos re-decoding on some ui interactions #10420
  • Fix video stuttering on Firefox & Safari when encountering b-frames #10405
  • Fix pixel picking ui width changing rapidly by padding values #10481
  • Fix: don't always pick / as the origin when creating new views #10514
  • Fix tooltips sometimes jumping around on screen #10527
  • Fix spurious decode errors with H.264 on native Viewer #10539
  • Fix H.264 with b-frames not decoding last few frames on native #10545
  • Fix visible time range documentation on property inheritance #10618
  • Remove redundant ffmpeg install check #10635
  • Make rerun menu scrollable if there is not enough space #10642

🌁 Viewer improvements

  • Basic ability to show components that only differ by archetype/field name on same entity #9877
  • Visualizers' data queries take component tags into account #9887
  • Allow editing existing recording properties, like name #10003
  • Fix issues when combining InstancePoses3D with transform properties of Boxes3D/Ellipsoids3D/Capsules3D #10010
  • Replace tinystl with stl_io to support more stl files #9997
  • Introduce variant UIs in re_component_ui #10034
  • Simple built-in URDF loader #10105
  • New VideoStream archetype for loose video samples #10126
  • Enable VideoStream H264 on the web #10189
  • Ensure LeRobot Dataloader logs tagged components #10213
  • Support loading ROS paths in URDF loader #10231
  • Move the default plot legend to bottom left instead of right #10233
  • Add FillMode to Capsules3D #10238
  • Make plot axis thinner #10234
  • Add a UI Slider to change speed of eye camera in 3D Views #10085 (thanks @jccampagne!)
  • Delay showing video wait-for-decoder spinner on seeks #10404
  • Rename RecordingProperties to RecordingInfo #10411
  • Sharper icons on low-dpi screens (which is most of them) #10508
  • Add tab-bar button to hide a view #10520
  • Improve error message for FFmpeg not found #10617
  • Add Eye Controls 3D (Camera) properties to blueprint for kind and speed #10445 (thanks @jccampagne!)

🚀 Performance improvements

  • Add detailed view of latency #10173
  • Update tower http and tonic dependencies, improving LogMsg decode speed #10209
  • Free up memory of inactive recordings #10537
  • Compaction: performance & quality improvements #10594
  • rrd stats: add transport-layer only fast path #10605
  • rrd stats: parallelize decompression #10606
  • Add documentation for all things compaction #10674

🧑‍🏫 Examples

  • Fix custom view example not instantiating visualizer by default #9762
  • Improve custom visualizer example #9994
  • New example: log and animate a URDF file #10210
  • Add Python example snippet for VideoStream using av to encode numpy images #10279
  • Standalone example for compressed video streams #10297
  • Add snippet demonstrating re-muxing of VideoStream data #10614

📚 Docs

  • Fix errors in getting-started/data-in documentation #10134 (thanks @wyfo!)
  • Make Rust getting-started example more idiomatic #10137 (thanks @wyfo!)
  • Show help clicking the ? as well as by just hovering it #10160
  • Provide descriptor-aware AnyValues example in snippets #10225
  • Update Video reference manual with information about VideoStream #10423
  • Publish JS docs #10410
  • Document changes to VisualizerOverride #10486
  • Improve documentation around AnyValues and custom data #10589
  • Add juxtaposition of SeriesLines and SeriesPoints to visualizer-overrides snippet #10637
  • Micro-batching snippets #10662

🖼 UI improvements

  • Introduce a DataFusion-backed table widget #9764
  • Use DataFusionTableWidget for table entries #9869
  • Selection ui now handles different images on the same entity path #9928
  • Add a light mode theme #9953
  • Save all selected recordings #9968
  • Use archetype field name in ui instead of component name #9983
  • Add keyboard navigation to the blueprint and streams tree #9960
  • Allow selecting multiple recordings #10004
  • Remove data_based_visualizability_filter from BarChart and EncodedImage #10029
  • Add support for variant UIs in the DataFusion table widget and show a button in the partition table recording links #10035
  • Switch icons to .svg #10055
  • Improve button tooltip help text #10103
  • Group components by archetype in the streams panel #10124
  • Group components by archetype in selection panel #10140
  • Zoom X/Y axes individually, in time series and bar chart view #10159
  • Toggle maximized view with Ctrl+M #10162
  • Add option to link the X-axis of multiple time plots #10146
  • Animate the maximization of a view #10163
  • Group dataframe table by archetype and use new table design #10149
  • Add "deep inspection" selection ui for the new VideoStream archetype #10239
  • Add duration field to the recording UI #10284 (thanks @pweids!)
  • Add catalog entry table #10290
  • Highlight help buttons until the user clicks one of them #10301
  • Highlight redap server list item if active #10340
  • Allow editing connection settings for an existing redap server #10300
  • Improved menu styling #10357
  • Disambiguate component entries in selection panel #10368
  • Display the partition ID in recording selection panel #10387
  • Show average FPS of selected video #10479
  • Rename the 'Welcome screen' option to 'Rerun examples' #10503
  • Show arrow data in header tooltips and ever more data when alt is down #10526
  • Add support for table entries in the redap browser #10569
  • Add a 'copy entry url' button next to the name of remote datasets and table #10571
  • Show loading screen when starting viewer with a URL #10587

🕸️ Web

  • Support callbacks in React #10518
  • Introduce RecordingOpen viewer event #10654

🧑‍💻 Dev-experience

  • Show stderr output of external dataloaders #10087
  • Dynamically Set Active Partition URL and Select Time Window #10557
  • Log a warning when starting the viewer from inside a Docker container #10306

🗣 Refactors

  • Add descriptors to all remaining blueprint components #9908
  • Require ComponentDescriptor in ChunkBuilder::with_component_batch* methods #10026
  • Make archetype_field_name mandatory in ComponentDescriptor #10082

📦 Dependencies

🤷‍ Other

  • Add migration tool for legacy .rrd files #9816
  • Add support for remote dataset blueprints #10128
  • Use new column name in blueprint::datatypes::ComponentColumnSelector #10215
  • Support u16/u32/u64 wide Rerun enum types in data definition #10252
  • rerun rrd route command to manipulate .rrds on transport level #10607

Contributors

jccampagne
Assets 22
Loading