Skip to content

Conversation

@frankmcsherry
Copy link
Member

This PR imagines deprioritizing the VecCollection type that is a stream around a Vec container.

Specifically, we group together the methods on VecCollection in their own mod vec, and migrate the vector specific parts of reduce.rs here. The Reduce (and others) traits that produce streams with vector containers are made inherent methods on VecCollection, and are not surfaced as traits.

There are trade-offs here:

  1. On the one hand, it is now easier to use these methods because there are no extension traits to import.
  2. On the other hand, it can be harder to go from an arrangement to "a stream of whatever, vec containers I guess".

I didn't go as far as doing the same thing with join.rs, as the output type there is inherently not an arrangement, and it's not as clear how to pilot the joining of arrangements to success without insisting on a container builder. Perhaps that's right, and the right thing to do, but also wanted to share this before going too deep.

Comment on lines -209 to +210
/// .reduce(move |_key, src, dst| dst.push((*src[0].0, 1)));
/// .reduce_abelian::<_,ValBuilder<_,_,_,_>,ValSpine<_,_,_,_>>("Reduce", |_key, src, dst| dst.push((*src[0].0, 1)))
/// .as_collection(|k,v| (k.clone(), v.clone()));
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To call out, this is an example where things are more tedious than before, as there isn't an easy way to get something that defaults to 1. any trace at all, and 2. vector output. This feels like an ergonomic nit we could fix, e.g. with some method on arrangements that are vec appropriate, so that type inference would drive it.

@frankmcsherry frankmcsherry force-pushed the deprioritize_vec branch 3 times, most recently from 715d66b to 8e4213f Compare February 7, 2026 20:36
@frankmcsherry frankmcsherry merged commit 96d839b into TimelyDataflow:master Feb 8, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant