Implement IndexedParallelIterator for one-dimensional arrays#815
Conversation
Impl this trait for 1D parallel iterators. One piece is missing - opt_len - but it's mostly used for .collect() optimization. We forgo this for now, because it would require some type system delusion to conditionally call the right drive method inside Parallel<ArrayView<..>>::drive_unindexed; conditional on if D is Ix1 or not.
|
|
||
| fn into_iter(self) -> Self::IntoIter { | ||
| self.0.into_iter() | ||
| } |
There was a problem hiding this comment.
It actually reduces to the default iterator as a base case, that could be a regression in performance from currently (default iterator uses logical order - our par iters in general have benefitted from not having to guarantee a visit order)
There was a problem hiding this comment.
Visit order doesn't matter for 1D, only implementation quality (unless we count preferring forward over reverse, that is positive vs negative stride, and we don't have evidence for that).
Before, unindexed, we had the view's self.into_iter().fold(..) as the final step, using fold. In the new indexed case, we don't always access the iterator by using fold, and that's the possible quality problem.
…tors This enables the .collect() optimization in rayon. We implement the necessary call to the indexed parallel iterator driver, for one-dimensional array views.
When fold_with is used, use Zip::fold_while to fold the array view's parallel iterator. Note that in some cases, the IntoIterator of the view is used instead.
bluss
force-pushed
the
parallel-array-view-1d
branch
from
ef48aff to
21d59da
Compare
|
These implementations logically fit, but the performance of the enabled methods is not good - so not having these is a way to push users towards using |
1 participant
The rayon IndexedParallelIterator trait maps well to any one-dimensional indexed
sequence, and we can implement this trait as a special case (of the general
unindexed parallel iterator of an arbitrary array).
Implementing this is logical, it fits, but we still will recommend ndarray's
Zipfor lock step iteration. The reason is performance.
Zipwill do it better.