Panics when using enter_region instead of enter

[Kixiron]

I've tried to make a minimal reproduction of this, but I can't seem to. It's happened multiple times at multiple places throughout this particular project too, so it's not an isolated thing for me at least.

When entering a named region, if I use .enter_region() instead of .enter() here a panic within ddflow's arrangement code occurs

Panic Backtrace

thread 'timely:work-0' panicked at 'assertion failed: input.frontier().frontier().iter().all(|t1|\n
                  input_frontier.iter().any(|t2:\n
&G::Timestamp|\n                                                                         t2.less_equal(t1)))', C:\Users\Chase\.cargo\git\checkouts\differential-dataflow-d065d23d797aa027\2b12f29\src\operators\arrange\arrangement.rs:594:21
stack backtrace:
   0: std::panicking::begin_panic_handler
             at /rustc/a2f8f6281817d430e20726128b739d3c6708561c\/library\std\src\panicking.rs:493
   1: core::panicking::panic_fmt
             at /rustc/a2f8f6281817d430e20726128b739d3c6708561c\/library\core\src\panicking.rs:92
   2: core::panicking::panic
             at /rustc/a2f8f6281817d430e20726128b739d3c6708561c\/library\core\src\panicking.rs:50
   3: differential_dataflow::operators::arrange::arrangement::{{impl}}::arrange_core::{{closure}}::{{closure}}<timely::dataflow::scopes::child::Child<timely::dataflow::scopes::child::Child<timely::worker::Worker<timely_communication::allocator::generic::Generic
             at C:\Users\Chase\.cargo\git\checkouts\differential-dataflow-d065d23d797aa027\2b12f29\src\operators\arrange\arrangement.rs:594
   4: timely::dataflow::operators::generic::operator::{{impl}}::unary_frontier::{{closure}}::{{closure}}<timely::dataflow::scopes::child::Child<timely::dataflow::scopes::child::Child<timely::worker::Worker<timely_communication::allocator::generic::Generic>, usi
             at C:\Users\Chase\.cargo\git\checkouts\timely-dataflow-4c0cc365061cd263\1cbfc99\timely\src\dataflow\operators\generic\operator.rs:352
   5: timely::dataflow::operators::generic::builder_rc::{{impl}}::build::{{closure}}::{{closure}}<timely::dataflow::scopes::child::Child<timely::dataflow::scopes::child::Child<timely::worker::Worker<timely_communication::allocator::generic::Generic>, usize>, us
             at C:\Users\Chase\.cargo\git\checkouts\timely-dataflow-4c0cc365061cd263\1cbfc99\timely\src\dataflow\operators\generic\builder_rc.rs:123
   6: timely::dataflow::operators::generic::builder_rc::{{impl}}::build_reschedule::{{closure}}<timely::dataflow::scopes::child::Child<timely::dataflow::scopes::child::Child<timely::worker::Worker<timely_communication::allocator::generic::Generic>, usize>, usiz
             at C:\Users\Chase\.cargo\git\checkouts\timely-dataflow-4c0cc365061cd263\1cbfc99\timely\src\dataflow\operators\generic\builder_rc.rs:163
   7: timely::dataflow::operators::generic::builder_raw::{{impl}}::schedule<usize,closure-0>
             at C:\Users\Chase\.cargo\git\checkouts\timely-dataflow-4c0cc365061cd263\1cbfc99\timely\src\dataflow\operators\generic\builder_raw.rs:203
   8: timely::progress::subgraph::PerOperatorState<usize>::schedule<usize>
             at C:\Users\Chase\.cargo\git\checkouts\timely-dataflow-4c0cc365061cd263\1cbfc99\timely\src\progress\subgraph.rs:655
   9: timely::progress::subgraph::Subgraph<usize, usize>::activate_child<usize,usize>
             at C:\Users\Chase\.cargo\git\checkouts\timely-dataflow-4c0cc365061cd263\1cbfc99\timely\src\progress\subgraph.rs:338
  10: timely::progress::subgraph::{{impl}}::schedule<usize,usize>
             at C:\Users\Chase\.cargo\git\checkouts\timely-dataflow-4c0cc365061cd263\1cbfc99\timely\src\progress\subgraph.rs:304
  11: timely::progress::subgraph::PerOperatorState<usize>::schedule<usize>
             at C:\Users\Chase\.cargo\git\checkouts\timely-dataflow-4c0cc365061cd263\1cbfc99\timely\src\progress\subgraph.rs:655
  12: timely::progress::subgraph::Subgraph<tuple<>, usize>::activate_child<tuple<>,usize>
             at C:\Users\Chase\.cargo\git\checkouts\timely-dataflow-4c0cc365061cd263\1cbfc99\timely\src\progress\subgraph.rs:338
  13: timely::progress::subgraph::{{impl}}::schedule<tuple<>,usize>
             at C:\Users\Chase\.cargo\git\checkouts\timely-dataflow-4c0cc365061cd263\1cbfc99\timely\src\progress\subgraph.rs:304
  14: timely::worker::{{impl}}::step::{{closure}}
             at C:\Users\Chase\.cargo\git\checkouts\timely-dataflow-4c0cc365061cd263\1cbfc99\timely\src\worker.rs:693
  15: core::option::Option<mut alloc::boxed::Box<Schedule, alloc::alloc::Global>*>::map<mut alloc::boxed::Box<Schedule, alloc::alloc::Global>*,bool,closure-0>
             at C:\Users\Chase\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\option.rs:487
  16: timely::worker::Wrapper::step
             at C:\Users\Chase\.cargo\git\checkouts\timely-dataflow-4c0cc365061cd263\1cbfc99\timely\src\worker.rs:693
  17: timely::worker::Worker<timely_communication::allocator::generic::Generic>::step_or_park<timely_communication::allocator::generic::Generic>
             at C:\Users\Chase\.cargo\git\checkouts\timely-dataflow-4c0cc365061cd263\1cbfc99\timely\src\worker.rs:375
  18: timely::execute::execute::{{closure}}<tuple<>,closure-0>
             at C:\Users\Chase\.cargo\git\checkouts\timely-dataflow-4c0cc365061cd263\1cbfc99\timely\src\execute.rs:285
  19: timely_communication::initialize::initialize_from::{{closure}}<timely_communication::allocator::generic::GenericBuilder,tuple<>,closure-1>
             at C:\Users\Chase\.cargo\git\checkouts\timely-dataflow-4c0cc365061cd263\1cbfc99\communication\src\initialize.rs:285
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
thread 'tests::optimization_test' panicked at 'Worker panic: Any', C:\Users\Chase\.cargo\git\checkouts\timely-dataflow-4c0cc365061cd263\1cbfc99\communication\src\initialize.rs:323:26
stack backtrace:
   0: std::panicking::begin_panic_handler
             at /rustc/a2f8f6281817d430e20726128b739d3c6708561c\/library\std\src\panicking.rs:493
   1: core::panicking::panic_fmt
             at /rustc/a2f8f6281817d430e20726128b739d3c6708561c\/library\core\src\panicking.rs:92
   2: core::option::expect_none_failed
             at /rustc/a2f8f6281817d430e20726128b739d3c6708561c\/library\core\src\option.rs:1300
   3: core::result::Result<tuple<>, alloc::boxed::Box<Any, alloc::alloc::Global>>::expect<tuple<>,alloc::boxed::Box<Any, alloc::alloc::Global>>
             at C:\Users\Chase\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\result.rs:997
   4: timely_communication::initialize::{{impl}}::drop<tuple<>>
             at C:\Users\Chase\.cargo\git\checkouts\timely-dataflow-4c0cc365061cd263\1cbfc99\communication\src\initialize.rs:323
   5: core::ptr::drop_in_place<timely_communication::initialize::WorkerGuards<tuple<>>>
             at C:\Users\Chase\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\ptr\mod.rs:179
   6: sruth::tests::optimization_test
             at .\src\lib.rs:495
   7: sruth::tests::optimization_test::{{closure}}
             at .\src\lib.rs:52
   8: core::ops::function::FnOnce::call_once<closure-0,tuple<>>
             at C:\Users\Chase\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\ops\function.rs:227
   9: core::ops::function::FnOnce::call_once
             at /rustc/a2f8f6281817d430e20726128b739d3c6708561c\library\core\src\ops\function.rs:227
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Changing the .enter_region() back to an .enter() fixes the problem immediately, so I'm not sure why it happens.
.cleanup() is called here and to reproduce the panic all you need to do is change the .enter() to an .enter_region() and run cargo test.

Currently I'm using timely@1cbfc9 and differential-dataflow@2b12f2 on Windows 10 v2004 using rustc version 1.51.0-nightly (a2f8f6281 2021-01-27)

[frankmcsherry]

Thanks for the report! enter_region is pretty simple, so .. I'm not clear what the problem could be right away. I'll check out what I think is boilerplate in enter and see if any of it was actually critical for enter_region.

[frankmcsherry]

I didn't find anything wrong with enter_region, but try out patching this in and see if it improves things for you: TimelyDataflow/timely-dataflow#360. It fixes up a frontier reporting bug that could have caused a frontier regression that was hidden by enter. Specifically, entry introduces a map operator which would be enough to avoid the issue fixed above, but enter_region would not introduce that operator and suffer.

[frankmcsherry]

It has landed in main, so testing it should be even easier now!

[Kixiron]

It looks like it's working, thanks!