[Compiler Refactor 2] Change operator executor interface to use port index#1793
Merged
Conversation
zuozhiw
force-pushed
the
zuozhi-op-interface
branch
from
fa89952 to
428e911
Compare
zuozhiw
force-pushed
the
zuozhi-op-interface
branch
from
428e911 to
f3c77e9
Compare
zuozhiw
changed the title
zuozhiw
changed the title
Xiao-zhen-Liu
pushed a commit
that referenced
this pull request
Jan 27, 2023
…1812) This PR fixes an issue introduced in #1793 . In some scenarios, the `inputPortMapping` is not correctly passed into the data processor due to an initilaization order issue of a lazily evaluated variable. This causes the join operator to not produce results because it wrongly thinks all the input data are from the build side. This PR does a temporary hot-fix of this issue, this issue will be completely solved once #1807 is merged into master.
yangzhang75
pushed a commit
to yangzhang75/texera
that referenced
this pull request
Jun 22, 2026
In this PR, we changed the operator executor interface to use
input/output port index instead of link to other operators.
original interface:
```
def processTuple(
tuple: Either[ITuple, InputExhausted],
input: LinkIdentity,
pauseManager: PauseManager,
asyncRPCClient: AsyncRPCClient
): Iterator[(ITuple, Option[LinkIdentity])]
// LinkIdentity contains fromOperatorID and toOperatorID
```
new interafce:
```
def processTuple(
tuple: Either[ITuple, InputExhausted],
input: Int,
pauseManager: PauseManager,
asyncRPCClient: AsyncRPCClient
): Iterator[(ITuple, Option[Int])]
```
The motivation for this change is to make each operator more
independent: an operator should independently run on its own and do not
care which operator it's connected to.
Before this change, an operator is aware of the specific input and
output operators, which violates this independence principle.
After the change, an operator only cares if an input data is coming from
a specific port (for multi-input operators such as join), and the output
data is sent to a specific port (for multi-output operators such as
split)
yangzhang75
pushed a commit
to yangzhang75/texera
that referenced
this pull request
Jun 22, 2026
…pache#1812) This PR fixes an issue introduced in apache#1793 . In some scenarios, the `inputPortMapping` is not correctly passed into the data processor due to an initilaization order issue of a lazily evaluated variable. This causes the join operator to not produce results because it wrongly thinks all the input data are from the build side. This PR does a temporary hot-fix of this issue, this issue will be completely solved once apache#1807 is merged into master.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In this PR, we changed the operator executor interface to use input/output port index instead of link to other operators.
original interface:
new interafce:
The motivation for this change is to make each operator more independent: an operator should independently run on its own and do not care which operator it's connected to.
Before this change, an operator is aware of the specific input and output operators, which violates this independence principle.
After the change, an operator only cares if an input data is coming from a specific port (for multi-input operators such as join), and the output data is sent to a specific port (for multi-output operators such as split)