Architecture LinuxCNC-block-diagram - update

[zz912]

I tried to update the block diagram of the LinuxCNC architecture. I would like the image to include both the ideal state and the current unwanted state that has arisen.

I am currently observing that some problems are being solved at the frontend level, instead of being solved at the background level.

When I started with LinuxCNC, I was missing a display where the HAL is. So I drew it.

If anyone knows how other parts of LCNC should work that would be good to draw. Feel free to send them to me drawn with a pencil on paper and photographed with a mobile phone. I can then draw them in the image.

I will be happy if there is a discussion about my proposal.

[andypugh]

I am surprised to see spindles in EMCIO. I am pretty sure those are in EMCMOT.
In 2,9 there is src/emc/iotask/iocontrol.cc but that is gone in master (and doesn't do very much at all in 2.9)
764655e

So the whole EMCIO block is probably not there any more.

[zz912]

I am interested only aboat 2.10 version. I made quick modification:

If I understand correctly, here are the HAL pins of the TASK:

linuxcnc/src/emc/task/taskclass.cc

Lines 46 to 70 in 764655e

	struct iocontrol_str {
	hal_bit_t *user_enable_out; /* output, TRUE when EMC wants stop */
	hal_bit_t *emc_enable_in; /* input, TRUE on any external stop */
	hal_bit_t *user_request_enable; /* output, used to reset ENABLE latch */
	hal_bit_t *coolant_mist; /* coolant mist output pin */
	hal_bit_t *coolant_flood; /* coolant flood output pin */
	hal_bit_t *lube; /* lube output pin */
	hal_bit_t *lube_level; /* lube level input pin */
	// the following pins are needed for toolchanging
	//tool-prepare
	hal_bit_t *tool_prepare; /* output, pin that notifies HAL it needs to prepare a tool */
	hal_s32_t *tool_prep_pocket;/* output, pin that holds the pocketno for the tool table entry matching the tool to be prepared,
	only valid when tool-prepare=TRUE */
	hal_s32_t *tool_from_pocket;/* output, pin indicating pocket current load tool retrieved from*/
	hal_s32_t *tool_prep_index; /* output, pin for internal index (idx) of prepped tool above */
	hal_s32_t *tool_prep_number;/* output, pin that holds the tool number to be prepared, only valid when tool-prepare=TRUE */
	hal_s32_t *tool_number; /* output, pin that holds the tool number currently in the spindle */
	hal_bit_t *tool_prepared; /* input, pin that notifies that the tool has been prepared */
	//tool-change
	hal_bit_t *tool_change; /* output, notifies a tool-change should happen (emc should be in the tool-change position) */
	hal_bit_t *tool_changed; /* input, notifies tool has been changed */
	// note: spindle control has been moved to motion
	} * iocontrol_data; //pointer to the HAL-struct

[andypugh]

Yes, though the lube pins don't exist any more. (They didn't have any useful behaviour)

The tool-change pins are missing from the diagram. (change / changed and number for both tools and pockets). They always have been despite being the main point of emcio.

[zz912]

@c-morley disagrees with me that HALUI should not be associated with GUI

#3580 (comment)

HAL and NML are for two different uses and are not really interchangeable. NML and ZMQ are similar and could be interchanged.

So yes if NML was easier to use, I might have used it instead of ZMQ for this pr.

The reasons for having the GUI communicate to HALUI:

• It's the defacto standard way to connect HAL to a control panel.
• it does not preclude using sinething else, where as if you put it in task you are quite stuck with what we decide.
• it's small/simple enough I can see it through
• it should not break configuration that use the current way, at least for now.
• there is information that task/motion don't know, such as jog rate. Currently if you use HALUI and a GUI then there are two jog rates.

I understand the functional reasons, but I would be interested in others' opinions from the perspective of preserving the architecture.

[rmu75]

If we are going to replace this diagram I suggest to

• make it more correct than the current version and
• use a diagram generation tool like graphviz or mermaid or something the current documentation generation pipeline understands already (ar there any? can somebody in the know comment?) maybe draw.io?
• check in sources of diagram and svg / png formats

[c-morley]

HALUI should not really be in this view.
HALUI and the GUI act the same in this overview.

[zz912]

If we are going to replace this diagram I suggest to

• make it more correct than the current version and

• use a diagram generation tool like graphviz or mermaid or something the current documentation generation pipeline understands already (ar there any? can somebody in the know comment?) maybe draw.io?

• check in sources of diagram and svg / png formats

Do you have a model of what you would like the LCNC architecture to look like? Do you know of any open-source projects with architecture?

Do you prefer graphviz or mermaid?

I have no experience with software architecture, but I am convinced that we need it. At least do it in reverse.

[zz912]

HALUI should not really be in this view. HALUI and the GUI act the same in this overview.

HALUI is part of LCNC. It is an inseparable part of Gmoccapy.

We can discuss:

• where should it be in the architecture?
• how should it communicate with TASK?
• whether and how should it communicate with GUI?
• whether the GUI should also have its own HAL layer?

I am interested in both states. How it is and how it should ideally be.

I want to update this picture mainly because of HALUI. HALUI is a specific component and therefore I would like to know its current function and whether we want to continue using it this way.

The abbreviation HALUI is "HAL User Interface", but at the moment HALUI serves more as an engine for Gmoccapy.

If I removed HALUI from the architecture, I would de facto stick my head in the sand like an ostrich.

Maybe no one will want to discuss it.
Maybe the discussion will not change anything.
Or maybe it will move us forward.

[rmu75]

I didn't find the source of the architecture diagram, I guess at some point in time it originated in xfig. Some translations also translated this diagram and even colorized it. With a vector format, we could do better and make that consistent.

I played around a bit with draw.io, will attach a draft later.

[c-morley]

This diagram is to show the basic architecture of linuxcnc. It is not to show a typical setup. HALUI is equivalent to the GUI and so just adds detail that is not needed.

I would suggest making another diagram that shows a typical setup maybe including Mesa card/ spindle control/ ethernet servo? In this diagram one might show task/motion as just one box called motion controller.

If Gmoccapy requires HALUI to be useful then IMHO a undisearable limiting choice has been made. One should be able to use a different program the HALUI such as panelui or a custom program.

[andypugh]

HALUI is part of LCNC.

It is.

It is an inseparable part of Gmoccapy.

It really shouldn't be....
I was going to argue that GUIs shouldn't have HAL pins at all, but then realised that Touchy, at least, couldn't work without them as it relies on hardware buttons and a jogwheel. Similarly Gmoccapy needs HAL pins for the hardware buttons.

I am curious what Gmoccapy relies on in HALUI that isn't available to it from the Python interface? Maybe this is a throwback to the fact that Mocca was written in Pascal ( https://forum.linuxcnc.org/41-guis/1813-new-gui-for-emc-available-for-testing?limit=6&start=0#1813 ) and we have no LinuxCNC Pascal interface.

[Sigma1912]

@andypugh

There are ideas to use HALUI pins to partially replace native GUI pins.

This discussion might shed some light on the topic:
#3504

[zz912]

Similarly Gmoccapy needs HAL pins for the hardware buttons.

I agree that some of the HAL pins created in Gmoccapy make sense, for example this:

linuxcnc/src/emc/usr_intf/gmoccapy/gmoccapy.py

Lines 6188 to 6197 in 781023e

	def _make_hal_pins(self):
	# generate the horizontal button pins
	for h_button in range(0, 10):
	pin = self.halcomp.newpin("h-button.button-{0}".format(h_button), hal.HAL_BIT, hal.HAL_IN)
	hal_glib.GPin(pin).connect("value_changed", self._button_pin_changed)
	# generate the vertical button pins
	for v_button in range(0, 7):
	pin = self.halcomp.newpin("v-button.button-{0}".format(v_button), hal.HAL_BIT, hal.HAL_IN)
	hal_glib.GPin(pin).connect("value_changed", self._button_pin_changed)

But creating pins like these seems weird to me:

linuxcnc/src/emc/usr_intf/gmoccapy/gmoccapy.py

Lines 6271 to 6276 in 781023e

	# make pins to react to tool_offset changes
	pin = self.halcomp.newpin("tooloffset-x", hal.HAL_FLOAT, hal.HAL_IN)
	hal_glib.GPin(pin).connect("value_changed", self._offset_changed, "tooloffset-x")
	pin = self.halcomp.newpin("tooloffset-z", hal.HAL_FLOAT, hal.HAL_IN)
	hal_glib.GPin(pin).connect("value_changed", self._offset_changed, "tooloffset-z")
	self.halcomp.newpin("tool-diameter", hal.HAL_FLOAT, hal.HAL_OUT)

For the other pins created by Gmoccapy, I can't determine whether they are there just because of Python interface shortcomings, or whether they have a purpose.

I am curious what Gmoccapy relies on in HALUI that isn't available to it from the Python interface?

I dont know what is available to it from the Python interface.

Here are halui pins in Gmoccapy:

linuxcnc/src/emc/usr_intf/gmoccapy/gmoccapy.py

Lines 3797 to 3812 in 781023e

	def _update_halui_pin(self):
	if self.spindle_override != self.stat.spindle[0]['override']:
	self.initialized = False
	self.widgets.spc_spindle.set_value(self.stat.spindle[0]['override'] * 100)
	self.spindle_override = self.stat.spindle[0]['override']
	self.initialized = True
	if self.feed_override != self.stat.feedrate:
	self.initialized = False
	self.widgets.spc_feed.set_value(self.stat.feedrate * 100)
	self.feed_override = self.stat.feedrate
	self.initialized = True
	if self.rapidrate != self.stat.rapidrate:
	self.initialized = False
	self.widgets.spc_rapid.set_value(self.stat.rapidrate * 100)
	self.rapidrate = self.stat.rapidrate
	self.initialized = True

I don't know if this is about using hal signals in the glade file. But they are called hal:

linuxcnc/src/emc/usr_intf/gmoccapy/gmoccapy.glade

Lines 180 to 202 in 781023e

	<object class="EMC_Action_Open" id="hal_action_open"/>
	<object class="EMC_Action_Reload" id="hal_action_reload"/>
	<object class="EMC_Action_Step" id="hal_action_step"/>
	<object class="EMC_Action_Stop" id="hal_action_stop"/>
	<object class="EMC_Stat" id="hal_status">
	<signal name="all-homed" handler="on_hal_status_all_homed" swapped="no"/>
	<signal name="hard-limits-tripped" handler="on_hal_status_limits_tripped" swapped="no"/>
	<signal name="interp-idle" handler="on_hal_status_interp_idle" swapped="no"/>
	<signal name="interp-run" handler="on_hal_status_interp_run" swapped="no"/>
	<signal name="line-changed" handler="on_hal_status_line_changed" swapped="no"/>
	<signal name="metric-mode-changed" handler="on_hal_status_metric_mode_changed" swapped="no"/>
	<signal name="mode-auto" handler="on_hal_status_mode_auto" swapped="no"/>
	<signal name="mode-manual" handler="on_hal_status_mode_manual" swapped="no"/>
	<signal name="mode-mdi" handler="on_hal_status_mode_mdi" swapped="no"/>
	<signal name="motion-mode-changed" handler="on_hal_status_motion_mode_changed" swapped="no"/>
	<signal name="not-all-homed" handler="on_hal_status_not_all_homed" swapped="no"/>
	<signal name="override-limits-changed" handler="on_hal_status_override_limits_changed" swapped="no"/>
	<signal name="state-estop" handler="on_hal_status_state_estop" swapped="no"/>
	<signal name="state-estop-reset" handler="on_hal_status_state_estop_reset" swapped="no"/>
	<signal name="state-off" handler="on_hal_status_state_off" swapped="no"/>
	<signal name="state-on" handler="on_hal_status_state_on" swapped="no"/>
	<signal name="tool-in-spindle-changed" handler="on_hal_status_tool_in_spindle_changed" swapped="no"/>
	</object>

[zz912]

It is an inseparable part of Gmoccapy.

I guess I'm wrong. Gmoccapy uses a function named "def _update_halui_pin(self):" , but it doesn't use halui pins.

Edit:
Definitely I am wrong. Gmoccapy can be loaded without HALUI.

[rmu75]

linuxcnc architecture test.drawio

[zz912]

To rmu75:
Thank you. I will explore drawio as soon as I can. I have currently dedicated more to LCNC than I can and I have a time deficit.

[satiowadahc]

My two cents, while draw.io has been around for a while mermaid or LaTeX might be preferable so it can be maintained with the source code.

[zz912]

I redrawn the diagram in drawio.

linuxcnc.architecture.drawio
LCNC_Architecture_C3 drawio.svg

[andypugh]

I don't think that there is a PID in Motion, the ones I know of are in HAL components. (there is a simple one in stepgen as well as the the standalone PID)

[andypugh]

I don't think that there is a PID in Motion,

(Possibly spindle-synch motion)

[c-morley]

Again, I believe you are adding too much information for what this diagram is for.
This diagram is in code notes, not integrator notes, nor user notes.
Keep it simple as it was and then build a different purpose built diagram for interrogators or users or what ever your target audience is.
I understand the info you are adding is helpful to interrogators but then you need to tailor it to interrogators.
eg. Interrogators do not need detail about NML or task, developers do. Users only need to know HAL and INI. etc

[c-morley]

I don't think that there is a PID in Motion,

(Possibly spindle-synch motion)

spindle sync does not use PID

[c-morley]

I must say though - the drawings do look much nicer!

[rmu75]

That really starts to look very nice -- thanks a lot!. Some comments:

• Task and motion don't talk NML with each other, it is more of a "single producer single consumer" queue between non-rt and rt domains. Status is communicated back via some structure IIRC with no explicit locking, will check that. All NML stuff is strictly in the non-realtime domain.

• PID is in hal

• I think it wouldn't hurt if you could draw an additional rectangle that delimits or groups the interface components (like mesa, gpio, lpt port) from the external stuff like buttons / contactors / ...

• There is a typo in "bus convertor".

[zz912]

Now I'm at the stage where I redrawn the image in drawio and added HALUI there. I consider adding HALUI very important because many Pull Requests are created that do not respect the original concept of HALUI and GUI. I myself was confused about this, thinking that HALUI was part of Gmoccapy. There may be a change, for example, HALUI may be canceled, or HALUI and GUI may communicate with each other, or ...... But now at least we have the current state.

I've done the simplest thing. The difficult part is just beginning.

Chris:

Again, I believe you are adding too much information for what this diagram is for.

Give me a chance to convince you that the amount of information I want to put in this diagram is right. I understand that the more information I put in this diagram, the more work I will have to maintain it. On the other hand, I believe that the maintenance of this diagram outweighs its benefits. I expect that developers will contact me and I will modify the diagram in the future. If I stop enjoying it, it will be easy to simplify the diagram.

Chris:

This diagram is in code notes, not integrator notes, nor user notes.

I agree. I made extra diagram for integrators and begginers. #3738

rmu75:

Task and motion don't talk NML with each other, it is more of a "single producer single consumer" queue between non-rt and rt domains. Status is communicated back via some structure IIRC with no explicit locking, will check that. All NML stuff is strictly in the non-realtime domain.

I would be happy if you could help me with this.

rmu75:

PID is in hal

I thought so, but I didn't solve it during the phase of redrawing the diagram into drawio.

rmu75:

I think it wouldn't hurt if you could draw an additional rectangle that delimits or groups the interface components (like mesa, gpio, lpt port) from the external stuff like buttons / contactors / ...

My goal is to make a diagram where the heart of LCNC is the TASK module and the interface is the HAL layer. This diagram should be intended mainly for developers. Moreover, the boundary from TASK to HAL is always the same, or similar. The boundary from HAL to hardware is unique for each integration. I do not want to draw the boundary from HAL to hardware here, or only very simplified as in the upper part of the figure (non-realtime hardware). I want to delete the lower part of the figure "Example of Hardware". I think that most LCNC applications do not use an encoder on the axis.

If someone wants to draw a diagram for integrators with specific hardware, I can do it, but it would be another diagram tied to this core.

[rmu75]

Task and motion don't talk NML with each other, it is more of a "single producer single consumer" queue between non-rt and rt domains. Status is communicated back via some structure IIRC with no explicit locking, will check that. All NML stuff is strictly in the non-realtime domain.

I would be happy if you could help me with this.

just remove "NML" from the arrows or replace with "FIFO" (first in first out).

I think it wouldn't hurt if you could draw an additional rectangle that delimits or groups the interface components (like mesa, gpio, lpt port) from the external stuff like buttons / contactors / ...

My goal is to make a diagram where the heart of LCNC is the TASK module and the interface is the HAL layer. This diagram should be intended mainly for developers. Moreover, the boundary from TASK to HAL is always the same, or similar. The boundary from HAL to hardware is unique for each integration. I do not want to draw the boundary from HAL to hardware here, or only very simplified as in the upper part of the figure (non-realtime hardware). I want to delete the lower part of the figure "Example of Hardware". I think that most LCNC applications do not use an encoder on the axis.

If someone wants to draw a diagram for integrators with specific hardware, I can do it, but it would be another diagram tied to this core.

don't overthink this, i just meant another rectangle just like the "JOINT 0" group around "encoder counter", "D/A converter" (notice also typo there), I would call that "INTERFACE". and perhaps add "field I/O" and connect the switches box.

I would have done that myself but the .drawio you attached seems to be my initial draft?

[smoe]

I just came across Mermaid https://github.com/mermaid-js/mermaid as a graph description library that reportedly can also be integrated with Asciidoc. This would ease quite some of our problems with whatever we need as an external tool to edit the current graphs. Please kindly check this out.

[zz912]

I'm sorry I interrupted work on this PR. I have some family problems. So I don't know when I'll be able to return to LCNC. I'm sorry I left some unfinished work here, but family is more important. Feel free to close my PR or continue on them.

[rmu75]

I just came across Mermaid https://github.com/mermaid-js/mermaid as a graph description library that reportedly can also be integrated with Asciidoc. This would ease quite some of our problems with whatever we need as an external tool to edit the current graphs. Please kindly check this out.

I played around with mermaid and found it was not a good fit for this type of diagram.

Where is the problem with SVGs that are generated by drawio? This doesn't need to be automated, that diagram will not be updated on a regular basis, and a manual step to generate SVG to include in asciidoc would be acceptable IMO.

draw.io can be used online and on desktop and is free (apache license), so changing a drawio diagram is a nuisance at worst. cf visio, illustrator or other proprietary stuff, those would be a real hurdle.

[smoe]

I'm sorry I interrupted work on this PR. I have some family problems. So I don't know when I'll be able to return to LCNC. I'm sorry I left some unfinished work here, but family is more important. Feel free to close my PR or continue on them.

Don't worry. We likely finish this somehow. It will just take a bit longer. Best wishes!

[zz912]

My last version, if somebody want continue.
LCNC_Architecture_C3.drawio

[grandixximo]

@smoe what do we need for this to be accepted? are we ok with draw.io ? ZMQ is not in yet, waiting for that before updating docs? do you want a PR for a fix without ZMQ to reflect current status?

[zz912]

For quick acceptance it is not a problem to delete the bridge module and the "zmq" arrows. Then the upper part of the "NON-REALTIME" diagram should be ok.

The MOTION area is taken from the original image. It does not contain AXIS + JOINTS. I have no idea how outdated it is.

[smoe]

@smoe what do we need for this to be accepted? are we ok with draw.io ? ZMQ is not in yet, waiting for that before updating docs? do you want a PR for a fix without ZMQ to reflect current status?

I have no say in this. I am happy for anything that is better than the current content. Just make sure both the drawio and the svg are in the right place.

[grandixximo]

Ok, sounds great, Robert told me in Discord/IRC he will do it.
quote from rs

regarding the diagram, I will integrate that, it needs figuring out some things like path, name of file and what other stuff needs changing

@smoe you should join us there sometime ;-)

[Zia-research]

Apologies for coming at this sideways — I didn't know this PR existed and
posted on the forum first.

Briefly: I'm new to LinuxCNC, coming from other CNC and PLC systems, and
while learning it I redrew the Code Notes block diagram against current
master. I used Claude extensively to do it. Precisely because of that, the
rule throughout was that nothing is claimed without being read in the
source, every statement carries a file:line citation, and a script
re-checks all 111 of them. LINUXCNC-FINDINGS.md records the method and
every result in full — including its own corrections — so anyone who wants
to can judge both for themselves. Once the findings were settled I asked it
to write the fixes as well: three small documentation patches against
master, applying cleanly. They sit in the repo, not submitted anywhere.

It landed on the same conclusions several of you reached here — EMCIO gone,
spindle in motion, PID in HAL, joints rather than axes — and on a number of
others besides.

The diagram itself, as published and corrected side by side:
https://zia-research.github.io/linuxcnc-audit/sheets/linuxcnc-code-notes-errata.html

What would help me most: could someone who knows this code look at it and
tell me whether the work is any good? If it holds up, I can point to the
rest of what it turned up, and you can decide whether any of it is useful
to you.

Everything else — findings, patches, verifier:
https://github.com/Zia-research/linuxcnc-audit

PS: I also used AI to help me with english :-)

[grandixximo]

@Zia-research I went through the three patches in upstream/ against master and checked the claims at the cited locations: the buffer types in cms_cfg.cc, the OVERRIDE_LIMITS mask in command.c, the 76/73 command count, the ENABLE and STEP rejections, and the G33/G33.1/G64/G96 interpreter checks. Everything I verified was exact. The lcec.0.activate example in the HAL manual was my error, thanks for catching it.

The patches look good to me. Please send them as a PR and we will have a look there. One detail for the review: in the PAUSE paragraph, "(and jerk)" only applies to the cruckig planner; the default tpmod planner is trapezoidal.

On the diagram: Sheet B of your errata sheet (the corrected diagram, second one on the page) is the most accurate rendering of the architecture diagram I have seen, and plain handwritten SVG carries no tooling dependency, which fits this codebase well. drawio has seen some friction here (single-line unformatted SVG output, out-of-tree web editor). @rmu75 @smoe, you have been steering the diagram question in this issue: what do you think of extracting Sheet B as a standalone .svg to replace LinuxCNC-block-diagram-small.png, versus continuing with the drawio version?

[zz912]

Hi @Zia-research,

First of all, thanks for taking the time to work on this. When I opened this PR, I thought updating the architecture diagram would be a relatively small task. As I dug deeper into the LinuxCNC architecture and the discussions around it, I realized it is a much larger effort than I originally expected, and I don't currently have enough time to see it through.

Since you're now working in the same area, I wonder if it would make sense to close this PR and let you open a new one based on your own work. I think that would keep the review process cleaner and make it easier to follow the discussion.

Please feel free to reuse anything from this PR that you find useful—diagrams, ideas, comments, or anything else. I'd be happy if any of it helps move the documentation forward.

Thanks again for picking this up, and I look forward to seeing where you take it.

[Zia-research]

Disclaimer. This text, the findings behind it and the diagrams are AI-generated —
directed, reviewed and published by me. I said so when I first posted here; I repeat it
because what follows includes corrections to my own earlier claims. Every statement
carries a file:line citation and a script re-reads each one against the source. That is
the only guarantee I can offer, and the correction below shows it is not sufficient on
its own.

---

@grandixximo @zz912 — thank you both.

The jerk point. You were right, and more right than I first credited: "the default
tpmod planner is trapezoidal" is the mechanism. My first correction had the jerk path
being attempted, ruckig refusing, and the planner reverting — that is wrong. The path is
never entered: [TRAJ]PLANNER_TYPE defaults to 0 (emccfg.h:57, initraj.cc:157), is
forced back to 0 when jerk is below 1.0 (initraj.cc:159-162), and tp.c takes the
S-curve branch only for type 1 (tp.c:3660,3664). Patch corrected and pushed. One
refinement to your phrasing: cruckig is not a second planner — it sits inside tpmod
(tp.c:43, :2795), so it is a setting, not a different module.

This is the second time you have caught the same shape of error: the citations held up
both times, the reasoning around them did not.

@zz912 — thank you. Let me be clear about scope, so I don't promise more than I can
deliver. What I have ready is the three documentation patches — text corrections,
reviewed above. I will open those as their own PR; they stand alone and don't depend on
what happens here. Whether to close this PR is your call and the maintainers': the
diagram effort is larger than what I can take on.

Sheet B. @grandixximo put that question to @rmu75 and @smoe, and it is theirs to
answer. If it is useful, I am glad to contribute the SVG under GPL-2.0 as a donation with
no strings — take it, adapt it, or drop it.

What I cannot honestly offer is to maintain it. I am not a LinuxCNC developer, and I
cannot myself judge whether a finding is right — the correction above came from your
review, not from mine. Keeping an architecture diagram correct as the code moves needs
someone able to make that judgement. Better said now than discovered when the figure has
gone stale under my name.

Three things found while checking the thread.

1. This PR edits LinuxCNC-block-diagram.png, but code-notes.adoc:103 renders
   LinuxCNC-block-diagram-small.png, and nothing in the repository references the
   non--small file. As committed, it changes nothing a reader sees.

2. @rmu75 asked whether the docs pipeline already understands a diagram tool. It does —
   Graphviz, docs/src/Submakefile:1328,1331 — and code-notes.adoc already renders
   three .dot-derived SVGs (lines 221, 225, 752); the .dot files are tracked and the
   .svg are not, so they are genuinely built. That answers "is there something
   already", not "which should we use": Documentation - add diagram "Architecture - Context diagram" #3781 set a merged drawio precedent.

3. @rmu75, on the arrows — your first description, "single producer single consumer
   queue", matches the code. The "FIFO" shorthand from the follow-up would not:
   motion_struct.h:21 holds a single emcmot_command_t, not an array, under
   command_mutex, with a commandNum→commandNumEcho handshake and a 1 s timeout
   (usrmotintf.cc:98-120). And on the status direction you said you would check: no
   mutex, but not unsynchronised either — a seqlock, head/tail at
   motion.h:581,666, written around the controller body at control.c:245,275.

Note: GitHub Pages has been failing to deploy on their side today, so the rendered
sheets may lag a few hours behind the repository files. The files themselves are current.

Next: I will open a PR with the three patches, re-checked against master when I do.

[Zia-research]

Hello everyone,

I started this trip into LinuxCNC for my own understanding, before building a test machine. I ended up posting here with a large case of impostor syndrome — hence my repeated mentions of AI. This is my first time contributing to a project and my first time using GitHub at all, so please tell me if I am going about any of this the wrong way.

I have learned a great deal. I also spent two weeks of holiday, full time, on LinuxCNC.

I let the AI guide me through opening a repository, and through opening #4349 after @grandixximo's invitation to send the patches as a PR — I am still surprised I did that. I still owe @grandixximo a reply on the review — both of the errors he found should be corrected, if the AI is right :-) The corrections exist in my own repository, but the PR still shows the old text, because I don't know whether to force-push the branch — which I am told marks his comments as outdated — or to add another commit on top. And should I answer eight review comments inline, or in one message? I would rather be told than guess and make more work for the reviewer.

@zz912 — you asked back in August whether it would make sense to close this PR and let me open a new one based on my own work, and I never really answered. Honestly: I don't know, and I am not sure it is mine to decide. I don't really know what to do with any of this next, mostly because I understand too little about how things work here. What I can say is that I took you up on the other half of your message — the context diagram below is redrawn from #3781, and your work is the base of it. If a new PR is what you and the maintainers want, I will open one; I would just rather not open it on my own judgement.

I kept going in the meantime. This figure mixes two images, and is inspired by #3781: that one I redrew my own way, and merged it with the rebuilt version of the diagram in the docs.

The three pieces, if you want to look:

• the system overview — the figure itself:
  https://zia-research.github.io/linuxcnc-audit/sheets/linuxcnc-system-overview.svg
  and its editable source:
  https://github.com/Zia-research/linuxcnc-audit/blob/main/sheets/linuxcnc-system-overview.drawio
• the context diagram, redrawn from Documentation - add diagram "Architecture - Context diagram" #3781 (this one has no drawio yet):
  https://zia-research.github.io/linuxcnc-audit/sheets/linuxcnc-context-diagram.html
• the findings the figures are built on:
  https://github.com/Zia-research/linuxcnc-audit/blob/main/LINUXCNC-FINDINGS.md
• (the rest of it: https://zia-research.github.io/linuxcnc-audit/)

The working format is a .drawio file.

What I would like to ask is what I should do with all of this.

I hope the content of the system overview is correct — could you please check it? It is still a work in progress, there are typos etc. And even if it is correct, I am not happy with it: I find the overall understanding weak. Too much information? Maybe fine for someone who already knows the system? A different presentation? I don't know.

Can you check whether the picture is true? Or at least not that false!

I would like to keep improving this image. It is the thing I missed when I wanted to dig deeper.

I hope I will get some feedback and suggestions. Thank you in advance.

[grandixximo]

You are not doing this wrong. Two weeks of holiday spent reading our source is more than most people give, and it is appreciated.

Practical bits first.

Force-push or commit on top. Force-push is fine. My comments going outdated is correct, the source changed, so I read it again. That is how it works. If you or the AI mess it up we will tell you.

Eight inline replies or one message. Inline, one short reply per thread, so each resolves on its own. One summary comment at the end only if something needs saying across all of them.

#3718 vs a new PR. Open a fresh one if you think you can stick with it to the end, or say so and give someone else permission to take it over and carry it. Both are fine, what does not work is a PR that sits there waiting on someone who is not coming back. Nobody needs permission to open a PR here, it is not a commitment to anything else, it is just where the review happens.

Is the picture true. As things stand today, it looks sensible. But that is the problem with a structure diagram of master: master moves. Your halmrt box was right three days ago, Bertho has changed it since. Some of what is coming sits in open PRs, some only exists in maintainers' heads as future restructuring. Nobody has the full picture; the project is many small pieces from many people and it is dynamic. So the diagram is a snapshot by nature. Fine as a draft to be maintained alongside the source, but someone has to keep doing it, and someone with an overall understanding of everything, which we don't really have. It is a community effort. 2.9 is static, a diagram of 2.9 would stay correct.

Maintaining it. You said you cannot commit to that. That is fine and honest. Say the word and someone will take it over. Don't be too scared either: we mostly know what we are doing, something unacceptable will not be merged.

Presentation. c-morley already said it earlier in this thread and I agree: one diagram per audience. The code notes one is for developers, users and integrators need a different and much simpler one. Your figure is trying to be both, that is why it reads heavy. Pick one reader and cut everything that reader does not need.

drawio. I don't know that it is the tool we want. There has been some pushback and some acceptance.

On the AI side: cut the walls of text. Read what the AI writes, understand it yourself, then shorten it. A short disclaimer at top or bottom is enough, you don't need to repeat it every post. It is fine to contribute, fine to arrive wanting to learn new things. What is not fine is sending text for other people to read that you have not read and understood yourself; a translation of it in your case. We are respectful of other people's time and we appreciate the time you are dedicating here, but the respect goes both ways. Some contributors do a touch and go, some stay, how much you invest is up to you. You will waste some of our time along the way, everyone does, don't beat yourself up. We all learn a thing or two with time, no shame in that.

[grandixximo]

Complaining about walls of texts, I built one myself, sorry...
TL;DR
Welcome to Open Souce, it's a mess, we learn, we respect, we deal with it...

[zz912]

Regarding draw.io: I would not consider #3781 a precedent for using draw.io.

I chose draw.io simply because it was the first diagramming tool I tried that worked reasonably well for me. I didn't want to spend my limited time evaluating and comparing different diagramming tools — my goal was to work on the architecture itself, not on choosing the perfect drawing software.

So I wouldn't read anything more into my choice of draw.io than that.

---

A little bit of context from my side:

My role in this project is basically that of an unofficial tester. The maintainers here are friendly and quite open to different opinions. They respect the fact that I sometimes have different views — and also that I sometimes get things wrong. 🙂

One advantage of using AI is that you can ask it to make your responses polite and non-confrontational. I had a few faux pas in the past when I relied on an ordinary translator, so I appreciate that aspect of AI. In fact, I am using AI to write this message to you as well. 🙂

I also consider this PR partly my own mistake. I overestimated how much time and effort I could afford to put into it. Nobody here blames me for that, but I try to keep the number of PRs I start and then cannot finish to a minimum. The difficult part is that it is not always possible to estimate in advance how complicated a problem will turn out to be.

In my opinion, it is usually better to split larger subjects into several smaller PRs or Issues, so that each one remains reasonably short and easy to follow. It is quite common here for an Issue to branch into several related Issues when that makes the discussion clearer.

I think the same approach could work here.

You cannot simply take over and close this PR yourself, but once you create a new PR, you can mention at the beginning that it is a continuation of #3718. Then we can add a note here saying that the discussion and further work have moved to your PR, and I can close this one. Or, alternatively, one of the maintainers with the necessary permissions can do it.

So I don't see closing #3718 as a problem. I think it would actually make things clearer once there is a new PR that represents the work you want to continue.

As for the architecture diagram itself, I don't see a problem with you offering to create it without committing to maintain it indefinitely. Ultimately, it is up to the maintainers to decide whether they want to merge it or not.

On the other hand, I would try to do the work in a way that makes it as well documented as possible, so that someone else can take it over later with the minimum amount of effort.

I think that is a reasonable approach for community-maintained documentation. You don't necessarily have to be the person who maintains it forever — but making the work easy for the next person to understand and continue is very valuable.

---

One more suggestion regarding your context diagram. https://zia-research.github.io/linuxcnc-audit/sheets/linuxcnc-context-diagram.html

I think it would be better to make this an official LinuxCNC Issue or PR rather than keeping the discussion only on your website. And please mention @zz912 in that Issue or PR. That would generate a GitHub notification/email for me, so I would know that you are asking for my input.

Would you please do that?

If the current material was produced with AI, I suspect this should be quite easy to turn into an Issue or PR — AI can probably do the conversion for you without much trouble. 🙂

In any case, I would suggest making the Issue or PR only about the context diagram, and not mixing it with the other architecture diagrams or broader architectural questions. That would make it much easier to review and discuss.

I think your own website is a good place for publishing your own views and documenting your research, especially when those views may not necessarily represent the consensus of the LinuxCNC community.

I have my own website as well https://zz912.webnode.cz/, where I describe my personal experience with LinuxCNC. For example, I recommend the combination of an OptiPlex 9010 + Gmoccapy + Mesa 7i96S. That is simply the setup I have personally tested and found to work well. For a beginner, that can be useful information about one possible way to get started. But if there are 100 people in the LinuxCNC community, there will probably be 101 opinions about the best hardware/software combination. 🙂

I see your website in a similar way: it is a very good place to document your research, experiments, and conclusions. But if you want the LinuxCNC project itself to consider something for inclusion in the documentation, I think the official Issue/PR is the right place for that discussion.

So in this particular case, I would really appreciate it if you could create an official Issue or PR specifically for the context diagram and mention @zz912 there. That way I can take a proper look at it and give you my feedback.

---

Thanks for taking the time to go through my work and for proposing improvements. I appreciate it.

Regarding the three actors, though, my intention was actually almost the opposite of yours. The Operator, Integrator and Developer are the most important parts of my context diagram. I wanted to show the LinuxCNC ecosystem from the point of view of these three roles and how they interact with the different parts of LinuxCNC.

So I don't think removing the actors would be an improvement to my diagram. It would turn it into a different kind of diagram, which can certainly be useful — but it would answer a different question.

The current LinuxCNC documentation itself describes this figure as a "Context diagram" and says that it presents the components and players of the LinuxCNC ecosystem and how they interact. It also explicitly says that it is not intended to explain LinuxCNC functionality. That was exactly the direction I was trying to follow.

There are, however, a couple of points in your review where I think you are absolutely right.

"halcmd / halshow — attach to HAL, not to the core"

I agree with you here. That is an error in my diagram. If you want to prepare a PR fixing this, please do so. I would be happy to see that correction made.

Regarding:

"Embedded tabs and panels have no link to HAL"

I agree that there is currently no such direct link that should be shown in the diagram. I deliberately did not draw one.

Part of the reason is that C-morley is currently working on ZMQ communication, so I think we should wait and see what becomes the preferred approach for creating and communicating with embedded panels in the future.

There was also a practical reason for my decision. When I was a beginner, I initially had the impression that HAL was somehow the universal solution for connecting everything to everything in LinuxCNC. I don't think that is a particularly helpful impression to give to beginners.

HAL is a realtime part of the system, whereas embedded panels are a user-interface mechanism. Personally, I don't see putting a realtime layer between the UI and everything else as something we should encourage unless there is a real need for it.

But I don't think this discussion belongs in a C1 context diagram anyway. The diagram should show the relevant relationships at a useful level of abstraction, rather than prescribe which communication mechanism should be used for every possible implementation.

So I think your observations are useful, but I would separate the two things:

• correcting an objectively wrong connection, such as halcmd / halshow → HAL, is definitely worthwhile;
• deciding whether an Embedded Panel should communicate through HAL, ZMQ, or something else is an architectural design question that I don't think should be encoded into the C1 context diagram.

In short, thank you for the proposed changes. I think some of them are valuable, but for the context diagram I would keep the three actors. Removing them would make the diagram clearer as an internal technical map, but it would also remove the main purpose for which I created this particular diagram.

[zz912]

I am closing #3718 because the further work on the architecture documentation is now being continued in #4377.

Thanks @Zia-research for taking this forward.

[Zia-research]

Thanks @zz912 — closing is fine by me.

One precision for the record: #4377 replaces the Code Notes block diagram (LinuxCNC-block-diagram-small.png). It does not carry the context diagram, the three actors, or the halcmd / halshow correction you accepted.

I will open a separate one for the context diagram in the course of next week, and mention you there as you asked.