[Security hardening] Require explicit opt-in for workflow shell steps

[PascalThuet]

Summary

Workflow shell steps currently execute local shell commands from workflow YAML. This is a powerful and useful capability, but catalog-installed or downloaded workflows should make that execution boundary explicit.

Why

A workflow definition can contain arbitrary shell commands. Users should have a clear prompt or policy gate before a workflow with shell execution runs, especially when the workflow came from a catalog, URL, or third-party source.

Proposed direction

• Add a workflow-level permission such as requires.permissions.shell: true.
• Reject or pause before running shell steps unless the workflow declares the capability and the user opts in.
• Surface the exact command or a summarized command list before execution.
• Keep local/development workflows ergonomic, but make downloaded/catalog workflows visibly executable.

Acceptance criteria

• Workflows with type: shell require an explicit declaration or opt-in.
• Existing bundled workflows without shell steps continue to run unchanged.
• Tests cover shell steps with and without the permission declaration.
• Documentation explains that shell workflows execute local code with user privileges.

[mnriem]

This is why the community catalog is read-only and thus requires anyone to look at the code and see what it does. I would like to keep the core simple and not add this complexity to this particular step. Feel free to create your own step if you feel more comfortable with this gated approach. And yes we can then list that step in the community workflow step catalog (you'll be the first). We are in the process of making sure you can create, list, add, discovery them!

[PascalThuet]

Thanks, that makes sense from a core-simplicity perspective.

One clarification: I agree the community catalog being read-only reduces supply-chain risk for the catalog itself, but it does not create a runtime boundary once a user installs or runs a workflow. A catalog workflow using the built-in shell step can still execute local commands with the user’s privileges.

It looks like the requires.permissions.shell: true declaration is already enforced, so I’m happy to narrow this issue. The remaining concern is less “add a new gated step” and more “make shell execution visibly explicit for catalog/downloaded workflows”, e.g. documentation, a warning, or showing the shell commands before execution.

If the preferred direction is to keep runtime prompting out of core, I can reframe this as a docs/catalog metadata improvement instead.

[mnriem]

If you require a prompt you should use the combination of a gate with an if step that would route the request according to what you have specified. So yea I am fine to put stronger language around workflows in general, but I do not want to micro manage each step that is what the primitives and the extensibility model should drive

[PascalThuet]

That makes sense, thanks for clarifying the intended model.

I’ll reframe this away from core runtime prompting or per-step permission management. The better fit seems to be documenting the security expectations around workflows and recommending that workflow authors compose gate + if around shell steps when they want explicit user approval.

I still think it’s worth making shell execution more visible at the workflow/catalog level, since shell runs local commands with the user’s privileges. I can open a smaller docs/catalog PR that:

• states this clearly in the workflow docs;
• strengthens the publishing checklist for workflows using shell;
• recommends gates before destructive or sensitive shell commands;
• avoids adding special-case behavior to the core shell step.

Would that be aligned with what you have in mind?

[D7x7z49]

@PascalThuet

You looking for this WORKFLOWASLIST -> https://github.com/D7x7z49/waf

Seems our brains just had a meeting without us. I’m currently designing waf as an execution unit, and I’ve already got a demo to prove it.

https://github.com/D7x7z49/waf/blob/8f03ba1c422bd280bbb73f3c7414f679df9706ed/SYNTAX.ebnf

(* ========================================================================= *)
(* WORKFLOWASLIST FORMAL SYNTAX SPECIFICATION                                *)
(* ========================================================================= *)
(*
   REFERENCE.
      ISO/IEC 14977:1996(E) - Extended Backus-Naur Form
   NOTE.
      This EBNF specification is for documentation only.
      WorkflowAsList requires no parser. LLM reads and follows directly.

   DESIGN PRINCIPLES.
   - Concise. Information Theory. Remove redundancy, preserve meaning.
   - Practical. Set Theory. Bounded scope, minimal complete.
   - Elegant. Logical Flow. Logical coherence, consistent ordering.
   - Uniform. Text Stream. All content is text, identical syntax for same things.
*)

(* ========================================================================= *)
(* SECTION 1. TOP-LEVEL STRUCTURE                                            *)
(* ========================================================================= *)

workflow = { line } ;
line     = line_content , eol ;
eol      = "\n" | "\r\n" ;

(* ========================================================================= *)
(* SECTION 2. LINE CONTENT                                                    *)
(* ========================================================================= *)

(*
   LINE CONTENT.
   - Top-level lines are empty, comment, import, variable, or step.
   - Indented lines are only comment or step. Each indent level adds two spaces.
*)

line_content = empty_line
             | comment_line
             | import_line
             | variable_line
             | step_line
             | indented_content ;

indented_content = indent , ( comment_line | step_line ) ;
indent           = { "  " } ;

(* ========================================================================= *)
(* SECTION 3. EMPTY LINE                                                      *)
(* ========================================================================= *)

empty_line = ;

(* ========================================================================= *)
(* SECTION 4. COMMENT LINE                                                    *)
(* ========================================================================= *)

comment_line = "#" , plain_text ;

(* ========================================================================= *)
(* SECTION 5. IMPORT LINE                                                     *)
(* ========================================================================= *)

(*
   FORMAT.
      Path is local (relative or absolute) or a remote URL.
      Remote URLs start with http. or https.
      Resolution is runtime-handled.
   USAGE.
      The alias creates a namespace.
      Internal tags are accessed via alias.identifier, e.g. ${lib.tag}.
*)

import_line = ">" , "(" , identifier , ")" , path ;

path = plain_text ;

(* ========================================================================= *)
(* SECTION 6. VARIABLE LINE                                                   *)
(* ========================================================================= *)

variable_line = ":" , identifier , "=" , text ;

(* ========================================================================= *)
(* SECTION 7. STEP LINE                                                       *)
(* ========================================================================= *)

(*
   Step execution is determined by control prefix.
   Exclamation invokes shell and returns success or failure.
   Question invokes agent and returns true or false.
   Both forms produce a boolean result.

   If reference is present, the boolean result controls jump.
   True causes jump to the target tag.
   False continues to next step.

   Reference in this context resolves to a step tag only.
   It is not used as a value.
*)

step_line = "-" , [ tag ] , [ control_prefix ] , text ;

tag            = "(" , identifier , ")" ;
control_prefix = ( "@" , reference , ( "!" | "?" ) ) | ( "!" | "?" ) ;

(* ========================================================================= *)
(* SECTION 8. TEXT AND PLAIN TEXT                                             *)
(* ========================================================================= *)

text         = { text_element } ;
text_element = plain_text | substitution ;

(* ========================================================================= *)
(* SECTION 9. SUBSTITUTION                                                    *)
(* ========================================================================= *)

(*
   Substitution resolves a reference to text.
   The result is inserted into the surrounding text.

   If exclamation is present, a shell transformation is applied.
   The reference value is passed as input to the command.
   The command output replaces the substitution.

   Exclamation here does not produce a boolean result.
   It performs text transformation only.
*)

substitution = "${" , reference , [ "!" , text ] , "}" ;

(* ========================================================================= *)
(* SECTION 10. IDENTIFIERS AND REFERENCES                                     *)
(* ========================================================================= *)

(*
   Reference is use-side dotted path, used in ${ref} and @ref.
   Identifier is definition-side name, used in tag, variable, and alias.
*)

reference  = identifier , { "." , identifier } ;
identifier = letter , { letter | digit | "-" } ;

(* ========================================================================= *)
(* SECTION 11. BASIC CHARACTER CLASSES                                        *)
(* ========================================================================= *)

letter = "A" - "Z" | "a" - "z" ;
digit  = "0" - "9" ;

plain_text   = { ? regex:[^\r\n] ? } ;

(*
   Symbol is any printable ASCII character except letter, digit, and newline.
   Space is allowed.
*)

symbol = ? printable ASCII character except letter, digit, newline ? ;

(* ========================================================================= *)
(* SECTION 12. CONSTRAINTS AND RULES                                          *)
(* ========================================================================= *)

(*
   - RULE 1.  Every non-empty line starts with : or - or > after whitespace removal.
   - RULE 2.  Step syntax is - [(tag)] [@ref] [!/?] text.
   - RULE 3.  If @ref present, ! or ? must follow immediately.
   - RULE 4.  ! means shell command, ? means boolean question.
   - RULE 5.  No special prefix means plain agent message.
   - RULE 6.  Text is never quoted. It runs to end of line.
   - RULE 7.  Variable values are text. No quoting or type distinction is required.
   - RULE 8.  ! text in steps and ${ref ! text} share the same text definition.
   - RULE 9.  ? constrains agent to reply true or false.
   - RULE 10. > (alias) path imports a .workflow.list file under a namespace.
   - RULE 11. Nesting uses 2 spaces per level. Only steps and comments may indent.
   - RULE 12. Variables and imports are always top-level, no indentation.
   - RULE 13. identifier is for definition, tag, variable, alias.
   - RULE 14. reference is for use, ${ref}, @ref. It supports dot-path access.
*)

(* ========================================================================= *)
(* SECTION 13. EXAMPLES                                                       *)
(* ========================================================================= *)

(* EXAMPLE 1. Variable definition
   :retries = 0
   :greeting = hello
*)

(* EXAMPLE 2. Plain step (agent message)
   - Read project structure
*)

(* EXAMPLE 3. Step with tag
   - (analyze) Analyze source code
*)

(* EXAMPLE 4. Step with shell command
   - ! ls -al
*)

(* EXAMPLE 5. Step with boolean question
   - ? Data backed up, continue?
*)

(* EXAMPLE 6. Step with jump target and question
   - (check) @retry ? Errors fixed?
*)

(* EXAMPLE 7. Step with jump target and command
   - @init ! python setup.py
*)

(* EXAMPLE 8. Variable with substitution
   :result = ${analyze ! grep "error"}
   :count = ${retries}
*)

(* EXAMPLE 9. Import with alias
   > (utils) ./lib/utils.workflow.list
   > (remote) https://example.com/wf/shared.workflow.list
*)

(* EXAMPLE 10. Nested steps with inline substitution
   - (diff) ! git diff --cached
   - (gen) Generate commit message for. ${diff}
   - @gen ? Accept this message?
   - ! git commit -m "${gen}"
*)

(* EXAMPLE 11. Full workflow with import and dot notation
   :attempt = 0
   > (tasks) ./common/tasks.workflow.list
   - (init) Initialize environment
      - ! python setup.py
   - (run) Run main process
      - ? Ready to proceed?
      - @tasks.run ! execute.sh
   - @init ? Setup correct?
*)

(* ========================================================================= *)
(* END OF SPECIFICATION                                                       *)
(* ========================================================================= *)

[D7x7z49]

@mnriem @DouweM

I've implemented a DSL. It's still rough. But I promise if this DSL gets some adoption, it can solve the cost problem caused by low cache hit rates. This is a win-win for LLM API providers and agent users.

Often there are many mechanical, repetitive actions. But if you let agents execute them freely, they produce very different behaviors. That makes cache hit rates very low. In reality, in our development workflows, after we pick a programming language and coding paradigm, most processes are almost identical.

So if we can reach some kind of tacit consensus, we can share similar caches with each other. Actively embracing this tacit agreement brings us shared cost advantages.

[D7x7z49]

For the workflow problem, it is really about removing uncertainty from tasks. This structure naturally fits a tree, not a graph. We only need to classify the child messages inside a message list in a simple way. Then, without using the agent's own index, we can compute what context to give to the agent purely through algorithms. See more in https://github.com/D7x7z49/llm-context-idea/blob/main/idea/issue-tree.md.

And I think the list structure is very important. It is a natural message list. Its indentation, meaning multi-level lists, can express stack or function call behavior. It is a natural way to express branching in an agent's message list.

Why don't I use YAML or Markdown? I made a standalone WAF DSL because I believe you cannot give all information, steps or the whole workflow to an agent at once. You must use a half-hidden, progressive disclosure. If an agent knows too much, it gets distracted.

WAF does not use many documents. It uses steps, one action at a time. Step prompts discourage long text. For longer text, either read a file via shell, or break it into more steps. That is why WAF forbids line breaks. The same goes for command lines. For complex commands, write a script file instead of a long pipeline. Pipelines are hard to make idempotent, and hard to debug and iterate. What we chase here is reproducibility and compounding gains. This helps cache hit rates. Later, WAF-generated script files become assets, what we can call "experience", reusable between agents. The key point is prompt-as-file and script-as-file. The focus is files.

WAF files work with OS scheduled tasks to run periodic agent calls easily. Both Windows and Linux have scheduling mechanisms. On Windows, you have Task Scheduler. On Linux, you have cron or systemd timers.

[D7x7z49]

Currently, the WAF runtime I use is Pydantic AI. In fact, WAF can use any Code Agent as its runtime. The project is at demo stage now. I used Pydantic AI as the runtime for quick prototyping.

If the feature in pydantic/pydantic-ai#5266 gets implemented, I feel it will be very convenient for development.

[PascalThuet]

Thanks for sharing ! This looks like a separate topic from the workflow shell security boundary I'm raising here. Probably best discussed in a dedicated issue or in the Discussions so this thread can stay focused on @mnriem 's response

[github-actions]

This issue has been automatically marked as stale because it has not had any activity for 60 days. It will be closed in 30 days if no further activity occurs.