Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions shared/tree-sitter-extractor/src/extractor/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ impl<'a> AstNode for Node<'a> {

impl AstNode for yeast::Node {
fn kind(&self) -> &str {
yeast::Node::kind(self)
yeast::Node::kind_name(self)
}
fn is_named(&self) -> bool {
yeast::Node::is_named(self)
Expand Down Expand Up @@ -882,7 +882,6 @@ fn emit_extras_in(visitor: &mut Visitor, node: Node<'_>) {
}

fn traverse_yeast(tree: &yeast::Ast, visitor: &mut Visitor) {
use yeast::Cursor;
let mut cursor = tree.walk();
visitor.enter_node(cursor.node());
let mut recurse = true;
Expand Down
22 changes: 7 additions & 15 deletions shared/yeast-macros/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -41,22 +41,14 @@ pub fn query(input: TokenStream) -> TokenStream {
/// (kind "literal") - leaf with static content
/// (kind #{expr}) - leaf with computed content (expr.to_string())
/// (kind $fresh) - leaf with auto-generated unique name
/// {expr} - embed a Rust expression returning Id
/// {..expr} - splice an iterable of Id (in child/field position)
/// field: {..expr} - splice into a named field
/// {expr}.map(p -> tpl) - apply tpl to each element; splice result
/// {expr}.reduce_left(f -> init, acc, e -> fold)
/// - fold with per-element init; splice 0 or 1 result
/// {expr} - embed a Rust expression, dispatched via
/// the `IntoFieldIds` trait: `Id` pushes a
/// single id; iterables (`Vec<Id>`,
/// `Option<Id>`, iterator chains) splice
/// their elements
/// field: {expr} - extend a named field with `{expr}`'s ids
/// ```
///
/// Chain syntax after `{expr}` or `{..expr}`:
/// - `.map(param -> template)` — one output node per input element.
/// - `.reduce_left(first -> init, acc, elem -> fold)` — fold left; the first
/// element is converted by `init`, subsequent elements are folded by `fold`
/// with the accumulator bound to `acc`. An empty iterable yields nothing.
/// - Chains always splice (the result is iterable).
/// - Multiple chains can be chained, e.g. `.map(...).reduce_left(...)`.
///
/// Can be called with an explicit context or using the implicit context
/// from an enclosing `rule!`:
///
Expand Down Expand Up @@ -100,7 +92,7 @@ pub fn trees(input: TokenStream) -> TokenStream {
/// rule!(
/// (query_pattern field: (_) @name (kind)* @repeated (_)? @optional)
/// =>
/// (output_template field: {name} {..repeated})
/// (output_template field: {name} {repeated})
/// )
///
/// // Shorthand: captures become fields on the output node
Expand Down
213 changes: 34 additions & 179 deletions shared/yeast-macros/src/parse.rs

Large diffs are not rendered by default.

42 changes: 26 additions & 16 deletions shared/yeast/doc/yeast.md
Original file line number Diff line number Diff line change
Expand Up @@ -214,7 +214,7 @@ yeast::tree!(ctx,
```rust
yeast::trees!(ctx,
(assignment left: {tmp} right: {right})
{..body}
{body}
)
```

Expand Down Expand Up @@ -256,12 +256,26 @@ occurrences of the same `$name` within one `BuildCtx` share the same value:

### Embedded Rust expressions

`{expr}` embeds a Rust expression that returns a single node `Id`:
`{expr}` embeds a Rust expression whose value is appended to the
enclosing field (or to the rule body's id list). Dispatch happens via
the [`IntoFieldIds`] trait, which is implemented for:

- `Id` — pushes the single id.
- Any `IntoIterator<Item: Into<Id>>` — extends with all yielded ids
(covers `Vec<Id>`, `Option<Id>`, iterator chains, etc.).

So the same `{expr}` syntax handles single ids, splices, and zero-or-many
options uniformly:

```rust
(assignment
left: {some_node_id} // insert a pre-built node
right: {rhs} // insert a captured value (inside rule!)
left: {some_node_id} // a single Id
right: {rhs} // a captured value (inside rule!)
)

yeast::trees!(ctx,
(assignment left: {tmp} right: {right})
{extra_nodes} // splices a Vec<Id>
)
```

Expand All @@ -277,21 +291,17 @@ expressions (with `let` bindings) work too:
})
```

`{..expr}` splices a `Vec<Id>` (or any iterable of `Id`); the contents
are likewise a Rust block, so the splice can be the result of arbitrary
computation:
Inside `rule!`, captures are Rust variables — `{name}` works for
single, optional, and repeated captures alike:

```rust
yeast::trees!(ctx,
(assignment left: {tmp} right: {right})
{..extra_nodes} // splice a Vec<Id>
rule!(
(assignment left: @lhs right: _* @parts)
=>
(assignment left: {lhs} right: (block stmt: {parts}))
)
```

Inside `rule!`, captures are Rust variables, so `{name}` inserts a
single capture (`Id`) and `{..name}` splices a repeated capture
(`Vec<Id>`).

### Raw captures (`@@name`)

The default `@name` capture marker is *auto-translated*: in OneShot
Expand All @@ -302,15 +312,15 @@ already conforms to the output schema.
For rules that need the raw (input-schema) capture — typically to read
its source text or to translate it explicitly with mutable context
state between calls — use `@@name` instead. The body sees the original
input-schema `NodeRef`:
input-schema `Id`:

```rust
yeast::rule!(
Comment thread
tausbn marked this conversation as resolved.
(assignment left: (_) @@raw_lhs right: (_) @rhs)
=>
{
// raw_lhs is untranslated: read its original source text.
let text = ctx.ast.source_text(raw_lhs.into());
let text = ctx.ast.source_text(raw_lhs);
// rhs is already translated by the auto-translate prefix.
tree!((call
method: (identifier #{text.as_str()})
Expand Down
26 changes: 0 additions & 26 deletions shared/yeast/src/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -158,15 +158,6 @@ impl<'a, C> BuildCtx<'a, C> {
self.ast
.create_named_token_with_range(kind, generated, self.source_range)
}

/// Prepend a value to a field of an existing node.
pub fn prepend_field(&mut self, node_id: Id, field_name: &str, value_id: Id) {
let field_id = self
.ast
.field_id_for_name(field_name)
.unwrap_or_else(|| panic!("build: field '{field_name}' not found"));
self.ast.prepend_field_child(node_id, field_id, value_id);
}
}

impl<C: Clone> BuildCtx<'_, C> {
Expand All @@ -176,9 +167,6 @@ impl<C: Clone> BuildCtx<'_, C> {
/// (translation is not meaningful when input and output share a
/// schema).
///
/// Accepts any value convertible to [`Id`] (including [`crate::NodeRef`]),
/// so manual rules can pass capture bindings directly without unwrapping.
///
/// Errors if this `BuildCtx` was constructed by hand (without a
/// translator handle) — for example, in unit tests that don't go
/// through the rule driver.
Expand All @@ -189,20 +177,6 @@ impl<C: Clone> BuildCtx<'_, C> {
None => Err("translate() called on a BuildCtx without a translator handle".into()),
}
}

/// Translate an optional capture, returning the first translated id or
/// `None`. Convenience for `?`-quantifier captures (`Option<NodeRef>`).
///
/// If the underlying translation produces multiple ids for a single
/// input, only the first is returned. For most use cases (e.g.
/// translating a single type annotation) this is what you want; if
/// you need all ids, use [`translate`] directly.
pub fn translate_opt<I: Into<Id>>(&mut self, id: Option<I>) -> Result<Option<Id>, String> {
match id {
Some(id) => Ok(self.translate(id)?.into_iter().next()),
None => Ok(None),
}
}
}

impl<C> std::ops::Deref for BuildCtx<'_, C> {
Expand Down
46 changes: 9 additions & 37 deletions shared/yeast/src/captures.rs
Original file line number Diff line number Diff line change
Expand Up @@ -54,37 +54,15 @@ impl Captures {
self.captures.entry(key).or_default().push(id);
}

pub fn map_captures(&mut self, kind: &str, f: &mut impl FnMut(Id) -> Id) {
if let Some(ids) = self.captures.get_mut(kind) {
for id in ids {
*id = f(*id);
}
}
}

/// Apply a fallible function to every captured id (across all keys),
/// replacing each id with the results. A function returning an empty
/// vector removes the capture; returning multiple ids splices them
/// into the capture's value list (suitable for `*`/`+` captures).
/// Stops and returns the error on the first failure.
pub fn try_map_all_captures<E>(
&mut self,
mut f: impl FnMut(Id) -> Result<Vec<Id>, E>,
) -> Result<(), E> {
for ids in self.captures.values_mut() {
let mut new_ids = Vec::with_capacity(ids.len());
for &id in ids.iter() {
new_ids.extend(f(id)?);
}
*ids = new_ids;
}
Ok(())
}

/// Like [`try_map_all_captures`] but leaves captures whose name appears
/// in `skip` untouched. Used by the `rule!` macro to support `@@name`
/// (raw) captures alongside the default auto-translated `@name`
/// captures.
/// Apply a fallible function to every captured id, replacing each id
/// with the results. A function returning an empty vector removes
/// the capture; returning multiple ids splices them into the
/// capture's value list (suitable for `*`/`+` captures). Captures
/// whose name appears in `skip` are left untouched. Stops and
/// returns the error on the first failure.
///
/// Used by the `rule!` macro's auto-translate prefix to translate
/// every capture except those marked `@@name` (raw).
pub fn try_map_captures_except<E>(
&mut self,
skip: &[&str],
Expand All @@ -102,12 +80,6 @@ impl Captures {
}
Ok(())
}
pub fn map_captures_to(&mut self, from: &str, to: &'static str, f: &mut impl FnMut(Id) -> Id) {
if let Some(from_ids) = self.captures.get(from) {
let new_values = from_ids.iter().copied().map(f).collect();
self.captures.insert(to, new_values);
}
}

pub fn merge(&mut self, other: &Captures) {
for (key, ids) in &other.captures {
Expand Down
8 changes: 0 additions & 8 deletions shared/yeast/src/cursor.rs

This file was deleted.

19 changes: 7 additions & 12 deletions shared/yeast/src/dump.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use std::fmt::Write;

use crate::{schema::Schema, Ast, Node, NodeContent, CHILD_FIELD};
use crate::{schema::Schema, Ast, Id, Node, NodeContent, CHILD_FIELD};

/// Options for controlling AST dump output.
pub struct DumpOptions {
Expand Down Expand Up @@ -34,16 +34,11 @@ impl Default for DumpOptions {
/// method:
/// identifier "foo"
/// ```
pub fn dump_ast(ast: &Ast, root: usize, source: &str) -> String {
pub fn dump_ast(ast: &Ast, root: Id, source: &str) -> String {
dump_ast_with_options(ast, root, source, &DumpOptions::default())
}

pub fn dump_ast_with_options(
ast: &Ast,
root: usize,
source: &str,
options: &DumpOptions,
) -> String {
pub fn dump_ast_with_options(ast: &Ast, root: Id, source: &str, options: &DumpOptions) -> String {
let mut out = String::new();
dump_node(ast, root, source, options, 0, None, &mut out);
out
Expand All @@ -53,7 +48,7 @@ pub fn dump_ast_with_options(
///
/// Any node that does not match the expected type set for its parent field is
/// rendered with a trailing `" <-- ERROR: ..."` annotation on the same line.
pub fn dump_ast_with_type_errors(ast: &Ast, root: usize, source: &str, schema: &Schema) -> String {
pub fn dump_ast_with_type_errors(ast: &Ast, root: Id, source: &str, schema: &Schema) -> String {
dump_ast_with_type_errors_and_options(ast, root, source, schema, &DumpOptions::default())
}

Expand All @@ -63,7 +58,7 @@ pub fn dump_ast_with_type_errors(ast: &Ast, root: usize, source: &str, schema: &
/// rendered with a trailing `" <-- ERROR: ..."` annotation on the same line.
pub fn dump_ast_with_type_errors_and_options(
ast: &Ast,
root: usize,
root: Id,
source: &str,
schema: &Schema,
options: &DumpOptions,
Expand Down Expand Up @@ -176,7 +171,7 @@ fn expected_for_field<'a>(

fn dump_node(
ast: &Ast,
id: usize,
id: Id,
source: &str,
options: &DumpOptions,
indent: usize,
Expand Down Expand Up @@ -315,7 +310,7 @@ fn dump_node(
/// Dump a leaf node inline (no newline prefix, caller provides context).
fn dump_node_inline(
ast: &Ast,
id: usize,
id: Id,
source: &str,
options: &DumpOptions,
type_check: Option<(
Expand Down
Loading
Loading