### 5.1 Tokenize
-Raw input becomes an array of typed tokens via a single-pass lexer (`Lexer.tokenize`, `StringScanner`-based). Three token types:
+Raw input becomes an array of typed tokens via a single-pass lexer (`Lexer.tokenize`, `StringScanner`-based). Four token types:
| Type | Carries | Example |
|---|---|---|
| `word` | text, lowercased later | `delete`, `then` |
| `literal` | exact original text **and** a kind | `path("~/Old.log")`, `string("My Notes.txt")` |
+| `command` | text, lowercased later; a word that also names an executable found on `$PATH` | `ls`, `git` |
| `punct` | a single character | `.` `,` `;` `?` |
+`command` is a subtype of `word` (`token.word?` is true for both, so classifier/verb matching still sees it) but never a `:literal`, so it can't fill a rule's `:slot` — a command name used as an argument stays a plain word, not a slot value.
+
**Literal kinds**, tried in this priority order at each token boundary: `string` (quoted), `url`, `env` (`$HOME`, `${Path}`), `glob` (`*.tmp`, `{a,b}`), `path` (`~/Old.log`, `./src`), `flag` (`-r`, `--force`), `number` (`3.5`, `500ms`, `50%` — tried before `file` so decimals aren't read as filenames), `file` (`notes.txt`, `.bashrc`).
Literals are never embedded back into a string with markers — they're carried as token data, so nothing downstream can accidentally lowercase, split, or "correct" a filename, and no restore/unmasking step is needed. `token.key` returns the word text or the literal's **kind** (not its value), so `delete old.log` and `delete report.pdf` normalize to the identical match string `delete file` for Tier 1/2 comparison, while `token.text` on a literal still holds the real value for execution.
-Known false-positive guards: `e.g.`/`i.e.` are kept as single `word` tokens (not misread as a `.g`/`.e` file with a sentence break); a bare `?` is never a glob character, so it always ends a question.
+Known false-positive guards: a bare `?` is never a glob character, so it always ends a question. (The `e.g.`/`i.e.` abbreviation guard described in the original source listing below is **not present** in the current `nlshell.rb` implementation — a gap to revisit.)
Full source (drop-in, no dependencies beyond Ruby's `strscan` stdlib):
`Lexer.sentences(tokens)` splits on `punct` tokens matching `. ? ! ;` (terminator stays attached to the sentence it ends). This is safe because periods inside literals (`old.log`, `3.5`, `e.g.`) never reached the token stream as bare punctuation — they were consumed by the literal/abbreviation patterns in Stage 5.1.
-### 5.5 Split into clauses **[A]**
+### 5.5 Split into clauses
+
+Implemented as `Lexer.clauses`, driven by the ruleset rather than hardcoded:
-Not yet implemented in code; policy to build against:
+- Splits a sentence into clauses on connective token sequences declared via `connectives([...])` (builtins: `["after", "that"]`, `["and", "then"]`, `["then"]`, `["next"]`).
+- A bare `and` **without** a preceding comma is ambiguous ("copy a and b to c" is one command). The code only splits on it when the token right after `and` (or `, and`) is in the ruleset's `verb_patterns` set (a known verb word) **or** is a `:command` token (a recognized `$PATH` executable, e.g. "delete a.txt and ls") — not "post-canonicalization" since Tier 2 canonicalization doesn't exist yet; otherwise it's treated as a list argument within one clause.
+- Each resulting clause is classified independently (§7 node types) via `Classifier.classify`.
+
+**[A] still open, not yet implemented:** the REPL currently classifies and executes each clause as it's reached, rather than resolving the whole Utterance first. The plan-then-execute policy below remains a v-next item:
-- Split a sentence into clauses on connective words: `then`, `and then`, `after that`, `next`, or `,` immediately followed by `and`.
-- A bare `and` **without** a preceding comma is ambiguous ("copy a and b to c" is one command). Only split on it when the token immediately after `and` matches a known action word post-canonicalization (§6.4 step 3); otherwise treat it as a list argument within one clause.
-- Each resulting clause is classified independently (§7 node types: action, chain, loop, conditional, query, teach) and enters the tiered pipeline (§6.1) separately.
- For a multi-clause Utterance: resolve every clause first (may hit Tier 3 per clause), assemble the full plan, show it once, then execute. Query and confirmed-teach clauses may run immediately since they don't mutate state; action clauses wait for the full plan to validate. If any clause fails to resolve, execute nothing.
### 5.6 Fragments and the context register **[A]**
---
-## 9. Worked Example Rule Set (Ruby DSL, starter vocabulary)
+## 9. Rule Set DSL
+
+### 9.1 Current implementation
+
+`Ruleset.create(name) { ... }` (see `nlshell.rb`). Rules are grouped per-tool rather than declared as standalone `rule(...) { action ... }` calls as originally sketched (§9.2): each `tool` block holds its own `rule` pattern(s), an optional `verb_synonyms([...])` list, a `permission` (`:yolo` runs immediately, `:ask` confirms with the user first, and is now the default — a tool must opt into `:yolo`), and a single `action` block whose keyword args must match every rule's `:slot` names exactly (`ToolBuilder#build` raises otherwise). `synonym`/`chain`/`query` DSL calls from the original sketch don't exist yet — only `tool`/`rule`/`verb_synonyms`/`permission`/`action`, plus top-level `types`/`connectives`/`classifiers` for lexer/classifier configuration. There's no separate `verbs()` declaration: a word is a verb (and recognized for `and <verb>` chain-splitting) only once a `tool` rule or taught rule actually starts with it — no pre-declaring verbs ahead of a real rule.
+
+`verb_synonyms` expands each declared `rule` once per synonym, swapping only the leading word (`verb_synonyms %w[remove]` on `rule "delete :target"` also registers `"remove :target"`) — a convenience for tools with several verbs that all take identical slots, without repeating the full pattern per verb.
+
+`teach`/`remember` aren't a `classifiers()` kind — they're ordinary builtin `tool`s (§6.3 is superseded: teaching is just an action, not a separate node type). `teach "<trigger>" means "<target>"` registers a new rule pointing at whatever tool the target phrase already resolves to; `remember "<term>" means "<canonical>"` writes into a runtime `Synonyms` store for future Tier 2 use. `teach` and `verb_synonyms` share one mechanism, `Ruleset.register_rule(rules, pattern, tool, reference:)`: it compares `Ruleset.slot_names(pattern)` against the reference pattern's and only writes `pattern -> tool` into the rules store on a match (warning on shadowing either way) — so a taught trigger can't bind a tool whose action expects different slots, same as an author-declared synonym can't.
+
+```ruby
+Ruleset.create("builtins") do
+ types(string: /.../, path: /.../, ...) # literal patterns, feeds Lexer.configure
+ connectives [["after", "that"], ["and", "then"], ["then"], ["next"]]
+ classifiers(conditional: [["if"]], loop: [["for", "each"]], query: [["what"]])
+
+ tool("delete") do
+ permission :ask
+ rule "delete :target"
+ verb_synonyms %w[remove]
+ action { |target:| puts "Deleting #{target}" }
+ end
+end
+```
+
+### 9.2 Original sketch **[A]** — superseded by §9.1
+
+Kept for historical context on the target vocabulary breadth; `synonym`/`chain`/`query` and per-action `rule` calls shown here don't match the current DSL.
```ruby
RuleSet.define do
Everything under `lib/` past `lexer.rb` is unimplemented; this document is the spec for building it.
+**Current state:** none of the above split has happened yet — Lexer, Fuzzy, Classifier, Tier 1, the `Ruleset` DSL (§9.1), builtin tools, and the REPL loop all live in a single top-level `nlshell.rb` prototype. Implemented so far: tokenize/validate/downcase/sentence-split/clause-split (§5.1-5.5), classification, Tier 1 exact/fuzzy-word matching, and a REPL that dispatches matched Tier 1 actions with `:ask`/`:yolo` confirmation. Still unbuilt: Tier 2, Tier 3 client, query resolver, teach flow, SQLite stores (§8), and the context register (§5.6).
+
---
## 13. Deployment / First Run **[A]**
# Scan the environment for possible shell commands
ShellCommands = Set.new
-ENV.fetch("PATH", "").split(File::PATH_SEPARATOR).reverse.each do |dir|
+ENV.fetch("PATH", "").split(File::PATH_SEPARATOR).each do |dir|
next unless File.directory?(dir)
- Dir.each_child(dir) do |name|
- path = File.join(dir, name)
- ShellCommands << name if File.file?(path) && File.executable?(path)
+
+ begin
+ Dir.each_child(dir) do |name|
+ path = File.join(dir, name)
+ ShellCommands << name if File.file?(path) && File.executable?(path)
+ end
+ rescue Errno::EACCES
+ next # unreadable PATH entry, skip it
end
end
# text: the word/punct as typed, or the literal's exact value
# kind: literal kind (:path, :string, ...) or nil
Token = Struct.new(:type, :text, :kind) do
- # :command matches as a word (it's just a word that shadows a PATH entry);
- # it still isn't a :literal, so it can't fill a rule's :slot.
+ # :command matches as a word (it's just a word that shadows a PATH entry).
+ # It isn't a :literal, but like any non-punct token it can still fill a
+ # rule's :slot (Tier1.match only excludes punctuation there).
def word? = type == :word || type == :command
+ def command? = type == :command
def literal? = type == :literal
def punct? = type == :punct
length = 1
elsif t.text == "," && n&.text == "and" && tokens[i + 2]&.word? && verbs.include?(tokens[i + 2].text)
length = 2
+ elsif t.text == "and" && n&.command?
+ length = 1
+ elsif t.text == "," && n&.text == "and" && tokens[i + 2]&.command?
+ length = 2
end
# return the length
# vocabulary - so there's no ambiguity to resolve, and slot values (which
# never go through this) can't be touched.
module Fuzzy
- MAX_DISTANCE = 2
+ # Short words ("is", "who") have too few possible typos to fuzz at all;
+ # medium words get one typo's worth of slack. Scaled by the expected word
+ # (b), not the input, so tolerance reflects the known vocabulary word.
+ SHORT_WORD_MAX_LENGTH = 3
+ MEDIUM_WORD_MAX_LENGTH = 6
def self.close?(a, b)
- a == b || distance(a, b) <= MAX_DISTANCE
+ a == b || distance(a, b) <= max_distance(b)
+ end
+
+ def self.max_distance(word)
+ # Exact match only below SHORT_WORD_MAX_LENGTH, else scale with length.
+ return 0 if word.length <= SHORT_WORD_MAX_LENGTH
+ return 1 if word.length <= MEDIUM_WORD_MAX_LENGTH
+
+ 2
end
# Levenshtein edit distance.
end
end
-# Stage: CLS. Tags a clause as teach/conditional/loop/query/action by
-# matching its leading words against configured phrases, checked in the
-# configured order (first match wins).
+# Stage: CLS. Tags a clause as conditional/loop/query/action by matching its
+# leading words against configured phrases, checked in the configured order
+# (first match wins). teach/remember aren't a kind - they're builtin actions.
module Classifier
class << self
attr_accessor :patterns
end
end
+# Tier 2 canonicalization store (SS6.4 step 3): term -> canonical word. Tier 2
+# itself isn't built yet; "remember" populates this for when it is.
+module Synonyms
+ class << self
+ attr_accessor :terms
+ end
+
+ self.terms = {}
+
+ def self.canonical(word)
+ terms.fetch(word, word)
+ end
+end
+
+# Stage: EX. Runs a resolved tool's action, isolating the REPL from any
+# exception an action raises so one bad command can't crash the shell.
+module Executor
+ def self.call(tool, args)
+ tool.action.call(**args.transform_values(&:text))
+ true
+ rescue StandardError => e
+ puts " error running #{tool.name}: #{e.message}"
+ false
+ end
+end
+
# Stage 0 result (spec SS5.7). Everything downstream (Tier 1/2/3, executor)
# operates on this, never on raw text or loose token/clause arrays.
# runs it, and whether running it needs user confirmation first.
Tool = Struct.new(:name, :rules, :action, :permission)
- # Collects rule()/action()/permission() calls inside a tool(...) do...end
- # block, then validates them into a Tool on build.
+ # A pattern's :slot names (the "target" in "copy :target to :destination").
+ def self.slot_names(pattern)
+ pattern.split.each_with_object(Set.new) { |w, s| s << w.delete_prefix(":").to_sym if w.start_with?(":") }
+ end
+
+ # Registers pattern -> tool in rules, after checking pattern's :slot names
+ # match reference's. The one mechanism behind both author-declared verb
+ # synonyms (ToolBuilder#verb_synonyms) and user-taught ones (the "teach"
+ # tool), so neither path can bind a pattern whose slots don't line up with
+ # what the tool's action actually expects. Returns false on a slot mismatch.
+ def self.register_rule(rules, pattern, tool, reference:)
+ return false unless slot_names(pattern) == slot_names(reference)
+
+ prior = rules[pattern]
+ warn "warning: rule #{pattern.inspect} reassigned from tool #{prior.name.inspect} to #{tool.name.inspect}" if prior && prior != tool
+
+ rules[pattern] = tool
+ true
+ end
+
+ # Collects rule()/action()/permission()/verb_synonyms() calls inside a
+ # tool(...) do...end block, then validates them into a Tool on build.
class ToolBuilder
PERMISSIONS = %i[yolo ask].freeze
def initialize(name)
@name = name
@patterns = []
- @permission = :yolo
+ @verb_synonyms = []
+ @permission = :ask # fail-safe default: a tool must opt into :yolo
@action = nil
end
@patterns << pattern
end
+ # Extra leading verbs for every rule() already declared on this tool, e.g.
+ # verb_synonyms %w[remove erase] on "delete :target" also registers
+ # "remove :target" and "erase :target" - without repeating the full
+ # pattern per synonym.
+ def verb_synonyms(words)
+ raise ArgumentError, "tool #{@name.inspect}: verb synonyms can't be a :slot" if words.any? { |w| w.start_with?(":") }
+
+ @verb_synonyms.concat(words)
+ end
+
def permission(level)
raise ArgumentError, "tool #{@name.inspect}: unknown permission #{level.inspect}" unless PERMISSIONS.include?(level)
raise ArgumentError, "tool #{@name.inspect} has no rule" if @patterns.empty?
raise ArgumentError, "tool #{@name.inspect} has no action" unless @action
- slots = slot_sets
+ patterns = @patterns + synonym_patterns
+ slots = patterns.map { |pattern| Ruleset.slot_names(pattern) }
raise ArgumentError, "tool #{@name.inspect}: rules disagree on slot names" unless slots.uniq.length == 1
raise ArgumentError, "tool #{@name.inspect}: action params don't match rule slots" unless slots.first == required_keywords
- Tool.new(@name, @patterns, @action, @permission)
+ Tool.new(@name, patterns, @action, @permission)
end
private
- # Each rule's slot names (the ":target" in "copy :target to :destination"),
- # so build() can confirm every rule in this tool agrees on them.
- def slot_sets
- @patterns.map do |pattern|
- pattern.split.each_with_object(Set.new) { |w, s| s << w.delete_prefix(":").to_sym if w.start_with?(":") }
+ # Every declared verb synonym applied to every canonical rule, swapping
+ # only the leading word - same shape as hand-writing one rule() per verb.
+ def synonym_patterns
+ @verb_synonyms.flat_map do |verb|
+ @patterns.map { |pattern| ([verb] + pattern.split[1..]).join(" ") }
end
end
class RulesModule
def initialize
- @verbs = Set.new
@rules = {}
end
register(builder.build)
end
- def verbs(list)
- @verbs = Set.new(list)
- end
-
def literal_patterns = @types
def connective_patterns = @connectives
def rule_index = @rules
# Every rule's leading word is a verb by construction (ToolBuilder
- # rejects a rule that opens on a :slot), plus any verbs() declared for
- # words not yet backed by a tool - so chaining still recognizes them.
+ # rejects a rule that opens on a :slot) - so chaining recognizes it the
+ # moment a tool or taught rule exists for it, never before.
def verb_patterns
- @verbs | @rules.keys.map { |pattern| pattern.split.first }
+ Set.new(@rules.keys.map { |pattern| pattern.split.first })
end
# :action phrases come from verbs, not the classifiers() call, so the
private
- # Last tool to claim a rule pattern wins; an earlier claimant is
- # overwritten with a warning so shadowing is never silent.
+ # Last tool to claim a rule pattern wins; shadowing is never silent
+ # (Ruleset.register_rule warns). Every pattern in tool.rules already
+ # shares the same slot names (build() checked), so the first is a valid
+ # reference for the rest.
def register(tool)
- tool.rules.each do |pattern|
- prior = @rules[pattern]
- warn "warning: rule #{pattern.inspect} reassigned from tool #{prior.name.inspect} to #{tool.name.inspect}" if prior && prior != tool
-
- @rules[pattern] = tool
- end
+ tool.rules.each { |pattern| Ruleset.register_rule(@rules, pattern, tool, reference: tool.rules.first) }
end
end
["next"],
]
- # Checked in this order; first matching kind wins.
+ # Checked in this order; first matching kind wins. teach/remember are not a
+ # kind of their own - they're builtin tools, classified as :action like any
+ # other verb (see the "teach"/"remember" tools below).
classifiers(
- teach: [
- ["teach"], ["learn"], ["remember"]
- ],
conditional: [
["if"], ["unless"], ["when"]
],
tool("delete") do
permission :ask
-
+
rule "delete :target"
- rule "remove :target"
+ verb_synonyms %w[remove]
action do |target:|
puts "Deleting #{target}"
rule "archive :target"
+ verb_synonyms %w[zip compress]
+
action do |target:|
puts "Archiving #{target}"
end
end
- tool("copy-from-to") do
+ tool("copy-to") do
permission :yolo
-
+
rule "copy :target to :destination"
rule "copy from :target to :destination"
action do |target:, destination:|
- puts "Copying from #{target} to #{destination}"
+ puts "Copying #{target} to #{destination}"
end
end
- tool("move-from-to") do
+ tool("move-to") do
permission :ask
rule "move :target to :destination"
- rule "move from :target to :destination"
-
+
action do |target:, destination:|
- puts "Moving from #{target} to #{destination}"
+ puts "Moving #{target} to #{destination}"
+ end
+ end
+
+ tool("teach") do
+ permission :ask
+
+ rule "teach :trigger means :target"
+
+ action do |trigger:, target:|
+ tool = Tier1.rules[target]
+
+ # Target phrase must already be a known rule; teach that one first.
+ unless tool
+ puts " don't know how to #{target.inspect} yet - teach that one first"
+ next
+ end
+
+ # Same slot-check-and-register mechanism as author-declared verb
+ # synonyms (Ruleset::ToolBuilder#verb_synonyms).
+ unless Ruleset.register_rule(Tier1.rules, trigger, tool, reference: target)
+ puts " #{trigger.inspect} doesn't use the same slots as #{target.inspect}"
+ next
+ end
+
+ Classifier.configure(builtins) # refresh so the new leading word classifies as :action
+ puts " learned: #{trigger.inspect} -> #{tool.name}"
end
end
- # Verbs not yet backed by a tool; "and <verb>" chaining still recognizes them.
- verbs %w[run create rename search find locate pick transfer]
+ tool("remember") do
+ permission :yolo
+
+ rule "remember :term means :canonical"
+
+ action do |term:, canonical:|
+ Synonyms.terms[term] = canonical
+ puts " remembered: #{term.inspect} means #{canonical.inspect}"
+ end
+ end
end
Lexer.configure(builtins)
Classifier.configure(builtins)
Tier1.configure(builtins)
-# Start the REPL now
-loop do
- print ":> "
- text = gets
- exit 0 if text.nil?
+# Only run the REPL when executed directly, so tests can require this file
+# for its modules/ruleset without blocking on stdin.
+if __FILE__ == $PROGRAM_NAME
+ loop do
+ print ":> "
+ text = gets
+ exit 0 if text.nil?
+
+ begin
+ utterance = Utterance.build(text.chomp)
+ rescue Utterance::Rejected => e
+ puts e.message
+ next
+ end
+ next if utterance.nil? # empty/punctuation-only input: no-op
- begin
- utterance = Utterance.build(text.chomp)
- rescue Utterance::Rejected => e
- puts e.message
- next
- end
- next if utterance.nil? # empty/punctuation-only input: no-op
+ if utterance.clauses.any? { |clause| clause.kind.nil? }
+ puts "couldn't tell what kind of request this is, please rephrase"
+ next
+ end
- if utterance.clauses.any? { |clause| clause.kind.nil? }
- puts "couldn't tell what kind of request this is, please rephrase"
- next
- end
+ utterance.clauses.each_with_index do |clause, j|
+ puts "clause #{j} [#{clause.kind}]: #{clause.tokens.map(&:to_s).join(' ')}"
+ next unless clause.kind == :action
- utterance.clauses.each_with_index do |clause, j|
- puts "clause #{j} [#{clause.kind}]: #{clause.tokens.map(&:to_s).join(' ')}"
- next unless clause.kind == :action
+ node = Tier1.lookup(clause.tokens)
+ puts " #{node ? "tier1 hit: #{node}" : 'tier1 miss'}"
+ next unless node
- node = Tier1.lookup(clause.tokens)
- puts " #{node ? "tier1 hit: #{node}" : 'tier1 miss'}"
- next unless node
+ tool = node[:tool]
+ # :ask tools must be confirmed before running; :yolo tools run immediately.
+ if tool.permission == :ask
+ print " run #{tool.name}? [y/N] "
+ next unless gets&.chomp&.downcase == "y"
+ end
- tool = node[:tool]
- # :ask tools must be confirmed before running; :yolo tools run immediately.
- if tool.permission == :ask
- print " run #{tool.name}? [y/N] "
- next unless gets&.chomp&.downcase == "y"
+ Executor.call(tool, node[:args])
end
-
- tool.action.call(**node[:args].transform_values(&:text))
+ puts "match: #{Lexer.match_text(utterance.clauses.flat_map(&:tokens))}"
end
- puts "match: #{Lexer.match_text(utterance.clauses.flat_map(&:tokens))}"
-
- # Later stages (Tier 2/3, executor) take a clause's token array. The
- # executor reads literal.text directly; no restore step.
end
--- /dev/null
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "minitest/autorun"
+require "open3"
+require_relative "../nlshell"
+
+class LexerTest < Minitest::Test
+ def test_tokenizes_words_and_literals
+ tokens = Lexer.tokenize('delete "My Notes.txt" in ~/Old.log')
+ assert_equal [:word, :literal, :word, :literal], tokens.map(&:type)
+ assert_equal "My Notes.txt", tokens[1].text
+ end
+
+ def test_recognizes_shell_commands_as_command_tokens
+ tokens = Lexer.tokenize("git status")
+ assert_equal :command, tokens.first.type
+ assert tokens.first.word?
+ end
+
+ def test_unbalanced_quote_is_detected
+ tokens = Lexer.tokenize('delete "unterminated')
+ assert Lexer.unbalanced_quote?(tokens)
+ end
+
+ def test_sentences_split_on_boundaries
+ tokens = Lexer.tokenize("delete old.log. archive new.log.")
+ sentences = Lexer.sentences(tokens)
+ assert_equal 2, sentences.length
+ end
+
+ def test_clauses_split_on_connective
+ tokens = Lexer.downcase_words(Lexer.tokenize("delete old.log then archive new.log"))
+ clauses = Lexer.clauses(tokens)
+ assert_equal 2, clauses.length
+ end
+
+ # Regression: "and <command>" must split like "and <verb>", not crash
+ # (Token had no command? method).
+ def test_clauses_split_on_and_command
+ tokens = Lexer.downcase_words(Lexer.tokenize("delete old.log and ls"))
+ clauses = Lexer.clauses(tokens)
+ assert_equal 2, clauses.length
+ end
+end
+
+class FuzzyTest < Minitest::Test
+ def test_exact_match_is_always_close
+ assert Fuzzy.close?("delete", "delete")
+ end
+
+ def test_short_words_require_exact_match
+ refute Fuzzy.close?("is", "it")
+ end
+
+ def test_medium_words_tolerate_one_typo
+ assert Fuzzy.close?("dlete", "delete")
+ end
+
+ # Regression: "wipe" (edit distance 2 from "while") must not fuzzy-match a
+ # short/medium classifier keyword it isn't actually a typo of.
+ def test_unrelated_short_word_is_not_close
+ refute Fuzzy.close?("wipe", "while")
+ end
+end
+
+class ClassifierTest < Minitest::Test
+ def test_classifies_known_verb_as_action
+ tokens = Lexer.downcase_words(Lexer.tokenize("delete old.log"))
+ assert_equal :action, Classifier.classify(tokens)
+ end
+
+ def test_classifies_query_keyword
+ tokens = Lexer.downcase_words(Lexer.tokenize("what is the time"))
+ assert_equal :query, Classifier.classify(tokens)
+ end
+
+ def test_unknown_leading_word_is_unclassified
+ tokens = Lexer.downcase_words(Lexer.tokenize("bork the garble"))
+ assert_nil Classifier.classify(tokens)
+ end
+end
+
+class Tier1Test < Minitest::Test
+ def test_lookup_extracts_slot_values
+ tokens = Lexer.downcase_words(Lexer.tokenize("delete old.log"))
+ node = Tier1.lookup(tokens)
+ refute_nil node
+ assert_equal "delete", node[:name]
+ assert_equal "old.log", node[:args][:target].text
+ end
+
+ def test_lookup_misses_on_wrong_arity
+ tokens = Lexer.downcase_words(Lexer.tokenize("delete"))
+ assert_nil Tier1.lookup(tokens)
+ end
+
+ # delete's "remove" verb_synonym must resolve to the same tool as "delete".
+ def test_lookup_resolves_a_verb_synonym_to_the_same_tool
+ tokens = Lexer.downcase_words(Lexer.tokenize("remove old.log"))
+ node = Tier1.lookup(tokens)
+ assert_equal "delete", node[:name]
+ end
+end
+
+class RulesetTest < Minitest::Test
+ def test_tool_without_permission_defaults_to_ask
+ tool = nil
+ Ruleset.create("t") do
+ tool("noop") { rule "noop :x"; action { |x:| x } }
+ end.tap { |rs| tool = rs.rule_index["noop :x"] }
+
+ assert_equal :ask, tool.permission
+ end
+
+ def test_tool_rejects_mismatched_slots_and_action_params
+ error = assert_raises(ArgumentError) do
+ Ruleset.create("t") do
+ tool("bad") { rule "bad :x"; action { |y:| y } }
+ end
+ end
+ assert_match(/don't match/, error.message)
+ end
+
+ def test_tool_rejects_rule_starting_with_slot
+ assert_raises(ArgumentError) do
+ Ruleset.create("t") do
+ tool("bad") { rule ":x bad"; action { |x:| x } }
+ end
+ end
+ end
+
+ def test_verb_synonyms_expand_into_additional_patterns
+ rs = Ruleset.create("t") do
+ tool("zap") { rule "zap :x"; verb_synonyms %w[nuke incinerate]; action { |x:| x } }
+ end
+
+ assert_equal %w[zap\ :x nuke\ :x incinerate\ :x], rs.rule_index.keys
+ assert_same rs.rule_index["zap :x"], rs.rule_index["nuke :x"]
+ end
+
+ def test_verb_synonyms_rejects_a_slot
+ assert_raises(ArgumentError) do
+ Ruleset.create("t") do
+ tool("zap") { rule "zap :x"; verb_synonyms [":nope"]; action { |x:| x } }
+ end
+ end
+ end
+end
+
+class SharedRegistrationTest < Minitest::Test
+ Tool = Struct.new(:name)
+
+ def test_slot_names_extracts_colon_prefixed_words
+ assert_equal Set[:target, :destination], Ruleset.slot_names("copy :target to :destination")
+ end
+
+ def test_register_rule_accepts_matching_slots
+ rules = {}
+ tool = Tool.new("delete")
+
+ assert Ruleset.register_rule(rules, "wipe :target", tool, reference: "delete :target")
+ assert_equal tool, rules["wipe :target"]
+ end
+
+ # Same mechanism used by ToolBuilder#verb_synonyms (author time) and the
+ # "teach" tool (runtime) - a pattern binding different slots than its
+ # reference must be rejected either way.
+ def test_register_rule_rejects_mismatched_slots
+ rules = {}
+ tool = Tool.new("delete")
+
+ refute Ruleset.register_rule(rules, "wipe :thing", tool, reference: "delete :target")
+ assert_nil rules["wipe :thing"]
+ end
+end
+
+class ExecutorTest < Minitest::Test
+ Tool = Struct.new(:name, :action)
+
+ def test_returns_true_on_success
+ tool = Tool.new("noop", ->(target:) { target })
+ token = Lexer::Token.new(:literal, "old.log", :file)
+
+ assert Executor.call(tool, { target: token })
+ end
+
+ # A raising action must not propagate - the REPL loop depends on this to
+ # avoid crashing on a single bad command.
+ def test_returns_false_and_does_not_raise_on_error
+ tool = Tool.new("boom", ->(target:) { raise "kaboom" })
+ token = Lexer::Token.new(:literal, "old.log", :file)
+
+ refute Executor.call(tool, { target: token })
+ end
+end
+
+# End-to-end smoke tests driving the actual REPL process over stdin/stdout.
+class ReplIntegrationTest < Minitest::Test
+ NLSHELL = File.expand_path("../nlshell.rb", __dir__)
+
+ def run_repl(input)
+ Open3.capture3("ruby", NLSHELL, stdin_data: input).first
+ end
+
+ def test_teach_registers_a_new_rule_tool_resolves
+ out = run_repl(%(teach "zap :target" means "delete :target"\ny\nzap old.log\ny\n))
+ assert_match(/learned: "zap :target" -> delete/, out)
+ assert_match(/tier1 hit.*name=>"delete"/, out)
+ end
+
+ def test_remember_stores_a_synonym
+ out = run_repl(%(remember "wipe" means "delete"\n))
+ assert_match(/remembered: "wipe" means "delete"/, out)
+ end
+
+ def test_declining_confirmation_skips_execution
+ out = run_repl(%(delete old.log\nn\n))
+ refute_match(/Deleting old\.log/, out)
+ end
+end
projects / proto / nlshell.git / commitdiff