From fc90904f0563f728c0e3ee450c53b96e0bb0ca22 Mon Sep 17 00:00:00 2001 From: Mike Lowis Date: Thu, 1 Oct 2026 21:12:28 -0400 Subject: [PATCH] Added tests and fixed some bugs. Added verb synonyms and removed duplicate code paths --- nl-shell-harness-spec.md | 51 +++++-- nlshell.rb | 277 +++++++++++++++++++++++++++------------ test/nlshell_test.rb | 221 +++++++++++++++++++++++++++++++ 3 files changed, 456 insertions(+), 93 deletions(-) create mode 100644 test/nlshell_test.rb diff --git a/nl-shell-harness-spec.md b/nl-shell-harness-spec.md index 82763d4..330e478 100644 --- a/nl-shell-harness-spec.md +++ b/nl-shell-harness-spec.md @@ -77,19 +77,22 @@ Everything downstream (classifier, Tier 1/2/3, executor) operates on an **Uttera ### 5.1 Tokenize -Raw input becomes an array of typed tokens via a single-pass lexer (`Lexer.tokenize`, `StringScanner`-based). Three token types: +Raw input becomes an array of typed tokens via a single-pass lexer (`Lexer.tokenize`, `StringScanner`-based). Four token types: | Type | Carries | Example | |---|---|---| | `word` | text, lowercased later | `delete`, `then` | | `literal` | exact original text **and** a kind | `path("~/Old.log")`, `string("My Notes.txt")` | +| `command` | text, lowercased later; a word that also names an executable found on `$PATH` | `ls`, `git` | | `punct` | a single character | `.` `,` `;` `?` | +`command` is a subtype of `word` (`token.word?` is true for both, so classifier/verb matching still sees it) but never a `:literal`, so it can't fill a rule's `:slot` — a command name used as an argument stays a plain word, not a slot value. + **Literal kinds**, tried in this priority order at each token boundary: `string` (quoted), `url`, `env` (`$HOME`, `${Path}`), `glob` (`*.tmp`, `{a,b}`), `path` (`~/Old.log`, `./src`), `flag` (`-r`, `--force`), `number` (`3.5`, `500ms`, `50%` — tried before `file` so decimals aren't read as filenames), `file` (`notes.txt`, `.bashrc`). Literals are never embedded back into a string with markers — they're carried as token data, so nothing downstream can accidentally lowercase, split, or "correct" a filename, and no restore/unmasking step is needed. `token.key` returns the word text or the literal's **kind** (not its value), so `delete old.log` and `delete report.pdf` normalize to the identical match string `delete file` for Tier 1/2 comparison, while `token.text` on a literal still holds the real value for execution. -Known false-positive guards: `e.g.`/`i.e.` are kept as single `word` tokens (not misread as a `.g`/`.e` file with a sentence break); a bare `?` is never a glob character, so it always ends a question. +Known false-positive guards: a bare `?` is never a glob character, so it always ends a question. (The `e.g.`/`i.e.` abbreviation guard described in the original source listing below is **not present** in the current `nlshell.rb` implementation — a gap to revisit.) Full source (drop-in, no dependencies beyond Ruby's `strscan` stdlib): @@ -234,13 +237,16 @@ end `Lexer.sentences(tokens)` splits on `punct` tokens matching `. ? ! ;` (terminator stays attached to the sentence it ends). This is safe because periods inside literals (`old.log`, `3.5`, `e.g.`) never reached the token stream as bare punctuation — they were consumed by the literal/abbreviation patterns in Stage 5.1. -### 5.5 Split into clauses **[A]** +### 5.5 Split into clauses + +Implemented as `Lexer.clauses`, driven by the ruleset rather than hardcoded: -Not yet implemented in code; policy to build against: +- Splits a sentence into clauses on connective token sequences declared via `connectives([...])` (builtins: `["after", "that"]`, `["and", "then"]`, `["then"]`, `["next"]`). +- A bare `and` **without** a preceding comma is ambiguous ("copy a and b to c" is one command). The code only splits on it when the token right after `and` (or `, and`) is in the ruleset's `verb_patterns` set (a known verb word) **or** is a `:command` token (a recognized `$PATH` executable, e.g. "delete a.txt and ls") — not "post-canonicalization" since Tier 2 canonicalization doesn't exist yet; otherwise it's treated as a list argument within one clause. +- Each resulting clause is classified independently (§7 node types) via `Classifier.classify`. + +**[A] still open, not yet implemented:** the REPL currently classifies and executes each clause as it's reached, rather than resolving the whole Utterance first. The plan-then-execute policy below remains a v-next item: -- Split a sentence into clauses on connective words: `then`, `and then`, `after that`, `next`, or `,` immediately followed by `and`. -- A bare `and` **without** a preceding comma is ambiguous ("copy a and b to c" is one command). Only split on it when the token immediately after `and` matches a known action word post-canonicalization (§6.4 step 3); otherwise treat it as a list argument within one clause. -- Each resulting clause is classified independently (§7 node types: action, chain, loop, conditional, query, teach) and enters the tiered pipeline (§6.1) separately. - For a multi-clause Utterance: resolve every clause first (may hit Tier 3 per clause), assemble the full plan, show it once, then execute. Query and confirmed-teach clauses may run immediately since they don't mutate state; action clauses wait for the full plan to validate. If any clause fails to resolve, execute nothing. ### 5.6 Fragments and the context register **[A]** @@ -399,7 +405,34 @@ CREATE TABLE audit_log ( --- -## 9. Worked Example Rule Set (Ruby DSL, starter vocabulary) +## 9. Rule Set DSL + +### 9.1 Current implementation + +`Ruleset.create(name) { ... }` (see `nlshell.rb`). Rules are grouped per-tool rather than declared as standalone `rule(...) { action ... }` calls as originally sketched (§9.2): each `tool` block holds its own `rule` pattern(s), an optional `verb_synonyms([...])` list, a `permission` (`:yolo` runs immediately, `:ask` confirms with the user first, and is now the default — a tool must opt into `:yolo`), and a single `action` block whose keyword args must match every rule's `:slot` names exactly (`ToolBuilder#build` raises otherwise). `synonym`/`chain`/`query` DSL calls from the original sketch don't exist yet — only `tool`/`rule`/`verb_synonyms`/`permission`/`action`, plus top-level `types`/`connectives`/`classifiers` for lexer/classifier configuration. There's no separate `verbs()` declaration: a word is a verb (and recognized for `and ` chain-splitting) only once a `tool` rule or taught rule actually starts with it — no pre-declaring verbs ahead of a real rule. + +`verb_synonyms` expands each declared `rule` once per synonym, swapping only the leading word (`verb_synonyms %w[remove]` on `rule "delete :target"` also registers `"remove :target"`) — a convenience for tools with several verbs that all take identical slots, without repeating the full pattern per verb. + +`teach`/`remember` aren't a `classifiers()` kind — they're ordinary builtin `tool`s (§6.3 is superseded: teaching is just an action, not a separate node type). `teach "" means ""` registers a new rule pointing at whatever tool the target phrase already resolves to; `remember "" means ""` writes into a runtime `Synonyms` store for future Tier 2 use. `teach` and `verb_synonyms` share one mechanism, `Ruleset.register_rule(rules, pattern, tool, reference:)`: it compares `Ruleset.slot_names(pattern)` against the reference pattern's and only writes `pattern -> tool` into the rules store on a match (warning on shadowing either way) — so a taught trigger can't bind a tool whose action expects different slots, same as an author-declared synonym can't. + +```ruby +Ruleset.create("builtins") do + types(string: /.../, path: /.../, ...) # literal patterns, feeds Lexer.configure + connectives [["after", "that"], ["and", "then"], ["then"], ["next"]] + classifiers(conditional: [["if"]], loop: [["for", "each"]], query: [["what"]]) + + tool("delete") do + permission :ask + rule "delete :target" + verb_synonyms %w[remove] + action { |target:| puts "Deleting #{target}" } + end +end +``` + +### 9.2 Original sketch **[A]** — superseded by §9.1 + +Kept for historical context on the target vocabulary breadth; `synonym`/`chain`/`query` and per-action `rule` calls shown here don't match the current DSL. ```ruby RuleSet.define do @@ -479,6 +512,8 @@ nl-shell/ Everything under `lib/` past `lexer.rb` is unimplemented; this document is the spec for building it. +**Current state:** none of the above split has happened yet — Lexer, Fuzzy, Classifier, Tier 1, the `Ruleset` DSL (§9.1), builtin tools, and the REPL loop all live in a single top-level `nlshell.rb` prototype. Implemented so far: tokenize/validate/downcase/sentence-split/clause-split (§5.1-5.5), classification, Tier 1 exact/fuzzy-word matching, and a REPL that dispatches matched Tier 1 actions with `:ask`/`:yolo` confirmation. Still unbuilt: Tier 2, Tier 3 client, query resolver, teach flow, SQLite stores (§8), and the context register (§5.6). + --- ## 13. Deployment / First Run **[A]** diff --git a/nlshell.rb b/nlshell.rb index 371aa48..09137e8 100755 --- a/nlshell.rb +++ b/nlshell.rb @@ -6,11 +6,16 @@ require "strscan" # Scan the environment for possible shell commands ShellCommands = Set.new -ENV.fetch("PATH", "").split(File::PATH_SEPARATOR).reverse.each do |dir| +ENV.fetch("PATH", "").split(File::PATH_SEPARATOR).each do |dir| next unless File.directory?(dir) - Dir.each_child(dir) do |name| - path = File.join(dir, name) - ShellCommands << name if File.file?(path) && File.executable?(path) + + begin + Dir.each_child(dir) do |name| + path = File.join(dir, name) + ShellCommands << name if File.file?(path) && File.executable?(path) + end + rescue Errno::EACCES + next # unreadable PATH entry, skip it end end @@ -19,9 +24,11 @@ module Lexer # text: the word/punct as typed, or the literal's exact value # kind: literal kind (:path, :string, ...) or nil Token = Struct.new(:type, :text, :kind) do - # :command matches as a word (it's just a word that shadows a PATH entry); - # it still isn't a :literal, so it can't fill a rule's :slot. + # :command matches as a word (it's just a word that shadows a PATH entry). + # It isn't a :literal, but like any non-punct token it can still fill a + # rule's :slot (Tier1.match only excludes punctuation there). def word? = type == :word || type == :command + def command? = type == :command def literal? = type == :literal def punct? = type == :punct @@ -148,6 +155,10 @@ module Lexer length = 1 elsif t.text == "," && n&.text == "and" && tokens[i + 2]&.word? && verbs.include?(tokens[i + 2].text) length = 2 + elsif t.text == "and" && n&.command? + length = 1 + elsif t.text == "," && n&.text == "and" && tokens[i + 2]&.command? + length = 2 end # return the length @@ -166,10 +177,22 @@ end # vocabulary - so there's no ambiguity to resolve, and slot values (which # never go through this) can't be touched. module Fuzzy - MAX_DISTANCE = 2 + # Short words ("is", "who") have too few possible typos to fuzz at all; + # medium words get one typo's worth of slack. Scaled by the expected word + # (b), not the input, so tolerance reflects the known vocabulary word. + SHORT_WORD_MAX_LENGTH = 3 + MEDIUM_WORD_MAX_LENGTH = 6 def self.close?(a, b) - a == b || distance(a, b) <= MAX_DISTANCE + a == b || distance(a, b) <= max_distance(b) + end + + def self.max_distance(word) + # Exact match only below SHORT_WORD_MAX_LENGTH, else scale with length. + return 0 if word.length <= SHORT_WORD_MAX_LENGTH + return 1 if word.length <= MEDIUM_WORD_MAX_LENGTH + + 2 end # Levenshtein edit distance. @@ -188,9 +211,9 @@ module Fuzzy end end -# Stage: CLS. Tags a clause as teach/conditional/loop/query/action by -# matching its leading words against configured phrases, checked in the -# configured order (first match wins). +# Stage: CLS. Tags a clause as conditional/loop/query/action by matching its +# leading words against configured phrases, checked in the configured order +# (first match wins). teach/remember aren't a kind - they're builtin actions. module Classifier class << self attr_accessor :patterns @@ -258,6 +281,32 @@ module Tier1 end end +# Tier 2 canonicalization store (SS6.4 step 3): term -> canonical word. Tier 2 +# itself isn't built yet; "remember" populates this for when it is. +module Synonyms + class << self + attr_accessor :terms + end + + self.terms = {} + + def self.canonical(word) + terms.fetch(word, word) + end +end + +# Stage: EX. Runs a resolved tool's action, isolating the REPL from any +# exception an action raises so one bad command can't crash the shell. +module Executor + def self.call(tool, args) + tool.action.call(**args.transform_values(&:text)) + true + rescue StandardError => e + puts " error running #{tool.name}: #{e.message}" + false + end +end + # Stage 0 result (spec SS5.7). Everything downstream (Tier 1/2/3, executor) # operates on this, never on raw text or loose token/clause arrays. @@ -298,15 +347,36 @@ module Ruleset # runs it, and whether running it needs user confirmation first. Tool = Struct.new(:name, :rules, :action, :permission) - # Collects rule()/action()/permission() calls inside a tool(...) do...end - # block, then validates them into a Tool on build. + # A pattern's :slot names (the "target" in "copy :target to :destination"). + def self.slot_names(pattern) + pattern.split.each_with_object(Set.new) { |w, s| s << w.delete_prefix(":").to_sym if w.start_with?(":") } + end + + # Registers pattern -> tool in rules, after checking pattern's :slot names + # match reference's. The one mechanism behind both author-declared verb + # synonyms (ToolBuilder#verb_synonyms) and user-taught ones (the "teach" + # tool), so neither path can bind a pattern whose slots don't line up with + # what the tool's action actually expects. Returns false on a slot mismatch. + def self.register_rule(rules, pattern, tool, reference:) + return false unless slot_names(pattern) == slot_names(reference) + + prior = rules[pattern] + warn "warning: rule #{pattern.inspect} reassigned from tool #{prior.name.inspect} to #{tool.name.inspect}" if prior && prior != tool + + rules[pattern] = tool + true + end + + # Collects rule()/action()/permission()/verb_synonyms() calls inside a + # tool(...) do...end block, then validates them into a Tool on build. class ToolBuilder PERMISSIONS = %i[yolo ask].freeze def initialize(name) @name = name @patterns = [] - @permission = :yolo + @verb_synonyms = [] + @permission = :ask # fail-safe default: a tool must opt into :yolo @action = nil end @@ -318,6 +388,16 @@ module Ruleset @patterns << pattern end + # Extra leading verbs for every rule() already declared on this tool, e.g. + # verb_synonyms %w[remove erase] on "delete :target" also registers + # "remove :target" and "erase :target" - without repeating the full + # pattern per synonym. + def verb_synonyms(words) + raise ArgumentError, "tool #{@name.inspect}: verb synonyms can't be a :slot" if words.any? { |w| w.start_with?(":") } + + @verb_synonyms.concat(words) + end + def permission(level) raise ArgumentError, "tool #{@name.inspect}: unknown permission #{level.inspect}" unless PERMISSIONS.include?(level) @@ -332,20 +412,21 @@ module Ruleset raise ArgumentError, "tool #{@name.inspect} has no rule" if @patterns.empty? raise ArgumentError, "tool #{@name.inspect} has no action" unless @action - slots = slot_sets + patterns = @patterns + synonym_patterns + slots = patterns.map { |pattern| Ruleset.slot_names(pattern) } raise ArgumentError, "tool #{@name.inspect}: rules disagree on slot names" unless slots.uniq.length == 1 raise ArgumentError, "tool #{@name.inspect}: action params don't match rule slots" unless slots.first == required_keywords - Tool.new(@name, @patterns, @action, @permission) + Tool.new(@name, patterns, @action, @permission) end private - # Each rule's slot names (the ":target" in "copy :target to :destination"), - # so build() can confirm every rule in this tool agrees on them. - def slot_sets - @patterns.map do |pattern| - pattern.split.each_with_object(Set.new) { |w, s| s << w.delete_prefix(":").to_sym if w.start_with?(":") } + # Every declared verb synonym applied to every canonical rule, swapping + # only the leading word - same shape as hand-writing one rule() per verb. + def synonym_patterns + @verb_synonyms.flat_map do |verb| + @patterns.map { |pattern| ([verb] + pattern.split[1..]).join(" ") } end end @@ -356,7 +437,6 @@ module Ruleset class RulesModule def initialize - @verbs = Set.new @rules = {} end @@ -378,19 +458,15 @@ module Ruleset register(builder.build) end - def verbs(list) - @verbs = Set.new(list) - end - def literal_patterns = @types def connective_patterns = @connectives def rule_index = @rules # Every rule's leading word is a verb by construction (ToolBuilder - # rejects a rule that opens on a :slot), plus any verbs() declared for - # words not yet backed by a tool - so chaining still recognizes them. + # rejects a rule that opens on a :slot) - so chaining recognizes it the + # moment a tool or taught rule exists for it, never before. def verb_patterns - @verbs | @rules.keys.map { |pattern| pattern.split.first } + Set.new(@rules.keys.map { |pattern| pattern.split.first }) end # :action phrases come from verbs, not the classifiers() call, so the @@ -399,15 +475,12 @@ module Ruleset private - # Last tool to claim a rule pattern wins; an earlier claimant is - # overwritten with a warning so shadowing is never silent. + # Last tool to claim a rule pattern wins; shadowing is never silent + # (Ruleset.register_rule warns). Every pattern in tool.rules already + # shares the same slot names (build() checked), so the first is a valid + # reference for the rest. def register(tool) - tool.rules.each do |pattern| - prior = @rules[pattern] - warn "warning: rule #{pattern.inspect} reassigned from tool #{prior.name.inspect} to #{tool.name.inspect}" if prior && prior != tool - - @rules[pattern] = tool - end + tool.rules.each { |pattern| Ruleset.register_rule(@rules, pattern, tool, reference: tool.rules.first) } end end @@ -438,11 +511,10 @@ builtins = Ruleset.create("builtins") do ["next"], ] - # Checked in this order; first matching kind wins. + # Checked in this order; first matching kind wins. teach/remember are not a + # kind of their own - they're builtin tools, classified as :action like any + # other verb (see the "teach"/"remember" tools below). classifiers( - teach: [ - ["teach"], ["learn"], ["remember"] - ], conditional: [ ["if"], ["unless"], ["when"] ], @@ -457,9 +529,9 @@ builtins = Ruleset.create("builtins") do tool("delete") do permission :ask - + rule "delete :target" - rule "remove :target" + verb_synonyms %w[remove] action do |target:| puts "Deleting #{target}" @@ -471,79 +543,114 @@ builtins = Ruleset.create("builtins") do rule "archive :target" + verb_synonyms %w[zip compress] + action do |target:| puts "Archiving #{target}" end end - tool("copy-from-to") do + tool("copy-to") do permission :yolo - + rule "copy :target to :destination" rule "copy from :target to :destination" action do |target:, destination:| - puts "Copying from #{target} to #{destination}" + puts "Copying #{target} to #{destination}" end end - tool("move-from-to") do + tool("move-to") do permission :ask rule "move :target to :destination" - rule "move from :target to :destination" - + action do |target:, destination:| - puts "Moving from #{target} to #{destination}" + puts "Moving #{target} to #{destination}" + end + end + + tool("teach") do + permission :ask + + rule "teach :trigger means :target" + + action do |trigger:, target:| + tool = Tier1.rules[target] + + # Target phrase must already be a known rule; teach that one first. + unless tool + puts " don't know how to #{target.inspect} yet - teach that one first" + next + end + + # Same slot-check-and-register mechanism as author-declared verb + # synonyms (Ruleset::ToolBuilder#verb_synonyms). + unless Ruleset.register_rule(Tier1.rules, trigger, tool, reference: target) + puts " #{trigger.inspect} doesn't use the same slots as #{target.inspect}" + next + end + + Classifier.configure(builtins) # refresh so the new leading word classifies as :action + puts " learned: #{trigger.inspect} -> #{tool.name}" end end - # Verbs not yet backed by a tool; "and " chaining still recognizes them. - verbs %w[run create rename search find locate pick transfer] + tool("remember") do + permission :yolo + + rule "remember :term means :canonical" + + action do |term:, canonical:| + Synonyms.terms[term] = canonical + puts " remembered: #{term.inspect} means #{canonical.inspect}" + end + end end Lexer.configure(builtins) Classifier.configure(builtins) Tier1.configure(builtins) -# Start the REPL now -loop do - print ":> " - text = gets - exit 0 if text.nil? +# Only run the REPL when executed directly, so tests can require this file +# for its modules/ruleset without blocking on stdin. +if __FILE__ == $PROGRAM_NAME + loop do + print ":> " + text = gets + exit 0 if text.nil? + + begin + utterance = Utterance.build(text.chomp) + rescue Utterance::Rejected => e + puts e.message + next + end + next if utterance.nil? # empty/punctuation-only input: no-op - begin - utterance = Utterance.build(text.chomp) - rescue Utterance::Rejected => e - puts e.message - next - end - next if utterance.nil? # empty/punctuation-only input: no-op + if utterance.clauses.any? { |clause| clause.kind.nil? } + puts "couldn't tell what kind of request this is, please rephrase" + next + end - if utterance.clauses.any? { |clause| clause.kind.nil? } - puts "couldn't tell what kind of request this is, please rephrase" - next - end + utterance.clauses.each_with_index do |clause, j| + puts "clause #{j} [#{clause.kind}]: #{clause.tokens.map(&:to_s).join(' ')}" + next unless clause.kind == :action - utterance.clauses.each_with_index do |clause, j| - puts "clause #{j} [#{clause.kind}]: #{clause.tokens.map(&:to_s).join(' ')}" - next unless clause.kind == :action + node = Tier1.lookup(clause.tokens) + puts " #{node ? "tier1 hit: #{node}" : 'tier1 miss'}" + next unless node - node = Tier1.lookup(clause.tokens) - puts " #{node ? "tier1 hit: #{node}" : 'tier1 miss'}" - next unless node + tool = node[:tool] + # :ask tools must be confirmed before running; :yolo tools run immediately. + if tool.permission == :ask + print " run #{tool.name}? [y/N] " + next unless gets&.chomp&.downcase == "y" + end - tool = node[:tool] - # :ask tools must be confirmed before running; :yolo tools run immediately. - if tool.permission == :ask - print " run #{tool.name}? [y/N] " - next unless gets&.chomp&.downcase == "y" + Executor.call(tool, node[:args]) end - - tool.action.call(**node[:args].transform_values(&:text)) + puts "match: #{Lexer.match_text(utterance.clauses.flat_map(&:tokens))}" end - puts "match: #{Lexer.match_text(utterance.clauses.flat_map(&:tokens))}" - - # Later stages (Tier 2/3, executor) take a clause's token array. The - # executor reads literal.text directly; no restore step. end diff --git a/test/nlshell_test.rb b/test/nlshell_test.rb new file mode 100644 index 0000000..5102ce2 --- /dev/null +++ b/test/nlshell_test.rb @@ -0,0 +1,221 @@ +#!/usr/bin/env ruby +# frozen_string_literal: true + +require "minitest/autorun" +require "open3" +require_relative "../nlshell" + +class LexerTest < Minitest::Test + def test_tokenizes_words_and_literals + tokens = Lexer.tokenize('delete "My Notes.txt" in ~/Old.log') + assert_equal [:word, :literal, :word, :literal], tokens.map(&:type) + assert_equal "My Notes.txt", tokens[1].text + end + + def test_recognizes_shell_commands_as_command_tokens + tokens = Lexer.tokenize("git status") + assert_equal :command, tokens.first.type + assert tokens.first.word? + end + + def test_unbalanced_quote_is_detected + tokens = Lexer.tokenize('delete "unterminated') + assert Lexer.unbalanced_quote?(tokens) + end + + def test_sentences_split_on_boundaries + tokens = Lexer.tokenize("delete old.log. archive new.log.") + sentences = Lexer.sentences(tokens) + assert_equal 2, sentences.length + end + + def test_clauses_split_on_connective + tokens = Lexer.downcase_words(Lexer.tokenize("delete old.log then archive new.log")) + clauses = Lexer.clauses(tokens) + assert_equal 2, clauses.length + end + + # Regression: "and " must split like "and ", not crash + # (Token had no command? method). + def test_clauses_split_on_and_command + tokens = Lexer.downcase_words(Lexer.tokenize("delete old.log and ls")) + clauses = Lexer.clauses(tokens) + assert_equal 2, clauses.length + end +end + +class FuzzyTest < Minitest::Test + def test_exact_match_is_always_close + assert Fuzzy.close?("delete", "delete") + end + + def test_short_words_require_exact_match + refute Fuzzy.close?("is", "it") + end + + def test_medium_words_tolerate_one_typo + assert Fuzzy.close?("dlete", "delete") + end + + # Regression: "wipe" (edit distance 2 from "while") must not fuzzy-match a + # short/medium classifier keyword it isn't actually a typo of. + def test_unrelated_short_word_is_not_close + refute Fuzzy.close?("wipe", "while") + end +end + +class ClassifierTest < Minitest::Test + def test_classifies_known_verb_as_action + tokens = Lexer.downcase_words(Lexer.tokenize("delete old.log")) + assert_equal :action, Classifier.classify(tokens) + end + + def test_classifies_query_keyword + tokens = Lexer.downcase_words(Lexer.tokenize("what is the time")) + assert_equal :query, Classifier.classify(tokens) + end + + def test_unknown_leading_word_is_unclassified + tokens = Lexer.downcase_words(Lexer.tokenize("bork the garble")) + assert_nil Classifier.classify(tokens) + end +end + +class Tier1Test < Minitest::Test + def test_lookup_extracts_slot_values + tokens = Lexer.downcase_words(Lexer.tokenize("delete old.log")) + node = Tier1.lookup(tokens) + refute_nil node + assert_equal "delete", node[:name] + assert_equal "old.log", node[:args][:target].text + end + + def test_lookup_misses_on_wrong_arity + tokens = Lexer.downcase_words(Lexer.tokenize("delete")) + assert_nil Tier1.lookup(tokens) + end + + # delete's "remove" verb_synonym must resolve to the same tool as "delete". + def test_lookup_resolves_a_verb_synonym_to_the_same_tool + tokens = Lexer.downcase_words(Lexer.tokenize("remove old.log")) + node = Tier1.lookup(tokens) + assert_equal "delete", node[:name] + end +end + +class RulesetTest < Minitest::Test + def test_tool_without_permission_defaults_to_ask + tool = nil + Ruleset.create("t") do + tool("noop") { rule "noop :x"; action { |x:| x } } + end.tap { |rs| tool = rs.rule_index["noop :x"] } + + assert_equal :ask, tool.permission + end + + def test_tool_rejects_mismatched_slots_and_action_params + error = assert_raises(ArgumentError) do + Ruleset.create("t") do + tool("bad") { rule "bad :x"; action { |y:| y } } + end + end + assert_match(/don't match/, error.message) + end + + def test_tool_rejects_rule_starting_with_slot + assert_raises(ArgumentError) do + Ruleset.create("t") do + tool("bad") { rule ":x bad"; action { |x:| x } } + end + end + end + + def test_verb_synonyms_expand_into_additional_patterns + rs = Ruleset.create("t") do + tool("zap") { rule "zap :x"; verb_synonyms %w[nuke incinerate]; action { |x:| x } } + end + + assert_equal %w[zap\ :x nuke\ :x incinerate\ :x], rs.rule_index.keys + assert_same rs.rule_index["zap :x"], rs.rule_index["nuke :x"] + end + + def test_verb_synonyms_rejects_a_slot + assert_raises(ArgumentError) do + Ruleset.create("t") do + tool("zap") { rule "zap :x"; verb_synonyms [":nope"]; action { |x:| x } } + end + end + end +end + +class SharedRegistrationTest < Minitest::Test + Tool = Struct.new(:name) + + def test_slot_names_extracts_colon_prefixed_words + assert_equal Set[:target, :destination], Ruleset.slot_names("copy :target to :destination") + end + + def test_register_rule_accepts_matching_slots + rules = {} + tool = Tool.new("delete") + + assert Ruleset.register_rule(rules, "wipe :target", tool, reference: "delete :target") + assert_equal tool, rules["wipe :target"] + end + + # Same mechanism used by ToolBuilder#verb_synonyms (author time) and the + # "teach" tool (runtime) - a pattern binding different slots than its + # reference must be rejected either way. + def test_register_rule_rejects_mismatched_slots + rules = {} + tool = Tool.new("delete") + + refute Ruleset.register_rule(rules, "wipe :thing", tool, reference: "delete :target") + assert_nil rules["wipe :thing"] + end +end + +class ExecutorTest < Minitest::Test + Tool = Struct.new(:name, :action) + + def test_returns_true_on_success + tool = Tool.new("noop", ->(target:) { target }) + token = Lexer::Token.new(:literal, "old.log", :file) + + assert Executor.call(tool, { target: token }) + end + + # A raising action must not propagate - the REPL loop depends on this to + # avoid crashing on a single bad command. + def test_returns_false_and_does_not_raise_on_error + tool = Tool.new("boom", ->(target:) { raise "kaboom" }) + token = Lexer::Token.new(:literal, "old.log", :file) + + refute Executor.call(tool, { target: token }) + end +end + +# End-to-end smoke tests driving the actual REPL process over stdin/stdout. +class ReplIntegrationTest < Minitest::Test + NLSHELL = File.expand_path("../nlshell.rb", __dir__) + + def run_repl(input) + Open3.capture3("ruby", NLSHELL, stdin_data: input).first + end + + def test_teach_registers_a_new_rule_tool_resolves + out = run_repl(%(teach "zap :target" means "delete :target"\ny\nzap old.log\ny\n)) + assert_match(/learned: "zap :target" -> delete/, out) + assert_match(/tier1 hit.*name=>"delete"/, out) + end + + def test_remember_stores_a_synonym + out = run_repl(%(remember "wipe" means "delete"\n)) + assert_match(/remembered: "wipe" means "delete"/, out) + end + + def test_declining_confirmation_skips_execution + out = run_repl(%(delete old.log\nn\n)) + refute_match(/Deleting old\.log/, out) + end +end -- 2.55.0