Posts

  • Projectile 3.3

    Projectile 3.3 is out! That’s the fourth release this month, which probably tells you something about how much fun I’ve been having with Projectile lately.

    This one is mostly about a single theme - Projectile knowing more about your project without you having to tell it anything.

    Read More
  • Closing the Find-Usages Gap in CIDER

    Next up in the series on the notable changes in CIDER 2.0: cross-references. Or, as most people call the feature, “find usages” - for years the most commonly cited reason to run clojure-lsp alongside (or instead of) CIDER. Let’s talk about why that gap existed and how we finally closed it.

    Why runtime xref wasn’t enough

    CIDER has had runtime cross-referencing for a while: the cider/fn-refs op walks the loaded vars in your REPL and reports which functions reference the one at point. It’s a genuinely cool trick - the REPL literally knows your program - but as a “find usages” answer it has three structural problems:

    • It only sees loaded code. Namespaces you haven’t required yet - often most of the codebase - are invisible.
    • It reports functions, not occurrences. Each hit points at the caller’s definition, not the exact call site, and a function that calls yours three times shows up once.
    • It’s JVM-only, so ClojureScript users got nothing.

    clojure-lsp, by contrast, builds a static index of your whole project with clj-kondo’s analyzer and answers instantly, loaded or not. For occurrence-oriented questions (“show me every place this is used, so I can change all of them”), static analysis is simply the right tool. No amount of runtime cleverness fixes “the code isn’t loaded”.

    The fix: search the source

    So CIDER 2.0 does the obvious thing we should have done years ago: xref-find-references (M-?) now finds references by searching the project’s source files on disk. Unloaded code, cljs files, commented-out drafts - if the name occurs in the project, you’ll see the exact occurrence, in the standard xref UI you already use for everything else in Emacs. To borrow the franchise that has been handing programmers debugging metaphors for over two decades now: the runtime is the Matrix, a tidy compiled illusion of your program, and to see every place a thing is really used you sometimes have to unplug and look at the source itself.1

    Here’s the difference in one picture - the same query on orchard.misc/require-and-resolve, first in runtime mode, then in source mode:

    Find-usages of orchard.misc/require-and-resolve: the runtime search returns only the loaded callers, one per calling function, while the source search finds every occurrence across the project - including the alias-qualified uses in other namespaces and a cljc file the REPL never loaded

    The runtime knows about three callers - and points you at each caller’s definition. The source scan turns up all ten actual occurrences, across five files, including the ones written as misc/require-and-resolve in namespaces the REPL never loaded. That’s the gap, in one screenshot.

    Now, “search the source” makes it sound like a grep, and I want to be clear that it isn’t a dumb one. Say you’re chasing orchard.misc/require-and-resolve. The search runs in three stages:

    1. First CIDER asks the REPL to resolve the symbol at point to its fully qualified name, orchard.misc/require-and-resolve. The REPL is right there on the other end of the wire, so why guess when you can ask?
    2. A fast first pass (ripgrep, via Emacs’ own xref-matches-in-files) finds every file that so much as mentions require-and-resolve, purely to narrow the field.
    3. Then the Clojure-aware part. For each of those files, CIDER reads its (ns ...) form to learn how that file pulls in orchard.misc - aliased as [orchard.misc :as misc], brought in with :refer [require-and-resolve], or is this orchard.misc itself? - and builds a regexp that matches only the forms that file could legitimately use: the qualified orchard.misc/require-and-resolve, the aliased misc/require-and-resolve, or a bare require-and-resolve where the namespace declaration makes that valid. The requires inside the ns form are excluded, so the import line doesn’t show up as a “usage”.2

    Is any of this as smart as clj-kondo’s full analysis? No - it’s ultimately a syntactic search, so an identically named var in another namespace can still sneak through as a false positive. But because it’s ns-aware rather than a blind text match, a bare require-and-resolve in some file that never requires orchard.misc (and has a require-and-resolve of its own) won’t be mistaken for yours. For the daily “where is this used?” question it turns out to be remarkably close to the real thing in practice, it requires zero extra infrastructure, and it composes with what only CIDER has: the running REPL.

    That composition is configurable via cider-xref-references-mode:

    • source (the default) - occurrences from the project’s files.
    • runtime - the historical loaded-vars behavior.
    • both - source occurrences first, plus the runtime hits the scan can’t see. And there are such hits: references generated by macro expansion leave no textual trace in your source, but the runtime knows about them. Static analysis can’t ever tell you those; your REPL can.

    (There’s also cider-xref-fn-refs-in-source, C-c C-? s, when you want the source search explicitly, and outside a project the source mode gracefully falls back to the runtime search.)

    Beyond find usages: the who-* family

    While closing the gap, we went further and built out a whole family of SLIME-inspired cross-referencing commands under C-c C-w, most of them rendered as expandable trees:

    • cider-who-calls / cider-who-is-called - the call graph, upward and downward. Expand a caller to see its callers; spelunk as deep as you like:

      The cider-who-calls tree, expanded two levels up the call graph

    • cider-who-implements - a protocol’s implementing types (inline defrecord/deftype implementations included) or a multimethod’s dispatch values, each jumping to the implementation’s source. Multimethods are a nice case study in hybrid thinking: the method functions carry no source metadata at runtime, so CIDER locates the defmethod forms by - you guessed it - searching the source.
    • cider-type-protocols / cider-protocols-with-method - the reverse lookups: what does this type implement, and which protocols declare this method?
    • cider-who-macroexpands - a macro’s use sites, found via source search, because macro invocations are expanded away at compile time and the runtime literally cannot see them.

    Much of this is powered by new ops in cider-nrepl (and Orchard underneath), and much of the inspiration came straight from SLIME and swank-clojure, which offered who-calls back when Clojure itself was barely out of the crib. And here’s the part I love: the whole “go read the source instead of trusting the runtime” instinct was already there in swank-clojure’s implementation. Its who-calls would find candidate callers among the loaded vars, sure, but then it went and read the actual source form of each one off disk and walked the parsed code looking for your symbol, rather than believing whatever the compiled runtime claimed. It was still anchored to loaded code - it never scanned the whole project the way CIDER 2.0 does - but the core idea, that the source on disk is the ground truth and the runtime is just a convenient approximation, predates this release by about fifteen years. Good ideas don’t expire. Sometimes nothing beats revisiting the classics.3

    So do you still need clojure-lsp?

    If you were running clojure-lsp primarily for find-usages - the most common answer I heard when I asked - then CIDER now covers you out of the box. If you use it for project-wide renames, unused-var linting, or editing without a REPL, carry on; those are real strengths of static analysis and CIDER doesn’t try to replicate them. The two continue to work fine side by side, and the new async eldoc even yields politely so LSP-provided docs can compose with CIDER’s.

    My goal was never to “beat” clojure-lsp - it was to make a freshly installed CIDER answer the questions every Clojure programmer asks a dozen times a day, with no extra moving parts, and with the one advantage nobody else has: a live runtime on the other end of the wire.

    The full story is in the navigation docs. Keep hacking!

    1. As Morpheus puts it: “Unfortunately, no one can be told what the Matrix is. You have to see it for yourself.” Same with find-usages, really - I can tell you a var is used in seven places, but until you’ve seen the actual call sites you don’t really know what changing it will break. 

    2. Full disclosure: matching the aliased and namespace-qualified forms correctly only landed after 2.0 - the 2.0.x releases had a bug where the source scan quietly skipped files that referenced a var through its alias, which is of course the common case. The fix will ship in CIDER 2.1, which doesn’t have a release date yet. I’m hoping to get back into a rhythm of cutting a new CIDER release every month or two, so it shouldn’t be a long wait. 

    3. “The path of the One ends at the Source.” The Architect was talking about Zion, but he might as well have been describing every debugging session that ends with you finally opening the file and reading the code instead of theorizing about it. 

  • Modernizing CIDER's Completion

    CIDER’s code completion has quietly gotten quite good over the years, and I don’t think it gets enough credit. It’s all built on Emacs’s standard completion-at-point, so it works with whatever completion UI you prefer - the built-in one, Corfu, company - without any special setup. Under the hood compliment does the heavy lifting for Clojure (and clj-suitable for ClojureScript), which means smart, backend-driven matching: mai completes to map-indexed, cji to clojure.java.io, and an unimported BiFun to java.util.function.BiFunction. The candidates come back ranked by the backend and are context-aware - it knows when you’re inside a -> or completing a deftype field.

    Lately I’ve been giving the Emacs side of things some attention, to bring it in line with the modern completion stack so many of us now use - Vertico, Corfu, Consult, Marginalia and friends. Two changes are worth calling out.

    The first is about the symbol prompts. A number of CIDER commands ask you for a Clojure symbol when there’s nothing at point - cider-doc, cider-find-var and the like. Historically those prompts used the older completion machinery, so your completing-read UI didn’t kick in and you were left with TAB and a *Completions* buffer. There’s now cider-use-completing-read-for-symbol (off by default for now); turn it on and those prompts go through completing-read over a collection that queries the running REPL lazily as you type. Vertico, Ivy, Helm - whatever you drive completing-read with - just works, and the candidates carry their type and namespace.

    The second is smaller, but I like it a lot: annotations now line up in a proper column instead of trailing raggedly after each candidate.

    CIDER completion annotations

    This comes from an affixation-function, the richer successor to the old annotation-function, so every frontend that understands it - the built-in *Completions*, Corfu, Vertico - renders the aligned version. company keeps showing its own trailing annotations, same as before.

    Both changes are in the latest CIDER MELPA build and will ship in the next stable release. As always, I’d love to hear how they work out for you.

    That’s all I have for you today. Keep hacking!

    P.S. If you use Embark, I wrote up a fun way to act on Clojure symbols with it - documentation, jump-to-definition, inspect and so on - over on Emacs Redux.

  • Making CIDER More Discoverable

    This series about the notable changes in CIDER 2.0 continues with the change you’ll bump into first, whatever your workflow: the transient menus, and the broader push to make CIDER’s functionality discoverable.

    The problem: CIDER is huge

    CIDER has well over 300 interactive commands. I’ll admit something I’ve said before: there are features in CIDER that even I forget exist, and I wrote half of them. For users, historically, the options for finding functionality were:

    • memorize cryptic key chords (C-c C-w i, anyone?)
    • grep the (very long) manual
    • read the source
    • stumble on a feature by accident three years in and feel robbed

    That’s not great for a tool whose whole pitch is making you more productive. The Emacs answer to this problem was demonstrated years ago by Magit: transient menus, which turn every prefix into a self-documenting popup. It took us embarrassingly long to follow suit, but CIDER 2.0 finally does.

    Transient everywhere

    Every command group in CIDER now opens a transient menu: cider-eval-menu at C-c C-v, cider-doc-menu at C-c C-d, and likewise for test, namespace, macroexpand, profile, trace and references. A top-level cider-menu ties them all together, and even the debugger (? mid-session) and the inspector (m) got menus of their own. Jack-in and connect live in cider-start-menu at C-c C-x.

    Here’s the evaluation menu, which is a good illustration of the problem the menus solve - I doubt many people knew all of this was hiding behind C-c C-v:

    The CIDER evaluation transient menu, listing every evaluation command

    One design constraint was non-negotiable: your muscle memory is safe. These menus replace bare prefix keymaps, so every existing keybinding works exactly as before, at full speed - C-c C-v e still evaluates instantly, menu or no menu. And if you’d rather not see the menus at all unless you actually hesitate mid-chord, set transient-show-popup to a short delay and they’ll appear only in that moment of doubt - which is precisely when you need them.

    Transient also gave us something the old keymaps never could: arguments. Menus now carry flags for the things that vary per invocation - pick a pretty-printer with --print-fn=, set test selectors with --include=/--exclude= and reuse them across runs, toggle cider-ns-refresh’s modes explicitly, pass Clojure CLI aliases at jack-in time. All those “this command behaves differently with a prefix argument” paragraphs in the manual are becoming visible checkboxes instead.

    The test menu shows this nicely - set the selectors once and every run below them picks them up:

    The CIDER test transient menu with its include and exclude selector arguments

    Discovery beyond menus

    The menus are the headline, but the discoverability push in the 2.0 cycle went wider:

    • A new keybindings reference page collects every binding in one place, and the printable refcard was brought back up to date.
    • The REPL’s shouty welcome banner is gone, replaced by a one-line hint; the getting-started material now lives in a summonable reference card (C-c C-h, or the ,refcard REPL shortcut) - available when you want it, invisible when you don’t.
    • CIDER now warns (once per session) when you use a deprecated keybinding, so bindings can actually be retired someday without silently breaking people. M-x cider-list-deprecated-keybindings shows what’s on the way out.
    • The 1.22 cycle’s big audit already made the mode menus expose dozens of commands that were technically present but practically invisible; 2.0 builds on that foundation.
    • Even cider-doctor is discoverability of a sort - it surfaces the problems in your setup that you’d otherwise discover one confusing bug report at a time.

    The philosophy

    If I had to compress the 2.0 discoverability work into one sentence: the features were always there; now the tool tells you about them. Documentation is where knowledge goes to be forgotten - the only reliable place to teach a user about a feature is inside the workflow itself, at the moment of hesitation. Transient menus are exactly that, and Magit proved the pattern scales to enormous command sets.

    If some menu feels wrong - a missing command, a flag that should exist, a grouping that doesn’t match how you think - please file an issue. This part of CIDER is young and very much open to feedback.

    The keybindings docs have the full picture. Keep hacking!

  • clj-suitable 0.8.0: Closing the Gap with Compliment

    You had me at js/.

    – Jerry Maguire, on ClojureScript interop completion

    clj-suitable 0.8.0 is out! If the name doesn’t ring a bell, that’s rather the point - clj-suitable is the small library that quietly powers ClojureScript code completion in CIDER, Calva, and pretty much anything else that talks to a cljs REPL over nREPL. Think of it as the ClojureScript counterpart to what compliment1 does for Clojure. For years it lagged well behind its Clojure sibling, and this release is my attempt to finally close that gap - to make cljs completion, if you’ll forgive me, a touch more suitable.

    This is also a direct follow-up to the Piggieback work I wrote about a couple of weeks ago. Once I had the cljs REPL plumbing back in decent shape, fixing up the completion story sitting on top of it was the obvious next move.2

    1. The completion library, spelled with an i. Not the nice thing 

    2. One thing invariably leads to another with this stuff. You set out to fix 

    Read More

Subscribe via RSS | View Older Posts