Posts

  • Hard CIDER: Hard Restart

    Another day, another hard topic to tackle! How do you (hard) restart CIDER? That should be simple, right? Well, it really depends on your perspective and on your definition of a “restart”.

    CIDER has two commands with the word “restart” in their name - one is cider-restart and the other is sesman-restart. I’ve noticed that the majority of CIDER’s users define “restart” as:

    • Restart the nREPL server
    • Reconnect CIDER to the newly started nREPL server

    I’ve also noticed the majority of the users expect the command cider-restart to do exactly this. It might come as a shock to some people, but this command does something pretty different - it simply kills the current nREPL connection (which in CIDER is the same as a REPL buffer) and creates a new one to the same nREPL server. I like to think of this as “soft restart”. Still, the question remains - how can we do a “hard restart”?

    Enter sesman-restart. This command is a whole different beast - it will actually restart the underlying nREPL server and re-create all connections within the same CIDER session. I guess that’s more aligned with the expectation of most people.

    The way I see it there are two primary reasons for the confusion:

    • cider-restart used to behave differently before we introduced the concept of multiple connections grouped in a single session (the sesman library provides this session abstraction)
    • most people don’t understand very well the concept of sessions in CIDER and the relationship between CIDER and sesman. As a corollary - they won’t think to look for commands that are not prefixed with cider-.

    There’s also the fact that the name cider-restart is probably not ideal. Perhaps a better name would be cider-restart-repl or cider-restart-connection? Naming is hard…

    I’ll ruminate on the subject of what’s the best course of action. As usual - feedback and ideas from you would be welcome! In the mean time I plan to work on improving the connection management documentation, so it’s clearer how the two command differ.

    That’s all I have for you today. This episode was brought to you by Clojurists Together. They are awesome and so are all of you! Keep hacking!

  • State of CIDER Survey (2019)

    One of the hardest things for any project author is to figure out whether their perspective on a project is aligned with perspective of the project’s users. Henry Ford supposedly knew better than his customers what they needed, but if that’s true I’d say he was a rather special case. I’m often guessing what people need and would find useful, what their issues are and what tools are they using. From time to time I guess right, but often I guess wrong. A couple of notable examples immediately come to my mind.

    I always assumed that people didn’t care much about the minimum supported Emacs version, as it’s pretty easy to upgrade Emacs, so I was dropping support for older Emacsen quite aggressively. Last year, however, I got some feedback after my “Clojurists Together” funding round that people who were working in corporate environments were really suffering, because they couldn’t upgrade the software on their machines. I was enlightened.

    I always assumed that people don’t care much about changing the keybindings of some commands and I was doing this quite a lot in the early days of CIDER. Now I know better…

    Similarly when I introduced cider-nrepl many years ago I thought that having people set it up manually won’t be a big deal, and I grossly underestimated how complex this task was for many people. Fun trivia - automating this setup in CIDER 0.10 was one of the most celebrated changes ever.

    At some point I foolishly believed that everyone read CIDER’s manual before starting to use it. If only that were true!1

    I’ve always loved the concept of surveys like “State of Clojure” and the insight they bring. I’ve meant to do something similar for CIDER for quite a while now - probably for at least 5 years. For one reason or another, however, this task just kept sitting in my backlog. Until today…2

    Please, allow to me to introduce to you the first ever State of CIDER survey! It’s far from perfect, but it’s a start. I’d really like to connect with more of you - the people who are actually using CIDER. I want to understand what’s the environment you’re working in, I want to learn what’s your background and how are you using CIDER. I want to learn what do you like and what do you hate. Long story short - I want to align the things I’m working on with your needs.

    I also want to get a feeling for how many people would be consider supporting the project financially down the road. I’m hoping to inspire a few more companies and individuals to pitch in to our OpenCollective and GitHub Sponsors campaigns. I’m a dreamer, but I’m not the only one… The subject of the economics of OSS development is something I plan to discuss at length in a dedicated article.

    The survey will be open until the end of next week. Looking forward to your responses!

    1. Not that I ever read any manual before diving into something. 

    2. All the credit here goes to the “Clojurists Together”. Having goals to work towards makes a big difference! 

  • Hard CIDER: Understanding the Jack-in Process

    There are many mysteries in life, but few are as perplexing as the eternal question “What exactly happens during cider-jack-in?”. Today it’s time to unravel this mystery!

    The Basics

    At its heart cider-jack-in does something super simple - it just executes a shell command (e.g. lein repl) that starts an nREPL server, waits for this server to come to life, and then connects CIDER to it. It’s really as simple as this. If I had to explain it in more details I’d probably do it like:

    • CIDER shells out and runs a command like lein repl :headless.
    • CIDER waits for the nREPL server to start. CIDER figures out this by parsing the output from the command and waiting for a line like nREPL server started on port 53005 on host localhost - nrepl://localhost:53005 to appear there.
    • CIDER extracts the port of the nREPL from the preceding message.
    • It connects to the running nREPL server.

    Basically it’s a combination of auto-starting an nREPL server and doing cider-connect to it afterwards. 1 Or is it? If it were so simple why does the shell command executed by cider-jack-in for Leiningen project look like:

    [nREPL] Starting server via /usr/local/bin/lein update-in :dependencies conj \[nrepl\ \"0.6.0\"\] -- update-in :plugins conj \[cider/cider-nrepl\ \"0.23.0-SNAPSHOT\"\] -- repl :headless :host localhost
    

    It’s time to explain another confusing concept - namely auto-injection of Clojure dependencies.

    Auto-Injecting Dependencies

    While CIDER’s core functionality requires nothing more than an nREPL server, there are many advanced features2 that depend on the presence of additional nREPL middleware (e.g. cider-nrepl). In the early versions of CIDER (up to CIDER 0.11) users had to add those dependencies themselves, which was a painful and error-prone process. Fortunately today that’s handled auto-magically when you’re using cider-jack-in.

    If your project uses lein, boot or tools.deps (deps.edn), CIDER will automatically inject all the necessary nREPL dependencies (e.g. cider-nrepl or piggieback) when it starts the server. The injection process is extremely simple - CIDER simply passes the extra dependencies and nREPL configuration to your build tool in the command in runs to start the nREPL server. Here’s how this looks for tools.deps:

    $ clojure -Sdeps ‘ cider/cider-nrepl {:mvn/version “0.22.4”}}}’ -m nrepl.cmdline –middleware ‘[“cider.nrepl/cider-middleware”]’

    Here’s a tip for you - if you don’t want cider-jack-in to inject dependencies automatically, set cider-inject-dependencies-at-jack-in to nil. Note that you’ll have to setup the dependencies yourself (e.g. by adding them to your project) just as in CIDER 0.10 and older. There are few reason why you’d want to do something like this, but the option exists.

    Normally cider-jack-in would inject only cider-nrepl and cider-jack-in-cljs would add piggieback as well. The injection, however, mechanism is configurable and you can easily add more libraries there. Some CIDER extensions (e.g. sayid and clj-refactor) would use this mechanism to auto-inject their own dependencies.

    CIDER would also inject the most recent version of nREPL that it supports. This is a simple trick to override the version of nREPL bundled with your build tool, so you can gain access to the newest nREPL features. Generally that’s one aspect of CIDER’s inner workings that end-users will rarely have to think about.

    CIDER can also inject a Clojure dependency into your project, which is useful, for example, if your project defaults to an older version of Clojure than that supported by the CIDER middleware. Set cider-jack-in-auto-inject-clojure appropriately to enable this.

    There are other nuances of the auto-injection - e.g. for Leiningen you can inject both dependencies and plugins, but that’s outside the scope of this article.

    Now it’s time for a little quiz! Can you guess why CIDER does not currently support dependency auto-injection for Gradle projects? Yeah, you’re totally right! Unfortunately there’s no way to pass extra dependencies to Gradle via its command-line interface.

    Jacking-in without a Project

    If you try to run cider-jack-in outside a project directory, CIDER will warn you and ask you to confirm whether you really want to do this; more often than not, this is an accident. If you decide to proceed, CIDER will invoke the command configured in cider-jack-in-default. Prior to CIDER 0.17, this defaulted to lein but was subsequently switched to clj, Clojure’s basic startup command.

    Here’s another useless tip - you can set cider-allow-jack-in-without-project to t if you’d like to disable the warning displayed when jacking-in outside a project. I never run CIDER outside of projects, but you might have a different style and, as you probably noticed by now, CIDER is infinitely configurable.

    Customizing the Jack-in Command Behaviour

    We now made it to my favourite part of our exploration of CIDER’s jack-in - how can we tweak the shell command it runs? There are many reasons to want to do this - most often you’d like to include some extra profiles (e.g. the famous fig profile if you are using tools.deps), but I can imagine some people would have to do more creative tweaks as well.

    Let’s start with the simplest possible option, that’s idea for one-off customizations. You can use C-u M-x cider-jack-in (C-u C-c C-x j j) to specify the exact command that cider-jack-in would run. This option is very useful is you want to specify a something like a lein or deps.edn profile.

    Alternatively you can C-u C-u M-x cider-jack-in, which is a variation of the previous command. This command will first prompt you for the project you want to launch cider-jack-in in, which is pretty handy if you’re in some other directory currently. This option is also useful if your project contains some combination of project.clj, build.boot and deps.edn and you want to launch a REPL for one or the other.

    Keep in mind that the examples here use only cider-jack-in, but this behaviour is consistent for all cider-jack-in-* commands.

    You can further customize the command line CIDER uses for cider-jack-in by modifying the following string options:

    • cider-lein-global-options, cider-boot-global-options, cider-clojure-cli-global-options, cider-gradle-global-options: these are passed to the command directly, in first position (e.g., -o to lein enables offline mode).
    • cider-lein-parameters, cider-boot-parameters, cider-clojure-cli-parameters, cider-gradle-parameters: these are usually task names and their parameters (e.g., dev for launching boot’s dev task instead of the standard repl -s wait).

    Don’t forget that those variables can be configured on a per-project basis.

    A Note about tools.deps and Windows

    You’ve probably figured out by now that using tools.deps with Windows can be a bit painful. Fortunately CIDER can help with that.

    To use cider-jack-in with tools.deps on Windows set the cider-clojure-cli-command to "powershell". This happens by default if you are on Windows and no clojure executable is found. Using "powershell" will Base64 encode the clojure launch command before passing it to PowerShell and avoids shell-escaping issues.

    You might be wondering why CIDER uses clojure instead of clj, right? It’s pretty simple actually - clj simply wraps clojure with some terminal-friendly functionality (rlwrap), that explodes if you’re not in an actual terminal. Another great mystery has been resolved!

    Epilogue

    Phew! That’s a lot of information! I hope now you’ve got a much better idea how the jack-in process is working and that this will help you use CIDER more effectively down the road.

    There’s a lot more that can be said about jack-in, but I do feel we’ve covered everything essential. I’m also a bit lazy and I don’t feel like writing anymore. :-)

    That’s all I have for you today. This episode was brought to you by Clojurists Together. They are awesome and so are all of you! Keep hacking!

    1. With a pretty confusing Neuromancer/Matrix-inspired name. 

    2. “Advanced” is a bit of stretch here - there’s no ClojureScript support without Piggieback. 

  • Meta Redux: Year One

    I’ve started “Meta Redux” on my birthday one year ago. I had plenty of ambitious goals when I kicked off the new blog and now it’s time to examine how well I’ve done with respect to them.

    In total I wrote 52 articles, which conveniently means that on average I wrote one article per week. That’s way more than I used to be writing on the old blog before retiring it, so I’m reasonably happy with myself. As usual, however, there’s plenty of room for improvement. For one, I planned to write more essay-like articles in the spirit of nREPL Redux and Ruby’s Creed, but I ended up writing way more release announcements, tutorials and short articles instead. I also wrote pretty much nothing that wasn’t related to programming. I really want to change this down the road, as (believe it or not) I’m passionate about more things in life than programming.1

    One area where I miserably failed were my journal series “Meta Reduce”. I set out to do one article per week and I ended up doing less than one per month. On the bright side - that’s the only area where I managed to share some non-programming related thoughts. I also really struggled to find the right structure for those, but I’m sure I’ll get there eventually.

    Another (small) failure - “Meta Redux” consumed most of my writing time and energy and this impacted negatively my other blog, dedicated to Emacs. I guess it’s no secret that I’m crazy about Emacs and I love writing about it, so I’m quite displeased with how things went down there. I’ll have to find some kind of balance between the two of them going forward.

    So, what’s next? This year I’ve been spending a lot of time playing with Objectives and Key Results and I’ve been trying to put more of the things I’m doing in the OKR framework. That’s how I see my “Meta Redux” OKRs for the upcoming year:

    Objective: Share my interesting thoughts and ideas with a broader audience.

    Key Results:

    • Publish 1 “Meta Reduce” article per month.
    • Publish 10 essays.
    • Publish 10 non-programming articles.
    • Publish 64 articles in total.
    • Double the number of unique visitors to the site.

    Nothing super ambitious here, but now that I’ve put the goals in writing it’s much easier to measure my success with achieving them.

    Now for some random observations:

    • I’ve rarely written anything while traveling. I’m really struggling with writing when I don’t have a comfortable desk and a real keyboard. Not to mention I have the laptop with the worst keyboard in the world…
    • I’ve started way more articles than I actually finished.
    • I spend more time editing something than I spend writing the initial draft.2
    • Switching back to Jekyll from Octopress really helped. Hooray for simple tools! I tried at some point using Hugo, but I found its tool-chain to be more complex than Jekyll’s. That’s partially related to my background as a Ruby developer, but it has more to do with the fact that the initial process of setting up a Hugo blog and customizing a theme there is more involved.
    • It’s really hard to find nice looking, free and well-maintained Jekyll themes. Most of the famous themes seem to be abandoned or with 20 competing forks. That’s a bummer, but at least the default Minima theme is quite compatible with my sense of aesthetics.
    • Writing some simple Jekyll helpers for Emacs (e.g. to link to another article) was a real productivity booster.
    • My readers are much better proofreaders than me.

    In the end of the day, however, for me the most important thing is that I’m happy with “Meta Redux”. It feels great to share my thoughts in writing. For some crazy reason I genuinely believe I’ve learned some importantly things in life (and on the job) that might be useful to others. I hope you enjoy my ramblings as much as I enjoy producing them! Until next time!

    1. That’s why I opted for programming-inspired, yet programming-neutral name for the blog. 

    2. Unless it’s some straightforward tutorial. 

  • CIDER 0.23 ("Lima")

    CIDER 0.22 was released just a month ago, but now I’m happy to announce the release of CIDER 0.23 (“Lima”). Unlike CIDER 0.22, Lima is a pretty small release. I was under a lot of pressure to ship 0.22 in time for Clojure/south and I didn’t have time to complete a few small tickets that I had originally planned to go into it. The new release basically addresses them, plus a few bugs that were reported in CIDER 0.22. You know what’s really good about small releases? It’s quite simple - you don’t need to write long blog posts about them.

    There’s not much in terms of new features to highlight - probably the most notable change is that CIDER now infers automatically figwheel builds and will present the available options to you with auto-completion when you need to select a build. Another small change is that now you can configure the position of evaluation results in source buffers - before they were always displayed at the end of the current line, but now you can display them right after the expression you’ve evaluated. Here’s the magic bit of Emacs configuration to make this happen:

    (setq cider-result-overlay-position 'at-point)
    

    We’ve also added the possibility to disable the new enhanced ClojureScript code completion, as some users reported it was causing issues for them that we’ve been having a hard time reproducing and identifying. If you want to go back to the basic ClojureScript completion from earlier versions of CIDER just use the following:

    (setq cider-enhanced-cljs-completion-p nil)
    

    Now it’s time for slightly bigger update - translating file paths! That’s a small feature that might be useful to people developing using Docker. For instance, suppose your app is running in a Docker container with your source directories mounted there. The navigation paths you’d get from nREPL will be relative to the source in the Docker container rather than the correct path on your host machine. If you wish to translate file paths from your running instance you may use the cider-path-translations defcustom to do so. You can add translation mappings easily by setting the following (typically in .dir-locals.el on a per-project basis):

    ((nil
      (cider-path-translations . (("/root" . "/Users/foo")
                                  ("/src/" . "/Users/foo/projects")))))
    

    Each entry will be interpreted as a directory entry so trailing slash is optional. Navigation to some definition will attempt to translate these locations, and if they exist, navigate there rather than report the file does not exist. In the example above, the .m2 directory is mounted at /root/.m2 and the source at /src. These translations would map these locations back to the user’s computer so that navigation to definition would work.

    There are some interesting updates on the documentations front as well. I’ve replaced all of the pages on the old documentation site https://cider.readthedocs.io with redirects to https://docs.cider.mx. 1 This marks the end of the run for the legacy documentation portal and hopefully it’s going to affect positively the Google page rank of the new one.2 I’ve also started work on a cider-nrepl section of the documentation portal. It’s still super basic, but it’s also the best documentation cider-nrepl has ever had.

    We’ve also fixed a bunch of bugs in Orchard and cider-nrepl, and shipped a new version of Piggieback, which doesn’t blow up in the absence of ClojureScript! Cool stuff, but I’m too lazy to write in details about any of this. I think it’s safe to say CIDER 0.23 is nice step on the path towards better stability and everyone should update to it right away. You can find a complete list of all the changes in CIDER 0.23 here.

    I’m quite pleased with this release and I’ll try to do a couple of more small releases until the end of the year. As usual I’d like to thank everyone who contributed to the developed of Lima and the awesome people at Clojurists Together for supporting the project. You’re awesome!

    That’s all from me for today. Until next release! Cheers!

    1. Don’t ask. There was simply no other way to do this. 

    2. Which is currently horrible. Please, update any links you might have to the CIDER documentation, so we can boost that page rank. 

Subscribe via RSS | View Older Posts