• Ruby, Where do We Go Now?

    This article is part of the “Meta Advent 2019” series. I’ve committed to writing a new blog post here every day until Christmas.

    A while ago I wrote an article, where I made the case Ruby seems to have lost its way. I thought I had exhausted the topic, but a few of the recent developments around the Ruby 2.7’s development lead me to revisit it.

    Ruby 2.7 has become infamous at this point for several highly controversial changes that were either quickly revised or removed. A classic tell-tale of good project management and a clear vision for the future, right?

    Let’s examine a few of those ill-fated changes in no particular order…

    Flip-flops Forever

    Remember how the flip-flop operator (..) was finally deprecated in Ruby 2.6 after years of discussions? Well, it just got (quietly) reinstated.

    I find it really amusing how this went down - 3 (!!!) people complained about the deprecation and Matz quickly responded with:

    I hear the negative feedback from the community. OK, I give up. The warning should be removed.

    – Matz

    There are numerous other instances where he ignored the complaints of way more people. Go figure. As usual, there was no real discussion on the ticket - just a decision. Not to mention that no one even thought to try to solicit more community feedback on the subject. I learned about the removal of the deprecation when someone posted this on Twitter…

    None of this is new, in Ruby-land it’s business as usual. It’s just becoming progressively more frustrating for me.

    Numbered Parameters Revisited

    Remember the highly controversial numbered block parameter syntax?

    # now
    [1, 2, 3].each { |i| puts i }
    (1..10).map { |i| i * 3 }
    (1..9).each_slice(3).map { |x, y, z| x + y + z }
    # Ruby 2.7
    [1, 2, 3].each { puts @1 }
    (1..10).map { @1 * 3 }
    (1..9).each_slice(3).map { @1 + @2 + @3 }

    After quite the backlash this syntax was revised to something more reasonable:

    [1, 2, 3].each { puts _1 }
    (1..10).map { _1 * 3 }
    (1..9).each_slice(3).map { _1 + _2 + _3 }

    A great success, right? Matz made a couple of interesting comments on this mega-thread. My favourite one was:

    Yes, I admit { @1[@2] = “Go Fish: #{@2}” } is cryptic. But {@1 * @2} is not. So use numbered parameters with care (just like other features in Ruby). The possibility to make code cryptic itself should not be the reason to withdraw a feature.

    The problem is introducing it or this could break existing code. That is the problem. Numbered parameters are a compromise to simplify block expression without breaking compatibility.

    FYI, I am not fully satisfied with the beauty of the code with numbered parameters, so I call it a compromise.

    – Matz

    So, he accepted something that he wasn’t perfectly happy about. OK, I can understand the need for compromises from time to time. But I totally hate this line of reasoning “The possibility to make code cryptic itself should not be the reason to withdraw a feature.”. I think that adding features for the sake of adding them is not a reason to add something either.

    I also think that we should be careful with the impact of features on tools and with all the accidental complexity they bring (e.g. now Ruby parsers are more complex). Not to mention the fact that no one could make a strong case for introducing this feature in the first place. I’m still not sure what’s the grand problem it solves…

    The Totally Useless Pipeline Operator

    This was probably the most ridiculous part of Ruby 2.7’s development cycle. At some point the pipeline operator landed in master:

    # now
      .bar(1, 2)
    # Ruby 2.7
      |> bar 1, 2
      |> display

    Everyone who has actually used the pipeline operator in Elixir (the source of inspiration for this change), however, knows that it behaves nothing like this. Here Ruby’s team basically came up with different syntax for . (method invocation) with a lower precedence that allowed you to do questionable things like:

    (a..b).each {}
    a..b |> each {}

    I won’t go into much details about this misguided operator. After all, a lot of great articles exist about it already.1 For me the important things here are:

    • This feature was not well thought-out
    • It generated a massive backlash
    • It was quickly reverted

    Sounds familiar, doesn’t it?

    The Rise and Fall of the Method Reference Operator

    Ruby 2.7 was supposed to introduce a method reference operator:

    [2, 4, 8, 16, 32].map { |n| Math.log2(n) }
    [2, 4, 8, 16, 32].map(&Math.:log2)

    Well, not anymore. This feature was recently reverted. There were some good reasons for it to be reverted, which serves to underline my point that a lot of changes to Ruby recently were made without thinking much about their implications. Matz, made one particularly interesting comment on this ticket:

    In recent years, we have added/discussed a lot of features inspired by functional programming. But those features are not designed with a grand design. Those are rather ad hoc additions. We felt the future, more functional Ruby should be designed with the big picture of functional Ruby programming, which we lack currently.

    Besides that, I don’t like the specific operator (.:) which looks like braille.

    So you don’t have to consider this revert as a rejection of this whole idea, but an invitation to the next try.

    – Matz

    I was really glad to see this, as it seems that Matz also understands that without a solid vision in the end we’ll just get one big mess. I’m a bit sad about the .: operator being reverted, as this was a feature I could see myself using, but that’s a small price to pay for a better future of Ruby.

    Parameter Forwarding Weirdness

    Just when I had some hope about the future I came across another “brilliant” addition to Ruby 2.7 - parameter forwarding. It’s best explained with an example:

    def foo(...)

    This defines a method foo, that can have any parameters whatsoever and simply forwards all of them to bar. I don’t know about you, but I always wanted something like this!2

    There are some plans to make it a bit more generic and support forwarding a subset of all parameters:

    def foo(a, b, ...)
      if a.even?
        bar(b, ...)

    Still, I think this is another random addition that doesn’t solve real problems and just adds mental and technical complexity.


    So, we made it to the end of this somewhat bitter and rantish article. So, what’s the takeaway from all of this, if any?

    It’s good that Ruby’s team listens to user feedback, but it might also be nice for them to actually start asking about feedback in some more open/structured manner prior to making language changes. Especially given their recent track record with the Ruby community… You can’t really expect everyone to be monitoring all tickets in Ruby’s issue tracker, right?3 I’ve often dreamt of a state where Ruby’s team actually announces in advance the focus for the next release (a.k.a. a roadmap), so interested parties can actually go and help with the tasks at hand. That should not require that much extra work - just a bit of good old planning. Some voting system on tickets and annual surveys might be a good idea as well - other programming communities are making good use of those.

    Many people are excited about Ruby 3.0, but I’m not. It’s a major release in theory, but in practice it will add relatively little to what we have today. It’s not even clear if the new concurrent programming model will make it there or not at this point.

    Ruby 2.7 has showed better than ever the chaos in Ruby’s development and the lack of a clear direction/destination. There are some signs that Matz understands this as well, so I’m still optimistic about the future.

    I think Matz should take a page from Rich Hickey’s4 book and take a bit of hammock-time to figure out where he wants to take Ruby next. So, Ruby, where do we go now?

    1. That’s my favourite one

    2. You can’t here my voice, but that’s sarcasm here. 

    3. Especially given the fact that the vast majority of ideas for language changes are just ignored and stay dormant for ages. 

    4. The author of Clojure. 

  • Pimp my Print Method: Prettier Clojure Built-in Types

    This article is part of the “Meta Advent 2019” series. I’ve committed to writing a new blog post here every day until Christmas.

    Trust me, it wasn’t easy to find the right name for this article. I started off with the pompous sounding “Improving the Printed Representation of Certain Clojure Built-in Object Types” and needed half an hour to come to the current title. Naming is hard!

    So, what’s the fancy sounding title all about? Have you noticed that many Clojure types are not represented in a particularly “user-friendly” way in the REPL?1 Here are a few examples:

    Clojure 1.10.1
    user=> +
    #object[clojure.core$_PLUS_ 0x2bc12da "clojure.core$_PLUS_@2bc12da"]
    user=> print-method
    #object[clojure.lang.MultiFn 0x3122b117 "clojure.lang.MultiFn@3122b117"]
    user=> (atom 5)
    #object[clojure.lang.Atom 0x17ae7628 {:status :ready, :val 5}]
    user=> (find-ns 'clojure.core)
    #object[clojure.lang.Namespace 0x5b78fdb1 "clojure.core"]

    Here we have a function, a multi-method, an atom and a namespace object. While when you’re used to Clojure their printed representations probably make sense to most people, they certainly are confusing to newcomers. The representation of + in particular is pure gold when it comes to clarity. The real question, however, is “Can we do better than this?”. Yes, we can!

    People using CIDER or editors backed by cider-nrepl (e.g. vim-fireplace and Calva) might be confused by the examples I gave, because they’d get something pretty different in their REPLs. Let’s do this one more time in CIDER’s REPL:

    user> +
    user> print-method
    #multifn[print-method 0x757ca063]
    user> (atom 5)
    #<Atom@3e319fc5: 5>
    user> (find-ns 'clojure.core)

    I don’t know about you, but I find this version much easier to comprehend. So, time for the crux of this article - how did we achieve this? It’s pretty simple actually - we simply leveraged print-method, a built-in Clojure multi-method that controls how an object gets printed. Here’s an excerpt from the actual implementation:

    (def ^:dynamic *pretty-objects*
      "If true, cider prettifies some object descriptions.
      For instance, instead of printing functions as
          #object[clojure.core$_PLUS_ 0x4e648e99 \"clojure.core$_PLUS_@4e648e99\"]
      they are printed as
      To disable this feature, do
          (alter-var-root #'cider.nrepl.print-method/*pretty-objects* not)"
    (defmacro def-print-method [dispatch-val arg & strings]
      `(defmethod print-method ~dispatch-val [~arg ~'^Writer w]
         (if *pretty-objects*
           (do ~@(map #(list '.write 'w %) strings))
           (#'clojure.core/print-object ~arg ~'w))))
    (defn- translate-class-name [c]
      (main/demunge (.getName (class c))))
    ;;; Atoms
    ;; Ex: #atom[{:foo :bar} 0x54274a2b]
    (def-print-method Atom c
      (pr-str @c)
      (format " 0x%x]" (System/identityHashCode c)))
    ;;; Function objects
    ;; Ex: #function[cider.nrepl.print-method/multifn-name]
    (def-print-method AFunction c
      (translate-class-name c)

    It’s a bit more complicated than it needs to be, mostly because of the wrapper def-print-method macro that makes each print-method aware of a dynamic var that controls the pretty-printing. It’s needed in a tool like CIDER, in case some users don’t like the different printing, but I guess most people won’t need this if they’re just tweaking something in their local setup.

    All in all - you need this code in some namespace that gets required early-on in your application and you’re done. That’s exactly what cider-nrepl does. While writing this article I also realized that I should really move this code to Orchard, as it’s not nREPL-specific in any way, and having it as part of Orchard would make it easier for more people to reuse the code directly.

    The technique is not without its flaws, though. Recently a CIDER user ran into the following problem:2

    (defrecord Foo []
      clojure.lang.IDeref (deref [x] x) )
    ;; =>
    ;; 2. Unhandled clojure.lang.ExceptionInfo
    ;; 1. Caused by java.lang.IllegalArgumentException
    ;;    Multiple methods in multimethod 'simple-dispatch' match dispatch value: class
    ;;    scratch.Foo -> interface clojure.lang.IDeref and interface
    ;;    clojure.lang.IPersistentMap, and neither is preferred

    Solving the problem is as easy as adding:

    (prefer-method print-method

    Still, I do believe that implementing IDeref in a record is a bad idea, and I’m also reasonably sure this is the only bug report we’ve got in the 5 years since this feature was added to CIDER. This leads me to believe it works fairly well for most people.

    That’s all I had for you today. I hope you managed to learn something useful. See you tomorrow!

    1. For my definition of “user-friendly” I guess. 

    2. This bug report was my inspiration to write the article you’re now reading. 

  • Documenting nREPL Middleware APIs

    This article is part of the “Meta Advent 2019” series. I’ve committed to writing a new blog post here every day until Christmas.

    One common problem with third-party nREPL middleware is that often they aren’t (properly) documented and you have to dig into the middleware code to figure out how to use it. Here I’m not referring to Clojure-level API documentation, but to nREPL operation (a.k.a. “op”) API documentation. Think things like eval, interrupt, etc - the different type of commands that a client would send to an nREPL server to make it do its bidding.1

    The funny thing is that nREPL had always had special provisions to ensure that op APIs were properly documented. Each middleware has to have a descriptor (think middleware metadata) which lists things like handled ops, their parameters, return values, etc. Here’s an example of a middleware descriptor:

    (set-descriptor! #'wrap-load-file
                     {:requires #{#'caught/wrap-caught #'print/wrap-print}
                      :expects #{"eval"}
                      :handles {"load-file"
                                {:doc "Loads a body of code, using supplied path and filename info to set source file and line number metadata. Delegates to underlying \"eval\" middleware/handler."
                                 :requires {"file" "Full contents of a file of code."}
                                 :optional (merge caught/wrap-caught-optional-arguments
                                                  {"file-path" "Source-path-relative path of the source file, e.g. clojure/java/io.clj"
                                                   "file-name" "Name of source file, e.g. io.clj"})
                                 :returns (-> (meta #'eval/interruptible-eval)
                                              (get "eval")

    You can read more about middleware descriptors here.

    nREPL leverages those descriptors in its own documentation and the result is pretty nice. This op API documentation is generated by a simple script that can output both AsciiDoc and Markdown.

    Is there a reason why third-party middleware libraries can’t apply the same approach? Not really. After a lot of procrastination I finally emulated all of this for cider-nrepl and its nREPL op API is finally well-documented. I had to tweak a bit the original script from nREPL, but the modifications were tiny2 and the result was pretty good. I was thinking it’d be cool if the documentation script was made more generic and reusable across projects (e.g. in the form of a Leiningen plugin), but I didn’t have time to pursue this idea.

    Actually, when it comes to cider-nrepl’s documentation, I did much more than documenting the nREPL op API. During the course of CIDER’s recent funding round by “Clojurists Together”, I actually created a documentation site for the project. It’s still pretty basic, but I hope to extend and polish it in the months to come. Feedback and help would be most welcome!

    Given the fact that middleware like cider-nrepl and refactor-nrepl are widely used today by many editors I think it’s really important to invest in documentation. That way client authors can focus on building great features for their users, instead of wasting their time figuring out how are they supposed to use some API. I doubt I’ll have time to tackle refactor-nrepl’s documentation any time soon, but anyone with enough desire to help out can easily replicate what I’ve done for cider-nrepl. Believe it or not - documenting APIs can be a lot of fun!

    That’s all I had for you today. See you tomorrow!

    1. You can read more those here

    2. Most notably - I had to exclude the built-in middleware prior to generating the output. 

  • RuboCop Meets Rake

    This article is part of the “Meta Advent 2019” series. I’ve committed to writing a new blog post here every day until Christmas.

    Another day, another (official) RuboCop extension. This time it’s focused on the legendary Rake.

    I’m happy to announce that for a while we (RuboCop’s team) have been working on rubocop-rake, which extends RuboCop with some extra knowledge of Rake. Currently it includes checks for things like duplicated tasks and namespaces, missing task descriptions, etc. A list of all cops is available here.

    Setting up rubocop-rake is really simple. Just add it to your .rubocop.yml and you’re good to go:

      - rubocop-other-extension
      - rubocop-rake

    By default all the cops are scoped to Rakefile and *.rake. I doubt you’ll ever have to change those defaults.

    Rake is a simple tool and I imagine that rubocop-rake will always remain a small extension. Still, any help with it would be appreciated, as there’s always room for improvement.

    That’s all from me for now. I hope you’ll enjoy rubocop-rake. I’m looking forward to hearing what you think about it. Until next time! Keep hacking!

  • The Ultimate Calendar for Geeks

    This article is part of the “Meta Advent 2019” series. I’ve committed to writing a new blog post here every day until Christmas.

    You know what’s the most important thing during the Advent? You’re totally right - that’s having a good calendar where you can constantly check the time left until Christmas. Today we’ve got all sorts of fancy calendars - GUI apps, web apps, mobile apps… I hate them all!1 Perhaps I’m just a bitter man stuck in the past, but I really prefer to interact with simpler and more hacker-friendly calendars. My favourite calendars are one that comes with Emacs2 and the command-line cal/ncal tool.

    1. Especially the built-in macOS calendar app. 

    2. You’re shocked, right? 

    Read More

Subscribe via RSS | View Older Posts