Posts
Projectile Introduces Significant Caching Improvements
One of Projectile’s key features has always been the ability to cache the files in a project, so next time need to do something with them you’d skip the potentially expensive project file indexing step. The feature has always worked reasonably well, but it suffered from a couple of design flaws that I’ve been meaning to address for a while:
- The cache for all projects is stored in the same file, which means that often you need to load information that you don’t really need. Also - storing all the cache data in the same file means that there’s a higher chance that the cache would get corrupted for one reason or another.
- When caching is enabled, Projectile loads the data from the cache file when
projectile-mode
is enabled. That causes two problems:- The mode’s init is slower than it could be
- The cache is not loaded unless you enable
projectile-mode
I’m happy to report that I finally got to doing something about the problems listed above!1 In a nutshell:
- Now each project gets its own cache file in the root of the project’s
directory. (named
.projectile-cache.eld
by default)2 I’ve opted for this approach over something like.emacs.d/cache
, as it avoids the need to hash the project paths. (so cache files could be mapped to project root paths) I might change this behavior in the future, depending on the user feedback I receive. - The cache is read from disk only when you trigger
projectile-find-file
for the first time. - Both reading and writing the cache file is much faster, as you’re only reading/writing the cache file for the currently visited project.
This means that Projectile’s loading time is much snappier if you’re a heavy cache user.
I’ve also addressed a common problem for people using Projectile over TRAMP - that writing to the cache file can be quite slow there if you’re working on huge project. The writing of the cache when individual files are added to it is now deferred using
run-with-idle-timer
3, which means that you’ll never experience a lock-up right after creating a new file within a project. Perhaps even more importantly - I’ve introduced the concept of “transient” cache. Basically it lives only in memory and is not persisted to a file.4 Going forward this is going to be the default caching behavior in Projectile, as I think it offers a better blend of performance and convenience. To enable the traditional “persistent” cache you’ll have to add this to your config:(setq projectile-enable-caching 'persistent)
To make things better I’ve also reorganized how known projects are initialized, so now no IO operations happen during Projectile’s initialization and it’s lightning fast! To illustrate this with a bit of code:
;; projectile-mode init before ;; ;; initialize the projects cache if needed (unless projectile-projects-cache (setq projectile-projects-cache (or (projectile-unserialize projectile-cache-file) (make-hash-table :test 'equal)))) (unless projectile-projects-cache-time (setq projectile-projects-cache-time (make-hash-table :test 'equal))) ;; load the known projects (projectile-load-known-projects) ;; update the list of known projects (projectile--cleanup-known-projects) (when projectile-auto-discover (projectile-discover-projects-in-search-path)) ;; projectile-mode init now ;; ;; nothing IO-related to show here
Last week I’ve started to prepare for the release of Projectile 2.9, which will bring a lot of bug fixes and improvements. I know that a lot of people have probably written off Projectile by now, but I still believe that it has a place in the broader Emacs ecosystem (as a community-driven and more feature-rich alternative of
project.el
) and the best is yet to come!That’s all I have for you today. Keep hacking!
P.S. I’d appreciate it if more Projectile users experimented with the current snapshot package, before Projectile 2.9 gets officially released.
-
See https://github.com/bbatsov/projectile/commit/5061bd8dcd9f4d0e874884272f88b10892d15da3. ↩
-
You’ll probably want to add this file to your
.gitignore
. Or not, as the paths stored in it are relative to the project’s root, so they can actually be reused across different machines. ↩ -
See https://github.com/bbatsov/projectile/commit/0efac68c82d8ebdb3fd83ea5e530c4b816c87f8f. ↩
-
See https://github.com/bbatsov/projectile/commit/a4a6cacd908bc614d60c6233a7f224baebfe1178. ↩
A Simpler Way to Deal with Java Sources in CIDER
For ages dealing with Java sources in CIDER has been quite painful.1 Admittedly, much of the problems were related to an early design decision I made to look for the Java sources only in the classpath, as I assumed that would be easiest way to implement this. Boy, was I wrong! Countless of iterations and refinements to the original solution later, working with Java sources is still not easy. enrich-classpath made things better, but it required changes to the way CIDER (nREPL) was being started and slowed down the first CIDER run in each project quite a bit, as it fetches all missing sources at startup. It’s also a bit trickier to use it with
cider-connect
, as you need to start nREPL together withenrich-classpath
. Fortunately, my good friend and legendary Clojure hacker Oleksandr Yakushev recently proposed a different way of doing things and today I’m happy to announce that this new approach is a reality!There’s an exciting new feature waiting for you in the latest CIDER MELPA build. After updating, try turning on the new variable
cider-download-java-sources
(M-x customize-variable cider-download-java-sources
and then toggle to enable it). Now CIDER will download the Java sources of third-party libraries for Java classes when:- you request documentation for a class or a method (
C-c C-d C-d
) - you jump to some definition (
M-.
) within a Java class
Note that
eldoc
won’t trigger the auto-download of Java sources, as we felt this might be harmful to the user experience.This feature works without
enrich-classpath
.2 The auto-downloading works for both tools.deps and Leiningen-based projects. In both cases, it starts a subprocess of eitherclojure
orlein
binary (this is the same approach that Clojure’s 1.12add-lib
utilizes).And that’s it! The new approach is so seamless that it feels a bit like magic.
This approach should work well for most cases, but it’s not perfect. You might have problems downloading the sources of dependencies that are not public (i.e. they live in a private repo), and the credentials are non-global but under some specific alias/profile that you start REPL with. If this happens to you, please report it; but we suspect such cases would be rare. The download usually takes up to a few seconds, and then the downloaded artifact will be reused by all projects. If a download failed (most often, because the library didn’t publish the
-sources.jar
artifact to Maven), CIDER will not attempt to download it again until the REPL restarts. Try it out in any project by jumping toclojure.lang.RT/toArray
or bringing up the docs forclojure.lang.PersistentArrayMap
.Our plan right now is to have this new feature be disabled by default in CIDER 1.17 (the next stable release), so we can gather some user feedback before enabling it by default in CIDER 1.18. We’d really appreciate your help in testing and polishing the new functionality and we’re looking forward to hear if it’s working well for you!
We also hope that other Clojure editors that use
cider-nrepl
internally (think Calva,iced-vim
, etc) will enable the new functionality soon as well.That’s all I have for you today! Keep hacking!
P.S. The State of CIDER 2024 survey is still open and it’d be great if you took a moment to fill it in!
- you request documentation for a class or a method (
State of CIDER Survey (2024)
TL;DR You’ll find the survey here.
It’s been a while since the first and only “State of CIDER” survey.1 Right after it happened in the end of 2019, the world went to shit and I kind of forgot about my intent to do the survey annually. 5 years later, it’s high time we get back on track!
CIDER has changed a lot in the past 5 years. nREPL has changed a lot. The development tooling Clojure ecosystem has changed a lot. Even Emacs has changed a lot. I’m guessing your usage of CIDER has changed a lot as well. If you’re still using Clojure and CIDER today, that is…
You can find the new survey here. We’ve made several updates to the questions - most notably there’s now a question about how frequently are you using certain CIDER features.
I’ve been thinking a lot lately that perhaps we went overboard with the feature-set in CIDER and that it might be prudent to trim the fat in CIDER 2.0, but we need data, so we can accurately know what’s being used and what’s not. I’ve also been thinking that mentioning many of the less known features in the survey might provoke some people to try them out. We’ll see.
There are other important questions that I like to get answers for, e.g.:
- Can we drop support for Emacs 26 (and maybe even Emacs 27)?
- Can we drop support for Java 8?2
- Do people think that CIDER improved in the last year?
- How often are people upgrading their CIDER installation?
- What problems are users encountering? What would they like to see added/changed/improved?
- Are people willing to support the project in one way or another?3
- Is Figwheel still a thing in ClojureScript?
Your responses will also be instrumental in shaping up the roadmap for CIDER 2.0.
Last time around we got 545 responses and I hope this time around we’ll do a bit better, as more responses mean that the data we got is more accurate. The new survey is off to a pretty slow start - less than 200 responses in the first week. It will likely stay open till the end of January, so I hope we are going to get 500+ responses in total this around time as well.
Thanks for taking the time to fill in the survey! This means a lot to me and our team! I’d really appreciate it if you shared the survey on your social media profiles, so more people see it and take part in it. Let’s make CIDER better together!
That’s all I have for you today. Keep hacking!
-
https://metaredux.com/posts/2019/11/02/state-of-cider.html ↩
-
Who am I kidding here… I’m well aware we’ll be stuck supporting it for all eternity… ↩
-
The financial support the project has received eroded quite a lot in the last couple of years, which was somewhat disappointing given all the work that went into CIDER and its ecosystem. ↩
RuboCop 1.62 Introduces (Experimental) Support for Prism
Recently I wrote that it was already possible to run RuboCop with Ruby’s new Prism parser, but that required a bit of manual work. I also outlined some plans to add built-in Prism support in RuboCop. 1 Today I’m happy to report that last week we’ve released RuboCop 1.62 which features (experimental) support for Prism!
Now using RuboCop with Prism is just a matter of adding Prism to your
Gemfile
(assuming you’re using Bundler, that is):gem 'prism'
and adding the following to your RuboCop configuration:
AllCops: ParserEngine: parser_prism TargetRubyVersion: 3.3
Magic! Initial benchmarks we’ve done suggest that using Prism results in about 70% speedup of the parsing speed, but I’d take those with a grain of salt as we’re very early in the process of optimizing RuboCop and
rubocop-ast
for Prism, plus the results you’d get would be strongly correlated with what you’re parsing.2 At any rate - it’s pretty certain that Prism is faster thanwhitequark/parser
.Why is the Prism support considered “experimental”? There a few reasons for this:
- Prism currently supports parsing only Ruby 3.3+, compared to
whitequark/parser
supporting Ruby 2.0+. This means Prism is still not a complete replacement forwhitequark/parser
in all use-cases. - There are a bunch of small issues with Prism’s
parser
translation that have to be fixed. In other words - for the time being this means that some cops won’t be working properly with Prism. You can monitor the list of outstanding Prism issues related to RuboCop here. - We might decide to make some changes to the configuration options. (although this seems unlikely)
I expect that as more people start to use RuboCop with Prism we’ll quickly identify and fix any outstanding problems. I’d encourage the more adventurous people to play a bit with RuboCop and Prism and report their findings. (e.g. issues discovered, benchmarks, etc)
RuboCop will keep supporting
whitequark/parser
for the foreseeable future, so there’s no rush for anyone to switch to using Prism today. Also, it’s not like we can stop supportingwhitequark/parser
until we switch to using Prism’s AST format natively - currently we’re using Prism as a backend forwhitequark/parser
.3That’s all I have for you today. Keep hacking!
- Prism currently supports parsing only Ruby 3.3+, compared to
Weird Ruby: The Double Aliased Enumerable Method
Ruby is famous (infamous?) for giving us many ways to do the same thing. One aspect of this is that quite a few methods in core classes have aliases, especially in the
Enumerable
module. E.g.:collect
->map
inject
->reduce
detect
->find
select
->find_all
Most Rubyists are probably aware that the “original” method names (
collect
,inject
, etc) were inspired by Smalltalk and later the aliases where added to reflect the naming trends in the functional programming community.Enumerable#select
is kind of special, as since Ruby 2.6 it has a second alias namedfilter
, thus making the only Ruby method I can think of in this elite category. As a reminder here’s how the methods behave:(1..10).find_all { |i| i % 3 == 0 } #=> [3, 6, 9] [1,2,3,4,5].select { |num| num.even? } #=> [2, 4] [:foo, :bar].filter { |x| x == :foo } #=> [:foo]
I think today
select
is the most popular of the 3, but I can imagine that over timefilter
will give it a run for its money.1The usage of aliases at this level has always been a bit controversial in the Ruby community. On one hand you make it easier for people coming from different backgrounds to find familiar methods, but on another you also confuse people a bit, as it’s not obvious that the different names are aliases. I’ve seen plenty of people asking what’s the difference between
inject
andreduce
and so on.What’s your preferred method to filter elements? Can you think of any other methods in the Ruby core classes that have multiple aliases?2 Please, share those in the comments!
That’s all I have for you today. Keep hacking!
P.S. As someone commented,
Hash#include?
is even more curious (weird?) as it has 3 (!!!) aliases:has_key?
key?
member?
Will we be able to top this?
Subscribe via RSS | View Older Posts