nix and ctags - Haskell

Welcome to the Functional Programming Zulip Chat Archive. You can join the chat here.

Torsten Schmits

does anyone have a setup for generating ctags for the entire dependency tree when using nix in some way? like codex does for stack projects

Sandy Maguire

i do this: hasktags **/*.hs. not nix specific, but works OK

Torsten Schmits

you mean for the local files in the project?

Alex Chapman

I just have hasktags, but yeah that's only for files in the project. Would be amazing to have ctags for the whole dependency tree.

Ben Kolera

How would hasktags work there since you wont necessarily have the source around? Tried the local hoogle for a nix haskell env?

Ben Kolera

The project specific hoogle is pretty rad, modulo the shitty broken file links to the nix store.

Alex Chapman

I have tried the local hoogle when playing with Reflex, and that's pretty nice. But it would be so much better to have text editor support for jumping to the code. Maybe I want text editor support for hoogle.

Alex Chapman

I would be happy to have the source around... but I guess nix doesn't keep it in the package?

Ben Kolera

It would only be in the nix store if you built from source. If you got a cached version, your machine would never see the source.

Ben Kolera

I don't know of any haskell tooling that will fetch sources for you. Not like how java has the pretty widely accepted pattern of putting source jars in maven.

Ben Kolera

Other than cabal fetch/unpack, ofc, but that's no where near automated for a whole project.

Alex Chapman

It feels like it should be another flag to the nix haskell package library, like doBenchmark and dontCheck: includeSources

Ben Kolera

Source derivations are much deeper in the nix space than just the haskell library.

Ben Kolera

Would still be lovely though if you could get to the source locally from hoogle.

Torsten Schmits

I built something for generating tags:
https://github.com/tek/thax

Works pretty well. With 4500 dependencies it takes about four seconds on my machine to re-run the derivation for all packages with everything cached. Of course you could generate dependency tags only when they change and do only the local source on save.

Suggestions appreciated!

ctags generation for haskell projects with nix dependency management - tek/thax
Alex Chapman

@Torsten Schmits Awesome! Works for me. Thanks :)

Ben Kolera

That looks like an awesome thing to build into a lorri setup so that it's always there in direnv.

Alex Chapman

/me adds lorri to his list of things to play with

Alex Chapman

My problem now is that there are too many ctags. E.g. following a tag for Parser gets me 22 results to sort through.

Torsten Schmits

Alex Chapman said:

My problem now is that there are too many ctags. E.g. following a tag for Parser gets me 22 results to sort through.

as opposed to what? having no dependency tags or using a different tool like codex?

Alex Chapman

The problem is that ctags isn't smart enough to know what definition I actually want to look up. If I look up show, it finds hundreds of instances of show to show me. It doesn't do type inference, figure out exactly what type's show instance I want, then go to that source code. Of course I never expected ctags to be able to do this. Maybe what I want next is the ability to fuzzy-find within a ctags lookup when there are many options.

Alex Chapman

Yeah, I want fzf within the tag disambiguation interface. I already have fzf for searching through all tags, via fzf.kak, but not for searching through the different options at the other end of the tag.

Alex Chapman

Kakoune is the best thing since sliced vim :)

Ben Kolera

I miss all of the shininess in spacemacs though. I need to recreate kakoune interactions (theres multicursors and a modal editing lib) in emacs so I can have the best of both worlds. :slight_smile:

Torsten Schmits

Alex Chapman said:

It doesn't do type inference

Your problem is that all our language servers are crap :slight_smile:

Torsten Schmits

Alex Chapman said:

Yeah, I want fzf within the tag disambiguation interface. I already have fzf for searching through all tags, via fzf.kak, but not for searching through the different options at the other end of the tag.

:Tags show does give you that functionality, unless I'm misunderstanding something

Alex Chapman

:Tags show does give you that functionality, unless I'm misunderstanding something

Which editor is that in?

Torsten Schmits

with the plugin fzf-vim

Alex Chapman

I guess I need to do some tinkering with my kakrc

Torsten Schmits

ah, you're using Kakoune as your primary editor

Torsten Schmits

did you use vim before?

Alex Chapman

It's awesome. I did use vim previously. I find that kakoune is to vim as vim is to all other editors, in terms of the editing language.

Torsten Schmits

ugh crap. I don't want to spend the next six months migrating my shit to a new editor :smiley:

Torsten Schmits

how do you write plugins in kakoune? like rplugins in neovim

Alex Chapman

I haven't written my own, but from what I can tell it's mostly about hooking shell commands into the editor. E.g: https://github.com/andreyorst/fzf.kak/blob/master/rc/fzf.kak

FZF for Kakoune. Mirror of https://gitlab.com/andreyorst/fzf.kak - andreyorst/fzf.kak
Torsten Schmits

looks like there is no support for plugins running in a subprocess with RPC, right?

Alex Chapman

I don't know. There's support for connecting to an lsp server, if that helps?

Alex Chapman

@Torsten Schmits I now use thax-generated tags hundreds of times a day. Thanks again for making it! I can use the same shortcut to jump to my own source code or to the source code of any Haskell function I use.

Torsten Schmits

@Alex Chapman that's wonderful! any suggestions for improvements?

I'm quite unhappy with the store hash in the path, but that's more of a presentation problem, (neo)vim's tag menu is not very configurable

Alex Chapman

Well my only annoyance is that to update the tags from my own project I need to run the whole tag process again, which takes around 30 seconds, so I avoid doing it too often, and so I can't look up my recently written functions.

Torsten Schmits

30 seconds seems steep, for me it takes about 5 seconds. is your dependency tree that huge? I think that the project I've just tested it on is reasonably large. is that an SSD?

Torsten Schmits

I trigger it whenever I save

Alex Chapman

I haven't tried to keep the dependency tree small, so it probably is pretty huge, though I don't have Pandoc at least. It does take at least 25 seconds, every time, and yes this is on an SSD. The tags file is 23M.

Torsten Schmits

hm, my tags file is 32M…what's your cpu?

Torsten Schmits

wonder how we could profile this

Alex Chapman

https://ark.intel.com/content/www/us/en/ark/products/97496/intel-core-i7-7820hq-processor-8m-cache-up-to-3-90-ghz.html

Intel® Core™ i7-7820HQ Processor (8M Cache, up to 3.90 GHz) quick reference guide including specifications, features, pricing, compatibility, design documentation, ordering codes, spec codes and more.
Torsten Schmits

when you run the derivation, do the 30 seconds pass entirely while nix is evaluating, or does it output what it is tagging?

Alex Chapman

Oh, it looks like the majority of the time is spent building my project. If I don't change any of my files then it only takes 3.7s.

Alex Chapman

So I guess I need my projectTags target to not include my own package, so I can run hasktags on my project without needing to compile it. I have it set up in my default.nix like this:

  pkg = modifiedHaskellPackages.callCabal2nix pkgname ./. { };
...
        projectTags = tags.combined.all { targets = [pkg] };

Is there a way I can turn pkg into a list of its dependencies, without including itself?

Torsten Schmits

by "building my project", do you mean compiling haskell?

Torsten Schmits

if so, that's wrong, it should only copy the sources and look at the nix deps, not realise the derivation

Torsten Schmits

the optimal case should be that hasktags is run on the current directory and the dep tree is checked for changes by nix

Alex Chapman

Yeah, it's compiling Haskell, because I gave the list of targets as a list containing only my package. Otherwise I would need to manually specify all of my dependencies.

Torsten Schmits

that's the intended way to use it, but it should not build the package. are you sure there's nothing else that could cause this? the tool only looks at the derivation's attributes to get the sources and dependencies.

Torsten Schmits

so I tested it like this:

let
  pkgs = import <nixpkgs> {};
  tags = import (fetchTarball "https://github.com/tek/thax/tarball/9ac46dfef0a99a74e65c838a89d0bbab00170d8b") {};
  p = pkgs.haskellPackages.callCabal2nix "thax-test" ./. {};
in {
  projectTags = tags.combined.all { targets = [p]; };
}

and the project isn't compiled when I run

nix-build -A projectTags

:shrug:

Torsten Schmits

maybe try this default.nix with a .cabalfile and one source file and see what happens, so that we can be sure it's not something external

Alex Chapman

It looks like it does build the package even in a simple test with nothing changed after a cabal init except the addition of your default.nix from above:

λ ~/src/test/ time nix-build -A projectTags
these derivations will be built:
  /nix/store/q4ci8784lmp9ic4nysf0hyxj0dvngww7-test-0.1.0.0-tags.drv
  /nix/store/3k1bqhlk2w2l8jhqzbz9gx1yiqbdaa4y-project-tags.drv
building '/nix/store/q4ci8784lmp9ic4nysf0hyxj0dvngww7-test-0.1.0.0-tags.drv'...
unpacking sources
unpacking source archive /nix/store/3frh2am7mzdk3qgshmlsy26c4hfnnryp-test
source root is test
building
building '/nix/store/3k1bqhlk2w2l8jhqzbz9gx1yiqbdaa4y-project-tags.drv'...
building
/nix/store/hyvqvkaya51kfsbj7cla5vb64jqjfprx-project-tags
nix-build -A projectTags  1.31s user 0.18s system 8% cpu 17.193 total
Alex Chapman
λ ~/src/test/ nix-build --version
nix-build (Nix) 2.3.3
λ ~/src/test/ cabal --version
cabal-install version 3.0.0.0
compiled using version 3.0.0.0 of the Cabal library
Alex Chapman

I'm not supposed to run it from within nix-shell am I?

Alex Chapman

(doing so doesn't seem to help)

Torsten Schmits

how do you figure it is built? there's no output referencing any haskell files

Alex Chapman
these derivations will be built:
  /nix/store/q4ci8784lmp9ic4nysf0hyxj0dvngww7-test-0.1.0.0-tags.drv
Torsten Schmits

well, that's the tags derivation

Alex Chapman

Oh, I see. Well then why is it slow? :/

Torsten Schmits

is it also slow if you use combined.packages?

Alex Chapman

Where should I use that? In the targets list?

Alex Chapman
λ ~/src/test/ time nix-build -A projectTags
these derivations will be built:
  /nix/store/ny62bvn7b5g35m6j5h821098h7fala3a-test-0.1.0.0-tags.drv
  /nix/store/v6dvswnhb0ydpcifv2nskl7balp76wzg-project-tags.drv
building '/nix/store/ny62bvn7b5g35m6j5h821098h7fala3a-test-0.1.0.0-tags.drv'...
unpacking sources
unpacking source archive /nix/store/h44kigiqmvsir82iwb3bv0pmrdji9m2s-test
source root is test
building
building '/nix/store/v6dvswnhb0ydpcifv2nskl7balp76wzg-project-tags.drv'...
building
/nix/store/3jzlz7hnmmjsa7jmgqk9ij4gfrac67vj-project-tags
nix-build -A projectTags  1.38s user 0.18s system 8% cpu 18.188 total
λ ~/src/test/ time nix-build -A projectTags
these derivations will be built:
  /nix/store/vyfcwyfia4n3r9f1sgmmlfbpbpzj44h2-test-0.1.0.0-tags.drv
  /nix/store/bk2fhhysakxazy809xcflxjzwgpss32c-project-tags.drv
building '/nix/store/vyfcwyfia4n3r9f1sgmmlfbpbpzj44h2-test-0.1.0.0-tags.drv'...
unpacking sources
unpacking source archive /nix/store/9ydv2j4yh0ng79gj7w4qpyf2g9y3m67k-test
source root is test
building
building '/nix/store/bk2fhhysakxazy809xcflxjzwgpss32c-project-tags.drv'...
building
/nix/store/xlg4fr4kf0v0x275z6fkml8q10hyvpni-project-tags
nix-build -A projectTags  1.33s user 0.17s system 8% cpu 16.877 total
Alex Chapman

Why do all the hashes change despite me not changing anything?

Torsten Schmits

projectTags = tags.combined.packages { targets = [xyz]; }

Torsten Schmits

Alex Chapman said:

Why do all the hashes change despite me not changing anything?

yeah that is worrying

Alex Chapman

Faster with combined.packages:

λ ~/src/test/ time nix-build -A projectTags
these derivations will be built:
  /nix/store/a0xhi7b18wlmwn33njxdbr2sm4dsmakx-test-0.1.0.0-tags.drv
building '/nix/store/a0xhi7b18wlmwn33njxdbr2sm4dsmakx-test-0.1.0.0-tags.drv'...
unpacking sources
unpacking source archive /nix/store/b510xvfch2lygwv72m2skl2cilrkjprz-test
source root is test
building
/nix/store/xwnps3ydyml8g5ycq13b1sn5m3k960fj-test-0.1.0.0-tags
nix-build -A projectTags  1.26s user 0.18s system 13% cpu 10.618 total
λ ~/src/test/ time nix-build -A projectTags
these derivations will be built:
  /nix/store/p1lw7lzlh50sbaxgs5dr48748mdqkpsf-test-0.1.0.0-tags.drv
building '/nix/store/p1lw7lzlh50sbaxgs5dr48748mdqkpsf-test-0.1.0.0-tags.drv'...
unpacking sources
unpacking source archive /nix/store/cmhw5hk4c3rvz6i67xbw643il54xifdw-test
source root is test
building
/nix/store/y534ck4pd40bzw3i2k3z111j6ji1c7a5-test-0.1.0.0-tags
nix-build -A projectTags  1.27s user 0.17s system 16% cpu 8.761 total
Torsten Schmits

I see now, this also happens for me, but only in the project where the package is in the root dir

Alex Chapman

I'm wondering whether the result symlink to the built package is part of the hashed package, so because it changes every time, the resultant package's hash changes every time...

Torsten Schmits

right, try it with --no-link

Torsten Schmits

yup, that fixes it for me

Alex Chapman

Yeah, much faster after the first build.

Torsten Schmits

how do you integrate the generated file? do you not use the command from the readme?

Alex Chapman

I use cp $(nix-build --no-link -A projectTags)/tags tags

Alex Chapman

So now if I change Main.hs, it takes longer to build again, ~10s, then back to ~2s on the next run.

Alex Chapman

Which would suggest it's compiling the package.

Torsten Schmits

since it's checking all the dependencies again

Torsten Schmits

but you could use combined.packages and then manually merge it with the file for the deps that can be generated with combined.deps

Torsten Schmits

I call it in the background from vim whenever I hit my "save all" mapping, so I don't really notice that there's a delay

Torsten Schmits

but if that's not your workflow, we can come up with something for the tool that's accomodating

Alex Chapman

Oh ok, I was thinking that yours took ~5s regardless.

Alex Chapman

Running it in the background would be fine, let me see if I can get kakoune to do that...

Torsten Schmits

well it's not taking 30 seconds, more like 8

Torsten Schmits

the project I've tested this on has its source files split across 14 packages, so not everything is retagged on every save

Torsten Schmits

maybe run hasktags manually to see how long that takes

Alex Chapman

(on just my own source files)

Torsten Schmits

right, my deps are also split, so the subtree that's examined on a change is smaller, maybe that's a reason

Torsten Schmits

I'll try to think of some way to avoid the deep dependency rechecking