Haskell/nix/HLS setup - Haskell

I am trying to figure a development workflow for Haskell packages using nix, emacs, hls, lsp...
I am struggling with the nix part and esp. setting up a nix-shell following the instructions here: https://input-output-hk.github.io/haskell.nix/tutorials/development/#how-to-get-a-development-shell
I have hoped from links to links on the internet, trying to form a clear mental picture of how those things interact but I am still pretty confused. What are resources that go beyond code snippets I could tap on?

nix is used to install dependencies.
the code in you default.nix or shell.nix is supposed to use the cabal2nix tools to analyze your cabal files and translate it to nix packages.
when you run nix-shell, all haskell packages are placed in the environment so that ghc can find them.

so if you start a shell with nix-shell, you can run, for example, ghcid and all the dependencies are there.
you can use nix-shell --run haskell-language-server-wrapper to run a single command with the full environment, like from your emacs language server config.

Thanks @Torsten Schmits I know what nix is used for and get the general idea of how to use it, but there's a lot of specific parts and details I don't get, and trying to follow the referenced link I ran into errors which I don't know how to troubleshoot. Hence my search for a detailed walkthrough.

can you provide some concrete errors?

Sure. So what I am trying to do is take an existing non-nixified package and add the needed nix bits to 1/ build it with nix and 2/ have all the needed dev tools (HLS, ghcid, hoogle, hlint, ormolu...) set up, in order to 3/ hack happily with emacs.
copy-pasting with minor adaptations the default.nix and shell.nix from https://input-output-hk.github.io/haskell.nix/tutorials/development tutorial, when I do nix-shell I run into the following error:

error: attribute 'haskell-nix' missing, at /home/curry/hydra-sim/default.nix:15:4

that means either that the haskell-nix nixpkgs are broken, the tutorial is outdated or your minor adaptations changed something that causes the haskell-nix attribute to be absent. can you post your nix file?

1. do you call this file with arguments?
2. if not, does it work with haskellNix.sources.nixpkgs?

I don't even understand the question :-D
What I did was:

nix-build -A hydra-sim.components.exes.hydra-sim

and I got a compilation failure which is probably a problem with the package I am trying to build and that I can solve.
Then I tried

nix-shell

and got the error I mentioned.
And now I am trying to run nix-shell -A shellFor without a shell.nix file and it is building a whole slew of packages and taking ages. I miss stack...

ah. can you post the shell.nix as well?

Sure, here it is

# shell.nix
{pkgs ? import <nixpkgs> {} }:

let
  hsPkgs = import ./default.nix { inherit pkgs; };
  haskellNix = import (builtins.fetchTarball https://github.com/input-output-hk/haskell.nix/archive/master.tar.gz) {};
in
  hsPkgs.shellFor {
    # Include only the *local* packages of your project.
    packages = ps: with ps; [
      hydra-sim
    ];

    # Builds a Hoogle documentation index of all dependencies,
    # and provides a "hoogle" command to search the index.
#    withHoogle = true;

    # You might want some extra tools in the shell (optional).

    # Some common tools can be added with the `tools` argument
    tools = { cabal = "3.2.0.0"; hlint = "2.2.11"; };
    # See overlays/tools.nix for more details

    # Some you may need to get some other way.
    buildInputs = with pkgs.haskellPackages;
      [ ghcid
        # (haskell-language-server.override { supportedGhcVersions = [ "884" "8102" ]; })
      ];



    # Prevents cabal from choosing alternate plans, so that
    # *all* dependencies are provided by Nix.
    exactDeps = true;
  }

I tried adding stuff for HLS but commented it out for now

{pkgs ? import <nixpkgs> {} }:

the problem is this, which you pass on to default.nix, overriding the default (after the ?) there. that replaces the nixpkgs from haskell-nix with the system default.

change it to

hsPkgs = import ./default.nix {};

and it should be fine.

I have this line inside the let block so I assume you meant I should simply remove the declaration? But then when I do that, I got another error:

error: undefined variable 'pkgs' at /home/curry/hydra-sim/shell.nix:23:24

no, no line to remove. just don't pass pkgs on to the import of default.nix.
and I don't see where you use pkgs in shell.nix:23

Ah I see. It's used for buildInputs

ah, yes

then you would have to use hsPkgs there too

Yeah, thanks a lot @Torsten Schmits

(I was AFK for a while) So I am into nix-shell and can access installed tools. However, _entering_ the shell is very slow which makes me wonder if this will be bearable when running commands from HLS, or even calling HLS wrapped from emacs. I am missing some important step to make this fast? There is this "materialized" concept which seems to be relevant but unsure about it.

it takes about one second for my haskell projects, how slow is it for you?

what do you mean by "running commands from HLS"?

Well, it takes about 1 minute or so, with some trace messages that might explain the slowness.
I mean nothing because of course HLS will be run from within nix-shell. But what about running ormolu, or ghcid, or cabal test?

one minute is alarming, if this happens on consecutive executions with no changes in between.
if you run ormolu through HLS, there should be no delay, since HLS is already running in that env.
if you run ghcid or cabal test manually, this is a concern for sure.

one reason for stuff being recalculated every time could be some kind of self-reference, like using the project directory in an inappropriate way, but it's hard to say

there are warnings at startup about missing sha256 which might be cause of slowness, I will investigate a bit more. At least, I can drop into a nix-shell and it works, I will see what happens when configuring a wrapper in emacs
I must admit I have always refrained from using nix before and am a total noob, even though I am understand the concept and the basic principles.

is there substantial output on every invocation? if so, post it!

I will! I am now upgrading the compiler to ghc8103 so it will take a while :)

I guess I am not hitting IOHK's binary caches anymore...

well compiling ghc should take an hour :sweat_smile:

I have not done that for years!

How can I fix the version of HLS I want to use? I tried this

    buildInputs = with pkgs.haskellPackages;
      [ ghcid
         (haskell-language-server.override { supportedGhcVersions = [ "884" "8102" ]; })
      ];

but I got an error saying supportedGhcVersions was not a "parameter"

is that from the docs?

From here: https://haskell4nix.readthedocs.io/nixpkgs-users-guide.html?highlight=language%20server#how-to-install-haskell-language-server

ah, that is for a global installation from the default nixpkgs derivation. when you're using haskell.nix, you've already pinned ghc to a version and that is what's going to be used for HLS

(I assume)

ah yes, compiler-nix-name in your first snippet

Well, that's what I would expect too, but then when I add only haskell-language-server it installs hls-8.10.3 even though the compiler is ghc8.8.4 (hence why I upgraded ghc)

don't know what exactly haskell.nix injects into the pkgs set you're using there though. haskellPackages usually points to the default set for nixpkgs, which might be 8.10.3.

instead you should be using the package set that haskell.nix provides. let me check the docs

oh, maybe just try using hsPkgs.haskell-language-server

or project.ghc or something

I find it pretty weird that the example tells you to use import <nixpkgs> {}

I have my own little world for haskell dev btw: https://github.com/tek/tryp-hs

it's very specific to my own needs, but maybe you can find some inspiration :sweat_smile:

Thanks, definitely will have a look!

Also don't forget to set the lsp-haskell-server-wrapper-function emacs custom so that lsp-mode picks the right one

Yes, I have that on my radar, thanks @tristanC

On another note, I am surprised that packages got installed sequentially and not in parallel, as stack does. I spun up a VM with 16 cores to speed build and, well, I would expect faster turnaround :)

they usually are installed in parallel

:thinking: that's not what it is currently reporting to me, is there something to configure?

there's -j to specify the concurrency

A more general question: What are the advantages of using nix over using stack, or newer cabal which seems also to support well-isolated toolchain and packages? Assuming I am not _already_ using nix of course

I mean, for haskell development specifically.

Cached libraries for one iirc

ah, the default for max-jobs in nix.conf is 1, and that's what -j16 would override

Cabal caching over CI should work but doesn't without messing with timestamps but nix didn't have that issue last I checked

Nix alone loses incremental building though, unless you use nix just to bootstrap a shell with ghc, cabal, and packages

Ah yes, awesome @Torsten Schmits !

@codygman by caching you mean global caching, for example to share binaries across a team? Libraries are also cached by stack when working on a common stackage version.

Libraries are cached locally by stack, but you have to build them first. Correct me if I'm wrong, but you don't have to compile Haskell libraries pulled from cache.nixos.org

you can also create a bincache for your team

Well, perhaps. I am following this tutorial https://input-output-hk.github.io/haskell.nix/ and so far it has built a lot of libraries so perhaps it's not using the same cache? According to this section https://input-output-hk.github.io/haskell.nix/troubleshooting/#why-am-i-building-lots-of-haskell-packages it's expected.

it's very helpful if you, for example, work with code from git repos instead of stackage

you can add lots of cachixes from other people to your nix setup

I am interested in setting up team-wide cached binaries, this has been a major PITA in past projects as everyone and every machine kept rebuilding everything

but that requires you to use the same version of nixpkgs

if you're very disciplined with versions (nixpkgs especially), this can speed up builds massively, @Arnaud Bailly

myself I use a free tier cachix and push all my built projects regularly, and I barely have to rebuild anything on any of my machines

If you use a different version of ghc and lots of custom libraries you'll want to use cachix. You'll have to build once to populate cachix of course.

Basically it results in "only one person on your team has to build library dependencies" just like I'm guessing you could do with stack+bincache Torsten recommended

including dependencies

codygman said:

stack+bincache Torsten recommended

I meant nix bincache!

cachix looks neat!

for something entirely local to your network, you can use nix-copy-closure to copy the entire tree of dependencies to a central host, then add this host as a substituter or even remote build host

only needs ssh access

It is! I even use cachix in github actions to build my local machine libraries so they can spend cpu time

sneaky!

I tried the following

    buildInputs = with pkgs.haskellPackages;
      [ ghcid
        (hsPkgs.haskell-language-server.override { supportedGhcVersions = [ "884" "8102" ]; })
      ];

and now I have the following error:

error: attribute 'haskell-language-server' missing, at /home/curry/hydra-sim/shell.nix:27:11
(use '--show-trace' to show detailed location information)

yeah I already tried sifting through the haskell.nix code to figure out what exactly is in hsPkgs, but didn't find enlightenment. I'd suggest trying hsPkgs.ghc.haskell-lang...

and with this method, you can remove the .override { ... part

thanks to nix being untyped we have no simple way of inspecting what that set is :sweat_smile:

you could try nix eval 'builtins.attrNames (import ./default.nix {})' or something

or go to nix repl and do project = import ./.

then just project and you'll see a bit

thanks for your help @Torsten Schmits !

Another question about Nix and Haskell. When initialising a package set I get the following error message:

unpacking 'https://github.com/input-output-hk/haskell.nix/archive/master.tar.gz'...
unpacking 'https://github.com/input-output-hk/hackage.nix/archive/8ab47a84664f99da20ee5b956851a2431b9073d7.tar.gz'...
warning: dumping very large path (> 256 MiB); this may run out of memory

Should I be worried and if yes, what am I doing wrong?

So, I managed to have a nix setup working and replicate that to "bootstrap" a simple project. I am still stuck at the haskell-language-server stage, I don't know how to set the correct GHC Version to use (8.10.2), it insists in installing 8.10.3. Here is my shell.nix file:

# shell.nix
{pkgs ? import <nixpkgs> {} }:

let
  hsPkgs = import ./default.nix { };
  haskellNix = import (builtins.fetchTarball https://github.com/input-output-hk/haskell.nix/archive/master.tar.gz) {};
in
  hsPkgs.shellFor {
    # Include only the *local* packages of your project.
    packages = ps: with ps; [
     okiwi-aoc
    ];

    # Builds a Hoogle documentation index of all dependencies,
    # and provides a "hoogle" command to search the index.
#    withHoogle = true;

    # You might want some extra tools in the shell (optional).

    # Some common tools can be added with the `tools` argument
    tools = { cabal = "3.2.0.0"; hlint = "2.2.11"; };
    # See overlays/tools.nix for more details

    # Some you may need to get some other way.
    buildInputs = with pkgs.haskellPackages;
      [ ghcid
        haskell-language-server
      ];

    # Prevents cabal from choosing alternate plans, so that
    # *all* dependencies are provided by Nix.
    exactDeps = true;
  }

    buildInputs = with pkgs.haskellPackages;
      [ ghcid
        haskell-language-server
      ];

you're still using pkgs here, which is the global nixpkgs of your system environment.
you need to use what's in hsPkgs.

Arnaud Bailly said:

Should I be worried and if yes, what am I doing wrong?

that's normal; always happens when it unpacks a nixpkgs snapshot.

Yes, but replacing pkgs with hsPkgs simply does not work. I will fiddle with it a bit more, reading @Gabriel Gonzalez's tutorial to make sense of nix :)

the problem is:
haskell.nix starts out with a "vanilla" nixpkgs, which is obtained by importing the snapshot in default.nix. then it modifies that snapshot, especially the haskellPackages, to include the tools the library provides, and creates an API set when calling the project function.
in this API set (assigned to hsPkgs), there exists a haskell package set that corresponds to the compiler you selected with that one option. this is what you need to find within hsPkgs. maybe hsPkgs itself is that set, so you could try using with hsPkgs instead of with pkgs.haskellPackages.

otherwise you can try my nix repl suggestion and dig around in hsPkgs manually.

I have not used this much because I haven't done much haskell, but I have reason to believe this is a good tool for haskell on nix https://gitlab.com/fresheyeball/snowball

looks good

Thanks again for your help. I will need to stick to haskell.nix as the code I am working on is part of the iohk ecosystem but thanks for the suggestion @Mason Mackaman

So I ended up posting an issue on haskell.nix repo and the solution was to add haskell-language-server = "0.8.0" to the tools attribute set. :rolling_eyes: I will try to find a way to document that somewhere.

@Torsten Schmits finally managed to get a working nix/emacs/haskell/lsp env, after much struggling :)

Thanks a lot for your help

awesome!

Just to suggest an alternative here. I had a relatively painless environment setup using the haskell docker container paired with vscode + vscode container extensions. It was a really seamless experience for me

Ÿ

Thanks for the suggestion @Vance Palacio
I really wanted to go down the nix route and see how to configure a working dev environment for Haskell/Emacs/LSP using it. What I "normally" do is configure everything through scripting inside a disposable VM which completely obviates the need for nix, unless of course when one is using NixOS which I don't.