polysemy-http - Polysemy

I extracted some stuff from a project into a library for using http-client with semy:

https://github.com/tek/polysemy-http

I invite you all to shower me with criticism and provide suggestions for features and improvements!

hackage to follow soon.

NIce work Torsten, it looks like you've been pretty thorough!

I notice you've got your own polysemised co-log in there. Maybe it would be worth breaking that out into its own package?

I've done a similar thing in a project of my own, that I was going to break out into its own package as well.

I've taken a different approach, so it may be fun to compare.

I know there already is one such package - what do y'all think about it?

(a co-log polysemy package)

Mine uses co-log-polysemy, but puts some convenience and bells and whistles on top.

And integrates with formatting as well.

Here's the core of mine:

type WithLog' msg r = (HasCallStack, Member (Log msg) r)

type WithLog r = WithLog' (Msg Severity) r

log :: WithLog r => Severity -> Format (Sem r ()) a -> a
log sev m = withFrozenCallStack $
  runFormat m
    $ Colog.log
    . Msg sev callStack
    . TL.toStrict
    . TLB.toLazyText

logDebug :: WithLog r => Format (Sem r ()) a -> a
logDebug = withFrozenCallStack $ log Debug

logInfo :: WithLog r => Format (Sem r ()) a -> a
logInfo = withFrozenCallStack $ log Info

logWarning :: WithLog r => Format (Sem r ()) a -> a
logWarning = withFrozenCallStack $ log Warning

logError :: WithLog r => Format (Sem r ()) a -> a
logError = withFrozenCallStack $ log Error

logException :: (WithLog r, Exception e) => e -> Sem r ()
logException = withFrozenCallStack . logError string . displayException

ignoreLog :: (Applicative m, Member (Embed m) r) => Sem (Log msg ': r) a -> Sem r a
ignoreLog = runLogAction mempty

So I've stuck with co-log-polysemy's Log effect, and built the different severity functions on top of this. These are couple with Format from formatting, but you could do the same and still take Text. Torsten has defined his own Log effect (from https://github.com/tek/polysemy-http/blob/master/packages/polysemy-http/lib/Polysemy/Http/Data/Log.hs):

-- |An effect that wraps 'Colog.Polysemy.Log' for less boilerplate.
-- Constructors are manual because 'HasCallStack' is always in scope.
data Log :: Effect where
  Debug :: HasCallStack => Text -> Log m ()
  Info :: HasCallStack => Text -> Log m ()
  Warn :: HasCallStack => Text -> Log m ()
  Error :: HasCallStack => Text -> Log m ()
  ErrorPlus :: (HasCallStack, Traversable t) => Text -> t Text -> Log m ()

I'm not sure which of these approaches is better, if either is.

I've also got some stuff to add colour (if the terminal supports it), the time of the log message, and the thread id. Again though, mine is using formatting. I found it awkward to get it working with multi-threading via the Async effect, because if I ran the effects in the wrong order then I got my log messages jumbled, and if I got the log message time in the place where I interpreted the Log effect then the log message always had the thread id of the main thread, not the thread where the message was logged. In the end I needed to interpret things like this:

  logEnvStderr <- newLogEnv stderr
  runWithConfiguration cliInfo $ \config -> (do
    logInfo ("Version " % string) (showVersion
    version)
    logDebug ("Running with config:\n" % text) (pShow config)
    runCommand config
    ) & runFileIO
      & filterLogs (if config ^. scShowDebug
          then const True
          else msgSeverity >>> (<) Debug)
      & addThreadAndTimeToLog
      & asyncToIO
      & runLogAction (logTextStderr & cmap (fmtMessage
      logEnvStderr))
      & runContM
      & runM

You can see there that:

1. I filter the logs early, to (possibly) avoid overhead of debugging logs when debugging is disabled,
2. I call addThreadAndTimeToLog before calling asyncToIO,
3. I call runLogAction after calling asyncToIO.

Output looks like:
image.png
image.png

(sorry for the kind of derail :sweat_smile: )

Right, so getting back on topic... :)
Torsten, I see you have your effects in one place and your interpreters in another, e.g. Polysemy.Http.Data.Log and Polysemy.Http.Log. I tend to just define both the effect, interpreter, and other related stuff in the one module. I'm not saying I'm right, just wondering what the rationale, and tradeoffs are?

Alex Chapman said:

I notice you've got your own polysemised co-log in there. Maybe it would be worth breaking that out into its own package?

It's been a few months (I copied this Log from the project I first defined Http in) and I'm not quite sure what motivations I had, but I think it's mostly been that I wanted a terser api. I just copied the relevant parts from co-log-polysemy because they weren't exported. Not sure this warrants for a new package, though it would surely be convenient for myself, since I so far have three copies of this stuff in dependent projects, and it's only gonna grow :sweat_smile: I think the most sensible action would be to PR co-log-polysemyto make my _bells and whistles_ easier to add without making a bespoke effect.
Also I think it's very likely that there was just one tiny thing that annoyed me, like a weird looking compiler message, that ultimately pushed me to wrap the logDebug etc. combinators in an effect :see_no_evil:

Alex Chapman said:

Right, so getting back on topic... :smile:
Torsten, I see you have your effects in one place and your interpreters in another, e.g. Polysemy.Http.Data.Log and Polysemy.Http.Log. I tend to just define both the effect, interpreter, and other related stuff in the one module. I'm not saying I'm right, just wondering what the rationale, and tradeoffs are?

This is kind of an application of a more general rule I have, which is to put any data types into individual modules. I do that because it tends to confuse me a bit when files are large and mixed :halo: and also because of potential field name clashes (think TH lenses) with smart constructors etc. I admit that it seems pretty sensible for effects and interpreters, but I like to stick to those rules, they have grown organically. It's probably more significant in projects with a few hundred files than in this case.

for example I have a file Native.hs for the http-client interpreter and one Strict.hs for the in-memory interpreter. It's a personal taste question, I feel more organized that way. and since the effect data type is related to both those files, it would feel incorrect to me to put it in one of them.

btw, I'm struggling with cabal upload. So far I've only used stack, but now that I'm managing dependencies with nix exclusively, I'm trying it with bare-metal cabal, but it seems to be a bit tricky.

Packages using 'cabal-version: 2.0' and the autogenerated module Paths_* must
include it also on the 'autogen-modules' field besides 'exposed-modules' and
'other-modules'. This specifies that the module does not come with the package
and is generated on setup. Modules built with a custom Setup.hs script also go
here to ensure that commands like sdist don't fail.

is this an hpack problem? any ideas?

@Torsten Schmits how does generated .cabal look like?

@TheMatten https://github.com/tek/polysemy-http/blob/master/packages/polysemy-http/polysemy-http.cabal

I guess I would try to follow that suggestion and add generated-other-modules to polysemy-http-unit test suite

thanks, will try that! :heart:

yessss! thanks @TheMatten !!! :sunglasses:

oh btw. I added it to the library, not the test suite

So it didn't work in test suite? Or Cabal just doesn't care? :grinning:

I think the problem is that Paths_polysemy_http was not at all listed for the library. no idea why it was present in the test suite, I guess it's only added for executable components by default

(no idea what that module is about)

It let's you access some build-time info gathered from Cabal AFAIK - not sure why it's needed either, but it sounds like a pretty reasonable dependency given purpose

sounds plausible

any idea on how to hide the Paths_polysemy_http module from the documentation? I tried cabal v2-haddock --haddock-options='--hide Paths_polysemy_http' but it doesn't make a difference

ah, now it works. it's still listed, but there's no doc link. not sure that's better...

https://hackage.haskell.org/package/polysemy-http-0.1.0.0/candidate btw

published :tada:

yay, first issue!