When do you reach for an effect system - Haskell

We were discussing effect systems in Haskell on the video chat over the weekend.

I and some others had mentioned that we haven't been using effect systems in our applications. We've been getting by fine with IO, refs, and the like. In my case it's because my system is deployed on AWS Lambda and so my programs are small by necessity and I haven't felt a need for such a library.

I have experimented with them for my side projects but I've often heard people say they are complicated to learn and I was curious where the cliff happens in your project: when do you reach for an effect library? What's the trade off?

it's useful if you want to reinterpret the same computation in multiple contexts. e.g. for testing, or for visualization

and also if you're program is big and you want to test small parts (different effects) of it?

I also enjoy these two "cosmetic" benefits:

• your "business logic" is "cleaner" - you're not talking about http, file systems, etc
• each function plainly states what things it can do in its type signature - it's easier to take a quick look and imagine what a function might be doing vs an all encompassing IO

I guess it's also harder to reach for an effect that you shouldn't be doing in the particular place where you are at:
with IO you can do whatever effect you want, whereas when you have to write it out in your type signature you necessarily have to think twice before using some effect

but tbh I'm also not sure about the benefits and realities of these things - I'm a "very inexperienced" dev (~1yr professionally), so I would also love to see some concrete examples where effects have greatly helped with e.g. testing

Right, I'm trying to find out what the tipping point is in terms of "large" or "complex" where you would say, "Okay this is getting too hard to manage with IORefs and ReaderT, I need something more sophisticated."

The reason being that I think most people find effects difficult to understand when they're new to the Haskell ecosystem and putting them out there as a best practice should probably come with some kind of guideline about the complexity trade off, no?

Otherwise we may end up trapping some people in a long hurdle before they can even begin writing useful Haskell programs, right? For context I recall getting tripped up when trying to add logging and not having yet learned transformers and finding that a steep hill I just climbed was only the foot of the mountain. :)

I wonder if our advice should be like, "for this large of a project, stick with IO and IORefs" and "for projects larger than that consider effect libraries"

IMO io and iorefs are always bad solutions

that's not to say they shouldn't be used

but that they _need_ an abstraction boundary around them

it's ok if my MemoryManagement module uses iorefs, but if i have inlined that functionality everywhere else in my codebase, i have commited a true crime against the future maintainability of my software

effect systems are an OK-to-bad solution to "i want an abstraction boundary but i dont want to work too hard for it"

they don't prevent you from shooting yourself by making a bad abstraction

using io/refs is bad not because "io is fundamentally evil" but because it doesn't give you any way of changing that decision later

people fall into this alllll the time. as soon as you have some ioref in your RIO pattern, you end up seeing shit like this everywhere:

do
  config <- ask
  let ref = _myIORef config
  modifyIORef ref whatever

the problem is that this conceptual "i want to change ref" is now split up into three commands, two of which assume a great deal about the context they're running in, and the third assuming a great deal about how to get myIORef

by working directly in IO here, not only are your concepts diluted, but also you are just asking people to say "oh, you're already in IO here? i want to do some IO. i'll do it here"

and the second that happens, you can no longer change your mind about how these things are implemented

by leaking the fact that your state management happens in IO, you are tempted to do other IO there because it's convenient, and now the implementation detail of IO is part of the public interface

:100:

i guess this is a little off topic from the original prompt --- my new book is exactly on this topic of finding abstraction boundaries, so i'm a little primed :)

to answer the original question it's "always build an abstraction boundary, even for single module IO programs." that abstraction doesn't need to be big

avoid monadicness if you can. if you can't, look harder at trying to avoid it. if you REALLY cant, make a newtype monad around IO, don't give it a name that suggests it should be used everywhere in your app, and don't give it any ability to directly lift IO

i like to both not export the ctor, AND to give a WARNING pragma on the ctor so that even if it does get exported, people get a warning if they ever use it

in general, i think the problem is that we all forget about functional programming as soon as we get our hands on IO. the bad, old procedural urges come back and we write the same software we would have in PHP

So no program is too small for modeling effects? :thinking:

I might try giving it another go. For this system I'm building at work it's all AWS Lambda's which are conceptually and purposefully limited to a single function.

I do have a side project where I'm building an in-memory kv database where I'll probably do some more serious experimentation with it.

I'm getting increasingly confused what an "effect system" is meant to designate. At least as far as Haskell implementations are concerned, what more is there to the term than "another way to organize interfaces"?

@Lysxia Many people seem to use the term "effect" to mean many different things. From my rather limited perspective an "effect" is an algebra of some signature on a type m a, for an arbitrary monad m (or an "interface" on a monad as you say)

I mean, I would call this example from FCI "effect system" too:

class Monad m => Teletype m where
  read  :: m String
  write :: String -> m ()

mkInst ''Teletype

hello :: Teletype m => m ()
hello = do
  write "What's your name?: "
  name <- read
  write $ "Hi " ++ name ++ "!\n"

main :: IO ()
main = Teletype inst getLine putStr ==> hello

Even though it's literally just "explicit dictionaries", reusing stuff from typeclass resolution.

To me it seems that all these patterns and libraries boil down to "I want to parametrize code over specific implementation of some of it's parts", in an composable way.

@TheMatten what's not apparent in mtl (but is once you use something like your FCI library) is that the underlying mathematics with the free monad stuff and mtl is the same

i'm not sure what's special about the "monad" part of the "signature functor on monads" though. for example why is a monoid interface not an effect?

maybe it has to do with the fact that monads lend themselves to writing code in a familiar imperative syntax

That last point may be the thing - other languages may swap monads for e.g. abilities and handlers, compare them to e.g. exceptions and still call them effects

The one advice is that you can very much make a lot of mistakes (including things that were correct decisions that turned out to be mistakes) in where you draw the lines (especially wrt performance) so you really need to optimise for your code to be easy to understand and change later. Especially when you're in more of a discovery phase and don't really know all the answers yet.

The codebase that I currently work on is orders of magnitude slower than it needs to be processing the data that it does and is an order of magnitude harder to change than you expect because there is so much incidental complexity built up over many years of dev on a big codebase (with lots of abstractions lines which at least now are questionable) from lots of different people. It's lovely and testable but it still sucks so hard the company has given up on haskell and rewriting it all in go. :slight_smile:

Now there's an sensible argument to be made that having your code as pure as possible and as testable as possible so that you can have a chance to keep up with a fast moving product trajectory, but drawing the wrong lines in your system too early can kill you just as much. It's just a different ball of mud.

This is largely a people management thing. It's doco; it's making sure that devs actually understand the product goals and aren't just being shovelled crap; it's making sure that everyone knows that discovering the product is a learning process that requires change over time but should still minimise chaos along that journey; it's about being aware of when you are eating tech debt and critically looking at the abstractions (or lack thereof) that are causing it. All of those things matter in building a product, and I'm always suspicious of getting the programming in the small stuff too nailed down before you've got the product engineering part sorted.

I guess my tl;dr is that "no amount of haskell will save you if your product pipeline sucks, and over abstracting too early can definitely hurt you while you aren't clear on the boundaries and reqs yet. So please be careful under abstracting and having shitty imperative haskell or over abstracting to the point you draw lots of layers that don't match reality anymore . Both can be impossible to work with later on! "

Really great comment Ben. Cheers.

I would reach for eff (or a transformer stack) if I had non-determinism effects with complex interactions with other effects. For example: nondeterminism with global search state, but also local per-branch state. Otherwise I would stay with a RIO-like solution. If depending on IO so directly started bothering me, I would try hiding it behind a veil of polymorphism, in the style of a van Laarhoven Free Monad. http://r6.ca/blog/20140210T181244Z.html