Zulip API with `aeson` - Haskell

aeson makes it super easy to interact for JSON APIs - I'm building Zulip bindings and this is all I need to write to bind concrete endpoint:

newtype MessageHistory = MessageHistory{
    mhMessageHistory :: [MessageSnapshot]
  } deriving stock (Show, Generic)
    deriving FromJSON via ZulipJSON MessageHistory

data MessageSnapshot = MessageSnapshot{
    msTopic               :: Text
  , msContent             :: Text
  , msRenderedContent     :: Text
  , msPrevContent         :: Maybe Text
  , msPrevRenderedContent :: Maybe Text
  , msUserId              :: UserId
  , msContentHtmlDiff     :: Maybe Text
  , msTimestamp           :: Timestamp
  } deriving stock (Show, Generic)
    deriving FromJSON via ZulipJSON MessageSnapshot

type instance Args MessageHistory = ()

getMessageHistory :: MonadIO m
                  => MessageId -> ZulipT m (Either ZulipError MessageHistory)
getMessageHistory msgId = zulipGet ("messages/" ++ show msgId ++ "/history") ()

@TheMatten What's ZulipJSON? Does it do anything more than fieldLabelMomd of https://github.com/srid/zulip-archive/blob/d2a3267/src/Zulip/Internal.hs#L9-L13 ?

I wonder: are you writing Zulip bindings for creating a Zulip bot in Haskell? In any case, we could use your bindings in zulip-archive!

It uses custom genericParseJSON Options - less verbose that making instance directly

(I hope you are using req for http requests)

I use http-client :grimacing: - I found it to get the least in the way

And I've tried hreq and req

Come on, isn't req more modern? :-D

I'm not familiar with http-client - but req says "type-safe" - and that appealed to me

I found req to be pretty easy to write REST API clients, actually. Disclaimer: never used any other client library (aside from wreq) in the past.

For my purpose of having few basic combinators for actual interaction wrapped in custom transformer, I find it's powerful interface unnecessary - I'm not interacting with it directly that often

But the way I'm doing this, if it turns out to be a better alternative, It should be simple enough to switch to it

newtype RGB = RGB Text
  deriving (Show, IsString, FromJSON, ToJSON) via RGB

:face_palm:

Can we have like warning for this? :big_smile:

That reminds me of modern-uri's RText type: https://hackage.haskell.org/package/modern-uri-0.3.1.0/docs/Text-URI.html#t:RText

Yeah - I want to avoid pulling dependencies because of newtypes, at least for now
But like, I was wondering why I'm suddenly stuck in infinite loop for no good reason for at least a few minutes... :sweat_smile:

This message was sent from Haskell using Zulip API :big_smile:

Yay! :party_ball:

@TheMatten So what do you plan to create with it? Lambdabot-like?

Yeah - once I map rest of the API

zulip-api already seems to cover more of the API than hzulip, but being more exhaustive means that a lot of functions take whole records of arguments to be manageable, which are pretty inconvenient to fill in - I guess I'll need some sort of anonymous records with optional fields

I guess I will use https://hackage.haskell.org/package/named-0.3.0.1/docs/Named.html to avoid those ugly record arguments

bye bye helpful type errors

i hate to say it, but i think those boring haskell guys are right here

there is a lot of magic here in order to get trivial syntactic sugar

does it pay for itself?

Hm, for the public API of a module I might consider not prefixing the fields as aeson and others do: https://hackage.haskell.org/package/aeson-1.2.4.0/docs/src/Data-Aeson-Types-Internal.html#Options

Exporting a default value that users can override is a nice API pattern.

@Sandy Maguire You would be surprised how well it works - (!) just locates right named argument, and if it doesn't, it throws a nice error

It really pays for itself - filing in of those records gets crazy pretty quickly

If I wanted default values with records, I would have to make stuff optional and require users to use Just everywhere - I can't avoid prefixes because there's a lot of overlapping names and I want to follow API closely

This is nicer, has no prefixes, has default values and no inference problems AFAIK

Named library is great for actual function arguments, but for filling up records ... well, I have never used it for that purpose, so I wouldn't know. FWIW, Reflex uses Data.Default and lenses to solve the default value problem.

getMessages $ def & anchor .~ AOldest & numBefore .~ 10 ...

@TheMatten what's :t getMessages?

@Sridhar Ratnakumar

getMessages
  :: MonadIO m =>
     ("anchor" :? Anchor)
     -> ("useFirstUnreadAnchor" :? Bool)
     -> ("numBefore" :! GHC.Natural.Natural)
     -> ("numAfter" :! GHC.Natural.Natural)
     -> ("narrow" :? [Narrow])
     -> ("clientGravatar" :? Bool)
     -> ("applyMarkdown" :? Bool)
     -> ZulipT m (Either ZulipError Messages)

I mean, it hardly gets better with any other solution

(in Haskell - I want anonymous records soo much)

Okay, that looks nice. :? are optional arguments? I can see the value of it, especially if from the API user point of view you don't have to care about records. You just call the API method, pass it the parameters and get back result.

Interesting, I didn't know about :? and defaults from Named. This is one way to implement variable arguments with default values in Haskell.

I wonder if this trick can be used in GADT constructor arguments.

data ZulipApi res where
  GetMessages :: ("anchor" :? Anchor) -> ... -> ("applyMarkdown" :? Bool) -> ZulipApi Messages
  ...

Yeah, it should work

They're just functions after all (on expression side)

• $(makeApiMethods ''ZulipApi)
• ???
• Profit!

Thing is, sometimes I need to work around API's and Haskell's quirks, so it wouldn't be possible in general - But I guess it's pretty good already:

data Stream = Stream{
    sStreamId    :: StreamId
  , sName        :: Text
  , sDescription :: Text
  , sInviteOnly  :: Bool
  } deriving stock (Show, Generic)
    deriving FromJSON via ZulipJSON Stream

getStreams :: MonadIO m
           => "includePublic"          :? Bool
           -> "includeSubscribed"      :? Bool
           -> "includeAllActive"       :? Bool
           -> "includeDefault"         :? Bool
           -> "includeOwnerSubscribed" :? Bool
           -> ZulipT m (Either ZulipError [Stream])
getStreams (P p) (P s) (P a) (P d) (P o) =
  deField @"streams" $ zulipGet "streams" $ p <> s <> a <> d <> o

(I may be able to get rid of that pattern matching too at some point)

Oh, and I probably want to use MonadZulip that could be implemented over Sem etc.

@TheMatten Have you considered generically deriving an isomorphism between some actual record type and the datatype of interest?

@Asad Saeeduddin Not sure what you mean exactly :sweat_smile:

Actual record type as SomethingArgs I was using previously?

what i mean is if you've got some big complicated datatype like data MessagesArgs = MessagesArgs { a :: Int, bajillion :: Bool, fields :: String, ... }, and some record library that lets you construct R ( #a := 42, #bajillion := True, #fields := "", ... ), you can have a generic isomorphism that converts between the two representations, and generic lenses onto partial subsets of fields, and if it's a big sum type, generic prisms, etc.

Yeah - well, it doesn't really solve my problem of having to fully fill in those big records, every time I want to call some API function

Unless user would be using such R - but then Sandy's argument applies I guess, it just becomes overkill for what I need

@Jan van Brügge was there some progress on anonymous records in Haskell? :sweat_smile:

not yet, I had exams until this week, but wanted to work on it at ghc week

yes, the user has to be exposed to whatever solution helps them call the API more conveniently ultimately. that said, the user could start with a mempty value to avoid filling every field themselves

I am a bit struggeling with the formal definition, ie the extention of System FC

now that ghc week was canceled, I will work on it at home

@Asad Saeeduddin that's defaults above basically - which won't work for normal records without Maybes all over the place, or some funky anonymous records impl (I've tried implementing those myself - it's not that it's not possible, but it never ends up being friendly)

@Jan van Brügge I guess Purescript's implementation is informal too, right?

Could you elaborate on why it wouldn't work with normal records?

it works with normal records

defaults :: Foo
defaults = Foo
   { blah = 5
  , blam = 6
  }

myFunc :: Foo -> whatever

callMyFunc = myFunc defaults { blam = 8 }

Because not all fields are optional

I find named to be strictly better solution for my problem, because:

• result of :t is the same as of Haddock
• it describes arguments in signature itself, instead of separate datatype which does not have other use anyway
• it avoids Just wrapping on optional arguments
• type errors are great - (!) forces type of argument to Named immediately, tells user when name is wrong and given concrete type (which is my case), it infers return type
• optional arguments can be easily filled in using defaults
• I can have overlapping argument names in same module easily - so no need for ugly prefixes
• according to author, it has no runtime overhead
• higher-level interface could be written in terms of low-level bindings simply by filling in specific arguments - no need for creating new argument records
• no duplication of whole argument records between functions with slightly different arguments - named arguments are part of signature

Haha, 4th point is actually pretty important - trying to generalize interface to MonadZulip, I've realized that named now can't be sure if m is some transformer or yet another argument and so it won't work....
I guess I still need some sort of structure passed as an argument after all

Does anyone know of any language that has typed anonymous records with optional values? I'm not able to find anything...

I completely missed option of using HKD :sweat_smile: : https://www.reddit.com/r/haskell/comments/fiaddl/record_construction_vs_optional_named_parameters/fkitghn?utm_source=share&utm_medium=web2x
I guess this sorta solves my problem - I can use generics to transform record into query as previously, plus derive value filled with bottoms that I immediately replace with specific defaults before exposing
@Sandy Maguire I guess Boring Haskeller in you should be satisfied enough :wink:

Alright, so now:

getMessages with{
    gmAnchor    = FirstUnread
  , gmNumBefore = 10
  , gmNumAfter  = 10
  }

And what's cool, omitting required field results in:

    • Couldn't match type ‘'MissingRequired’ with ‘'Complete’
      Expected type: GetMessages 'Complete
        Actual type: GetMessages 'MissingRequired

Is there some reason for aeson's Parser to fail with mempty, instead of lifting mzero of inner Monoid as other Monads do?

There're few places where endpoint takes many arguments, all of which are optional (like changing settings) - should I go with Maybe Field and accept Just everywhere or something like list/map of ADT of unary constructors?

dependent-map?

DMap.fromList [Foo :=> Identity "stuff", ...] where

data Args a where
  Foo :: Args Text

But then now all arguments become optional.

I guess that's equivalent to the latter in my case

And wrapping with Identity it doesn't really bring benefits over Just "stuff" - I'm interested in reducing verbosity

Why not the lens approach reflex has taken? callApi $ def & gmLimit ?~ 10 & gmFrom .~ 120201

And then I have to decide between lens and optics - and some people may not be interested in using optics in their project at all... :slight_smile:
(Though I wouldn't mind supporting even both options as separate packages if someone finds them useful)

Wow - this zulip-api journey makes me think that structural typing is actually pretty useful feature to have - I guess Unison got it right in this area

Structural typing?

@Sridhar Ratnakumar see https://en.wikipedia.org/wiki/Structural_type_system and https://www.unisonweb.org/docs/language-reference#unique-types

Basically, two types are structurally equal if they have the same representation - as with Coercible types in Haskell, though e.g. Unison allows you to use (non-unique) structurally equal types interchangeably, without need for coercions

I'm not saying that I don't want nominal types - I want both

Oh, and namespaced constructors/fields :stuck_out_tongue_wink:

structural types for derivingvia would be a dream come true..