Thought I'd dedicate a topic for it.

https://github.com/srid/ema is a static site generator that does more than that; I pretty much use it as a light weight web framework for writing static-site'ish apps, like rendering my .org files as a nice online diary. So far, the main feature that has been perfected is the hot reload feature.

When you develop ema sites, and when you change either .html, .hs code (DSL included), or the .md or whatever files your site uses as a source, the web browser will instantly reload. And it won't do a 'refresh' - nor will there be a setInnerHTML (which is slow), but the DOM is actually patched (so it is damn fast) based on what Ema sends the browser.

I'm currently working on a documentation site for ema based on the principles laid out by https://diataxis.fr/ (PR open as draft).

As part of it -- and since hot reload for data (not only code) requires that you monitor your data (which in our case are markdown files) for changes -- I decided to pull out the fsnotify code (used by https://github.com/srid/orgself - an ema site) and re-use it for the documentation site. But for DRY, I made it a helper module Ema.Helper.FileSystem.

Using that, the main entry point of the documentation site looks like this:

main :: IO ()
main = do
  mainWith "docs"

mainWith :: FilePath -> IO ()
mainWith folder = do
  runEma render $ \model -> do
    LVar.set model =<< do
      putStrLn $ "Loading .md files from " <> folder
      mdFiles <- FileSystem.filesMatching folder ["**/*.md"]
      forM mdFiles readSource
        <&> Tagged . Map.fromList . catMaybes
    FileSystem.onChange folder $ \fp -> \case
      FileSystem.Update ->
        whenJustM (readSource fp) $ \(spath, s) -> do
          putStrLn $ "Update: " <> show spath
          LVar.modify model $ Tagged . Map.insert spath s . untag
      FileSystem.Delete ->
        whenJust (mkSourcePath fp) $ \spath -> do
          putStrLn $ "Delete: " <> show spath
          LVar.modify model $ Tagged . Map.delete spath . untag

Note that Data.LVar thing will be published to Hackage, and this is what I was talking about in an earlier topic: https://funprog.zulipchat.com/#narrow/stream/201385-Haskell/topic/TVar.20with.20change.20notification

(the LVar in conjunction with Ema's websocket machinary is how when you edit a .md file in VSCode, it propagates instantly to the browser view)

marketing messaging?

the dev experience should compare to that of https://kit.svelte.dev ... ema does support SSE (sever-side rendering)! SvelteKit is nice for writing static sites, but JS sucks - this is where ema comes in: get the SvelteKit like experience, but use Haskell instead of JS.

There is one catch; if you want client-side JS behaviour, you'll still have to write "low-level" JS ... but perhaps that can be addressed using either PureScript or Tweag's Asterius (depending on the trade-offs). That's for Ema's roadmap

type class

i've a history of abusing typeclasses, but this time hopefully I didn't overcomplicate it (for the static site generation case) because I find it to be the minimum necessary (and cannot be achieved more simply in other means; unless someone can demonstrate otherwise),

class Ema MyModel Route where
  -- Convert our route to browser URL, represented as a list of slugs
  encodeRoute = \case
    Index -> []  -- An empty slug represents the index route: index.html
    About -> ["about"]
  -- Convert back the browser URL, represented as a list of slugs, to our route
  decodeRoute = \case
    [] -> Just Index
    ["about"] -> Just About
    _ -> Nothing
  -- The third method is used during static site generation.
  -- This tells Ema which routes to generate .html files for.
  staticRoutes model =
    [Index, About]
  -- The fourth method is optional; if you have static assets to serve, specify
  -- them here. Paths are relative to current working directory.
  staticAssets Proxy =
    ["css", "images", "favicon.ico", "resume.pdf"]

The class itself,

-- | Enrich a model to work with Ema
class Ema model route | route -> model where
  -- How to convert URLs to/from routes
  encodeRoute :: route -> [Slug]
  decodeRoute :: [Slug] -> Maybe route

  -- | Routes to use when generating the static site
  --
  -- This is never used by the dev server.
  staticRoutes :: model -> [route]

  -- | List of (top-level) filepaths to serve as static assets
  --
  -- These will be copied over as-is during static site generation
  staticAssets :: Proxy route -> [FilePath]
  staticAssets Proxy = mempty

"type class" ~ "implicitly resolved record" - maybe it would make sense to pass in data Ema model route = Ema{..} instead?

That way, user wouldn't be restricted to single implementation for given model and route

An Ema{..} record with four fields of functions types?

TheMatten: That way, user wouldn't be restricted to single implementation for given model and route

Would you say that's the only downside of using type-classes here (also, what use cases exist for someone to create multiple implementations for a given model/route?) ... or are there others?

Yeah
One more downside is limited "derivability" - it's easier to write function modifying record, than it is to parameterize deriving mechanism by wanted modifications to default implementation

One problem with the record approach is that you now have to carry the record value all over the code base, or implement a ReaderT over IO.

Type class here seems more ergonomic to me; are there practical use cases where they start to become a pain in this case (static site generation)?

Does user usually need to use Ema constraint directly, or is it simply used by the implementation?

The constraint is utilized by the library (Ema), and not in the user code. The user code simplify defines the instances, and forgets about it.

This is the simplest 70-line example; let's use that as a reference: https://github.com/srid/ema/blob/b80a9327a6917cb8f7b10e3f7055d8e0a191aa72/src/Ema/Example/Ex02_Clock.hs

Notice line 68. the use of routeUrl which the user calls to get the URL for the route

This function, defined in Ema library, uses the constraint:

routeUrl :: forall a r. Ema a r => r -> Text

Ah - so if it's simply an internal problem, and you don't like Reader, there're alternatives like ImplicitParams

If we were to use a record, then the user would have to thread-in the record to routeUrl ... or be made aware of Reader

I mean look at the view function right now that user has to write:

render :: Ema.CLI.Action -> model -> route -> LByteString

If you add a Reader in there, just to be able to get URL to a route - isn't that an overkill, and extra overhead for the user to deal with?

Yeah, you're switching from static to dynamic resolution that way
In that case, you have to decide whether you want to give up on first-class Ema implementations or sligthly complicate things by doing something like type Ema model route = ?emaImpl :: EmaImpl model route

I guess class would be reasonable then, as long as required global uniqueness isn't something users will often run into

I'm not familiar with ImplicitParams; how would this look like from user point of view (when they are writing the render function)?

required global uniqueness isn't something users will often run into

This is what I'm wondering. Right now, each of your route maps to one implementation. But you can have multiple implementations for the given model (eg: markdown files) due to fundep. Are there any practical use cases where the user would want two or more site implementation for a given route type? What does that look like, in practice?

Nothing would change - ImplicitParams would simply let you introduce user-supplied record into scope as an constraint and users wouldn't even need to enable it themselves
There's no global uniqueness for ImplicitParams, which is technically what you want - at the same time, it means that you could technically swap implementation for some scope, and abuse of that may create confusing situations

As far as implementation goes, you would add ?emaImpl as an argument to selectors of your record

This is what I'm wondering. Right now, each of your route maps to one implementation. But you can have multiple implementations for the given model (eg: markdown files) due to fundep. Are there any practical use cases where the user would want two or more site implementation for a given route type? What does that look like, in practice?

Well, I imagine they may want to start with some pre-populated record that handles some stuff for them

Like "neuron model" or "org model", which could be extended for specific needs

An example of extending Ema MarkdownNotes NeuronRoute record?

I'm reminded of XMonad using records too. So may be composability is something I'm overlooking here. But I'm trying to figure out how composability specifically comes into play for static site generation.

E.g. I may want to implement aliases for routes
"composability model and route handling building blocks" - classes won't let you manipulate implementations of instances easily

Though I imagine examples where model and route would be extensible types by themselves - e.g. parameterized by some user-defined type

(BTW, short introduction to ImplicitParams - ?a :: B constraint means "implicit parameter ?a of type B in scope" and can be introduced by let ?a = .. in ..)

Aliases are a good example. So if somebody extends neuron v3 written in Ema as, import Neuron (Model, Route) -- they want to implement custom aliases on top of what the neuron ema site generates. What's the approach here? Assuming a) typeclass, b) records. What would the implementation look like ....

In the record case, I imagine them wrapping the route encoding functions, eg:

type RouteWithAliases = Either Route Alias
myEma = neuronEma { decodeRoute = decodeAlias (Neuron.decodeRoute neuronEma) ..}
...

Well, I don't know ... record composition looks complicated?

in type-class case,

data AliasableRoute r = AliasableRoute (Map Text r) r

lookupAlias :: ...

instance Ema model r => Ema model (AliasableRoute r) where
  decodeRoute slugs =
    case lookupAlias r of
        Just route -> Just route
        Nothing -> decodeRoute @model @r slugs
  -- Rest of the functions are delegated

Oh, also the render function has to handle the alias route. By generating a .html stub that simply redirects to target route.

Well, with classes, you'll have to wrap Model and Route in newtypes if you want to implement new instance based on them - plus this instance must be basically defined at compile-time:

newtype MyModel = MyModel Model
newtype MyRoute = MyRoute Route

instance Ema MyModel MyRoute where
  ..
  decodeRoute r = MyModel <$> decodeRoute case r of
    "yay" -> "happy"
    _ -> r

meanwhile with records, neuron's implementation is as easy to extend as any other value:

myEma :: IO (Ema Model Route)
myEma = do
  h <- getEnv "HAPPINESS_SOURCE"
  pure neuronEma{
      decodeRoute r = decodeRoute neuronEma if r == h then "happy" else r
    }

I'll have to think about this a bit more.

There is another issue here. Remember ema supports hot-reload of model. In general all the "state" used by your static site should be stored in the model. Aliases are part of such a state. Even if you put them in neuron.dhall they get loaded as part of the model.

And when you modify neuron.dhall to add/remove an alias, your site updates in real-time.

The problem with type-class based route decoding is that ... there is no state available. Route decoding doesn't depend on model, so it won't know anything about aliases. But then ... decodeRoute is used only in dev server. For static site generation, staticRoutes is used which does receive the model as an argument, can enable generation of the stub routes. Basically "dynamic" route decoding is impossible.

That does make me wonder ... maybe decodeRoute should take the (current) model as argument. It would be useful to handle 404 early. Like decode /notes/foo.md can fail if "foo.md" is not present in the model.

Actually, that breaks the isomorphic guarantee between dev server and static generation. That if r == h ... thingy would work in dev server, without any guarantee of working in static generation.

Having to wrap the route type, while it does seem like extra work, most importantly provides that guarantee actually. Because now your view function is forced to deal with it because Haskell will complain about non-exhaustive pattern matches.

The alias extension would then look like this,

data AliasableRoute r
  = Alias Text r
  | Normal r

type WithAlias model r = (Map [Slug] Route, r)

lookupAlias :: ...

instance Ema model r => Ema (WithAlias model r) (AliasableRoute r) where
  decodeRoute model slugs =
    case lookupAlias (getAliasTable model) slugs of
        Just (s, route) -> Just $ Alias s route
        Nothing -> decodeRoute @model @r slugs
  staticRoutes (aliases, model) =
     getAllAlises aliases <> staticRoutes model
  -- Rest of the functions are delegated

And your view function will be forced to deal with the new kinds of routes,

render cli (aliases, model) = \case
    Normal r -> Neuron.render cli model
    Alias alias r ->
      "<head><meta redirect..."

(Just added staticRoutes implementation; and that, in conjunction with view, guarantees that static site generation will create our alias HTMLs)

So I think records don't compose well whilst guaranteeing safety. Because "patching" a record value doesn't propagate to all the places where the type is used (like the render function using the route type). Type-classes guarantee type safety here, albeit at the expense of losing some ease of composability. I'm not 100% sure which way to sway, but haven't yet seen compelling evidence to seriously consider records yet.

Sridhar Ratnakumar said:

That does make me wonder ... maybe decodeRoute should take the (current) model as argument. It would be useful to handle 404 early. Like decode /notes/foo.md can fail if "foo.md" is not present in the model.

Wait, no it is not needed! I have to revert the commit.

decodeRoute and encodeRoute can just remain isomorphisms. Adding a model argument to one of them violates that property. And alias feature can be implemented in the staticRoutes method which does take that argument.

Record vs Type classes - an approach to choose

In summary, I think we should do the following in order to systematically arrive at a conclusion as to what to choose:

• Start with Type class design and examples
• Create an example that supposedly make records seem advantageous (eg: aliased routes)
• Code that example using type classes
• Now code that example using records
• Compare them both.
  • Do they both have same level of type-safety?
  • What's their API ergonomics?

Composability of type classes

Are records really "more" composable than type classes? I'm not sure that's true.

With records, you are simply composing them at the value level. With type classes, you compose them at the _type-level_.

Adding logging

Played a bit with co-log, but it doesn't work well with functions like withCurrentDirectory (i.e., taking functions as arguments that return IO action; and you want to have logging in those actions), so going back to good ol monad-logger but with monad-logger-extras sugar on top. I also had to pull unliftio as part of it.

https://github.com/srid/ema/pull/17/files

With records, you are simply composing them at the value level. With type classes, you compose them at the _type-level_.

Yeah - what I mean is that the latter is harder and can't be done in effectful context

@TheMatten I like the 'effectful context' descriptor. By which you mean - being able to pull values out of some closure (such as in IO monad) right?

The thing is - with Ema, all such values should be (by Ema's design) part of the model. The alias example is approached in the exact way; by putting Map Text Route in the model. This allows you to use Ema's hot reload easily with these values. Such as when your alias.json file changes, there is nothing special to do (thanks to the LVar in use).

It is similar to the state in TEA (The Elm Architecture) or MVU (Model View Update) .

Using a diagram from one of the MVU posts, image.png

In Ema, we have two circles only -- Model and View. There is no Update, because - these are static sites, with no two-way interactivity.

However, due to use of LVar the program can unilaterally (with no client involvement; hence not interactive) modify the model (say, from filesystem changes) - which modifications are propagated to the view when using the dev server, via hot reload.

The thing is - with Ema, all such values should be (by Ema's design) part of the model.

They are - it's just that the initial configuration of the model can be done at runtime - e.g. when starting the program

You can already do that, since runEma takes a LVar model -> IO () function (inside which you'd set / modify the LVar); and it is also a forever living IO action, to monitor file changes and such (it is run as in a different thread).

LVar published on hackage

https://hackage.haskell.org/package/lvar-0.1.0.0/docs/Data-LVar.html

Template repo is ready!

Start creating your Ema sites now: https://github.com/srid/ema-docs

Last step before announcement: finish documentation and write blog post.

Deriving automatically using Generics

data Route
  = Index
  | About
  deriving (Show, Enum, Bounded)

instance Ema Model Route where
  encodeRoute = \case
    Index -> mempty
    About -> one "about"
  decodeRoute = \case
    [] -> Just Index
    ["about"] -> Just About
    _ -> Nothing

Wouldn't it be cool if the above can be automatically derived using Generic? Should support nested routes too, eg:

data Route
  = Index
  = Blog BlogRoute

data BlogRoute
  = BlogIndex
  | BlogPost Slug

Then deriving instance Ema m Route should automatically create implementation that maps say Blog (BlogPost "foo") to /blog/foo URL and vice-versa.

:tada::tada: Ema pre-announcement

Ema is now ready for writing static websites with a delightful dev server. Give it a try: https://ema.srid.ca/ (docs are now sufficiently complete) <- this website itself was written and published with Ema; notice it uses Markdown underneath.

(I need to put a short video demo of hot-reload in frontpage)

Video demo https://ema.srid.ca/ema-demo.mp4

announced

https://lobste.rs/s/rkud9l/ema_haskell_static_site_generator_with

phew; now I can relax into other things.