https://github.com/srid/ema is a static site generator that does more than that; I pretty much use it as a light weight web framework for writing static-site'ish apps, like rendering my .org files as a nice online diary. So far, the main feature that has been perfected is the hot reload feature.
When you develop ema sites, and when you change either .html, .hs code (DSL included), or the .md or whatever files your site uses as a source, the web browser will instantly reload. And it won't do a 'refresh' - nor will there be a setInnerHTML (which is slow), but the DOM is actually patched (so it is damn fast) based on what Ema sends the browser.
I'm currently working on a documentation site for ema based on the principles laid out by https://diataxis.fr/ (PR open as draft).
As part of it -- and since hot reload for data (not only code) requires that you monitor your data (which in our case are markdown files) for changes -- I decided to pull out the fsnotify code (used by https://github.com/srid/orgself - an ema site) and re-use it for the documentation site. But for DRY, I made it a helper module Ema.Helper.FileSystem.
Using that, the main entry point of the documentation site looks like this:
main::IO()main=domainWith"docs"mainWith::FilePath->IO()mainWithfolder=dorunEmarender$\model->doLVar.setmodel=<<doputStrLn$"Loading .md files from "<>foldermdFiles<-FileSystem.filesMatchingfolder["**/*.md"]forMmdFilesreadSource<&>Tagged.Map.fromList.catMaybesFileSystem.onChangefolder$\fp->\caseFileSystem.Update->whenJustM(readSourcefp)$\(spath,s)->doputStrLn$"Update: "<>showspathLVar.modifymodel$Tagged.Map.insertspaths.untagFileSystem.Delete->whenJust(mkSourcePathfp)$\spath->doputStrLn$"Delete: "<>showspathLVar.modifymodel$Tagged.Map.deletespath.untag
the dev experience should compare to that of https://kit.svelte.dev ... ema does support SSE (sever-side rendering)! SvelteKit is nice for writing static sites, but JS sucks - this is where ema comes in: get the SvelteKit like experience, but use Haskell instead of JS.
There is one catch; if you want client-side JS behaviour, you'll still have to write "low-level" JS ... but perhaps that can be addressed using either PureScript or Tweag's Asterius (depending on the trade-offs). That's for Ema's roadmap
i've a history of abusing typeclasses, but this time hopefully I didn't overcomplicate it (for the static site generation case) because I find it to be the minimum necessary (and cannot be achieved more simply in other means; unless someone can demonstrate otherwise),
classEmaMyModelRoutewhere-- Convert our route to browser URL, represented as a list of slugsencodeRoute=\caseIndex->[]-- An empty slug represents the index route: index.htmlAbout->["about"]-- Convert back the browser URL, represented as a list of slugs, to our routedecodeRoute=\case[]->JustIndex["about"]->JustAbout_->Nothing-- The third method is used during static site generation.-- This tells Ema which routes to generate .html files for.staticRoutesmodel=[Index,About]-- The fourth method is optional; if you have static assets to serve, specify-- them here. Paths are relative to current working directory.staticAssetsProxy=["css","images","favicon.ico","resume.pdf"]
-- | Enrich a model to work with EmaclassEmamodelroute|route->modelwhere-- How to convert URLs to/from routesencodeRoute::route->[Slug]decodeRoute::[Slug]->Mayberoute-- | Routes to use when generating the static site---- This is never used by the dev server.staticRoutes::model->[route]-- | List of (top-level) filepaths to serve as static assets---- These will be copied over as-is during static site generationstaticAssets::Proxyroute->[FilePath]staticAssetsProxy=mempty
An Ema{..} record with four fields of functions types?
TheMatten: That way, user wouldn't be restricted to single implementation for given model and route
Would you say that's the only downside of using type-classes here (also, what use cases exist for someone to create multiple implementations for a given model/route?) ... or are there others?
Yeah
One more downside is limited "derivability" - it's easier to write function modifying record, than it is to parameterize deriving mechanism by wanted modifications to default implementation
Yeah, you're switching from static to dynamic resolution that way
In that case, you have to decide whether you want to give up on first-class Ema implementations or sligthly complicate things by doing something like type Ema model route = ?emaImpl :: EmaImpl model route
required global uniqueness isn't something users will often run into
This is what I'm wondering. Right now, each of your route maps to one implementation. But you can have multiple implementations for the given model (eg: markdown files) due to fundep. Are there any practical use cases where the user would want two or more site implementation for a given route type? What does that look like, in practice?
Nothing would change - ImplicitParams would simply let you introduce user-supplied record into scope as an constraint and users wouldn't even need to enable it themselves
There's no global uniqueness for ImplicitParams, which is technically what you want - at the same time, it means that you could technically swap implementation for some scope, and abuse of that may create confusing situations
This is what I'm wondering. Right now, each of your route maps to one implementation. But you can have multiple implementations for the given model (eg: markdown files) due to fundep. Are there any practical use cases where the user would want two or more site implementation for a given route type? What does that look like, in practice?
Well, I imagine they may want to start with some pre-populated record that handles some stuff for them
I'm reminded of XMonad using records too. So may be composability is something I'm overlooking here. But I'm trying to figure out how composability specifically comes into play for static site generation.
E.g. I may want to implement aliases for routes
"composability model and route handling building blocks" - classes won't let you manipulate implementations of instances easily
(BTW, short introduction to ImplicitParams - ?a :: B constraint means "implicit parameter ?a of type B in scope" and can be introduced by let ?a = .. in ..)
Aliases are a good example. So if somebody extends neuron v3 written in Ema as, import Neuron (Model, Route) -- they want to implement custom aliases on top of what the neuron ema site generates. What's the approach here? Assuming a) typeclass, b) records. What would the implementation look like ....
dataAliasableRouter=AliasableRoute(MapTextr)rlookupAlias::...instanceEmamodelr=>Emamodel(AliasableRouter)wheredecodeRouteslugs=caselookupAliasrofJustroute->JustrouteNothing->decodeRoute@model@rslugs-- Rest of the functions are delegated
Well, with classes, you'll have to wrap Model and Route in newtypes if you want to implement new instance based on them - plus this instance must be basically defined at compile-time:
There is another issue here. Remember ema supports hot-reload of model. In general all the "state" used by your static site should be stored in the model. Aliases are part of such a state. Even if you put them in neuron.dhall they get loaded as part of the model.
The problem with type-class based route decoding is that ... there is no state available. Route decoding doesn't depend on model, so it won't know anything about aliases. But then ... decodeRoute is used only in dev server. For static site generation, staticRoutes is used which does receive the model as an argument, can enable generation of the stub routes. Basically "dynamic" route decoding is impossible.
That does make me wonder ... maybe decodeRouteshould take the (current) model as argument. It would be useful to handle 404 early. Like decode /notes/foo.md can fail if "foo.md" is not present in the model.
Actually, that breaks the isomorphic guarantee between dev server and static generation. That if r == h ... thingy would work in dev server, without any guarantee of working in static generation.
Having to wrap the route type, while it does seem like extra work, most importantly provides that guarantee actually. Because now your view function is forced to deal with it because Haskell will complain about non-exhaustive pattern matches.
dataAliasableRouter=AliasTextr|NormalrtypeWithAliasmodelr=(Map[Slug]Route,r)lookupAlias::...instanceEmamodelr=>Ema(WithAliasmodelr)(AliasableRouter)wheredecodeRoutemodelslugs=caselookupAlias(getAliasTablemodel)slugsofJust(s,route)->Just$AliassrouteNothing->decodeRoute@model@rslugsstaticRoutes(aliases,model)=getAllAlisesaliases<>staticRoutesmodel-- Rest of the functions are delegated
So I think records don't compose well whilst guaranteeing safety. Because "patching" a record value doesn't propagate to all the places where the type is used (like the render function using the route type). Type-classes guarantee type safety here, albeit at the expense of losing some ease of composability. I'm not 100% sure which way to sway, but haven't yet seen compelling evidence to seriously consider records yet.
That does make me wonder ... maybe decodeRouteshould take the (current) model as argument. It would be useful to handle 404 early. Like decode /notes/foo.md can fail if "foo.md" is not present in the model.
Wait, no it is not needed! I have to revert the commit.
decodeRoute and encodeRoute can just remain isomorphisms. Adding a model argument to one of them violates that property. And alias feature can be implemented in the staticRoutes method which does take that argument.
Played a bit with co-log, but it doesn't work well with functions like withCurrentDirectory (i.e., taking functions as arguments that return IO action; and you want to have logging in those actions), so going back to good ol monad-logger but with monad-logger-extras sugar on top. I also had to pull unliftio as part of it.
@TheMatten I like the 'effectful context' descriptor. By which you mean - being able to pull values out of some closure (such as in IO monad) right?
The thing is - with Ema, all such values should be (by Ema's design) part of the model. The alias example is approached in the exact way; by putting Map Text Route in the model. This allows you to use Ema's hot reload easily with these values. Such as when your alias.json file changes, there is nothing special to do (thanks to the LVar in use).
However, due to use of LVar the program can unilaterally (with no client involvement; hence not interactive) modify the model (say, from filesystem changes) - which modifications are propagated to the view when using the dev server, via hot reload.
You can already do that, since runEma takes a LVar model -> IO () function (inside which you'd set / modify the LVar); and it is also a forever living IO action, to monitor file changes and such (it is run as in a different thread).
Ema is now ready for writing static websites with a delightful dev server. Give it a try: https://ema.srid.ca/ (docs are now sufficiently complete) <- this website itself was written and published with Ema; notice it uses Markdown underneath.
Support generation of non-HTML files (like RSS, etc.) Live server should respect different route/file types Make Route type polymorphic enough to treat static routes as the same as other (generate-...
Thought I'd dedicate a topic for it.
https://github.com/srid/ema is a static site generator that does more than that; I pretty much use it as a light weight web framework for writing static-site'ish apps, like rendering my .org files as a nice online diary. So far, the main feature that has been perfected is the hot reload feature.
When you develop ema sites, and when you change either .html, .hs code (DSL included), or the .md or whatever files your site uses as a source, the web browser will instantly reload. And it won't do a 'refresh' - nor will there be a setInnerHTML (which is slow), but the DOM is actually patched (so it is damn fast) based on what Ema sends the browser.
I'm currently working on a documentation site for ema based on the principles laid out by https://diataxis.fr/ (PR open as draft).
As part of it -- and since hot reload for data (not only code) requires that you monitor your data (which in our case are markdown files) for changes -- I decided to pull out the fsnotify code (used by https://github.com/srid/orgself - an ema site) and re-use it for the documentation site. But for DRY, I made it a helper module
Ema.Helper.FileSystem
.Using that, the main entry point of the documentation site looks like this:
Note that
Data.LVar
thing will be published to Hackage, and this is what I was talking about in an earlier topic: https://funprog.zulipchat.com/#narrow/stream/201385-Haskell/topic/TVar.20with.20change.20notification(the LVar in conjunction with Ema's websocket machinary is how when you edit a .md file in VSCode, it propagates instantly to the browser view)
marketing messaging?
the dev experience should compare to that of https://kit.svelte.dev ... ema does support SSE (sever-side rendering)! SvelteKit is nice for writing static sites, but JS sucks - this is where ema comes in: get the SvelteKit like experience, but use Haskell instead of JS.
There is one catch; if you want client-side JS behaviour, you'll still have to write "low-level" JS ... but perhaps that can be addressed using either PureScript or Tweag's Asterius (depending on the trade-offs). That's for Ema's roadmap
type class
i've a history of abusing typeclasses, but this time hopefully I didn't overcomplicate it (for the static site generation case) because I find it to be the minimum necessary (and cannot be achieved more simply in other means; unless someone can demonstrate otherwise),
The class itself,
"type class" ~ "implicitly resolved record" - maybe it would make sense to pass in
data Ema model route = Ema{..}
instead?That way, user wouldn't be restricted to single implementation for given
model
androute
An
Ema{..}
record with four fields of functions types?Would you say that's the only downside of using type-classes here (also, what use cases exist for someone to create multiple implementations for a given model/route?) ... or are there others?
Yeah
One more downside is limited "derivability" - it's easier to write function modifying record, than it is to parameterize deriving mechanism by wanted modifications to default implementation
One problem with the record approach is that you now have to carry the record value all over the code base, or implement a ReaderT over IO.
Type class here seems more ergonomic to me; are there practical use cases where they start to become a pain in this case (static site generation)?
Does user usually need to use
Ema
constraint directly, or is it simply used by the implementation?The constraint is utilized by the library (Ema), and not in the user code. The user code simplify defines the instances, and forgets about it.
This is the simplest 70-line example; let's use that as a reference: https://github.com/srid/ema/blob/b80a9327a6917cb8f7b10e3f7055d8e0a191aa72/src/Ema/Example/Ex02_Clock.hs
Notice line 68. the use of
routeUrl
which the user calls to get the URL for the routeThis function, defined in Ema library, uses the constraint:
Ah - so if it's simply an internal problem, and you don't like
Reader
, there're alternatives likeImplicitParams
If we were to use a record, then the user would have to thread-in the record to
routeUrl
... or be made aware of ReaderI mean look at the view function right now that user has to write:
If you add a Reader in there, just to be able to get URL to a route - isn't that an overkill, and extra overhead for the user to deal with?
Yeah, you're switching from static to dynamic resolution that way
In that case, you have to decide whether you want to give up on first-class
Ema
implementations or sligthly complicate things by doing something liketype Ema model route = ?emaImpl :: EmaImpl model route
I guess class would be reasonable then, as long as required global uniqueness isn't something users will often run into
I'm not familiar with ImplicitParams; how would this look like from user point of view (when they are writing the
render
function)?This is what I'm wondering. Right now, each of your
route
maps to one implementation. But you can have multiple implementations for the givenmodel
(eg: markdown files) due to fundep. Are there any practical use cases where the user would want two or more site implementation for a givenroute
type? What does that look like, in practice?Nothing would change -
ImplicitParams
would simply let you introduce user-supplied record into scope as an constraint and users wouldn't even need to enable it themselvesThere's no global uniqueness for
ImplicitParams
, which is technically what you want - at the same time, it means that you could technically swap implementation for some scope, and abuse of that may create confusing situationsAs far as implementation goes, you would add
?emaImpl
as an argument to selectors of your recordWell, I imagine they may want to start with some pre-populated record that handles some stuff for them
Like "neuron model" or "org model", which could be extended for specific needs
An example of extending
Ema MarkdownNotes NeuronRoute
record?I'm reminded of XMonad using records too. So may be composability is something I'm overlooking here. But I'm trying to figure out how composability specifically comes into play for static site generation.
E.g. I may want to implement aliases for routes
"composability model and route handling building blocks" - classes won't let you manipulate implementations of instances easily
Though I imagine examples where
model
androute
would be extensible types by themselves - e.g. parameterized by some user-defined type(BTW, short introduction to
ImplicitParams
-?a :: B
constraint means "implicit parameter?a
of typeB
in scope" and can be introduced bylet ?a = .. in ..
)Aliases are a good example. So if somebody extends neuron v3 written in Ema as,
import Neuron (Model, Route)
-- they want to implement custom aliases on top of what the neuron ema site generates. What's the approach here? Assuming a) typeclass, b) records. What would the implementation look like ....In the record case, I imagine them wrapping the route encoding functions, eg:
Well, I don't know ... record composition looks complicated?
in type-class case,
Oh, also the render function has to handle the alias route. By generating a .html stub that simply redirects to target route.
Well, with classes, you'll have to wrap
Model
andRoute
in newtypes if you want to implement new instance based on them - plus this instance must be basically defined at compile-time:meanwhile with records, neuron's implementation is as easy to extend as any other value:
I'll have to think about this a bit more.
There is another issue here. Remember ema supports hot-reload of model. In general all the "state" used by your static site should be stored in the
model
. Aliases are part of such a state. Even if you put them inneuron.dhall
they get loaded as part of themodel
.And when you modify
neuron.dhall
to add/remove an alias, your site updates in real-time.The problem with type-class based route decoding is that ... there is no state available. Route decoding doesn't depend on
model
, so it won't know anything about aliases. But then ...decodeRoute
is used only in dev server. For static site generation,staticRoutes
is used which does receive the model as an argument, can enable generation of the stub routes. Basically "dynamic" route decoding is impossible.That does make me wonder ... maybe
decodeRoute
should take the (current) model as argument. It would be useful to handle 404 early. Likedecode /notes/foo.md
can fail if "foo.md" is not present in the model.Actually, that breaks the isomorphic guarantee between dev server and static generation. That
if r == h ...
thingy would work in dev server, without any guarantee of working in static generation.Having to wrap the route type, while it does seem like extra work, most importantly provides that guarantee actually. Because now your view function is forced to deal with it because Haskell will complain about non-exhaustive pattern matches.
The alias extension would then look like this,
And your view function will be forced to deal with the new kinds of routes,
(Just added
staticRoutes
implementation; and that, in conjunction with view, guarantees that static site generation will create our alias HTMLs)So I think records don't compose well whilst guaranteeing safety. Because "patching" a record value doesn't propagate to all the places where the type is used (like the render function using the route type). Type-classes guarantee type safety here, albeit at the expense of losing some ease of composability. I'm not 100% sure which way to sway, but haven't yet seen compelling evidence to seriously consider records yet.
Sridhar Ratnakumar said:
Wait, no it is not needed! I have to revert the commit.
decodeRoute and encodeRoute can just remain isomorphisms. Adding a model argument to one of them violates that property. And alias feature can be implemented in the
staticRoutes
method which does take that argument.Record vs Type classes - an approach to choose
In summary, I think we should do the following in order to systematically arrive at a conclusion as to what to choose:
Composability of type classes
Are records really "more" composable than type classes? I'm not sure that's true.
With records, you are simply composing them at the value level. With type classes, you compose them at the _type-level_.
Adding logging
Played a bit with
co-log
, but it doesn't work well with functions likewithCurrentDirectory
(i.e., taking functions as arguments that return IO action; and you want to have logging in those actions), so going back to good olmonad-logger
but withmonad-logger-extras
sugar on top. I also had to pullunliftio
as part of it.https://github.com/srid/ema/pull/17/files
Yeah - what I mean is that the latter is harder and can't be done in effectful context
@TheMatten I like the 'effectful context' descriptor. By which you mean - being able to pull values out of some closure (such as in IO monad) right?
The thing is - with Ema, all such values should be (by Ema's design) part of the
model
. The alias example is approached in the exact way; by puttingMap Text Route
in themodel
. This allows you to use Ema's hot reload easily with these values. Such as when youralias.json
file changes, there is nothing special to do (thanks to theLVar
in use).It is similar to the
state
in TEA (The Elm Architecture) or MVU (Model View Update) .Using a diagram from one of the MVU posts, image.png
In Ema, we have two circles only -- Model and View. There is no Update, because - these are static sites, with no two-way interactivity.
However, due to use of
LVar
the program can unilaterally (with no client involvement; hence not interactive) modify themodel
(say, from filesystem changes) - which modifications are propagated to the view when using the dev server, via hot reload.They are - it's just that the initial configuration of the model can be done at runtime - e.g. when starting the program
You can already do that, since
runEma
takes aLVar model -> IO ()
function (inside which you'dset
/modify
the LVar); and it is also a forever living IO action, to monitor file changes and such (it is run as in a different thread).LVar published on hackage
https://hackage.haskell.org/package/lvar-0.1.0.0/docs/Data-LVar.html
Template repo is ready!
Start creating your Ema sites now: https://github.com/srid/ema-docs
Last step before announcement: finish documentation and write blog post.
Deriving automatically using Generics
Wouldn't it be cool if the above can be automatically derived using
Generic
? Should support nested routes too, eg:Then
deriving instance Ema m Route
should automatically create implementation that maps sayBlog (BlogPost "foo")
to/blog/foo
URL and vice-versa.:tada::tada: Ema pre-announcement
Ema is now ready for writing static websites with a delightful dev server. Give it a try: https://ema.srid.ca/ (docs are now sufficiently complete) <- this website itself was written and published with Ema; notice it uses Markdown underneath.
(I need to put a short video demo of hot-reload in frontpage)
Video demo https://ema.srid.ca/ema-demo.mp4
announced
https://lobste.rs/s/rkud9l/ema_haskell_static_site_generator_with
phew; now I can relax into other things.
Think I might just ditch the type class, https://github.com/srid/ema/issues/35
TheMatten said:
I'll have to revisit my design choice to use typeclasses here, cf. https://github.com/srid/ema/issues/75