I don't think I'm really happy with Dhall. It seems to be I'd be spending non-trivial time trying to get Dhall syntax right, which means I might as well use Xml (which is more simpler in syntax). With Dhall you also have to define types in Dhall itself, in addition to Haskell types.
Today, I needed to use an Optional on a record with huge type… that I don’t have in Dhall code anywhere. Since Dhall won’t infer a Nothing _ on its own, I need some Nothing ./MyRecord.type. Could t...
i dont personally care if I have the xml -> types mapping done for me, writing my own stuff in this case is OK by me because I know i'm going to want to do a lot of custom stuff
For me types mapping is kind of important, philosophically. I really like tomland's bidirection mapping. Eventually I'd write a sub-command in my static generator, that will generate .toml entries (for new links, whatever) from the values of Haskell types. Neatly formatted too.
XML is flexible. If there is a notion of 'transformations' ... say you create a special node <footnote> or whatever, and somehow have that translate to custom HTML or whatever ...
BTW so I was thinking more about it, and I think the issue I have with relying on automatic codecs is that I do not necessarily desire to have the things i type so directly tied in to the data model
so at that point I have to have serveral data definitions of similar concepts; the one from which the XML is translated, and the one that I want to use in memory
so again like what I WANT to do is have an initial stage where i derive various data from all my site content, but the "link targets" map I would make would NOT be a direct mapping of the XML content; in the future i'd like to definine other things as having the ability to be linked to, too!
now if just use dhall to do what you're suggesting, then adding a new term would require me to both add the glossary term in the text and add link metadata via dhall
If using markdown, this would be equivalent to one file referring to a particular section header of another markdown file; and then the static site generator should check these links for validity.
This "linking between source files, or their section" seems like a nice idea to add to Rib (as long as it doesn't bring back the messy type class, I guess)
... even if it as trivial a check as scanning for links like [some text](/foo/bar.html#section1) and then ensuring that "section1" ID exists in the HTML generated by a/foo/bar.md
In your specific use case, I can definitely see not having to codify types for all and sundry.
But thinking of it in terms of 'transformations' (a. la. XSLT?) seems useful to me. So you have foo.xml - and you want to 'transform' that to foo.html.
bar.xml would contain a section ref to somewhere in inside foo.xml, and during its 'transformation' to bar.html, that reference too would automatically 'transform' to the corresponding a tag with 'href' whose value would be checked by the transformer to be valid.
Your XML format would perhaps be a superset of HTML, allowing you to use direct HTML syntax, but also escaping to custom abstractions (which would ultimately transform to HTML) you design
so my plan here is to follow what pollen does (because it works fine), and if I don't have any other behavior specified for an xml element, it just becomes html
running into a lot of situations where I wish xml were statically typed, and I could assume that for example every <term> element has both <name> and <def> children
BTW, I will probably be pausing on this project for a couple of weeks. I have some other stuff I need to work on in my free time. but I will certainly be back at it, hopefullly within a month!
I've decided that it is time to move away from TOML. A flat list of TOML entries a) can grow to be huge, b) not flexible enough (for eg., can't simply represent longform tidbits without hacks). My plan is to migrate links to good ol' markdown files (each link gets its own .md file, with its YAML metadata specifying the link properties). This way if one of the tidbit is a longform post, rather than a link, the very markdown file will specify its content, in addition to a longform property set to True.
Lol so basically the feature port from pollen to rib/haskell/xml is nearly complete, just a tiny bit remaining. But this code is cursed. I am going to need to ask around for some design advice for sure.
Feel free to ask for any sort of guidance. It would also, indirectly, help me with the guide-like documentation thingy I will be writing for Rib (using neuron, of course)
I don't think I'm really happy with Dhall. It seems to be I'd be spending non-trivial time trying to get Dhall syntax right, which means I might as well use Xml (which is more simpler in syntax). With Dhall you also have to define types in Dhall itself, in addition to Haskell types.
Actually the later is not true if we do this https://github.com/dhall-lang/dhall-haskell/issues/794
There is also TOML, which has an interesting Haskell library: https://kowainik.github.io/posts/2019-01-14-tomland
@Joel McCracken What libraries do you use to parse and validate Xml?
Here's what using Toml in Rib looks like: https://github.com/srid/website/pull/7/files
Toml is way better looking than writing Dhall: pasted image
https://github.com/srid/website/pull/8
Right now I am just doing manual parsing with xml-conduit
you can see some of what i was using here https://gitlab.com/JoelMcCracken/joelmccracken-hs/blob/master/src/Lib.hs#L87
i dont personally care if I have the xml -> types mapping done for me, writing my own stuff in this case is OK by me because I know i'm going to want to do a lot of custom stuff
ive thought abotu TOML, but for whatever reason it really didnt seem to be what I wanted
warning, if you decide to use xml, you are gonna be considered unbearably uncool
For me types mapping is kind of important, philosophically. I really like tomland's bidirection mapping. Eventually I'd write a sub-command in my static generator, that will generate .toml entries (for new links, whatever) from the values of Haskell types. Neatly formatted too.
That's what one thing I learned in Haskell, and particularly at Obsidian.
I would have went with Xml if it had excellent Haskell libraries (with type mapping, that is)
But it doesn't seem to.
So you want automatic bidirectional mapping?
One directional mapping, definitely. Bidirectional is a nice to have.
interesting
Types should be the boundary between what's on file system, and the code logic in Haskell.
Once all the data is loaded into memory, my code can just deal with Haskell types instead of having to deal with the underlying file format specifics.
Aeson is another example of this bidirectional conversion that everyone knows.
https://wiki.haskell.org/HXT/Conversion_of_Haskell_data_from/to_XML
yea
XML is flexible. If there is a notion of 'transformations' ... say you create a special node <footnote> or whatever, and somehow have that translate to custom HTML or whatever ...
And being able to _reference_ other nodes. So a <footnotes-list> would somehow collect all the footnote tags and list them,
And someone can embed markdown using its CDATA section if they really want, and still retain control over the rest of the structure.
Just need a nice Haskell library that does much of this.
that hxt arrow thing seems to aim to do this.
what you're saying is like 100% what I want to do
and the reason why i dont like e.g. markdown
I dont think it even has to have markdown embedded in CDATA; you could just treat it as text!
I dunno, maybe working on a decent aeson-like lib for xml in haskell might be a decent idea
I mean, if you are not so opposed to XML, i'd love to haave someone to work on this with
BTW so I was thinking more about it, and I think the issue I have with relying on automatic codecs is that I do not necessarily desire to have the things i type so directly tied in to the data model
so at that point I have to have serveral
data
definitions of similar concepts; the one from which the XML is translated, and the one that I want to use in memoryWhat sort of data do you plan to represent in Xml? Aside from text content (markdown, etc.) that is.
On my website, that's site metadata. And a list of links. Perhaps a personal twitter-like as well, in future.
well its hard to say; i have several things I want to implement. For example, flexibly linking from one page to subsections in another
so for example, something I have already done in pollen is use the aliases thing I mentioned before to make links that are checked at generation time
so like i can say that my fp-jargon.xml file can be linked via
jargon
orfp-jargon
HOWEVER I also added a sub-linking funcitonality
so that for example within the
@glossary{ ... }
section, terms can be linked toso i could link to the main page as
fp-jargon
, but linking to a specific section would be e.g.fp-jargon monads
so again like what I WANT to do is have an initial stage where i derive various data from all my site content, but the "link targets" map I would make would NOT be a direct mapping of the XML content; in the future i'd like to definine other things as having the ability to be linked to, too!
now if just use dhall to do what you're suggesting, then adding a new term would require me to both add the glossary term in the text and add link metadata via dhall
its kindof like lisp; text is data is code is text is data
or haskell, where funcitons and data are both first-class
So you want to be able to use hyperlinks to site content, but have the static site generator verify that those targets exist and are valid, right?
If using markdown, this would be equivalent to one file referring to a particular section header of another markdown file; and then the static site generator should check these links for validity.
This "linking between source files, or their section" seems like a nice idea to add to Rib (as long as it doesn't bring back the messy type class, I guess)
... even if it as trivial a check as scanning for links like
[some text](/foo/bar.html#section1)
and then ensuring that "section1" ID exists in the HTML generated bya/foo/bar.md
yes exactly, you can link to a file/section from any other file/section
so yeah you could validate with markdown
in pollen i made an
@elink[:to "http://theUrl.com"]{ someplace else}
thing for external linksso i could differentiate which was an internal link and which was external
i suppose you could do something like this though by inferring that link should be checked because it is relative
but... that still might not be right; again this is why i've chosen to go beyond markdown, its just not extensible in the way I want
In your specific use case, I can definitely see not having to codify types for all and sundry.
But thinking of it in terms of 'transformations' (a. la. XSLT?) seems useful to me. So you have foo.xml - and you want to 'transform' that to foo.html.
bar.xml would contain a section ref to somewhere in inside foo.xml, and during its 'transformation' to bar.html, that reference too would automatically 'transform' to the corresponding
a
tag with 'href' whose value would be checked by the transformer to be valid.Your XML format would perhaps be a superset of HTML, allowing you to use direct HTML syntax, but also escaping to custom abstractions (which would ultimately transform to HTML) you design
The HTML5 semantic elements kind of come closer to that design
so my plan here is to follow what pollen does (because it works fine), and if I don't have any other behavior specified for an xml element, it just becomes html
so if i typed <img src=".../whatr.png" /> and I dont have any special behavior for img, it will get passed through
but yeah, a series of transformations is a good model
a lot of what i am doing is overlapped by xslt
I have no interest in using xslt, and if the kind of behavior I want to implement is actually implementable, i would not want to learn it
if ur interested, here is my repo: https://gitlab.com/JoelMcCracken/joelmccrackencom
its still a major wip but it does build
Cool. It runs locally
Note: you do not necessarily have to use
readFileText
. prelude'sreadFile
or shake'sreadFile'
work too (but the later is preferred)cool
yeah right now its super inelegant; kinda still in a prototyping phase
ty
I usually make a thing work first, then refactor
running into a lot of situations where I wish xml were statically typed, and I could assume that for example every
<term>
element has both<name>
and<def>
childreni wonder if smth like a DTD could be parsed to determine the haskell data for an xml format
Did a writeup here: https://www.srid.ca/tidbits/dhall-toml-free-monad.html
So, i was talking it over with a buddy of mine and he suggested just using haskell direclty instead of xml
and , i sort of prototyped it out, and its not as bad as I thought it would be
so i think i'm going to try it
ive converted that glossary document I keep using as an example, and given some assumptions, i think i can make it work well
ehhh after spending a bit more time, though i really like the idea a lot, its not what I want
That's what I did as well. Used free monad to make management of data prettier
BTW, I will probably be pausing on this project for a couple of weeks. I have some other stuff I need to work on in my free time. but I will certainly be back at it, hopefullly within a month!
I've decided that it is time to move away from TOML. A flat list of TOML entries a) can grow to be huge, b) not flexible enough (for eg., can't simply represent longform tidbits without hacks). My plan is to migrate links to good ol' markdown files (each link gets its own .md file, with its YAML metadata specifying the link properties). This way if one of the tidbit is a longform post, rather than a link, the very markdown file will specify its content, in addition to a
longform
property set to True.https://github.com/srid/website/pull/13
cool; sometimes the best solution is the simplest.
I went back to Dhall for links! Newer Dhall provides TH for auto generating of Haskell types ... and I'm coming to accept the relative verbosity of Dhall syntax as an acceptable compromise over the benefits. https://github.com/srid/website/tree/a0e8db490c18d3205ea6953568771f4e2e57fb5a/content/tidbits/links
Lol so basically the feature port from pollen to rib/haskell/xml is nearly complete, just a tiny bit remaining. But this code is cursed. I am going to need to ask around for some design advice for sure.
I generally vastly prefer porting more code and then cleaning things once i have more examples
Feel free to ask for any sort of guidance. It would also, indirectly, help me with the guide-like documentation thingy I will be writing for Rib (using neuron, of course)