Secrets - Dhall

Welcome to the Functional Programming Zulip Chat Archive. You can join the chat here.

Barnabás Králik

Hi!

I'm evaluating Dhall for generating configs for Fedora CoreOS clusters. There is one topic that I haven't found any info on: that is secrets handling.
1) Ideally, the configuration generation should entail reading secrets from external databases and creating random secrets on-the-fly. As far as I understand this is contrary to Dhall's core concepts. Is there some "unsafe" hack to still do this by e.g. calling an external executable and using its output as import?
2) Is there any way to mark a value as 'secret', i.e. one that should not be shown in debug/error messages, just in the direct output?

Thanks for any input!

tristanC

Hello o/

1) You could cheat and read random Text using an http import, e.g. let secret = http://localhost:8080/secret-generator as Text in secret. Though this is unsafe as this requires an extra service to serve the data. Another solution would be to run a secret creation command before interpreting Dhall to import its output using an intermediate file. e.g. uuidgen > .build/secret.txt and then let secret = ./.build/secret.txt in secret
2) Have you tried using the --censor flag of the dhall-haskell implementation?

Barnabás Králik

2) Thanks! Should be sufficient.

1) The file-based attempt would be the best shot at this issue then. This would then require a two-stage compilation: firstly, I'll need to get a list of nonexistent secrets, then run the generator, then run dhall again with all the secrets readily available. What I don't see yet is how I'd be able to collect all the secrets scattered around the huge nested record that the configuration is. I would like to keep the configuration as low in syntactic noise as possible, so using some special type instead of records is a no-go. Is there some way to deconstruct any record into e.g. [{ a : Type, b : a }] ?

tristanC

Could you share a sample of that Fedora CoreOS cluster config looks like?

One way to manage such thing is by passing the secrets as a record, for example your configuration would look like this:

let Secrets = { root-password : Text, api-token : Text }
in \(secrets : Secrets) ->
    let config = {
         nested = {
            root-password = secrets.root-password
        }
     }
    in config

That way your configuration becomes a function that takes the secrets as an argument.

Then the first stage would ensure a secret record exists, for example echo "{ root-password = \"$(uuidgen)\", api-token = \"$(uuidgen)\" }" > .build/secrets.dhall

And the second stage would interpret the config function like so: dhall-to-yaml <<< './config.dhall ./.build/secrets.dhall'

tristanC

Actually it's probably better to generate a json or yaml file using whatever tool, then the second stage would be: dhall-to-yaml <<< "./config.dhall $(json-to-dhall ./.build/secrets.json)" (use yaml-to-dhall instead if needed)

Barnabás Králik

Sorry, I can't show anything right now that would add to this conversation.

So as a first step, I should have a record with all the required secrets as fields in it and a dhall file ("secrets database") that contains a value of this type. When extending the record type with a new field, the database would not match, thus I need to generate values for the missing files. How do I get to know which fields are missing? My only bad idea is specifying these as Optional and then adding a default None value, then doing something non-Dhall.

tristanC

Here is a more concrete example of that workflow. This input.dhall file defines a completable record (schema) of the input (used by this function. Then here is how to evaluate the configuration: procedure.

In other words, if your input type is defined in an input.dhall file, then using (yaml|json)-to-dhall './input.dhall' < generated-input result in a valid record with the optional attribute set to None.

A Kubernetes Operator for Zuul
A Kubernetes Operator for Zuul
A Kubernetes Operator for Zuul