I'm evaluating Dhall for generating configs for Fedora CoreOS clusters. There is one topic that I haven't found any info on: that is secrets handling.
1) Ideally, the configuration generation should entail reading secrets from external databases and creating random secrets on-the-fly. As far as I understand this is contrary to Dhall's core concepts. Is there some "unsafe" hack to still do this by e.g. calling an external executable and using its output as import?
2) Is there any way to mark a value as 'secret', i.e. one that should not be shown in debug/error messages, just in the direct output?
1) You could cheat and read random Text using an http import, e.g. let secret = http://localhost:8080/secret-generator as Text in secret. Though this is unsafe as this requires an extra service to serve the data. Another solution would be to run a secret creation command before interpreting Dhall to import its output using an intermediate file. e.g. uuidgen > .build/secret.txt and then let secret = ./.build/secret.txt in secret
2) Have you tried using the --censor flag of the dhall-haskell implementation?
1) The file-based attempt would be the best shot at this issue then. This would then require a two-stage compilation: firstly, I'll need to get a list of nonexistent secrets, then run the generator, then run dhall again with all the secrets readily available. What I don't see yet is how I'd be able to collect all the secrets scattered around the huge nested record that the configuration is. I would like to keep the configuration as low in syntactic noise as possible, so using some special type instead of records is a no-go. Is there some way to deconstruct any record into e.g. [{ a : Type, b : a }] ?
Could you share a sample of that Fedora CoreOS cluster config looks like?
One way to manage such thing is by passing the secrets as a record, for example your configuration would look like this:
let Secrets = { root-password : Text, api-token : Text }
in \(secrets : Secrets) ->
let config = {
nested = {
root-password = secrets.root-password
}
}
in config
That way your configuration becomes a function that takes the secrets as an argument.
Then the first stage would ensure a secret record exists, for example echo "{ root-password = \"$(uuidgen)\", api-token = \"$(uuidgen)\" }" > .build/secrets.dhall
And the second stage would interpret the config function like so: dhall-to-yaml <<< './config.dhall ./.build/secrets.dhall'
Actually it's probably better to generate a json or yaml file using whatever tool, then the second stage would be: dhall-to-yaml <<< "./config.dhall $(json-to-dhall ./.build/secrets.json)" (use yaml-to-dhall instead if needed)
Sorry, I can't show anything right now that would add to this conversation.
So as a first step, I should have a record with all the required secrets as fields in it and a dhall file ("secrets database") that contains a value of this type. When extending the record type with a new field, the database would not match, thus I need to generate values for the missing files. How do I get to know which fields are missing? My only bad idea is specifying these as Optional and then adding a default None value, then doing something non-Dhall.
Here is a more concrete example of that workflow. This input.dhall file defines a completable record (schema) of the input (used by this function. Then here is how to evaluate the configuration: procedure.
In other words, if your input type is defined in an input.dhall file, then using (yaml|json)-to-dhall './input.dhall' < generated-input result in a valid record with the optional attribute set to None.
Hi!
I'm evaluating Dhall for generating configs for Fedora CoreOS clusters. There is one topic that I haven't found any info on: that is secrets handling.
1) Ideally, the configuration generation should entail reading secrets from external databases and creating random secrets on-the-fly. As far as I understand this is contrary to Dhall's core concepts. Is there some "unsafe" hack to still do this by e.g. calling an external executable and using its output as import?
2) Is there any way to mark a value as 'secret', i.e. one that should not be shown in debug/error messages, just in the direct output?
Thanks for any input!
Hello o/
1) You could cheat and read random Text using an http import, e.g.
let secret = http://localhost:8080/secret-generator as Text in secret
. Though this is unsafe as this requires an extra service to serve the data. Another solution would be to run a secret creation command before interpreting Dhall to import its output using an intermediate file. e.g.uuidgen > .build/secret.txt
and thenlet secret = ./.build/secret.txt in secret
2) Have you tried using the
--censor
flag of the dhall-haskell implementation?2) Thanks! Should be sufficient.
1) The file-based attempt would be the best shot at this issue then. This would then require a two-stage compilation: firstly, I'll need to get a list of nonexistent secrets, then run the generator, then run dhall again with all the secrets readily available. What I don't see yet is how I'd be able to collect all the secrets scattered around the huge nested record that the configuration is. I would like to keep the configuration as low in syntactic noise as possible, so using some special type instead of records is a no-go. Is there some way to deconstruct any record into e.g. [{ a : Type, b : a }] ?
Could you share a sample of that Fedora CoreOS cluster config looks like?
One way to manage such thing is by passing the secrets as a record, for example your configuration would look like this:
That way your configuration becomes a function that takes the secrets as an argument.
Then the first stage would ensure a secret record exists, for example
echo "{ root-password = \"$(uuidgen)\", api-token = \"$(uuidgen)\" }" > .build/secrets.dhall
And the second stage would interpret the config function like so:
dhall-to-yaml <<< './config.dhall ./.build/secrets.dhall'
Actually it's probably better to generate a json or yaml file using whatever tool, then the second stage would be:
dhall-to-yaml <<< "./config.dhall $(json-to-dhall ./.build/secrets.json)"
(use yaml-to-dhall instead if needed)Sorry, I can't show anything right now that would add to this conversation.
So as a first step, I should have a record with all the required secrets as fields in it and a dhall file ("secrets database") that contains a value of this type. When extending the record type with a new field, the database would not match, thus I need to generate values for the missing files. How do I get to know which fields are missing? My only bad idea is specifying these as Optional and then adding a default None value, then doing something non-Dhall.
Here is a more concrete example of that workflow. This input.dhall file defines a completable record (schema) of the input (used by this function. Then here is how to evaluate the configuration: procedure.
In other words, if your input type is defined in an
input.dhall
file, then using(yaml|json)-to-dhall './input.dhall' < generated-input
result in a valid record with the optional attribute set to None.Thank you!