Does it make any sense to use trivial (i.e. constant/nonvarying) generators for types that functions/classes under test are parametric with respect to?

the type system already guarantees the function can't screw up by examining the values of those types, so you can basically put unit in there and still discover any bugs there are to find, no?

e.g. if testing the functor laws for (a -> b) -> f a -> f b, they'll use integers for a and b. what I'm wondering is if its' safe for me to be stupider and just use (). will i still uncover whatever bugs there are to be found?

Isn't it that for such type of testing, we want "counting" generator? We don't really care about the content, we care about it's uniqueness and position in resulting value (you can only move value around and apply arguments to it if it's polymorphic, can't you?). Simply spitting out unique values would uncover "moving around" immediately, while random generator could potentially need more runs in case it "accidentally" generates same values in multiple places.

very interesting. ok, so now that you've invalidated that hypothesis, how about the "dual" hypothesis: does using a generator that never produces the same thing twice for those type parameters work?

everything here is polymorphic! but if i have (badly) implemented insert in such a way that it only works once. in this case, a constant generator will say "yup, everything is fine!"

let's assume you're right that parametricity solves this problem for us! even then, you're also going to want to test code that is concrete, where a deterministic generator can always be detected, and then hide functionality or w/e

Seems like this is the weakness of not having dependent or refinement types to serve as proofs. Property tests generate values so that we can assert invariants on the domain and codomain of our functions which types alone are not able to express, no?

I think polymorphism alone is not sufficient to describe such properties otherwise we would be able to prove a functor is valid in Haskell alone, wouldn't we?

Does it make any sense to use trivial (i.e. constant/nonvarying) generators for types that functions/classes under test are parametric with respect to?

the type system already guarantees the function can't screw up by examining the values of those types, so you can basically put unit in there and still discover any bugs there are to find, no?

is there something prompting you to suspect otherwise?

Just my inexperience with property based testing :)

most of the "xyz-classes" collections of laws I've seen online use something like an integer for types that the class is parametric wrt

e.g. if testing the functor laws for

`(a -> b) -> f a -> f b`

, they'll use integers for`a`

and`b`

. what I'm wondering is if its' safe for me to be stupider and just use`()`

. will i still uncover whatever bugs there are to be found?well.. I would still use something with more than one value

Ok, why is that?

if you want to check

`fmap f . fmap g $ xs == fmap (f . g) xs`

you would need to use some`f`

, and if that`f`

is also`Identity`

it would matterbecause

`Identity ()`

has only one (not containing bottom) value, so it will always be`==`

to itselfthe

`fmap`

implementation never analyzes the`a`

or`b`

, that's guaranteed by the type system. so why would it matter whether I have a`Bool`

or a`()`

?right. and there is actually only one way to write the fmap function for

`Identity`

just based on the types I thinkIf you erase the newtype, that's:

I don't think it's possible to screw that up and still typecheck

so you wouldn't be testing it for

`Identity`

anywaybut I think this argument can be transferred to something more complex, e.g.

`Maybe`

?hmm. with

`Maybe`

can't I just return`Nothing`

for the`Just`

case and screw up?if I have that bad implementation, and I'm only ever using

`()`

s for the type parameter to`Maybe`

, i think I'd still catch itLike I said, I actually don't know whether this generalizes, that's why i'm asking. It just sounds like something that might make sense.

I'd also wager on "it doesn't matter", I'm just trying to "play devil's advocate"

yes, that's exactly what i'm looking for, thanks

e.g. does something go wrong if we apply this principle to

`Traversable`

? Is`Identity`

the only applicative I need to test with?for this using

`[()] -> Maybe ()`

will never be able to catch that this implementation is "wrong":whereas

`[Integer] -> Maybe Integer`

has a chance of doing sowhat is wrong with it?

like, what's the law we're testing?

wrong in the sense that it doesn't do the same thing as

`Data.Maybe.listToMaybe`

:`Nothing`

so the law requires an

`Eq`

constraint on`a`

?yes

like the

`fmap`

onehmm ok. I guess that's a good counterexample.

i'm having some trouble figuring out how to write the law

`listToMaybe [] = Nothing`

is law 0that's the problem with the example - the law is the implementation itself, basically

`listToMaybe (x : xs) = Just x`

i guess that's the second one?

this gives us a counterexample for the fmap thing as well I believe

for

`fmap :: (a -> b) -> [a] -> [b]`

I meanlike so?

hmm, i'm not sure we can actually write that second case

oops

ah, ok

yeah, exactly. so that one would be fine if we were looking at

`()`

s all the timeIsn't it that for such type of testing, we want "counting" generator? We don't really care about the content, we care about it's uniqueness and position in resulting value (you can only move value around and apply arguments to it if it's polymorphic, can't you?). Simply spitting out unique values would uncover "moving around" immediately, while random generator could potentially need more runs in case it "accidentally" generates same values in multiple places.

very interesting. ok, so now that you've invalidated that hypothesis, how about the "dual" hypothesis: does using a generator that never produces the same thing twice for those type parameters work?

@TheMatten you beat me to it!

(using

`Const ()`

for testing`Traversable`

s should also exhibit the same flaw, I think)then you can't test the property of "being a function"

because you'll never encounter the same

`x`

twicegranted, that is a very weird thing to test

`unsafePerformIO`

stuff may need more than property testing :sweat_smile:not only for actual functions:

if you've implemented

`Map k v`

as`[(k, v)]`

under the hood, you may want a property that for all`Map`

s, for each`k`

there is at most one`v`

this thread is too long for me to work through, so at the risk of repeating something already said

no, you still want randomness!

maybe i am implementing a set and property testing it

probably i want a law that says

`insert a (insert b s) == insert b (insert a s)`

everything here is polymorphic! but if i have (badly) implemented

`insert`

in such a way that it only works once. in this case, a constant generator will say "yup, everything is fine!"there is a trivial model where you map everything to

`()`

; using a constant generator can't differentiate your model from that one@Sandy Maguire What about the non random generator that always generates distinct values discussed above (e.g. natural numbers 0..infinity)?

would that fail to detect an unlawful implementation in your set example?

then you don't get automatic testing of crazy values

maybe my code only works up to 100 or something?

Your code doesn't have the opportunity to only work up to 100

because it is parametric

not true

well, i guess true if you don't have any constraints

but constraints can do arbitrarily crazy things :)

If it isn't parametric in the relevant type then it's not relevant to what we're talking about

if you're talking fully

`forall a. a -> ...`

then yeah, sure, it's fineexcept if your code doesn't work now if you get two of the same value?

I don't quite understand that last bit

trying hard to construct an example

maybe it can't be done without an

`Eq`

I'm talking about this bit. I don't quite understand what this means.

okay well let's assume i have an eq on

`a`

i have a bad impl:

maybe this can't exist without an eq constraint, so maybe you're fine

let's assume you're right that parametricity solves this problem for us! even then, you're also going to want to test code that is concrete, where a deterministic generator can always be detected, and then hide functionality or w/e

Seems like this is the weakness of not having dependent or refinement types to serve as proofs. Property tests generate values so that we can assert invariants on the domain and codomain of our functions which types alone are not able to express, no?

They're not proofs but they give us some amount of certainty that for a sampling of the domain our invariant holds.

I think polymorphism alone is not sufficient to describe such properties otherwise we would be able to prove a functor is valid in Haskell alone, wouldn't we?