I'm failing some of the tests, but it's with a runtime error. When I tried it with the input from the test on my laptop, it got the correct answer after a few seconds. I think what's happening is that my code is being killed because it's too slow. I profiled with the following:
echo '4757362 10000' | ./superdigit +RTS -p
And got the following profile:
Fri Sep 25 21:40 2020 Time and Allocation Profiling Report (Final)
superdigit +RTS -p -RTS
total time = 1.46 secs (1456 ticks @ 1000 us, 1 processor)
total alloc = 1,084,500,184 bytes (excludes profiling overheads)
COST CENTRE MODULE SRC %time %alloc
digits.revDigits Main superDigit.hs:(7,5)-(9,57) 98.2 96.4
makeP Main superDigit.hs:17:1-36 0.9 3.1
individual inherited
COST CENTRE MODULE SRC no. entries %time %alloc %time %alloc
MAIN MAIN <built-in> 53 0 0.0 0.0 100.0 100.0
CAF Main <entire-module> 105 0 0.0 0.0 99.9 100.0
digitSum Main superDigit.hs:14:1-23 113 1 0.0 0.0 0.0 0.0
main Main superDigit.hs:36:1-22 106 1 0.0 0.0 99.9 100.0
interactLn Main superDigit.hs:34:1-39 107 1 0.1 0.0 99.9 100.0
doIt Main superDigit.hs:(28,1)-(30,32) 108 1 0.0 0.0 99.9 100.0
superDigit Main superDigit.hs:(23,1)-(25,39) 111 3 0.0 0.0 99.0 96.9
digitSum Main superDigit.hs:14:1-23 114 0 0.8 0.4 99.0 96.9
digits Main superDigit.hs:(5,1)-(9,57) 115 2 0.0 0.2 98.2 96.5
digits.revDigits Main superDigit.hs:(7,5)-(9,57) 116 70006 98.2 96.4 98.2 96.4
makeP Main superDigit.hs:17:1-36 112 1 0.9 3.1 0.9 3.1
readInts Main superDigit.hs:20:1-37 110 0 0.0 0.0 0.0 0.0
readInts Main superDigit.hs:20:1-37 109 1 0.0 0.0 0.0 0.0
CAF GHC.IO.Handle.FD <entire-module> 101 0 0.0 0.0 0.0 0.0
CAF GHC.IO.Handle.Text <entire-module> 99 0 0.1 0.0 0.1 0.0
CAF GHC.IO.Encoding <entire-module> 91 0 0.0 0.0 0.0 0.0
CAF Text.Read.Lex <entire-module> 84 0 0.0 0.0 0.0 0.0
CAF GHC.Conc.Signal <entire-module> 82 0 0.0 0.0 0.0 0.0
CAF GHC.IO.Encoding.Iconv <entire-module> 68 0 0.0 0.0 0.0 0.0
So it seems like the problem is my digits functions is too slow. Any suggestions on speeding it up? Do I need to resort to arrays? I don't even think hackerrank will let me use packages other than base?
Wouldn't reverse show up in the profile if that was the problem though? (I'm actually a bit confused about why is doesn't) but it seems like it's spending all the time in the subfunction revDigits, which doesn't do the actual reversing
you're using foldl1 on a list, which is unsafe.
and I don't understand your solution given the problem. why are you operating on integers instead of strings?
well if you're going for performance this will only get you so far, since you're operating on boxed Ints. for a bare-metal experience you'll need something like UArray
{-# LANGUAGE Strict #-}--{-# LANGUAGE BangPatterns #-}importqualifiedData.TextasTimportData.Text(Text)importqualifiedData.Text.IOasTimportData.Semigroup(stimes)importData.Char(digitToInt)importTextShowmakeP::Text->Integer->TextmakePnk=stimesknsuperDigit::Text->IntsuperDigitn|T.lengthn==1=digitToInt$T.headn|otherwise=superDigit.showt.T.foldl'(\nc->n+digitToIntc)0$ndoIt::Text->TextdoItinput=caseT.wordsinputof[n,k]->showt.superDigit$makePn(txtReadk)_->T.pack"invalid input"-- only called once for ktxtRead::Reada=>Text->atxtRead=read.T.unpack--txtShow :: Show a => a -> Text--txtShow = T.pack . showinteractLn::(Text->Text)->IO()interactLnf=T.putStrLn.f=<<T.getLine-- main = interactLn doItmain=T.putStrLn$doItinputinput=T.pack"7404954009694227446246375747227852213692570890717884174001587537145838723390362624487926131161112710589127423098959327020544003395792482625191721603328307774998124389641069884634086849138515079220750462317357487762780480576640689175346956135668451835480490089962406773267569650663927778867764315211280625033388271518264961090111547480467065229843613873499846390257375933040086863430523668050046930387013897062106309406874425001127890574986610018093859693455518413268914361859000614904461902442822577552997680098389183082654625098817411306985010658756762152160904278169491634807464356130877526392725432086439934006728914411061861235300979536190100734360684054557448454640750198466877185875290011114667186730452681943043971812380628117527172389889545776779555664826488520325234792648448625225364535053605515386730925070072896004645416713682004600636574389040662827182696337187610904694029221880801372864040345567230941110986028568372710970460116491983700312243090679537497139499778923997433720159174153 100000"
They say "don't try to guess what the perf problem is, use profiling". But I had the issue at one higher level up, I guessed that I needed to micro optimize instead of noticing the macro optimization of thinking about the problem differently.
So I'm working on the following hackerrank problem:
https://www.hackerrank.com/challenges/super-digit/problem
I have the following solution:
I'm failing some of the tests, but it's with a runtime error. When I tried it with the input from the test on my laptop, it got the correct answer after a few seconds. I think what's happening is that my code is being killed because it's too slow. I profiled with the following:
echo '4757362 10000' | ./superdigit +RTS -p
And got the following profile:
So it seems like the problem is my digits functions is too slow. Any suggestions on speeding it up? Do I need to resort to arrays? I don't even think hackerrank will let me use packages other than base?
You can try
DList
instead of plain list to avoid reversingWouldn't
reverse
show up in the profile if that was the problem though? (I'm actually a bit confused about why is doesn't) but it seems like it's spending all the time in the subfunction revDigits, which doesn't do the actual reversingHmm, yeah - I guess it then may be interesting to look at Core to see whether that arithmetics gets unboxed
you're using
foldl1
on a list, which is unsafe.and I don't understand your solution given the problem. why are you operating on integers instead of strings?
something along the schematic lines of
recurse . show . foldMap digitToInt
should be a complete solutionOh fromDigits is vestigial I think
I'm not using it
What's
recurse
?a placeholder for the name of the function that first checks whether the string length is 1 and contains the expression
foldMap (Sum . digitToInt) ?
oh wait
I'm getting confused
right, Sum
Oh, so you're saying I don't need to do all the converting
exactly!
That makes sense. Thanks! I'll try that
(also unsafe btw) :embarrassed:
digitToInt is partial?
yeah
ah well
Yeah, I passed a couple more cases. Thanks very much for your help! Now gotta debug the next issue haha
keep 'em coming!
New version, before I look into the next case
The issue is still speed, the next input is
I actually got a segfault when I tried to profile that
I guess it's time to switch to
text
:sweat_smile:True!
I'm curious why I'd be getting a segfault there. Super strange
you're parsing this behemoth into an integer!
Ah, of course
Ok, off to mess with it again
Ok, I now have
Still getting a long wait followed by a segfault
I have a feeling it has to do with
Show
andRead
, although I'm not sure how to avoid themI guess i'd better profile again
Ah, I think I'm going to have to set up a stack project in order to profile
text
?annoying
what about
cabal --enable-library-profiling
?I'm currently just using a single file, will that work for globally installed packages?
also, use
Text.foldl'
to avoid lazinessJames Sully said:
I guess not
Is that for if I have an existing cabal project?
yeah, wrong assumption
btw, are you compiling with
-O2
?With the strict fold, It gets the correct answer after a few seconds rather than segfaulting, but still not fast enough for hackerrank
Torsten Schmits said:
No, I didn't want to make assumptions about how hackerrank is compiling
I guess it's worth it for experimenting
-O2 Didn't seem to make a huge difference
man this is a heavy duty problem for something labelled "Difficulty: medium"
I'm gonna try profiling with stack
well if you're going for performance this will only get you so far, since you're operating on boxed
Int
s. for a bare-metal experience you'll need something likeUArray
no idea if ghc can optimize that away in your current implementation tho
I bet this is just read and show being slow. You can use text-show from hackage, not sure if you can use it on hacker rank
Oh, cool, I'll try that.
Although it doesn't seem like that's the issue in the profile? I'm not super confident in interpreting it
Don't forget foldl' and not foldl
I can't imagine the intended solution is Arrays, given the question is under
Practice > Functional Programming > Recursion
with medium difficulty haharight :smile:
Profile says makeP is slow, maybe make it's arguments strict with bangpatterns
try
{-# LANGUAGE Strict #-}
Data.Text.Strict version of getLine? I think that exists
also probably helpful to hardcode the input to rule out IO :smile:
Just waiting for
text-show
to buildprofile:
Unfortunately hackerrank won't let me use
text-show
I think I'm done with this one for now, this problem is wack
Oh I spoiled myself and looked at the solution. It's an algorithmic thing, not an optimization thing.
Goddamn hahaha
stimes is O(log n) btw, stimesIdempotent is O(1)
I like trying to make all of the problems optimization problems to practice benchmarking lol
Idk if you can use stimesIdempotent actually
It's just that
digitSum (k repeats of n) == k * digitSum n
this allows you to start the recursion with a much smaller number
final solution:
boooringggg :sweat_smile:
hahaha
At least I learned how to profile
and also not to
They say "don't try to guess what the perf problem is, use profiling". But I had the issue at one higher level up, I guessed that I needed to micro optimize instead of noticing the macro optimization of thinking about the problem differently.
Thanks very much for your help @Torsten Schmits @codygman !
it was a pleasure!