<bobthecow>
basically it's a bytestream representation that allows serializing and unserializing objects.
<bobthecow>
when they come back to life, they have the exact same methods and everything as before.
<bobthecow>
so it's not just a list to look things up in, it's a map from object identifiers to checksums.
<bobthecow>
instead of saying "does this checksum exist" you can ask "what was the last checksum for that"
<bobthecow>
and because it's a hash, it's actually faster than searching a string.
<bobthecow>
i didn't write this particular bit, but i can imagine the reasons are (1) PStore + Marshal are really performant in Ruby, (2) Hash lookups are also really performant in Ruby, (3) PStore is transactional, so it's threadsafe, (4) by using a generic "store" object type, nanoc can store the dependency graph, checksums, rule caches, and a bunch of other things in the same way, reducing bugs and increasing code reuse. See: https://github.com/nanoc/nan
<bobthecow>
musicmatze: sorry for the wall-o-text :)
<musicmatze>
bobthecow: Thanks for your explanation! One thing: (3) threadsafe... doesnt matter, ruby is not real multithreading... but yeah, makes sense to me!
skroon_ has quit [Ping timeout: 245 seconds]
skroon has quit [Ping timeout: 245 seconds]
<musicmatze>
Sadly, I cannot implement the caching like this in my ssg, I guess. Things are much different :-P
<musicmatze>
... but a sorted list of checksums would have one problem: Some time, the file with the checksums would get really really large, as we cannot figure out which checksum is old :-P ... I have to think about this... :-D
<bobthecow>
what language are you building yours in?
<musicmatze>
C
<bobthecow>
you could make a second file every time you compile.
<bobthecow>
then move it over the first after compiling.
<musicmatze>
that does not work with inc. generation, as not everything gets compiled when compiling. Some checksums would be lost...
<bobthecow>
ahh, right.
<bobthecow>
well, no.
<bobthecow>
for incremental generation, you'd still have to check the source files.
<bobthecow>
to know whether to recompile something.
<bobthecow>
and that's where the checksum comes from.
<bobthecow>
so keep track of 'em during that phase.
<musicmatze>
aaah, you're right!
<bobthecow>
for really optimal stuff, you could generate the new list of checksums, sort 'em, then walk through the old list and new list side by side and decide which of the new needs to be recompiled.
<musicmatze>
and that's easy to implement, as I don't need the stuff around the checksums. After the loading is done, I can store the "old" checksums in a simple list...
<bobthecow>
that would save you searching, because you'd only have to look at most one or two forward.
<musicmatze>
that's really great! I think I will talk to you again later because of this, but I have to go to work now!
<bobthecow>
what is the current state of your ssg?
<musicmatze>
The kernel is kinda ready, I have to write at least five modules to have a complete ssg which can be compared to jekyll or nanoc!
jadd_ has quit [Quit: Leaving...]
<bobthecow>
how's it compare performance-wise so far?
<bobthecow>
you might even be able to skip the incremental compiles and DAG stuff.
<musicmatze>
But there will be changes in the kernel if I realize I need a functionality it does not have yet.
<bobthecow>
as tempting as the millisecond compile times are.
<bobthecow>
:P
<guardian>
yeah my site compiles really slowly :(
<guardian>
but I know why
<bobthecow>
oh yeah?
<guardian>
my images go through nanoc's pipeline
<bobthecow>
mine's not horrible anymore.
<bobthecow>
oh. yeah. don't do that.
<guardian>
an image has 3 reps
<guardian>
thumbnail, default, original
<guardian>
original is pass through
<bobthecow>
honestly, i'd define a second nanoc site in a subdirectory, and have it output images into ../static/
<guardian>
thumbnail uses smart cropping —> it detects the region in the original image that has interest
<bobthecow>
then you only compile that when it changes.
<guardian>
and default is just keep aspect ratio and resize to max width
jadd_ has joined #nanoc
<ddfreyne>
guardian: You could probably also use Make to pre-render the images
<ddfreyne>
maybe compile then to an output_assets/ dir and don't let those go through nanoc at all
<ddfreyne>
Assets are a bit of a pain point in nanoc, but I don't have a proper way around that
<ddfreyne>
nanoc doesn't provide a "clean" command to rm -rf tmp output, and I want to keep it that way
<bobthecow>
ddfreyne: is it a dependency graph issue that makes nanoc poorly suited to an image workflow?
<bobthecow>
it seems to work out fine for me, but i've got a decidedly not image-heavy site.
<bobthecow>
it seems like checksums and the like could almost always make images not recompile.
<bobthecow>
since it's not like they depend on anything.
jadd_ has quit [Ping timeout: 245 seconds]
tbm_ has quit [Quit: .]
tbm_ has joined #nanoc
tbm_ is now known as tbm
jachymko has joined #nanoc
jadd_ has joined #nanoc
alerante has joined #nanoc
jugglinmike has joined #nanoc
jachymko has quit [Ping timeout: 245 seconds]
jachymko has joined #nanoc
<ddfreyne>
bobthecow: Yeah. Any changes to config.yaml/nanoc.yaml, lib and some other things will cause everything to be recompiled
<ddfreyne>
That is clearly subobtimal, but required if you don't want a "nanoc clean" command
<guardian>
the point imho is to get everything through nanoc's pipeline
<guardian>
as soon as you start hacking around nanoc, with Rake or Make, then it shows a deficiency :)
<ddfreyne>
guardian: It is not super easy to fix :(
<guardian>
I know
jadd_ has quit [Ping timeout: 268 seconds]
alerante has quit [Remote host closed the connection]
<ddfreyne>
Dependency tracking on config.yaml is possible (that would be the fine-grained dependencies that have been in the queue for a long time)
<ddfreyne>
For cahnges in lib, the situation is harder. I'm thinking of *requiring* a lib/ structure like lib/filters/, lib/helpers/ and then explicitly declaring which helpers are used.
yogsototh has quit [Remote host closed the connection]
yogsototh has joined #nanoc
<guardian>
I won't be against structure inside lib/