warner has quit [Quit: ERC (IRC client for Emacs 25.1.1)]
<jamiew>
TIL `Promise.all`, nice one. will that await for all promises to resolve before returning, or do it as they return?
lanlink has quit [Quit: lanlink]
<cwahlers_>
jamiew: Promise.all immediately returns a Promise. if you call .then() on that returned promise, its success callback will only be called once all promises resolve
Adbray has quit [Quit: Quit]
plexigras2 has quit [Ping timeout: 260 seconds]
Adbray has joined #ipfs
jesse22_ has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<cwahlers_>
so yes, "await Promise.all([ ... ])" waits for all promises to resolve
ygrek has quit [Ping timeout: 240 seconds]
Taoki has joined #ipfs
TheTallesT has joined #ipfs
danordille has quit [Quit: My MacBook Air has gone to sleep. ZZZzzz…]
appa_ has quit [Ping timeout: 244 seconds]
IRCsum has quit [Remote host closed the connection]
IRCsum has joined #ipfs
appa_ has joined #ipfs
IRCsum has quit [Remote host closed the connection]
appa_ has quit [Read error: No route to host]
dimitarvp has quit [Quit: Bye]
appa_ has joined #ipfs
redfish has quit [Ping timeout: 256 seconds]
IRCsum has joined #ipfs
IRCsum has quit [Remote host closed the connection]
mindCrime has quit [Ping timeout: 260 seconds]
mrhavercamp has joined #ipfs
user_51 has joined #ipfs
lassulus_ has joined #ipfs
user51 has quit [Ping timeout: 256 seconds]
lassulus has quit [Ping timeout: 256 seconds]
lassulus_ is now known as lassulus
MDude has quit [Ping timeout: 268 seconds]
mindCrime has joined #ipfs
fredthomsen has quit [Quit: Leaving.]
mindCrime has quit [Ping timeout: 244 seconds]
ygrek has joined #ipfs
Jesin has quit [Quit: Leaving]
mrhavercamp has quit [Ping timeout: 265 seconds]
mindCrime has joined #ipfs
<midnightlightnin>
Hey all! Are there any tools already existing that help you manage the pinned items on your IPFS node? I've been doing some development of sites hosted there, and as I test and fix bugs, I realize I've got a ton of intermediary pinned content that I don't really need to keep around any more. However, using the `ipfs pin ls` command gives just the hashes of the content (not the name of the file that became that pinned
<midnightlightnin>
item through `ipfs add`, nor the date it was added as a local pin, etc.), and to do any pruning, I have to go visit each hash one-by-one on my local node, figure out if I want to keep it, and then `ipfs pin rm` it.
mindCrime has quit [Ping timeout: 256 seconds]
<midnightlightnin>
If there were some way to "label" a pin on your local node (or "tag" them), that would help with this issue. Or, that could be done with some second layer application; has anyone heard of a "IPFS pin management" tool already built?
omnigoat has joined #ipfs
mandric has joined #ipfs
warner has joined #ipfs
muvlon_ has quit [Quit: Leaving]
danordille has joined #ipfs
BeerHall has joined #ipfs
redfish has joined #ipfs
[itchyjunk] has quit [Remote host closed the connection]
saki has quit [Read error: Connection reset by peer]
TUSF has joined #ipfs
<TUSF>
midnightlightnin: you may want to check out the mfs. see the commands under "ipfs files"
<TUSF>
it's an api for dealing with ipfs files, through a sort of virtual filesystem.
danordille has quit [Quit: My MacBook Air has gone to sleep. ZZZzzz…]
<midnightlightnin>
Thanks! I did try the "files" command a bit, though it seems the basis of that virtual filesystem is COPYING the content from a hash, so the actual data would take up two places on your hard drive (the "pinned" area, and the "files" area)? Plus, it does let you clearly indicate what you want to keep (by copying it and naming it), but it then doesn't have a "show me all other things I have pinned that I've not yet added
<midnightlightnin>
to the virtual filesystem" to find orphans that are chewing up resources unnecessarily.
nublar has quit [Ping timeout: 256 seconds]
<TUSF>
the files api doesn't have a separate file store, so it would still only take up the same space
<TUSF>
that said, to look at all your pinned files, i suggest using "`ipfs pin ls --type=recursive"
<TUSF>
won't have any filenames, but it'll excuse all the smaller blocks you don't have *directly* pinned.
<TUSF>
exclude*
MDude has joined #ipfs
TUSF has quit [Read error: Connection reset by peer]
sanderp_ has quit [Read error: Connection reset by peer]
sanderp has quit [Ping timeout: 248 seconds]
DJ-AN0N has quit [Quit: DJ-AN0N]
makergrl has joined #ipfs
abueide has quit [Remote host closed the connection]
abueide has joined #ipfs
mrhavercamp has joined #ipfs
rendar has joined #ipfs
<shoku>
Hey guys, is it possible to get an ipfs hash for a file I don't have on my machine, but I see on a website? ex: I'm seeing a jpg I like and I wanna get the hash. Can I do that without dling the file?
mauz555 has quit [Read error: Connection reset by peer]
pecastro has joined #ipfs
shguwu has joined #ipfs
lanlink has joined #ipfs
mauz555_ has quit [Remote host closed the connection]
Mottengrotte has joined #ipfs
kkarhan has joined #ipfs
kkarhan has left #ipfs [#ipfs]
kkarhan has joined #ipfs
mauz555 has joined #ipfs
Alpha64 has quit [Quit: Alpha64]
kkarhan has left #ipfs ["-"]
user51 has joined #ipfs
user_51 has quit [Ping timeout: 240 seconds]
[itchyjunk] has quit [Remote host closed the connection]
ygrek has quit [Remote host closed the connection]
abueide has quit [Ping timeout: 256 seconds]
mauz555 has quit [Remote host closed the connection]
<lidel>
midnightlightnin, if you care about filenames, you can safely ignore low-level pinning and just use MFS. Files from MFS are implicitly pinned, so GC won't remove them. See discussion at: https://github.com/ipfs/ipfs-gui/issues/10#issuecomment-373928862
<lidel>
midnightlightnin, on the other hand, if you need to manage big pinsets, MFS may not be the right tool, in that case you need ipfs-cluster: https://cluster.ipfs.io/
<midnightlightnin>
lidel: Good to know MFS files are implicitly pinned. From the example in the linked discussion, they suggest instead of `ipfs pin /ipfs/Qm....` use `ipfs files cp -r /ipfs/Qm... /my-app/`, which works for the use case of "I want to help a friend seed their website" (where the friend can tell me the IPFS hash of the content), but if I'm the original creator of the content, and the source file is sitting on my hard
<midnightlightnin>
drive, I'd be doing `ipfs add -r ./my-awesome-webapp` to get it into IPFS. The only parallel command for that I see in the "files" grouping would be `ipfs files write`, but that command has no option for a recursive add (doing `ipfs files write --create /my-awesome-webapp ./my-awesome-webapp` error's out with "argument 'data' does not support directories").
<midnightlightnin>
The general feel of that linked issue thread is that there would ideally be a GUI/utility built on top of those API calls to make it more user friendly, and I could cobble together a shell script that recursively walked the directory in the local filesystem and did a a `ipfs files write` call for each file it found.
<midnightlightnin>
And a side-question: Given that I've been doing development as successive rounds of `ipfs add -r`, if I go through the work of going back through the projects I've published to IPFS and add them all to my node's MFS tree, can I safely do an `ipfs pin ls` and run `ipfs pin rm` on all of them? Will IPFS be smart enough to keep the pins from the MFS files, or does `ipfs pin rm` trump and will remove any MFS files that
<midnightlightnin>
reference that hash?
<lidel>
midnightlightnin, MFS will keep blocks around, pin rm only removes the explicit, low-level pin flag
<lidel>
midnightlightnin, as for adding files from your local hard drive, ipfs add + ipfs files cp will do the trick
malaclyps has quit [Ping timeout: 240 seconds]
malaclyps has joined #ipfs
<midnightlightnin>
Okay, so do I need to know the difference between "will keep blocks around" (what MFS does to mark content to keep around forever) and a "low-level pin flag"?
<midnightlightnin>
The issue with "ipfs add -r" followed by "ipfs files cp" is the nature of development doing that action over and over again, I don't wand the "ipfs add -r" to pin the intermediary versions after they're obsolete. Doing `ipfs add -r --pin=false ./my-webapp` could work, provided I do a `ipfs files cp` quickly after that command, so the data's still in cache?
<lidel>
midnightlightnin, you need to add without pinning: ipfs add -r --pin false
<lidel>
that way the only thing keeping data around is its presence in MFS
<lidel>
if you remove references from MFS, old data will be GC'd
<lidel>
you can also add with pin, and remove explicit pin after you finish cp to MFS
<lidel>
that being said, if you need anything more advanced, I'd look at https://cluster.ipfs.io/
saki has joined #ipfs
R[m] has joined #ipfs
tsglove has joined #ipfs
R[m] has left #ipfs [#ipfs]
lanlink has quit [Ping timeout: 240 seconds]
mandric has quit [Ping timeout: 244 seconds]
Steverman has joined #ipfs
hinderfully has quit [K-Lined]
<cipres>
is there any documentation on the repository versions used by each version of the go-ipfs daemon ? like an history of the repo format etc..
<cipres>
i've seen that between 0.4.15 and 0.4.16 there's been a version update it seems
dethos has quit [Ping timeout: 256 seconds]
<jamiew>
midnightlightnin: to simplify step 2 slightly you can use `ipfs add -q` -- output will only be the hash
<jamiew>
`-q -r` in this case specifically
<makeworld[m]>
is there any format where i can have files available on disk and am still able to reproduce the original ipfs hashes (eg. to republish them)?
guideline has quit [Ping timeout: 256 seconds]
mindCrime has joined #ipfs
itaipu has joined #ipfs
<cipres>
when you run ipfs daemon with the --migrate flag, does it do the repo migration by running the 'fs-repo-migrations' program or is it done by go-ipfs itself ?
<cipres>
=> Looking for suitable fs-repo-migrations binary.
<cipres>
got the answer sorry
shizy has joined #ipfs
[itchyjunk] has joined #ipfs
IRCsum has joined #ipfs
BeardRadiation has joined #ipfs
tcfhhrw has joined #ipfs
Davester1 has joined #ipfs
Mottengrotte has quit [Quit: Mottengrotte]
BeardRadiation has quit [Remote host closed the connection]
BeardRadiation has joined #ipfs
vivus has joined #ipfs
pcardune has joined #ipfs
crislar[m] has joined #ipfs
pcardune has quit [Remote host closed the connection]
pcardune has joined #ipfs
tcfhhrw has quit [Ping timeout: 260 seconds]
BeardRadiation has quit [Remote host closed the connection]
tcfhhrw has joined #ipfs
BeardRadiation has joined #ipfs
pcardune has quit [Remote host closed the connection]
mindCrime has quit [Ping timeout: 240 seconds]
tdiesler has joined #ipfs
dethos has joined #ipfs
biopandemic has joined #ipfs
saki has quit [Read error: Connection reset by peer]
mrhavercamp has quit [Ping timeout: 240 seconds]
fazo96 has quit [Quit: Konversation terminated!]
mandric has joined #ipfs
<swedneck[m]>
how can i check whether ipfs-cluster is working?
pcardune has joined #ipfs
pcardune has quit [Remote host closed the connection]
<swedneck[m]>
actually nvm, i can't seem to make it works anymore
<swedneck[m]>
it just shuts down..
pcardune has joined #ipfs
aalex__ has joined #ipfs
pcardune has quit [Remote host closed the connection]
BeardRadiation has quit [Remote host closed the connection]
ylp has quit [Quit: Leaving.]
ulrichard has quit [Remote host closed the connection]
xzha has quit [Remote host closed the connection]
pcardune has joined #ipfs
matt-h has quit [Quit: Leaving]
tcfhhrw has quit [Ping timeout: 265 seconds]
mindCrime has joined #ipfs
konubinix has quit [Remote host closed the connection]
cris_thor has joined #ipfs
mandric has quit [Ping timeout: 240 seconds]
matt-h has joined #ipfs
Caterpillar has quit [Quit: You were not made to live as brutes, but to follow virtue and knowledge.]
mandric has joined #ipfs
Caterpillar has joined #ipfs
Caterpillar has quit [Client Quit]
Davester1 has quit [Quit: Connection closed for inactivity]
tsglove has quit [Quit: Leaving]
Caterpillar has joined #ipfs
mandric has quit [Ping timeout: 244 seconds]
biopandemic has quit [Quit: Quit.]
biopandemic has joined #ipfs
ericwooley has joined #ipfs
<lidel>
makeworld[m], check ipfs add --nocopy, it uses filestore, which means it does not copy data into IPFS repo, but refers to files from your system (but it is up to you to ensure those files do not disappear in future).
PyHedgehog has quit [Quit: Connection closed for inactivity]
<swedneck[m]>
does nocopy put symlinks to everything added in .ipfs?
The_8472 has quit [Ping timeout: 240 seconds]
<swedneck[m]>
if not that's something i'd like to see, not sure if anyone else feels that's useful though
pcardune has quit [Remote host closed the connection]
pcardune has joined #ipfs
<nixze>
swedneck[m]: not sure I understand the question, nocopy (or rather the filstore) keeps links from hash to file block, so leaf blocks are not added to the block store, but I'm assuming that any higher level information is put in the block store
The_8472 has joined #ipfs
<swedneck[m]>
i'm thinking that if you had ~/.ipfs/mfs/ with symlinks to all the files added to ipfs with --nocopy, it'd be very easy to know what you've added to mfs
pcardune has quit [Ping timeout: 260 seconds]
<swedneck[m]>
actually, is mfs the same as filestore?
TUSF has joined #ipfs
<nixze>
swedneck[m]: filestore which is needed for nocopy is not the same as mfs
<swedneck[m]>
then i meant filestore, sorry
<nixze>
(confusing names)
<nixze>
swedneck[m]: you can run ipfs filestore verify - that dumps all source paths and their block hash (not file hash)
<nixze>
I have just been investigating nocopy usage the past week and this is just what I have found ... hope I'm wrong somehow.
<TUSF>
There should be a way to register aliases, like with git.
<swedneck[m]>
the thing i'd want before i use filestore is just being able to know where the file is on my system, and if it can't deal with moving anyways then a static symlink should be fine
<swedneck[m]>
a flag for files that tells you they're added to ipfs filestore would be neat at some point, maybe a milestone for the year 2035?
<nixze>
swedneck[m]: if you want to know which files are in the filestore then "ipfs filestore verify" can tell you, it is slow and impractical for that purpose - but still possible.
<swedneck[m]>
except it doesn't tell me where the file is
<nixze>
swedneck[m]: hu?
<swedneck[m]>
it tells me filestore has a certain file, but not where on my filesystem it is
<nixze>
ok, true it tells you where the file was when it was added
aalex__ has quit [Quit: This computer has gone to sleep]
<swedneck[m]>
it does?
<nixze>
hmm true
<nixze>
relative path used for the add command, but the store must already know this somehow
<swedneck[m]>
--help says <path>, but it doesn't give me a path, only a filename
<nixze>
my bad, sorry
<nixze>
example output: ok zb2rhYT4cdbaPcJWkpESwYLb3BdoVQbpKs2MXRA2xnBqHJiMA 262144 gentoo-distfiles/releases/x86/12.1/livedvd-x86-amd64-32ul-2012.1.iso 2414870528
<swedneck[m]>
doesn't show that for me
<swedneck[m]>
oh wait it shows it in relation to ~/ ?
<nixze>
in my case the files are added with "ipfs add -w -r --nocopy --local /home/distfiles/gentoo-distfiles/*"
<nixze>
hmm yes it must be, since gentoo-distfiles are not part of the IPFS object in my case