sz0 has quit [Quit: Connection closed for inactivity]
mr_pink_ has joined #ipfs
<whyrusleeping>
darkdrgn2k[m]: back, computer decided it really wanted to update
<whyrusleeping>
darkdrgn2k[m]: if you abandon a request, the node will cache anything it has already fetched for a little while
mr_pink_ has quit [Quit: mr_pink_]
<HeavyMetal[m]>
whyrusleeping: if the request is for a 10MB file and then it abandon the request after 1MB will it still cache the whole 10MB or just 1MB?
<whyrusleeping>
just 1MB
<HeavyMetal[m]>
I see
dnel has quit [Read error: Connection reset by peer]
dnel has joined #ipfs
ulrichard has joined #ipfs
lanlink has quit [Ping timeout: 256 seconds]
<JCaesar>
There is a minimum, though… was that 1MB?
<JCaesar>
(well, depends on how the file was added, but the default…)
<isanisan>
Hi everybody! I've started keeping basically all of my non-sensitive data in my local IPFS node, but I now have a problem: I can't find a way to backup the data because doing `ipfs pin add` on another machine is really, really slow. I've been trying to solve this problem for a while now and am not sure what to do.
<isanisan>
So, my first question is: why is it slow? I want to copy, like 100GB of data, but data is only moving a at few KB per *minute*. How can I debug this?
<isanisan>
Second question is: I'd really like to just bypass bitswap and just manually upload a set of blocks from one node to another. Is there an easy way to do this?
<whyrusleeping>
isanisan: just to check, youre using ipfs 0.4.15 on both nodes, right?
gde33 has quit [Ping timeout: 240 seconds]
<JCaesar>
I don't know how to debug the whole thing, but one way to make the process a tiny bit less painful is to ipfs files cp the object first and then get it without occupying the global pin lock via ipfs refs -r
<isanisan>
I am not! I have been running 0.4.13 on both nodes, but I'm currently installing 0.4.15 on my laptop (I'm using nix, so it's building).
<isanisan>
I was just reading the changelogs for 0.4.{14,15} and didn't see anything that seemed relevant to this, and I've been having this problem since before 0.4.14 was released.
fazo96 has joined #ipfs
rendar has joined #ipfs
<isanisan>
Here's a related question: is it possible to just manually syncronize blocks from ~/.ipfs with rsync or something?
<isanisan>
My intuition tells me that this is probably not possible, but if it was, it would be a really easy solution to this problem.
hph^ has joined #ipfs
<whyrusleeping>
isanisan: you can do that, yeah
<whyrusleeping>
just rysync the ~/.ipfs/blocks/ directory over
<whyrusleeping>
and it should work fine
<whyrusleeping>
just make sure the daemons arent running while you do that
Fessus_ is now known as Fessus
hph^ has quit [Ping timeout: 255 seconds]
tobyai[m] has joined #ipfs
ssander_ has quit [Remote host closed the connection]
lanlink has joined #ipfs
<isanisan>
Really?? Sick! Thank you so much! I'm gonna try this now.
<whyrusleeping>
:)
<whyrusleeping>
my TODO list has two main items on it right now:
<whyrusleeping>
1) Fix network connectivity so IPNS sucks less
<whyrusleeping>
2) Make bitswap faster
konubinix has joined #ipfs
hph^ has joined #ipfs
ylp has joined #ipfs
ssander has joined #ipfs
Have-Quick has quit [Quit: Have-Quick]
AkhILman has quit [Ping timeout: 268 seconds]
Alpha64 has quit [Read error: Connection reset by peer]
hph^ has quit [Ping timeout: 256 seconds]
piti has joined #ipfs
buoyantair has joined #ipfs
Rusty78 has joined #ipfs
<Rusty78>
Quick question, I did some tests on multiple VPS servers and it seems that ipfs.io/ipfs/*multihash* is taking forever to find my files where earlier it took only a few seconds
<Rusty78>
Anyone else facing this issue? The other gateways are seemingly giving me the same issues, was there a big chance to the gateways recently?
<whyrusleeping>
there was, yeah
<whyrusleeping>
i don't know the details
<whyrusleeping>
but kyledrake and lgierth were switching a bunch of stuff around
<Rusty78>
whyrusleeping: Is it an open issue? I'm just wondering because I have a project built on IPFS and it worked really smoothly with fast propagation but kinda sucks without it :p
<Rusty78>
I can definitely wait if it's actively being looked at, just wondering if this is high on the list for issues to fix
<whyrusleeping>
Rusty78: you could definitely file an issue on ipfs/infrastructure
<whyrusleeping>
any reports of 'hey this doesnt work anymore' is good IMO
<Rusty78>
whyrusleeping: Will do, ty
hph^ has joined #ipfs
kvuser4 has joined #ipfs
vasco-santos has quit [Quit: Connection closed for inactivity]
hph^ has quit [Ping timeout: 276 seconds]
AkhILman has joined #ipfs
khernyo has joined #ipfs
gde33 has joined #ipfs
clownpriest has joined #ipfs
xzha has joined #ipfs
jungly has joined #ipfs
clemo has joined #ipfs
clownpriest has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
clownpriest has joined #ipfs
Krisostoomus has joined #ipfs
Krisostoomus has quit [K-Lined]
clownpriest has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
vasco-santos has joined #ipfs
}ls{ has joined #ipfs
jjy has joined #ipfs
jjy has quit [Client Quit]
bomb-on has joined #ipfs
plexigras has joined #ipfs
dnel has quit [Read error: Connection reset by peer]
dnel has joined #ipfs
cxl000 has joined #ipfs
kvuser4 has quit [Quit: Page closed]
rendar has quit [Ping timeout: 265 seconds]
bomb-on has quit [Quit: SO LONG, SUCKERS!]
xnbya2 has joined #ipfs
sknebel_ has joined #ipfs
Ekho- has joined #ipfs
DuClare_ has joined #ipfs
silotis_ has joined #ipfs
treora_ has joined #ipfs
rektide_ has joined #ipfs
spossiba_ has joined #ipfs
treora_ is now known as treroa
treroa is now known as treora_
nivekuil` has joined #ipfs
freeflying has joined #ipfs
riveter_ec2 has joined #ipfs
jhulten_ has joined #ipfs
kants_ has joined #ipfs
tobs_ has joined #ipfs
vimclippy_ has joined #ipfs
aananev has quit [*.net *.split]
silotis has quit [*.net *.split]
esph has quit [*.net *.split]
xnbya has quit [*.net *.split]
r0kk3rz has quit [*.net *.split]
Ekho has quit [*.net *.split]
treora has quit [*.net *.split]
riveter has quit [*.net *.split]
spossiba has quit [*.net *.split]
freeflying_ has quit [*.net *.split]
cypher has quit [*.net *.split]
furqan has quit [*.net *.split]
DuClare has quit [*.net *.split]
vimclippy has quit [*.net *.split]
sknebel has quit [*.net *.split]
nivekuil has quit [*.net *.split]
kants has quit [*.net *.split]
tobs has quit [*.net *.split]
rektide has quit [*.net *.split]
jhulten has quit [*.net *.split]
jhulten_ is now known as jhulten
tobs_ is now known as tobs
riveter_ec2 is now known as riveter
sudorobo has quit [Ping timeout: 260 seconds]
aananev has joined #ipfs
furqan has joined #ipfs
sudorobo has joined #ipfs
sudorobo has joined #ipfs
sudorobo has quit [Changing host]
fazo96 has quit [Ping timeout: 260 seconds]
cypher has joined #ipfs
esph has joined #ipfs
fazo96 has joined #ipfs
r0kk3rz has joined #ipfs
spossiba_ has quit [Changing host]
spossiba_ has joined #ipfs
bomb-on has joined #ipfs
rendar has joined #ipfs
treora_ is now known as treora
lanlink has quit [Ping timeout: 265 seconds]
gmoro has joined #ipfs
gmoro has quit [Remote host closed the connection]
gmoro has joined #ipfs
gabcoh has quit [Ping timeout: 240 seconds]
gmoro has quit [Remote host closed the connection]
gmoro has joined #ipfs
therabbit_ has joined #ipfs
tsglove has quit [Quit: Leaving]
todder has quit [Quit: is such a quitter...]
gabcoh has joined #ipfs
<darkdrgn2k[m]>
lidel`: Re companion. Can you clearify how it works. Does it just redirect all requests to a gatway or use a local gatway or what? Also is there any logging to actualy see what its doing (and not guess) When we where trying to figure out how to use out live stream through the plugin i twas very difficult to do so as we didnt know what was being redirected what was not.
<darkdrgn2k[m]>
also is there anyway to turn OFF the plguing, at times it would be good to turn it off and doesnt seem to be able to whout uninstalling it
reit has quit [Quit: Leaving]
<lidel>
darkdrgn2k[m], in short browser extension looks at all HTTP requests made by the browser and if URL path is a valid IPFS path (with correct CID) then it replaces such request to a request to local (external) gateway (usually users run go-ipfs on localhost)
mtodor has joined #ipfs
<lidel>
I think there is some logging in Browser Console, in Firefox you can open it via ctrl+shift+j or just click "Debug" next to extension in `about:debugging` screen
<darkdrgn2k[m]>
So what's embedded mode then
todder has joined #ipfs
<lidel>
embedded node is browserified js-ipfs running in the background page of the extension
<lidel>
it is mostly experimental, as it does not provide http gateway nor current Webextension APIs enable us to inject responses via custom protocol (this is being developed tho)
<lidel>
for now if you pick embedded node you will use public http gateway (ipfs.io) + you will get window.ipfs object provided by embedded js-ipfs node (so it is kind of a hybrid approach for now)
<lidel>
usually people run go-ifps on localhost as 'external' node, that way you get the most of the ecosystem
<lidel>
darkdrgn2k[m], we dont have explicit 'off' button yet (but it is being discussed here: https://github.com/ipfs-shipyard/ipfs-companion/issues/491), however you can disable redirect to custom gateway via button i browser action menu ("Switch to Public Gateway")
<darkdrgn2k[m]>
ok thanks for clarifying
<lidel>
if you open Page Console (ctrl+shift+k in firefox) and switch to Network tab, you will see which requesrs were redirected to custom gateway (in Domain column), in Chrome there is similar screen and there is additional 307 redirect that makes things even more clear
<darkdrgn2k[m]>
i was trying to get this to work with our LIFE FEED setup a few week ago when matt z was in town, but we couldnt get it to work / couldnt figure out if it was working!??!
<lidel>
by default these screens may not show XHR requests, you may want to tweak filters to display them too, if your page does the fetching via javascript
<darkdrgn2k[m]>
(i use f12 in network, and it does show ajax requests)
Mateon1 has quit [Ping timeout: 248 seconds]
Mateon3 has joined #ipfs
<darkdrgn2k[m]>
anyway from the sounds of it unless you have a local ipfs server installed, the pluging will just use a remote gateway, right?
<lidel>
yeah it should show which gateway is being used in the status after you click browser action icon
<darkdrgn2k[m]>
im going to try to test it later with the plugin on the other computer (dont want to install it on this one :P)
<lidel>
darkdrgn2k[m], quick test you can do is to load page when local gateway is used, then click "Switch to Public Gateway" and refresh page, to reload it via public gateway
Mateon3 is now known as Mateon1
<lidel>
popular problem is when your local gateway does not have probper CORS headers set up
<darkdrgn2k[m]>
i mean the xrm
<lidel>
and if public gw works and local one does not, it usually means CORS issue
<darkdrgn2k[m]>
(ipns over pubsub is a better when it doesnt stall.. but is not running on most getways since its experimental)
ericxtang has joined #ipfs
<darkdrgn2k[m]>
sometimes it much better. other times much wors
<darkdrgn2k[m]>
we have to run 30 second cunnk MINUM to allow enough time for this variance but today it seems it should be more like 40
<darkdrgn2k[m]>
it may also be that my test machines here is behind a nat..
ericxtang has quit [Remote host closed the connection]
user_51 has joined #ipfs
user51 has quit [Ping timeout: 248 seconds]
ericxtang has joined #ipfs
Rayben has joined #ipfs
Rayben has quit [Max SendQ exceeded]
hph^ has quit [Ping timeout: 245 seconds]
sknebel_ is now known as sknebel
]BFG[ has quit [Remote host closed the connection]
colatkinson has joined #ipfs
hph^ has joined #ipfs
rngkll has joined #ipfs
xylaoo has joined #ipfs
xylaoo has quit [K-Lined]
goiko_ has joined #ipfs
goiko_ has quit [Client Quit]
goiko has quit [Ping timeout: 248 seconds]
mindCrime has joined #ipfs
dnel has quit [Read error: Connection reset by peer]
mazeto has joined #ipfs
dnel has joined #ipfs
mindCrime has quit [Ping timeout: 256 seconds]
dimitarvp has joined #ipfs
clownpriest has joined #ipfs
hph^ has quit [Ping timeout: 260 seconds]
gmoro_ has joined #ipfs
hph^ has joined #ipfs
gmoro has quit [Ping timeout: 260 seconds]
rodolf0 has joined #ipfs
SunflowerSociety has quit [Ping timeout: 260 seconds]
gmoro_ has quit [Remote host closed the connection]
gmoro has joined #ipfs
hph^ has quit [Ping timeout: 240 seconds]
gmoro has quit [Remote host closed the connection]
gmoro has joined #ipfs
lanlink has joined #ipfs
gmoro has quit [Remote host closed the connection]
gmoro has joined #ipfs
buoyantair_ has joined #ipfs
buoyantair has quit [Ping timeout: 260 seconds]
mindCrime has joined #ipfs
<whyrusleeping>
darkdrgn2k[m]: is that transfer speed graph made while using mars.i.ipfs.team as the gateway?
<whyrusleeping>
if using mars, i was looking at disk write latency numbers for it earlier, and they are pretty bad. I'll see about spinning up a node with a more performant disk to see if things work better for you
<darkdrgn2k[m]>
No
Caterpillar has quit [Quit: You were not made to live as brutes, but to follow virtue and knowledge.]
dnel has quit [Quit: ZNC 1.6.5+deb1 - http://znc.in]
therabbit_ has quit [Remote host closed the connection]
<darkdrgn2k[m]>
Also would it help to ipfs add content on multiple servers
<darkdrgn2k[m]>
(and I assume scp would be faster then ipfs pin)
buoyantair has quit [Quit: Leaving]
<whyrusleeping>
forwarding 4001 should do the trick
<whyrusleeping>
pinning on multiple servers would help yes
<whyrusleeping>
though for this amount of data, pinning should be plenty fast enough
<darkdrgn2k[m]>
What's faster. Scp to server then ipfs add. Or just ipfs pin
ulrichard has quit [Remote host closed the connection]
<darkdrgn2k[m]>
And any way to check. If it worked
<darkdrgn2k[m]>
(ie why does address say 1042)
cmbrnt has quit [Ping timeout: 260 seconds]
Alpha64 has joined #ipfs
<jamiew>
so is igis getting some wind behind as of yesterday's GitHub+Microsoft news? :)
Caterpillar has joined #ipfs
buoyantair__ has joined #ipfs
<r0kk3rz>
people still really hate microsoft huh
buoyantair_ has quit [Ping timeout: 260 seconds]
carsonfarmer has joined #ipfs
Alpha64 has quit [Ping timeout: 260 seconds]
ylp has left #ipfs [#ipfs]
<whyrusleeping>
darkdrgn2k[m]: the address probably says port 1042 because of that being the outbound port peers see from your NAT
<whyrusleeping>
jamiew: i definitely poked Magik6k_ to make the front page of it work today because of that ;)
<jamiew>
it works, just a little slow :)
<whyrusleeping>
r0kk3rz: they havent really given any good examples of not fucking up a company post acquisition
<r0kk3rz>
this is true
<jamiew>
I'd argue that LinkedIn is just as fucked up as when MS acquired them
<jamiew>
so a net-neutral
<jamiew>
VSCode scored them some points in my book ¯¯\_(ツ)_/¯¯ and all the consistently solid MS Research work
<r0kk3rz>
indeed any major change in leadership of github could be disasterous
<whyrusleeping>
the agressiveness of them trying to get me to add people has definitely gone up
<r0kk3rz>
jamiew: yeah vscode is pretty good, generally ive been quite happy with their developer offering for years
<whyrusleeping>
jamiew: yeah, MS Research is definitely solid
<whyrusleeping>
just wish it wasnt all patented so hard
<whyrusleeping>
really cool research
<whyrusleeping>
but basically useless to anyone but MS
<r0kk3rz>
and half the time they dont put it in any products :)
<whyrusleeping>
yeah...
<whyrusleeping>
such a waste of lamports talent TBH
<jamiew>
well +1 igis, at least for syndication/backup
<jamiew>
I guess the neckbeard stack will be gittea backed by igis or similar?
xzha has quit [Ping timeout: 240 seconds]
<r0kk3rz>
gitlab is pretty good these days
<r0kk3rz>
and neckbeard friendly enough for gnome to use
mtodor has quit [Remote host closed the connection]
astrofog has joined #ipfs
mtodor has joined #ipfs
<mikeal_>
my only concern with the GitHub aquisition is the public data feeds and apis
<mikeal_>
They bought LinkedIn for the data, and it got harder to pull data out shortly after
<r0kk3rz>
i feel real sorry for any company with github private repos
jaycarpenter has joined #ipfs
<mikeal_>
Even kicked down the rate limits would make my life a lot harder
<mikeal_>
I think the GitHub EULA prevents them from looking too deeply at private data, and now with GDPR their hands are tied even more
<r0kk3rz>
eventually they might break the APIs but only once they have microsoft certified alternatives for everything
<mikeal_>
I recall a few years ago they wanted to know something about the private data and basically had to go and ask a bunch of their customers if they would participate in the research
mtodor has quit [Ping timeout: 260 seconds]
mtodor has joined #ipfs
mtodor has quit [Remote host closed the connection]
hph^ has quit [Read error: Connection reset by peer]
hph^ has joined #ipfs
fazo96 has quit [Quit: Konversation terminated!]
ssander has quit [Remote host closed the connection]
ssander has joined #ipfs
James_Epp has joined #ipfs
mazeto has quit [Ping timeout: 256 seconds]
Alpha64 has joined #ipfs
Encrypt has joined #ipfs
drrty has joined #ipfs
<darkdrgn2k[m]>
what specs do you need to run a gateway?
<voker57>
lots of RAM
DuClare_ has quit [Changing host]
DuClare_ has joined #ipfs
<voker57>
and automatic restart of IPFS when it runs out of RAM anyway
DuClare_ is now known as DuClare
<voker57>
other resources are not rly significant, depends on how much you want to serve
Alpha64 has quit [Ping timeout: 256 seconds]
jaycarpenter has quit [Quit: Page closed]
<ChrisMatthieu_>
I just recorded a new @computesio video demonstrating how to build a Decentralized API powered by Computes & IPFS! Working on blog post now... https://youtu.be/RBVDhKrP9io
_PiGreco_ has quit [Ping timeout: 260 seconds]
_PiGreco_ has joined #ipfs
drrty has quit [Ping timeout: 256 seconds]
lnostdal has quit [Ping timeout: 265 seconds]
rodolf0 has quit [Remote host closed the connection]
rodolf0 has joined #ipfs
petroav has joined #ipfs
rodolf0 has quit [Remote host closed the connection]
rodolf0 has joined #ipfs
drrty has joined #ipfs
ericxtang has quit [Remote host closed the connection]
<gozala>
lidel: is there docs somewhere for it ? Is it using gitremote-helpers ?
b5 has quit [Ping timeout: 260 seconds]
drrty has quit [Max SendQ exceeded]
zautomata has joined #ipfs
<lidel>
gozala, no idea, I did not play with it yet, but I dont think there is anything more than the repo at https://github.com/ipfs-shipyard/IGiS (opening an issue about lack of docs may be a good step)
zautomata has quit [Changing host]
zautomata has joined #ipfs
<gozala>
lidel: it looks like a viewer maybe, I don’t see how I would add ipfs remots
<lidel>
..and I thought being blocked by GFW was the sign we've made it
<lidel>
koalalorenzo[m], alas, nsfw links are against our Code of Conduct: https://git.io/vVBS, please don't paste them in future
<koalalorenzo[m]>
Sorry 🙂It has nothing NSFW inside, just text. Still 😃 my bad!
petroav has quit [Ping timeout: 264 seconds]
<lidel>
:)
<lidel>
darkdrgn2k[m], fyi opened a PR with a fix for the bug we discussed earlier today: https://github.com/ipfs-shipyard/ipfs-companion/pull/494 (it is a workaround for a weird edge case, so any feedback - or questions - would be appreciated)