urnix has quit [Remote host closed the connection]
<dignifiedquire>
daviddias: :D:D
<dignifiedquire>
not yet, working on i
<daviddias>
Wasn't it the problem ??
ggoZ has joined #ipfs
ryepdx has quit [Ping timeout: 250 seconds]
_Vi has quit [Ping timeout: 260 seconds]
<dignifiedquire>
daviddias: I got it
<dignifiedquire>
needed to replace all the process.cwd
<dignifiedquire>
nooo
<dignifiedquire>
it's failing again -.-
<dignifiedquire>
wtf????
ryepdx has joined #ipfs
<daviddias>
Oh this is ridiculous, the reason why my internet was shut off is because they cut the signal to move to the new place, which will only happen tomorrow, and now I can't have Internet at home and because this place is like a faraday cage, I can use 4G inside
<daviddias>
What is the error now? I need to relocate to a coffee shop
<dignifiedquire>
same error
Boomerang has joined #ipfs
<dignifiedquire>
daviddias: looks lik the error has sth to do with clean up
<dignifiedquire>
if I run mocha on each of cli, core and http individually all pass fine now
cemerick has quit [Ping timeout: 276 seconds]
rhalff has quit [Ping timeout: 260 seconds]
rhalff has joined #ipfs
computerfreak has quit [Read error: Connection timed out]
computerfreak has joined #ipfs
afisk has quit [Remote host closed the connection]
zz_r04r is now known as r04r
ylp1 has quit [Ping timeout: 260 seconds]
ylp1 has joined #ipfs
pfraze has joined #ipfs
afisk has joined #ipfs
tmg has quit [Ping timeout: 250 seconds]
<daviddias>
Same error?
<daviddias>
Clean as in, move to aegir?
afisk has quit [Remote host closed the connection]
afisk has joined #ipfs
<dignifiedquire>
daviddias: made progress
<dignifiedquire>
but now I'm seeing sth interesting
<dignifiedquire>
daviddias: in cli-tests/test-id.js I get this output
<dignifiedquire>
daviddias: started using different folders repo-test-run-cli and repo-test-run-http and things are looking much better
pfraze has quit [Remote host closed the connection]
_Vi has joined #ipfs
pfraze has joined #ipfs
jaboja has joined #ipfs
<richardlitt>
nicolagreco: Added to calendar
<dignifiedquire>
daviddias: getting closer
<dignifiedquire>
daviddias: will have a post mortem for you soon
<haad_>
nicolagreco: it just gets stuck at "Starting IPFS daemon..."?
<nicolagreco>
haad_: yup
<daviddias>
dignifiedquire: sounds like ipfs-core tests aren't finished when the cli and http starts (with aegir) and that is causing them to try to reuse the same temp repo
<dignifiedquire>
daviddias: that's part of the equation, the other is bad global state
<haad_>
daviddias: on that ^ note, I have someone working to fix the npm2 vs. npm3 problem in ipfsd-ctl
<dignifiedquire>
daviddias: though shall not use process.env ...
<nicolagreco>
haad_: npm--version == 3.2.7 or similar
<haad_>
nicolagreco: :O
<haad_>
nicolagreco: hmmm... that's weird
<nicolagreco>
I am not on my laptop yet
<haad_>
nicolagreco: have you upgraded 0.4?
<haad_>
+to
<M-Sonata>
I'm getting an error when I try to mount:<br>10:31:35.798 ERROR node: error mounting: fusermount: fork/exec /bin/fusermount: cannot allocate memory mount_unix.go:101
ruby32 has quit []
Qwertie has quit [Ping timeout: 276 seconds]
zootella has joined #ipfs
Qwertie has joined #ipfs
ylp1 has quit [Quit: Leaving.]
Looking has quit [Ping timeout: 250 seconds]
mildred has quit [Quit: Leaving.]
Qwertie has quit [Ping timeout: 268 seconds]
<dignifiedquire>
daviddias: we need to talk
<voxelot>
uhoh, dignifiedquire is going to break up with you daviddias
mildred1 has quit [Ping timeout: 252 seconds]
<ipfsbot>
[js-ipfs] dignifiedquire pushed 1 new commit to aegir: https://git.io/vwnZJ
<ipfsbot>
js-ipfs/aegir 0848298 dignifiedquire: try to fix process.env and fail
Qwertie has joined #ipfs
Looking has joined #ipfs
<voxelot>
dignifiedquire: thanks for putting so much work into getting aegir on js-ipfs, was having some testing issues last night
<A124>
Found another problem.
<dignifiedquire>
voxelot: I can imagine..I've been having testing issues the whole day
<A124>
When IFPS is running, there are CPU context switches spikes, from average 3k per whole system to 16k, they are fairly regular in pattern (once oevery few seconds)
jedahan has joined #ipfs
<A124>
There is also problem in the CPU scheduler, which is (finally after years) put to light and finally patch should go in. That does consume a lot of cpu when context switches happen to different cores and stuff.
<A124>
So combination of those spikes + bad scheduler (the default CF on most systems) results in spikes and jitter in realtime applications plus wastes a lot of CPU.
<A124>
Yeah, I should find the reference. The CPU scheduler is presented on this years conference. But older kernels plus bad application would still suffer.
<A124>
Their measure performance hit was 10-20% usually, 20% for databases and up to 138x for scientific corner cases.
<A124>
ACTION notes, that he wanted to do multithreaded client for his application, but after getting more background, decided using polling, as a lot of connections and threads, hence context switching makes whole application both slower and more demanding on system and CPU. ... Depending on type of application and scenario, other people also use Threads + Fibers combination. (Few threads, each with few fibers.) Fibers are not pre-emptive, so less scheduling ove
<A124>
The scale is little confusing, but here is what is actually does.
Qwertie has quit [Ping timeout: 268 seconds]
<A124>
Spawns 50x process, and context switching goes from quiet 2k, to 3-20k madness.
* A124
wishes good morning as well. *looks at sun touching down*
Qwertie has joined #ipfs
<noffle>
hey why
<noffle>
voxelot: great!
nycoliver has quit [Ping timeout: 244 seconds]
<noffle>
it sounds like something I hit on js-ipfs before; glad you got it sorted out
* A124
notes it spawned 100 processes/threads more in the time from posting image to now.
zorglub27 has joined #ipfs
<whyrusleeping>
A124: whoa, thank you
<whyrusleeping>
these graphs are super nice
<A124>
You are very welcome, read my notes above, it goes more in depth with reference.
Tv` has joined #ipfs
<whyrusleeping>
yeah, catching up now
<whyrusleeping>
A124: we had a similar problem before...
Qwertie has quit [Ping timeout: 276 seconds]
<whyrusleeping>
i'm trying to remember what it was
<whyrusleeping>
it was exactly this though, a huge spike in context switches
jedahan has joined #ipfs
<A124>
Actually, before I shut down, and started again the spike was periodic and the rest silent, but at plus few k more switches just for ipfs.
Qwertie has joined #ipfs
<A124>
All in all, if you have networked application doing a lot of sockets / connections, if you spawn one connection for each, that results in problems.
<anaptrix>
A124: yeah... that could be part of it
<A124>
So depending on type of connection meta / data, polling or threads + polling or threads + fibers at least are appropriate.
<whyrusleeping>
which in theory should prevent us from opening too many sockets
<A124>
polling... one socket, multiple contexts = multiple connections.
<A124>
Of course for performance at least one thread for cpu code would benefit and load balance the connections among them.
<whyrusleeping>
we multiplex streams over individual sockets
<whyrusleeping>
so one socket per peer
<whyrusleeping>
A124: do you have a github account?
<A124>
Cannot comment more in depth, as I was concerned primarily at low resource highperf thing with small data blocks. So did use only single core plus polling, as network is the limit for me (latency).
<A124>
Not yet.
<whyrusleeping>
okay, i'll open an issue to help track this problem
Qwertie has quit [Ping timeout: 250 seconds]
<A124>
Excellent :)
<whyrusleeping>
if you don't want to make an account thats totally fine
<whyrusleeping>
thank you for the investigative work
<A124>
You can add the notes I said before with the references, of course.
Qwertie has joined #ipfs
Akaibu has quit [Quit: Connection closed for inactivity]
<dignifiedquire>
daviddias: my brain is gone, so I'm going to stop for now, would appreciate if you could start reviewing my aegir PR, will hopefully get this fixed tomorrow
<A124>
Got into the phase where noise gone down, still +2-3k switches just for IFPS alone, lot of processes, but except spikes it is not going above 5k
<A124>
Will read teh debugging, nost sure if able to do so though.
<whyrusleeping>
A124: no worries, i posted it as a general guide for everyone
<whyrusleeping>
A124: I also list out the information you can give me to be able to debug the issues
Qwertie has quit [Ping timeout: 268 seconds]
nunofmn1 has joined #ipfs
<haad_>
noffle: don't have a readme yet but the interface should be easy enough to understand the usage. if not see tests. you can use that with ipfs-log by doing: log.add(JSON.stringify(counter.payload)) and counter = Counter.from(JSON.parse(log.items[0]))
<haad_>
noffle, nicolagreco, voxelotm whoelsewasinterested: see https://github.com/haadcode/crdts for humble beginnings, feedback highly appreciated!
<A124>
Those spikes are indeed IPFS.
<haad_>
voxelot ^
<voxelot>
haad_ o/
<voxelot>
looking now
* Looking
now
nunofmn1 has quit [Ping timeout: 276 seconds]
<A124>
whyrusleeping Sorry for excessive quoting, if I should not quote that often tell me. ... Additional info: I do store (pinned) some different stuff from network, including the geoip database (recursively of course)
<ipfsbot>
[js-ipfs] dignifiedquire pushed 1 new commit to aegir: https://git.io/vwni9
<ipfsbot>
js-ipfs/aegir 103666f dignifiedquire: More fixes
<whyrusleeping>
A124: nah, quoting is fine
Qwertie has quit [Ping timeout: 268 seconds]
<A124>
Else, not doing anything with it in particular, just letting it run in swarm. Just seen spikes in cpu jitter (important for realtime stuff), so I looked also into context, and found this problem.
<A124>
But in general I felt the machine is slower with repetitive slow downs, which do correspond to the spikes, as found out later.
Qwertie has joined #ipfs
Looking has quit [Remote host closed the connection]
jedahan has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
Qwertie has quit [Ping timeout: 276 seconds]
Qwertie has joined #ipfs
<nicolagreco>
richardlitt jbenet, daviddias, dignifiedquire we are going to start the meeting as soon as we find juan
<richardlitt>
Sounds good to me. Don't see him yet.
<richardlitt>
Is he coming, in person?
<nicolagreco>
I think he is
<dignifiedquire>
didn't you say est 1pm?
<nicolagreco>
want to join us physically richardlitt ?
<richardlitt>
I'm up at Simon's Coffee on Mass Ave
s_kunk has quit [Ping timeout: 276 seconds]
<nicolagreco>
dignifiedquire: EDT sorry!
<richardlitt>
And my bike broke down half an hour ago, so it's at the shop. Wouldn't be able to make it until 12:45~
<richardlitt>
Whoa EDT is not EST
<richardlitt>
Ugh. Wow. I hate time zones.
<dignifiedquire>
I'm guessing Juan won't show up now, but in an hour?
<nicolagreco>
oh noes, I think it was my mistake
<nicolagreco>
dignifiedquire: juan just arrived
<dignifiedquire>
which is what I assumed
<dignifiedquire>
ok
<nicolagreco>
everyone is here
<dignifiedquire>
daviddias: are you here?
<nicolagreco>
s/everyone is here/everyone is here?/
<nicolagreco>
daviddias, dignifiedquire are you in?
<dignifiedquire>
I'm in
<nicolagreco>
give me one sec
<dignifiedquire>
my webcam is broken today :(
<voxelot>
can we get a stream link to hangout?
<voxelot>
my sounds is busted lol
<voxelot>
want to watch on my pc
Guanin_ has left #ipfs [#ipfs]
Qwertie has joined #ipfs
<A124>
whyrusleeping So I am trying to profile, but when run with the cpu profiling, the spikes and all are much smaller and less frequent, but still present.
<nicolagreco>
voxelot: join from the url above
<whyrusleeping>
A124: interesting...
<A124>
Any by less frequent I mean the startup spike noise, the first few (not sure how many) minutes
<voxelot>
nicolagreco: okay, just didn't want to take a spot just to listen since my pc cant speaker or cam
ianopolous has quit [Ping timeout: 240 seconds]
<nicolagreco>
no worries voxelot 3 of us are in one cam
<dignifiedquire>
voxelot: just join and mute yourself
<A124>
Which would correspond with threading problem. The rest is network interrupts, which does also correspond, having to switch between each different socket? (no idea)
reit has quit [Ping timeout: 276 seconds]
<A124>
Quote: "If you are on a regular desktop system, you will see that number of interrupts that your computer handles is relatively small. Even powerful servers handling millions of packets per second handle only tens of thousands of interrupts per second. Yet these interrupts consume CPU power and handling them properly undoubtedly helps to improve system's performance."
<noffle>
haad_: great!
<noffle>
haad_: how does ipfs-log fit in? you'll want to map e.g. increment()s to log entries, right?
<ipfsbot>
[js-ipfs] diasdavid created greenkeeper-ipfs-repo-0.6.3 (+1 new commit): https://git.io/vwn7H
<ipfsbot>
js-ipfs/greenkeeper-ipfs-repo-0.6.3 af7129b greenkeeperio-bot: chore(package): update ipfs-repo to version 0.6.3...
Qwertie has quit [Ping timeout: 250 seconds]
<A124>
whyrusleeping So IPFS itself adds 1500-2000 interrupts, which on system that whole runs with 500 without ipfs, is ... well, you know.
Qwertie has joined #ipfs
afisk has quit [Remote host closed the connection]
<ipfsbot>
[js-ipfs] diasdavid deleted greenkeeper-ipfs-repo-0.6.3 at af7129b: https://git.io/vwn5A
Stebalien has joined #ipfs
Qwertie has quit [Ping timeout: 268 seconds]
afisk has joined #ipfs
Qwertie has joined #ipfs
afisk has quit [Remote host closed the connection]
afisk has joined #ipfs
palkeo has joined #ipfs
palkeo has joined #ipfs
Qwertie has quit [Ping timeout: 276 seconds]
<M-Sonata>
Is there a way to add a large directory without using an enormous amount of RAM?
Qwertie has joined #ipfs
Guanin_ has joined #ipfs
ygrek_ has joined #ipfs
Qwertie has quit [Ping timeout: 276 seconds]
Qwertie has joined #ipfs
<ipfsbot>
[js-ipfs] dignifiedquire pushed 1 new commit to aegir: https://git.io/vwnNG
<haad_>
noffle: not sure yet. I think keeping the gcounter data structure pure is the base level. I'm thinking I'll build something like ipfs-gcounter on top of it that uses log to automate the logging. another thing I want to play with is that if you use the gcounter as it is now, the whole state will be serialized and it'll be a challenge for scaling. I wanna try out making the gcounter a op-based crdt, see how that would work.
Qwertie has joined #ipfs
<richardlitt>
anyone seen lgierth today?
yolo_scrub has joined #ipfs
exo_cortex has joined #ipfs
yolo_scrub has left #ipfs [#ipfs]
<exo_cortex>
hallo guys
computerfreak has quit [Remote host closed the connection]
<exo_cortex>
is it ok to ask for help ?
<daviddias>
Really impressed that I was able to do an hangout inside a moving train on the subway and sometimes I can't do it in a nice stable place
jaboja has quit [Ping timeout: 240 seconds]
Qwertie has quit [Ping timeout: 250 seconds]
<exo_cortex>
I tried out IPFS, started the init and daemon and added a folder. A friend tried to access it, (ipfs ls <hash>) but it doesn't seem to work.
<exo_cortex>
or does it just take very long ?
<dignifiedquire>
daviddias: nice
<Boomerang>
I think the first time you start your node it does take a bit of time to propagate to other nodes.
<exo_cortex>
does this mean, if I start the program later it will work faster ?
<voxelot>
dignifiedquire: thanks for taking notes, hangouts was bugging out for me
matoro has joined #ipfs
Boomerang has quit [Quit: Leaving]
Qwertie has quit [Ping timeout: 268 seconds]
<noffle>
haad_: isn't "state" equivalent to "all the ops"?
Qwertie has joined #ipfs
jedahan has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
nunofmn1 has joined #ipfs
Qwertie has quit [Ping timeout: 268 seconds]
jaboja has joined #ipfs
matoro has quit [Ping timeout: 244 seconds]
zorglub27 has joined #ipfs
urnix has quit []
Qwertie has joined #ipfs
Peer3Peer has joined #ipfs
nunofmn1 has quit [Ping timeout: 260 seconds]
Looking has joined #ipfs
matoro has joined #ipfs
Peer3Peer has quit [Client Quit]
reit has joined #ipfs
reit has quit [Client Quit]
Qwertie has quit [Ping timeout: 268 seconds]
<haad_>
noffle: yes, literally. ie. not the sum of all counts in case of the counter but all the counters and their values.
Qwertie has joined #ipfs
pfraze has quit [Remote host closed the connection]
Qwertie has quit [Ping timeout: 250 seconds]
<voxelot>
noffle haad_: something that interests me is stress testing how much data and fast data can be replicated
<noffle>
haad_: I wonder if approaching the crdts from an ops angle would map much more cleanly to the also-op-based ipfs-log
<voxelot>
is this something that will depend on how the crdts are built, or are there metrics for ideas written into the papers i could find
Qwertie has joined #ipfs
<voxelot>
i would assume network speed and data size would be a factor
ggoZ has joined #ipfs
<noffle>
voxelot: how the ops propagate over the swarm will probably be more important than anything the base crdt modules do
<noffle>
as you say
<voxelot>
makes sense
zorglub27 has quit [Quit: zorglub27]
<voxelot>
haad_: something you mentioned was i could think of these like a block chain, a linked list of merkle dag objects but 'without' consensus
<voxelot>
crdts dont provide authority on the state then?
Qwertie has quit [Ping timeout: 268 seconds]
<noffle>
voxelot: he may have been referring to ipfs-log, which is a merkle list that can fork (and thus doesn't require consensus or conflict resolution)
<noffle>
merkle linking isn't necessary for crdts (even op ones), but it's a convenient data structure for doing replication and layers onto ipfs nicely
Qwertie has joined #ipfs
<noffle>
and it grants the same integrity checking as a merkle dag
<noffle>
a crdt op log isn't immutable necesarily
<A124>
whyrusleeping Got 20 min of periodic cpuprof, etc, but I believe it would be replicable anywhere. But if not, you know where to get them.
<noffle>
but that's a nice property for us to add on (and cheap, since ipfs provides it)
<whyrusleeping>
A124: sweet, how are you generating those graphs?
<whyrusleeping>
if i wanted to do so myself
<whyrusleeping>
or better: what is your process of monitoring all these context switches?
<A124>
Calculate the change per second and that's it.
Qwertie has quit [Ping timeout: 268 seconds]
Qwertie has joined #ipfs
afisk has quit [Remote host closed the connection]
zorglub27 has joined #ipfs
_Vi has quit [Ping timeout: 252 seconds]
matoro has quit [Ping timeout: 260 seconds]
zorglub27 has quit [Client Quit]
Qwertie has quit [Ping timeout: 250 seconds]
Qwertie has joined #ipfs
<haad_>
voxelot: as noffle says, the biggest factor in crdts will be the network throughput. obviously the implementation also plays a role. as a reference, I've been benchmarking orbit-db and it can do about 70-80 ops/s atm. but there's a lot to optimize and I haven't tested how it scales in big networks (ie. many nodes sending/receiving to the same pubsub topic), I think that will be an interesting challenge for orbit-db.
<haad_>
voxelot: what do you mean by "authority on the state"?
pigeons has left #ipfs [#ipfs]
<noffle>
haad_: what is an 'op' in orbit-db's context?
<haad_>
noffle: read or write but I've been only benchmarking writes
<haad_>
noffle: oh, and "sync" which is the merge of logs
<haad_>
what I've been benchmarking is writes + syncs
Qwertie has quit [Ping timeout: 268 seconds]
Qwertie has joined #ipfs
jedahan has joined #ipfs
Qwertie has quit [Ping timeout: 268 seconds]
jedahan has quit [Client Quit]
Qwertie has joined #ipfs
afisk has joined #ipfs
afisk has quit [Remote host closed the connection]
afisk has joined #ipfs
nunofmn1 has joined #ipfs
Qwertie has quit [Ping timeout: 250 seconds]
jedahan has joined #ipfs
Qwertie has joined #ipfs
nunofmn1 has quit [Ping timeout: 240 seconds]
matoro has joined #ipfs
Qwertie has quit [Ping timeout: 268 seconds]
pfraze has joined #ipfs
<richardlitt>
whyrusleeping: I am still having to run `ulimit -n 1024` before doing anything
Qwertie has joined #ipfs
Encrypt has quit [Quit: Quitte]
TheWhisper_ has joined #ipfs
rendar has quit [Ping timeout: 252 seconds]
pfraze has quit [Ping timeout: 260 seconds]
Qwertie has quit [Ping timeout: 276 seconds]
Qwertie has joined #ipfs
TheWhisper has quit [Ping timeout: 244 seconds]
rendar has joined #ipfs
Qwertie has quit [Ping timeout: 268 seconds]
Qwertie has joined #ipfs
<mafintosh>
noffle: when are you coming to europe?
<mafintosh>
also any other ipfs people coming to squatconf?
Qwertie has quit [Ping timeout: 250 seconds]
ianopolous has quit [Ping timeout: 252 seconds]
<dignifiedquire>
mafintosh: lots of us ;) david, juan and myself
<mafintosh>
nice
<dignifiedquire>
and haad_
<mafintosh>
i'll see you all there then!
nunofmn1 has joined #ipfs
<dignifiedquire>
yes :)
<voxelot>
noffle, haad_: thanks for the description again! was in a meeting, going to ponder on i for a bit
Qwertie has joined #ipfs
<noffle>
mafintosh: I'm in berlin on either the 26 or 27 (need to double check)
<noffle>
I left my departure date open
binyom has joined #ipfs
Qwertie has quit [Ping timeout: 268 seconds]
<mafintosh>
noffle: great! epic hacking trip :)
jedahan has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
Qwertie has joined #ipfs
<ipfsbot>
[go-ipfs] noffle opened pull request #2596: Adds --local option to 'ipfs mount' (master...local-mount) https://git.io/vwcnf
Stebalien has quit [Ping timeout: 260 seconds]
Stebalien has joined #ipfs
jedahan has joined #ipfs
<noffle>
mafintosh: really excited to see more of europe
cowkeeper has quit [Ping timeout: 276 seconds]
Qwertie has quit [Ping timeout: 268 seconds]
Guest39213 has joined #ipfs
Qwertie has joined #ipfs
<mafintosh>
noffle: do you have any travels planned other than germany?
<whyrusleeping>
alu: have you tried programming in VR?
Qwertie has quit [Ping timeout: 268 seconds]
ralphtheninja has quit [Remote host closed the connection]
Qwertie has joined #ipfs
<yangwao>
mm
<yangwao>
any idea how works that extension of filesystems for ChromeOS ?
<yangwao>
I think this will could totally fit IPFS scenario :)
Qwertie has quit [Ping timeout: 268 seconds]
Qwertie has joined #ipfs
pfista has joined #ipfs
<whyrusleeping>
yangwao: proprietary google magic probably
<yangwao>
indeed
<richardlitt>
yangwao: I've got IPFS working in a unix shell in a chrom extension on ChromeOS
<richardlitt>
but, uh, it was kind of ridiculous.
<yangwao>
but I have it in my list to do one extension for it based on ipfs
<ianopolous>
whyrusleeping: Do you know what the state of IPFS FUSE bindings are? We're currently implementing our own FUSE bindings in Peergos and I imagine solving some of the same problems.
<yangwao>
richardlitt: me too man! :D
<richardlitt>
yangwao: o///
<yangwao>
richardlitt: oh, but you have x86 or arm chromeOS? because I have ARM
<richardlitt>
I had the ARM
<whyrusleeping>
ianopolous: they work on linux
<yangwao>
yay which one?
<richardlitt>
I gave it away.
<richardlitt>
Asus
<yangwao>
flip?
<richardlitt>
yep
<yangwao>
perfect HW!
<yangwao>
I think :)
<richardlitt>
Was the cheapest lightweight thing I could find
<yangwao>
and affordable
<yangwao>
yeah, like $200-300
<richardlitt>
Yeah. Ended up giving it to my dad, he didn't have a computer.
<ianopolous>
whyrusleeping: cool! did you need to increase the write buffer size the kernel uses? or was the default fine for IPFS?
<yangwao>
noice
<whyrusleeping>
ianopolous: the default is fine as far as i've noticed
jedahan has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
jedahan has joined #ipfs
<pfista>
ok this is probably a noob mistake (new to go) but does anyone know why when running this basic command "Success" isn't being printed out? http://pastebin.com/vxtXwBtq
computerfreak has joined #ipfs
jedahan has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<barndon>
Hey folks, I'm having trouble with adding files using js-ipfs-api. I've got a FileReader reading in a file with readAsArrayBuffer(), which I convert the result to a Buffer, and when passing to ipfs.add() I get the error "options.payload must be a string, a Buffer, or a Stream". I've also tried testing out add() by giving it a string, to which I get the error, "Can not add paths in node". Any thoughts? I'd appreciate any advice :D
jedahan has joined #ipfs
mildred has quit [Ping timeout: 240 seconds]
<noffle>
pfista: does execution reach your marshaller?
<pfista>
yes
ralphtheninja has joined #ipfs
<pfista>
noffle: if i return ("text" + fsck.result) in the marshaller the output when i run the command is "text%"
Stebalien has quit [Ping timeout: 250 seconds]
<pfista>
returning (fsck.result) prints nothing
<whyrusleeping>
pfista: the field 'result' needs to be capitalized
<whyrusleeping>
go uses case to signify whether or not a field is exported (public)
<whyrusleeping>
the json marshaler, since its in a different package, can only access exported fields of structs its given
<pfista>
whyrusleeping: Got it! I need to read through effective go xD
<pfista>
thanks
<whyrusleeping>
no problemo!
<noffle>
nice!
zootella has quit [Quit: zootella]
jedahan has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
jedahan has joined #ipfs
<A124>
Quote: A decade ago engineers tackled the C10K scalability problems that prevented servers from handling more than 10,000 concurrent connections. This problem was solved by fixing OS kernels and moving away from threaded servers like Apache to event-driven servers like Nginx and Node.
<A124>
whyrusleeping Tons of threads is no way to go in P2P, at least in my opinion. It is fundamentaly flawed from the very foundation of the communications.
<A124>
Also 500MB mem is not little.
heyrhett has joined #ipfs
<whyrusleeping>
A124: agreed
jedahan has quit [Client Quit]
afisk has quit [Remote host closed the connection]
Looking has quit [Ping timeout: 250 seconds]
<whyrusleeping>
ipfs *should* only have one thread per core of your machine
<whyrusleeping>
anything more is the go runtime doing wonky things
<whyrusleeping>
A124: i just remembered the last time we ran into this context switching issue
<whyrusleeping>
and I actually remembered it because you mentioned the C10K problem
<whyrusleeping>
could you set the env var IPFS_REUSEPORT to false ?
<whyrusleeping>
and then run the daemon
<A124>
Yeah, what would it do?
ashark has quit [Ping timeout: 250 seconds]
<A124>
Will have to find out how to pass env to container, haha.
herzmeister has joined #ipfs
<rendar>
whyrusleeping: what async i/o lib is Go using under the hoods? libevent? or its own impementation?
Stebalien has joined #ipfs
<rendar>
i mean, the library that should call epoll() etc
<whyrusleeping>
rendar: so go's net lib is really dumb and doesnt let us easily set socket options
pfraze has quit [Remote host closed the connection]
matoro has quit [Ping timeout: 252 seconds]
joeghi has quit [Quit: Leaving]
<A124>
whyrusleeping IPFS_REUSEPORT=false?
<whyrusleeping>
yeap
<whyrusleeping>
make sure to run it in the shell before starting the daemon
<A124>
I will export it, rather
<whyrusleeping>
yeah
<richardlitt>
haad_: ping
Stebalien has quit [Ping timeout: 276 seconds]
afisk has joined #ipfs
<A124>
whyrusleeping So, running, about +30 processes (got some instrumentation), So not that crazy on this side, context switching and interrupts that do correspond go up as before, but little less crazy and little less taking cpu. So the threads changed, but else not much.
afisk has quit [Ping timeout: 276 seconds]
<A124>
Memory 300MB, but had to make another container, only sharing data. But should be fairly similar.
matoro has joined #ipfs
<Igel>
ne ipfs @ mit
<pfista>
does anyone know where datastore/LOCK is managed? i know where repo.lock is set but can't find it for the datastore
<pfista>
trying to get the lockfile paths
Encrypt has quit [Quit: Quitte]
<whyrusleeping>
pfista: thats part of leveldb
_rht has quit [Quit: Connection closed for inactivity]
Stebalien has joined #ipfs
<pfista>
where is level db?
zootella has joined #ipfs
<rendar>
whyrusleeping: why the use of epoll is wrong?!
<pfista>
whyrusleeping: fsrepo/defaultds.go? is that the only place? I'm trying to programmatically get the lockfile from leveldb
<pfista>
path
zootella has quit [Ping timeout: 268 seconds]
Stebalien has quit [Ping timeout: 260 seconds]
<pfista>
or should I just hardcode the lockfile paths in
Stebalien has joined #ipfs
ianopolous has quit [Ping timeout: 260 seconds]
<alu>
IPFS goes crazy on this network sometimes when im testing
<alu>
the CTO of the office stopped by n thought i was torrenting stuff but i just explained to him im building decentralized metaverse applications on the blockchain
Stebalien has quit [Ping timeout: 246 seconds]
ggoZ has quit [Ping timeout: 260 seconds]
<richardlitt>
Woo MIT meetuup happening now
<richardlitt>
Alu that's pretty awesome
<alu>
u at the meetup?
<richardlitt>
Yup!
<richardlitt>
Juan is using my computer to present
<achin>
\o/
<alu>
nice lol
<alu>
livestream pls
<achin>
clearly i didn't make it up there, but woo for meetups!
<alu>
;_;
<richardlitt>
Sorry! We are taking video though
<richardlitt>
Achin - one day!
<achin>
yes, one day!
<achin>
midweek stuffs is really bad for me, but fridays would be much better. easier to take the afternoon off
<richardlitt>
Yeah
nunofmn1 has quit [Ping timeout: 268 seconds]
<alu>
cant wait for video
Looking has joined #ipfs
rendar has quit [Quit: std::lower_bound + std::less_equal *works* with a vector without duplicates!]
computerfreak has quit [Remote host closed the connection]
jo_mo has joined #ipfs
jo_mo has left #ipfs [#ipfs]
<noffle>
whyrusleeping: I'm fiddling with gx-go, and notice that 'gx import'ing adds my gx/ipfs/Qm* package to both $GOPATH/src/gx as well as src/gx in my project dir