hotpancakes has quit [Read error: Connection reset by peer]
Contigi777 has joined #ruby-lang
seank___ has joined #ruby-lang
hotpancakes has joined #ruby-lang
znz_jp has quit [Quit: kill -QUIT $$]
Guest62726 has quit [Ping timeout: 252 seconds]
Contigi has quit [Ping timeout: 240 seconds]
seank__ has quit [Ping timeout: 256 seconds]
hotpanca_ has joined #ruby-lang
slumos is now known as slumos[away]
Olipro- has joined #ruby-lang
hotpancakes has quit [Ping timeout: 246 seconds]
hotpancakes has joined #ruby-lang
hotpanca_ has quit [Ping timeout: 245 seconds]
LMity has joined #ruby-lang
znz_jp has joined #ruby-lang
yfeldblum has quit [Remote host closed the connection]
yfeldblum has joined #ruby-lang
RickHull has joined #ruby-lang
michael_mbp has quit [Excess Flood]
LMity has quit [Ping timeout: 264 seconds]
michael_mbp has joined #ruby-lang
djbkd has quit [Quit: Leaving...]
djbkd has joined #ruby-lang
<RickHull>
hello folks, I'm playing around with oauth (v1) for the first time
<RickHull>
i want to poll the bitbucket REST API
<RickHull>
i'm having trouble understanding who my "user" is, and what the callback url should be
<RickHull>
i don't really have a user, and i just want to authenticate to bitbucket
workmad3 has joined #ruby-lang
skade has joined #ruby-lang
<RickHull>
my app has the key and secret. these correspond to my personal bitbucket account
<RickHull>
so when my app authenticates using oauth, what is the callback_url?
Technodrome has quit [Quit: Technodrome]
<RickHull>
to make an analogy, I have used AWS REST API. I provide the secret and the key, and then I can make authenticated / authorized calls to the REST API
<RickHull>
i never had to provide a callback URL
<RickHull>
i'm having trouble understanding what is necessary here
workmad3 has quit [Ping timeout: 246 seconds]
RobertBirnie has quit [Ping timeout: 256 seconds]
<RickHull>
there is no user, the purpose is for automation
gwendall_ has quit [Remote host closed the connection]
crankharder has quit [Ping timeout: 245 seconds]
<RickHull>
i'd just as soon redirect a phantom user to disney.com, is that sensible?
kerunaru has joined #ruby-lang
<RickHull>
or is the "callback_url" simply the actual bitbucket url that my app would like to poll?
ledestin has joined #ruby-lang
Contigi has joined #ruby-lang
Olipro- has quit [Ping timeout: 245 seconds]
Cloudhand has quit [Ping timeout: 256 seconds]
vondruch_ is now known as vondruch
Contigi777 has quit [Ping timeout: 255 seconds]
kerunaru has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
Contigi has quit [Quit: Leaving]
dhjondoh has quit [Quit: dhjondoh]
djbkd has quit [Quit: My people need me...]
dhjondoh has joined #ruby-lang
djbkd has joined #ruby-lang
OnBrokenWings has joined #ruby-lang
fsvehla has joined #ruby-lang
Olipro- has joined #ruby-lang
<RickHull>
anybody ever used oauth before?
RobertBirnie has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
bb010g has joined #ruby-lang
skade has quit [Quit: Computer has gone to sleep.]
djbkd has quit [Quit: Leaving...]
iamninja_ has quit [Read error: Connection reset by peer]
fsvehla has quit [Quit: fsvehla]
iamninja_ has joined #ruby-lang
fujimura has quit [Ping timeout: 250 seconds]
djellemah has joined #ruby-lang
fujimura has joined #ruby-lang
fujimura has quit [Remote host closed the connection]
fujimura has joined #ruby-lang
fujimura has quit [Read error: Connection reset by peer]
fujimura has joined #ruby-lang
OnBrokenWings has quit [Ping timeout: 244 seconds]
OnBrokenWings has joined #ruby-lang
crankharder has joined #ruby-lang
brownbathrobe has joined #ruby-lang
oh_hai_mark has joined #ruby-lang
michael_mbp has quit [Excess Flood]
michael_mbp has joined #ruby-lang
crankharder has quit [Ping timeout: 245 seconds]
brownbathrobe has quit [Ping timeout: 245 seconds]
jas02 has joined #ruby-lang
hahuang65 has quit [Ping timeout: 240 seconds]
elia has joined #ruby-lang
vulix has joined #ruby-lang
bruno- has joined #ruby-lang
bruno- is now known as Guest53574
Mon_Ouie has quit [Ping timeout: 248 seconds]
Guest53574 has quit [Ping timeout: 276 seconds]
SylarRuby has joined #ruby-lang
rippa has joined #ruby-lang
cornerma1 has joined #ruby-lang
elia has quit [Quit: (IRC Client: textualapp.com)]
cornerman has quit [Ping timeout: 248 seconds]
cornerma1 is now known as cornerman
houhoulis has quit [Remote host closed the connection]
workmad3 has joined #ruby-lang
workmad3 has quit [Ping timeout: 250 seconds]
postmodern has joined #ruby-lang
yfeldblum has quit [Remote host closed the connection]
kerunaru has joined #ruby-lang
relix has joined #ruby-lang
arBmind1 has quit [Quit: Leaving.]
djbkd has joined #ruby-lang
oh_hai_mark has quit [Remote host closed the connection]
crankharder has joined #ruby-lang
solars has joined #ruby-lang
crankharder has quit [Ping timeout: 245 seconds]
hahuang65 has joined #ruby-lang
skade has joined #ruby-lang
<maloik>
RickHull: if you sign in from your app with bitbucket, you'll be redirected to their app, sign in, then they'll redirect back to that callback url
<maloik>
you then check the params to figure out who the user is, if they logged in successfully etc
<maloik>
there's really a bunch of great tutorials up about devise and services like twitter, github etc
<RickHull>
maloik: thanks, hmm
<RickHull>
i'm not sure my use case fits that model, but i'm probably misunderstanding
<maloik>
oauth is the same, always
ta has quit [Remote host closed the connection]
<maloik>
the only difference, as far as I understood it, is that you don't necessarily want to save the user to your database
<RickHull>
ok, then maybe it's just a question of adapting the terminology
<RickHull>
i don't have users, i don't have an app with URLs
<RickHull>
callbacks or otherwise
<RickHull>
i have a daemon, basically
<maloik>
then how would someone log in?
<RickHull>
that polls bitbucket, and then takes action when bitbucket changes
<RickHull>
i have no users, no one logs in
<maloik>
... then why ask about oauth?
<RickHull>
i am using oauth so that my app can log in to bitbucket
<RickHull>
to access private repos
<RickHull>
purely for automation purposes
<maloik>
that's not how oauth works
<RickHull>
i.e. this repo changed, pull the changes, then take action
<maloik>
you'll most likely need to get some sort of api token in bitbucket
<RickHull>
hmm
<maloik>
maybe a secret key and token pair
<RickHull>
i have a secret key and access id
<RickHull>
the bitbucket docs pointed me to oauth stuff
<RickHull>
maybe i'm barking up the wrong tree?
hahuang65 has quit [Ping timeout: 245 seconds]
<maloik>
okay to be clear, "thats not how oauth works" was a bit of a lame comment... what I meant was that the stuff you've been pasting is how oauth in terms of apps works, as in the flow I described earlier with the redirects
<RickHull>
i understood that i would use oauth to get a token, and then use the token to do authorized / authenticated stuff
<RickHull>
yeah, the stuff i posted is in terms of the docs they gave me
<RickHull>
which i posted because, they don't make sense to me
<RickHull>
for my use case
<RickHull>
maybe i need a different approach?
<maloik>
did you look at webhooks and git hooks?
<RickHull>
i did
<maloik>
okay just checking
<RickHull>
problem is, the webhooks need a public IP or DNS
<RickHull>
and my automation server has neither
<RickHull>
so my automation server will just poll the REST API for changes
<RickHull>
i'd much rather go with the webhooks, but it's not an option given the constraints
<RickHull>
so, i'm polling the REST API on a private repo for changes
<RickHull>
bitbucket says to use oauth
<RickHull>
and it talks about callback_urls and users, and i'm not sure what i'm supposed to plug in
<maloik>
I'm not sure, briefly looking around doesnt tell me much
<maloik>
step 3 you could fake by manually visiting that url, granting the application access, and from then on you should be able to make api calls using the first two steps to get a token
<RickHull>
I doubt it, but I'm just trying to understand the landscape
Technodrome has joined #ruby-lang
solars has quit [Ping timeout: 245 seconds]
Guest20 has joined #ruby-lang
elia has joined #ruby-lang
<maloik>
not sure how they check it or if they check it at all
yfeldblum has joined #ruby-lang
<maloik>
cant look into it more right now though, have some actual work to do :(
yfeldblum has quit [Read error: Connection reset by peer]
solars has joined #ruby-lang
OnBrokenWings has quit [Quit: Leaving]
yfeldblum has joined #ruby-lang
<maloik>
ssh pair@home.jeroenj.be -p 9022
_djbkd has quit [Read error: Connection reset by peer]
_djbkd has joined #ruby-lang
gwendall has joined #ruby-lang
ta has joined #ruby-lang
Mon_Ouie has joined #ruby-lang
ta_ has joined #ruby-lang
SylarRuby has quit [Remote host closed the connection]
stan has joined #ruby-lang
_djbkd has quit [Ping timeout: 256 seconds]
djbkd_ has joined #ruby-lang
djbkd has quit [Ping timeout: 264 seconds]
piotrj has joined #ruby-lang
intinig has joined #ruby-lang
exons has quit [Ping timeout: 250 seconds]
charliesome has quit [Quit: zzz]
exons has joined #ruby-lang
intinig has quit [Ping timeout: 250 seconds]
piotrj has quit [Remote host closed the connection]
ta_ has quit [Remote host closed the connection]
<yorickpeterse>
morning
workmad3 has joined #ruby-lang
skade has quit [Quit: Computer has gone to sleep.]
stevednd has quit [Ping timeout: 252 seconds]
stevednd has joined #ruby-lang
hotpancakes has quit [Remote host closed the connection]
workmad3 has quit [Ping timeout: 250 seconds]
Mon_Ouie has quit [Ping timeout: 256 seconds]
fujimura_ has joined #ruby-lang
fujimura has quit [Ping timeout: 265 seconds]
intinig has joined #ruby-lang
skade has joined #ruby-lang
lloyd is now known as Lloyd
piotrj has joined #ruby-lang
crankharder has joined #ruby-lang
crankharder has quit [Ping timeout: 250 seconds]
hahuang65 has joined #ruby-lang
hahuang65 has quit [Ping timeout: 255 seconds]
Guest20 has quit [Read error: Connection reset by peer]
intinig has quit [Read error: Connection reset by peer]
intinig has joined #ruby-lang
stardiviner has joined #ruby-lang
intinig has quit [Read error: Connection reset by peer]
intinig has joined #ruby-lang
postmodern has quit [Quit: Leaving]
jsaak has quit [Ping timeout: 244 seconds]
intinig has quit [Remote host closed the connection]
gwendall has quit []
arBmind has joined #ruby-lang
skade has quit [Quit: Computer has gone to sleep.]
skade has joined #ruby-lang
isale-eko has joined #ruby-lang
k3asd` has joined #ruby-lang
hotpancakes has joined #ruby-lang
intinig has joined #ruby-lang
stef204 has joined #ruby-lang
intinig has quit [Remote host closed the connection]
hotpancakes has quit [Ping timeout: 256 seconds]
ta_ has joined #ruby-lang
intinig has joined #ruby-lang
mikecmpbll has joined #ruby-lang
fujimura has joined #ruby-lang
fujimura_ has quit [Read error: Connection reset by peer]
<chris2>
what a shitload of shitposts to ruby-talk
bruno- has joined #ruby-lang
riotjones has quit [Ping timeout: 256 seconds]
bruno- is now known as Guest74468
<yorickpeterse>
que?
riotjones has joined #ruby-lang
<chris2>
some guy spamming
<yorickpeterse>
Ah
Guest74468 has quit [Ping timeout: 245 seconds]
auzty has quit [Quit: Leaving]
fujimura has quit [Remote host closed the connection]
hendranata_ has quit [Ping timeout: 264 seconds]
fujimura has joined #ruby-lang
fujimura has quit [Ping timeout: 276 seconds]
<Eising>
hey
<Eising>
ehm
<Eising>
I have this templating system I've made. It sort of converts mustache templates into webforms, which is pretty handy if you're working with router configurations for example
<vulix>
converting mustaches into web forms, what in the...
<Eising>
:D
<yorickpeterse>
bah, fucking cookies and iframes
<yorickpeterse>
customer apparently embeds this survey app we have in an iframe
workmad3 has joined #ruby-lang
<yorickpeterse>
but said app needs a cookie
<yorickpeterse>
which doesn't work from an iframe because yay third-party cookies
<Eising>
so, the thing is, I want to be able to make it more flexible, by allowing some ruby code in the templates, or actually, around the templates, but I can't figure out how I can do this securely, or at least in a way, where I can limit what classes/libraries/gems/functions are available
<Eising>
sort of like what eval.in does
intinig has quit [Remote host closed the connection]
hahuang65 has joined #ruby-lang
riotjones has quit [Ping timeout: 245 seconds]
<Eising>
there's a lot about this I'm unsure about. Including storing ruby code in my db...
workmad3 has quit [Ping timeout: 256 seconds]
hahuang65 has quit [Ping timeout: 276 seconds]
piotrj has quit [Read error: Connection reset by peer]
piotrj has joined #ruby-lang
jas02 has left #ruby-lang [#ruby-lang]
intinig has joined #ruby-lang
skade has quit [Quit: Computer has gone to sleep.]
<havenwood>
PKL: So now, inside the #map block what logic do you want to go with to decide whether to return `a.capitalize` or just `a` for each iteration?
<havenwood>
for each word*
shinnya has quit [Ping timeout: 255 seconds]
robbyoconnor has quit [Quit: Konversation terminated!]
voxxit[away] is now known as voxxit
<PKL>
in the map block
<PKL>
hmmm, I was going to say it needs to check that it's not at the beginning or end but that won't work...
piotrj has joined #ruby-lang
StephenOTT has joined #ruby-lang
<havenwood>
does it matter if it's at the end for capitalizing?
<PKL>
if I'm using the criteria of 4 characters or less it wouldn't capitalize "Kwai"
<PKL>
but that's not a good way of thinking about it
<havenwood>
PKL: getting that criteria working might be a good exercise though
hahuang65 has joined #ruby-lang
skade has quit [Ping timeout: 250 seconds]
<PKL>
havenwood: I can put a.capitalize if a.length <4?
<havenwood>
PKL: That would be capitalizing the short strings, right? It is mapping, so you'll need to return `a` even if you don't `a.capitalize` or you'll map to `nil`s.
<Senjai>
Please at least try googling before asking :(
banister has quit [Max SendQ exceeded]
<RobertBirnie>
well i'm using that already :)
<RobertBirnie>
if source_file.flock(File::LOCK_NB|File::LOCK_EX)
<RobertBirnie>
`xz #{source_file}`
<RobertBirnie>
but xz will sit forever waiting for my ruby script to unlock the file
<Senjai>
What is xz?
bungoman_ has joined #ruby-lang
<RobertBirnie>
its a compression app, like bzip
<RobertBirnie>
much better compression ratio
<Senjai>
RobertBirnie: Why are you flocking it, cant it do its own locking if it has to
<RobertBirnie>
i'm guessing xz does lock, since it waits on a locked file. I'm also guessing that if the file's already locked that it'll sit around forever
bungoman has quit [Ping timeout: 240 seconds]
Cloudzy has joined #ruby-lang
<RobertBirnie>
i want to flock it in ruby so that it'll skip already locked files
<pipework>
Why use locks for that?
<pipework>
Why not just get a list of files and use that instead?
<RobertBirnie>
i guess i could just check for a lock and then continue
<pipework>
I don't think locks were meant to be used in this fashion.
<Senjai>
pipework: They weren't
<Senjai>
RobertBirnie: It would wait on a locked file, because all process have to wait on locked files, regardless if they want a lock or not
<Senjai>
Thats the point of.. locking files
<RobertBirnie>
yeah, but i don't want to baby sit the script if it hits a locked file. its compressing thousands of files daily via a regular cronjob
<RobertBirnie>
"compress, move, delete"
<Senjai>
RobertBirnie: So, test if the file is locked
Cloudhand has quit [Ping timeout: 245 seconds]
<Senjai>
if it is, move a long
<Senjai>
Testing if a file is locked != locking a file
<pipework>
Still not sure why you can't acquire a list of files and work off that.
<Senjai>
pipework: He probably does, but the files are probably shared between processes
<Senjai>
which is why hes having issues
<Senjai>
He's going about testing files the wrong way though
<RobertBirnie>
pipework what do you mean? I am working off a big glob of files
alestuber has quit [Ping timeout: 276 seconds]
<pipework>
RobertBirnie: Why do you think you need to lock the files?
<pipework>
Are you actually doing operations on them, or are you just using it as a mechanism to mark the file as 'done' for some purpose?
<RobertBirnie>
so if script runs twice at once it won't start squashing itself
<pipework>
RobertBirnie: Then use a pidfile as a lock.
ascarter has joined #ruby-lang
<pipework>
Don't lock files. Have your script check for the pidfile, only continuing if it isn't present. Then have your script clean up that pidfile regardless of how it exited, unless you want to leave it there to signal for investigation.
|jemc| has joined #ruby-lang
baweaver has joined #ruby-lang
<Senjai>
pipework: We already establsihed he doesnt need to lock files
<Senjai>
You dont need to use pidfiles either
<Senjai>
because that would be changing another process
<Senjai>
to generate the pidfiles
<Senjai>
Just use lsof, fixed forever
<pipework>
Senjai: lsof for a whole shitload of files?
<pipework>
I don't know, man.
<RobertBirnie>
this scripts also threaded. it'd be a lot of lsof
<pipework>
Would rather have a pidfile, possibly actually use a lock on it if it were helpful.
banister has joined #ruby-lang
crankharder has quit [Ping timeout: 264 seconds]
<RobertBirnie>
i also worry about others using the files. this is a huge shared logging server so who knows what others are doing
<Senjai>
RobertBirnie: Returns false if File::LOCK_NB is specified and the operation would otherwise have blocked
<pipework>
RobertBirnie: Then consider reading/copying the files contents in and releasing a lock and then using the copies for what you backup.
<Senjai>
if File.flock(File::LOCK_NB)
<Senjai>
Aksi
<Senjai>
Also
<Senjai>
each thread should be given a list of files just for that thread
<Senjai>
they shouldn't be stepping on eachother
<Senjai>
That list should be prepared at startup
<RobertBirnie>
yeah, i'm using a Queue so they won't compete
<Senjai>
Dont even need a queue man
<Senjai>
Dir.glob, seperate it into thread_count even arrays
<Senjai>
pass each one to a worker
<Senjai>
fixed forever
<pipework>
What about when the files are being modified and appended to?
<Senjai>
I dont have enough context man
<pipework>
Neither do I.
<Senjai>
I'd use a makefile, or guard
<pipework>
I'd use cocaine.
<Senjai>
I'd do cocaine
<RobertBirnie>
i'm just compressing a large number of rolling log files :)
<Senjai>
because I never find myself in these terrible situations :P
<pipework>
If you find that the buffer size of your pipes is insufficient, either bump that up, or you can actually split the ls pipe into multiple processes, let me know if you need help with that.
<Senjai>
RobertBirnie: Everything you say screams "overcomplicated"
<Senjai>
No offence :P
<Senjai>
pipework: Interesting, never used paralell
unreal_ has joined #ruby-lang
<pipework>
I call it parallul.
<pipework>
Not because it sucks, but because I get massive lulz when I use it in front of people.
<pipework>
RobertBirnie: You should probably either declare that you're merely socializing or give us all the information.
<pipework>
If you simply just want to write some ruby to do what can be done with a one liner, that's okay. I support the need to flex your ruby muscle.
<RobertBirnie>
so we've got two main disks for logs /data/logs/ and /data/log_archives/
<RobertBirnie>
under /data/logs/ there are n number of directories
<pipework>
If it's going to be long, you might want to write it in one message. IRC can handle long messages.
unreal has quit [Ping timeout: 252 seconds]
<Senjai>
Or.. use gist.github.com
<RobertBirnie>
so i need to xz all the files in /data/logs and get them over to the archives directory, preserving filesystem hierarchy. and we're talking about roughly 10GB a minute data
piotrj has quit []
<pipework>
What should an individual archive contain?
<pipework>
Just the single log file(s), do the archives need to be nested under the file hierarchy, or can you merely unpack them in the right place?
<pipework>
Cause what I'm hearing is that you don't know that you want to ask for some help using awk.
<RobertBirnie>
its not deep nesting but they need to keep place
toretore has quit [Quit: Leaving]
apt-get__ has quit [Remote host closed the connection]
<Senjai>
RobertBirnie: rsync?
<Senjai>
lolz
<pipework>
'keep place' as in the hierarchy no matter how deep must be reflected within the archive itself?
ldnunes has quit [Quit: Leaving]
<pipework>
Is the archive a single log file?
<RobertBirnie>
no, rolling 1GB files
<pipework>
The size doesn't matter, and since they're logrotated, the rolling isn't applicable.
<pipework>
So yeah, that's pretty doable as a one-liner.
<pipework>
You can even expand it into a ruby script that just wraps the one-liner with sugars and stuff, but I'd head that way around instead of ruby script first.
<RobertBirnie>
how would you handle threading it? parallel is cool, but if it were to thread out based on all the files in the list it'd crush the box
alestuber has quit [Read error: Connection reset by peer]
alestuber has joined #ruby-lang
<pipework>
You limit the number of threads based on a default or user input in the command.
<pipework>
You'd have to solve that problem with your script just as much as you would a one-liner.
<RobertBirnie>
yeah, queue made it easy since i already had the script and was just modifying it
<pipework>
Yeah, parallel makes it easier.
<pipework>
It already only threads out according to your core count, iirc.
<pipework>
There's also --use-cpus-instead-of-cores if you have hyperthreading.
joaomdmoura has quit [Remote host closed the connection]