<fijal>
mattip: what's wrong with self-hosted mercurial, but use github for everything else?
<fijal>
we *essentially* have all the devops necessary, I think people are too scared of those solutions
<fijal>
or call me old-fashioned
xcm has quit [Killed (hitchcock.freenode.net (Nickname regained by services))]
xcm has joined #pypy
rjarry has quit [Read error: Connection reset by peer]
rjarry has joined #pypy
<antocuni>
fijal: from my POV, self-hosting means that we have to maintain it and that it will eventually break, even for just few days, like all the things that we maintain do
<fijal>
unlike bitbucket that breaks forever?
<fijal>
generally I agree, but it seems not to be a critical problem, is it?
<antocuni>
well, it never happened to me that I can't push or pull because bitbucket is down
<fijal>
right, but it will from the next year June or so
<fijal>
permanently, without fix
<antocuni>
not sure how to answer this
<antocuni>
you asked "what's wrong with self-hosted mercurial" and I just replied :)
BPL has joined #pypy
BPL has quit [Remote host closed the connection]
antocuni has quit [Ping timeout: 246 seconds]
<kenaan>
mattip default ca952577d5b5 /lib-python/2.7/test/test_dictviews.py: update test for 2.7.15, sync with forward port of bpo-18533 in 3ea4a70584df
BPL has joined #pypy
<fijal>
fair enough, my point was "it's just a different failure scenario"
<mjacob>
fijal: so you propose to use github issues and also accept pull requests through github?
<mattip>
I would like to use leverage the change to improve our interface with open source contributors
<mattip>
for whatever reason, people are used to a development platform that includes an integrated issue tracker and workflow management
<mattip>
it is already difficult to get involved in the pypy codebase, adding additional barriers will not help our cause
<mattip>
I don't understand how PRs would be managed via github, and I have not been successful in getting hggit to work on our codebase
rjarry has quit [Read error: Connection reset by peer]
rjarry_ has joined #pypy
<fijal>
mjacob: I'm not sure what I'm proposing
<fijal>
I'm just saying that "let's just use git" is not a great way forward
<fijal>
ideally I would want gitlab to work
<mattip>
+1
<mjacob>
it seems like the main heptapod (mercurial for gitlab) developer will be at a mercurial sprint in leipzig on 7th / 8th. is someone else from the pypy project interested to participate?
antocuni has joined #pypy
<mattip>
mjacob: is there more info about the sprint somewhere? I (and maybe some other PyPy people) will be at EuroScipy which ends Sept 6
<mjacob>
as most people have time on sep 7th / 8th, that date is already fixed. i'm trying to find out how many people are interested in also sprinting in the days before. after finding that out, there will be an announcement.
<mattip>
heh, I would need to organize flights, so I need to know soon
<mjacob>
mattip: since the 7th and 8th is fixed and you don't have time before, you can already book. :)
<mattip>
for sure it is going to happen?
<mjacob>
it's sure to happen on 7th / 8th
Rhy0lite has joined #pypy
xorAxAx has quit [Quit: Idle timeout reached: 172800s]
tsaka_ has quit [Ping timeout: 245 seconds]
<mattip>
maybe doable, a bit of a mess to organize
<arigato>
I don't have much comments to add, I'm on the opinion that git is a pile of hacks (even though it probably works well if you know it inside-out). I wouldn't be against trying to fix hggit for our use case; it seems like it's a small code
tsaka_ has joined #pypy
<arigato>
I don't know how others are feeling but I would be happy if we have a user-facing github account with all the usual bells and whistles, as long as it's also possible to use hg locally with no bells and whistles
<arigato>
but it's not for immediately now, so let's see how heptapod evolves in the meantime
<arigato>
bitbucket shutting down might create a lot of demand for heptapod
<antocuni>
arigato: I know it's hard to convince you :), but my first opinion on git was the same as yours. However, what I can say after having used it for several years is that at the end of the day if you stick to a "standard" workflow, it just works well
<antocuni>
sure, you can mess everything up in your local repo if you try hard enough
<antocuni>
but it's almost impossible to mess the remote repo by mistake, and the local development workflow is basically the same as with mercurial
<antocuni>
apart the notable exception of named branches, which we use a lot. I suppose we need to decide whether they are worth the trouble or not
<arigato>
I gave up after using git for one year (we converted our repo to mercurial)
<antocuni>
what was the problem?
<arigato>
our "first opinion" just stick and remains my opinion nowadays
<fijal>
I used to be more of anto opinion and now more of armin opinion
<fijal>
everything sort of works, except it comes with tons of footguns
<fijal>
and it does not work that well
<tumbleweed>
I'd echo what antocuni said. I was not an easy convert, but when you wrap your head around git, it really works well. And I feel a little powerless in other SCM tools, because I can't do things I can do with git
<fijal>
there is a couple of git commands that I just copy paste with complicated -- and stuff, since it does something basic but in a way that makes no sense
<fijal>
I'm also happy with forbidding people to do all the things that git allows you to do
<fijal>
like all the history editing
<cfbolz>
if people do it on their private forks, it's hard to forbid
<antocuni>
I'm genuinely interested in knowing a more concrete example of what are the problems, since apparently I never run into something which was so bad to want to me run away
<arigato>
maybe all I need is some wrapper around git that feels like hg (ignoring for a moment the branches)
<antocuni>
and note that you can edit history also in hg, I don't really see a difference there
<arigato>
I think we already had this discussion, I'll not repeat more than "please give me a solution that works for me with hg"
<arigato>
but I would be happy if that means fixing hggit for our use case
<fijal>
antocuni: named branches are a massive one
<fijal>
antocuni: being unable to manage local changes that you might want to keep around for a bit
<fijal>
diffs that make no sense to me, because the direction you merge is arbitrary
<fijal>
inability to look for all checkins on a given branch
<fijal>
that's from the top of my head what I fought about
<fijal>
but more importantly absolutely nonsense error messages that take me half an hour to dig myself out of the whole
<fijal>
er, hole
<antocuni>
ok, so basically most boils down to "named branches"; fair enough
<arigato>
note that we didn't use a lo of branches in the project where I tried for one year to use git
<fijal>
antocuni: no, I think the "overall experience" is more important to me
<fijal>
having to do diff reading, picking pieces from a diff, digging in history, all of that feels incredibly subpar
<fijal>
managing a set of changes as local changes is also a bit big for me
<antocuni>
what do you mean by "managing a set of changes as local changes"?
<fijal>
I have some changes locally (like hardcoding a path) that I don't want to commit
<fijal>
and I want to pull in changes from the server
<fijal>
this is *hard*
<antocuni>
I usually do git stash && git pull && git stash pop for that; but I'm curious how you do that in hg, because I don't know :)
<fijal>
hg pull && hg update works
<arigato>
uh, it doesn't work in git? I didn't even guess that
<fijal>
but overall the experience of using git - unhelpful error messages, basic commands being with five hundred different switches that you have to copy-paste etc.
<antocuni>
ok, then the solution I proposed is not "*hard*", honestly :)
<arigato>
(maybe the GUI I was using did the stashing behind the scenes?)
<fijal>
and the random behavior that sometimes just fucks up my local repo in a hard to fix way
<fijal>
additionally I also don't quite like the idea of supporting the single monopoly of keeping open source code
<antocuni>
It seems to me that you both tried to use git using an "hg mindset" and it didn't work well. But anyway, it's not that I want to convince you that git is good or better or whatever, I don't care much
<fijal>
yes, maybe!
<fijal>
but I like the way hg works
<fijal>
so yeah, maybe the answer is "like the way git works", but it seems both me and armin just don't
<antocuni>
I think that what we should evaluate for the best of pypy is the overall experience; I think that even if git has some quirks, git+github is the best overall solution we have on the table
<arigato>
I have nothing against it if we can make hggit work and if we preserve the named branches in git in a systematic way also going forward
<fijal>
I think the solution where we use git, but armin uses some other tool is a very bad one, in general
<fijal>
we gonna end up with svn setup with custom commits
<fijal>
I think I also want a more future proof solution than a random cloud provider - something we can host ourselves if we want to, even if we don't want to now
<antocuni>
I tend to agree. At the same time, if we stick to hg we end up in a situation in which we use hg and the rest of the world uses another tool, which has its own problems
<fijal>
yes, but also, monopolies are bad
<antocuni>
maybe, but I would not like PyPy to be the Don Quixote of open source
<fijal>
noone does want to be the Don Quixote
<mattip>
how would I go back in history and remove any use of .hgsubstate (for subrepos)? That is what is confusing hggit
<fijal>
but we're all better off
<arigato>
mattip: you can edit hggit's source code itself, it's not that big. comment out anything special about .hgsubstate
<mattip>
arigato: hmm, tried that but maybe I didn't remove enough of it. I will remove more
<tumbleweed>
I'd also agree that trying to pretend your using hg when using git is a recipe for disaster. you have to go all in and learn to love the git, or you'll hate it
<arigato>
thanks for the input
<tumbleweed>
hrm, I wonder if using git-notes to annotate branches would make sense
<tumbleweed>
help me understand the importance of named branches: My guess is: You blame, it gives you a commit, and you hg log -pr it to see what branch it comes from
<tumbleweed>
if the branch is short-lived enough, you can easily see a commit's context in a git log --graph
<tumbleweed>
and if it's long lived (e.g. py3.6) it probably doesn't matter as much which branch the commit happened in - the branch is about maintaining forks rather than feature development
<antocuni>
uhm, git-notes might be a good solution for the lack of named branches: especially because you can add metadata to commits even after they have been pushed, so we can think to have a system to automatically attach the "branch label" to commits whenever you do a PR, for example
<cfbolz>
tumbleweed: note that we also have branches that live for a month or two, with dev work there, tons of commits, that eventually get merged and closed
dddddd has joined #pypy
<tumbleweed>
cfbolz: yeah, that's where things get messier. esp if there are continual merges in from master. A rebase before merging could be logical there. Or even a squashed merge...
<antocuni>
tumbleweed: also, sometimes we do "hg merge default" inside an ongoing branch, so it is important to be able to distinguish whether the commit was done on the branch or just merged in
<tumbleweed>
git log --graph can help you understand that. Or you can rebase instead of merging, to avoid that confusion
<tumbleweed>
but then we start getting into bigger workflow changes to support a different VCS
<mattip>
IMO we do not want to squash commits, since sometimes bugs appear only months after commit so we need to go back and work out what happened
<tumbleweed>
with a github squashed PR workflow, you'd still have the original commits on the PR
<tumbleweed>
(and in the repo, if you don't delete the branch, but best practice is to delete the branch)
<cfbolz>
tumbleweed: ok, but squashed PRs don't work that well for big refactorings
<cfbolz>
which are the main sources of complicated bugs
<mattip>
but why go there in the first place? I don't mind if people see how often my commit message is "typo" or "whoops"
xcm has quit [Remote host closed the connection]
xcm has joined #pypy
dpn` has quit [Ping timeout: 250 seconds]
rguillebert has quit [Ping timeout: 250 seconds]
lauren has quit [Ping timeout: 250 seconds]
graingert has quit [Ping timeout: 250 seconds]
krono has quit [Ping timeout: 250 seconds]
Lightsword has quit [Ping timeout: 250 seconds]
tsaka_ has quit [Ping timeout: 268 seconds]
lordmauve has quit [Ping timeout: 250 seconds]
string has quit [Ping timeout: 250 seconds]
jeroud has quit [Ping timeout: 250 seconds]
dpn` has joined #pypy
Kronuz has quit [Ping timeout: 248 seconds]
Lightsword has joined #pypy
rguillebert has joined #pypy
lauren has joined #pypy
lordmauve has joined #pypy
string has joined #pypy
jeroud has joined #pypy
Kronuz has joined #pypy
graingert has joined #pypy
<tumbleweed>
cfbolz: totally
krono has joined #pypy
<tumbleweed>
mattip: if there is more value in seeing which feature is to blame, than which exact commit
<arigato>
that's why we need both branch names and individual commits
<arigato>
the idea of squashed commits but detailled pull requests is a bit similar, but I'm not sure how well it would work for the case where you have a long-lived branch and make/merge short-lived branches into it
<tumbleweed>
should work - those short-lived branches would be the squashed commits
<tumbleweed>
the long-lived branches would just be branches (that happen to not be master, but are a trunk of development nonetheless)
<antocuni>
moreover, where does the "detailed PR" data live? Is it on github or stored somehow inside the repo? I don't think it's a good idea to lock ourselves too much to a vendor
<tumbleweed>
it's available over git, in a different ref namespace
<antocuni>
ah ok, thant's better then :)
<arigato>
I guess it's a bit theoretical but then if you have long-, medium- and short-lived branches, it no longer works
krono has quit [Ping timeout: 250 seconds]
<arigato>
and it's a revert to the bad old days of svn, if it means we can't use regular merges any more for half the branches at the end
krono has joined #pypy
<antocuni>
what about the following: 1) we create a pre-commit git hook which prepends [branch-name] to all the commits we do; 2) we create a travis check which fails if the commits don't include the branch name
<antocuni>
(the travis checks runs BEFORE the PR is merged, so you notice immediately before pressing "merge")
<antocuni>
this would make "git blame" effectively working, and it would prevent accidental merges with no branch names
<arigato>
it's roughly what I wrote, but right now I think our best way forward is to wait and see how the mercurial hosting situation develops
<antocuni>
the only remaining missing functionality would be "hg log -b", which can be easily implemented on top of "git log --grep"
<arigato>
if you missed it, you managed to miss it twice already :-)
<antocuni>
ok sorry... I didn't understand the detail of what you wrote the first time I read it :). Now I think I do and yes, I think it's mostly the same
<tumbleweed>
the only issue with the pre-commit hook is what to do if a commit in the middle of a branch doesn't have this
<tumbleweed>
you don't really want to be rewriting history to fix that
<tumbleweed>
I guess you just live without it
<antocuni>
why not?
<tumbleweed>
if there are merges after that commit the rewrite gets painful
<antocuni>
as long as you rewrite it BEFORE merging to master
<antocuni>
I see
<tumbleweed>
doable, but a little sucky
tsaka_ has joined #pypy
<antocuni>
well, note that it shouldn't happen; in the ideal world, all the core committers are using the pre-commit hook; the main problem would be with external contributors
<tumbleweed>
right
<tumbleweed>
I was thinking that too
tsaka_ has quit [Ping timeout: 248 seconds]
ionelmc has joined #pypy
<tumbleweed>
on another topic: I was looking at packaging revdb, and I have a few questions, arigato:
<tumbleweed>
it's not published to pypi yet, are you intending to? do you not want to start cutting releases yet?
<tumbleweed>
it would make slightly more sense for us to run it under cpython3. But the tests are so intertwined with the pypy source that I don't think there's any testing I could do in python3. is using cpython3 here a bad idea?
<kenaan>
rlamy py3.6 50a7b57689ea /lib_pypy/_blake2/__init__.py: Fix handling of 1st argument to hashlib.blake2{b,s}() to match CPython 3.6.9 (bpo-33729)
<arigato>
tumbleweed: are you talking about hg/revdb? that's a standalone package that should work fine on top of any python (2/3/cpython/pypy)
<tumbleweed>
I am
<tumbleweed>
it has some prints that need porting to python3, but that's trivial
<tumbleweed>
but beyond that I can't run any tests, just smoke test it
<arigato>
confused, there are tests in the "test" subdirectory
<tumbleweed>
they import rpython
<arigato>
all of them?
<tumbleweed>
I don't think I could get any to work, but I can hack at that harder
<arigato>
OK, yes
<tumbleweed>
only one file doesn't improt rpython :P
<arigato>
no, the tests rely on the rpython directory
<arigato>
they are more tests for whether revdb works when translating small examples
<tumbleweed>
right
<arigato>
and then as a side-effect they also test running the code in hg/revdb on the result
<arigato>
maybe we could write a test that assumes a compiled pypy-revdb, and really tries to use it, testing all commands in order to check them and that they work in a complete pypy-revdb
<arigato>
(it's not done so far)
<tumbleweed>
yeah, I can have a shot at that
<arigato>
thanks!
<kenaan>
rlamy py3.6 4239b16d9453 /lib_pypy/_sha3/__init__.py: Prevent overflow in digest() (bpo-34922)
<ronan>
I think we should have a single base command to record and replay, i.e. 'revdb record' and 'revdb replay'
<ronan>
and then we'd have a single revdb app to test
<tumbleweed>
that's a nice idea
<ronan>
but then the question becomes how to distribute the revdb-record backend(s)
<tumbleweed>
that's currently an issue anyway. I was going to call it pypy-revdb and pypy3-revdb
<tumbleweed>
but that starts to require onboarding documentation
<ronan>
and also answering weird questions like can you record and/or replay py2 from py3 and vice-versa
<tumbleweed>
the binary it was recorded with is recorded in the log, isn't it?
<tumbleweed>
so not necessarily an issue there
<ronan>
yes, the revdb package is already quite agnostic regarding which instrumented interpreter it's talking to
<ronan>
I guess it just needs to grow the ability to control the recording
<arigato>
right, yes, that's as easy as starting the executable with the right environment settings
<ronan>
mattip: df138cf691c1 did weird things in lib-python/ Can you double-check?