cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://quodlibet.duckdns.org/irc/pypy/latest.log.html#irc-end ) | use cffi for calling C | if a pep adds a mere 25-30 [C-API] functions or so, it's a drop in the ocean (cough) - Armin
<mattip>
arigato also had problems logging in with bitbucket credentials.
<gracinet>
will tell you about that in a minute
<mattip>
I made an email account early on
<mattip>
gracinet: no hurry
<mattip>
gracinet: the other repos are much smaller, I assume they will go faster
<gracinet>
import started
<gracinet>
mattip: yes, I've done some testing of that some time ago. Some take like 10m on my workstation, but nothing on the same scale as the main repo
<Dejan>
heptapod.net ?
<gracinet>
cfbolz: we had an issue with accounts created from the SSO connection, I'll check yours and arigato's
<mattip>
Dejan: check foss.heptapod.net/pypy in a few hours
<gracinet>
mattip: during the test run a few days ago, Bitbucket had a big hickup, I had to start over
<gracinet>
(response HTTP 410, apparently it happens)
<mattip>
fingers crossed
<gracinet>
cfbolz: your account is not tied to Bitbucket at all. Are you able to login with normal email+password ?
<cfbolz>
yes
<gracinet>
ok then you have to make the link from your settings page (detailed instruction in a few secs)
<mattip>
cfbolz: I tried to make a distinction between long-lived named branches for core devs
<cfbolz>
ok
<mattip>
and short-lived topic branches for occasional contributors
<cfbolz>
just saying we should discuss it
<mattip>
yup. I don't think the core devs want to change their workflow
<gracinet>
mattip: a thing that should probably be underlined is that *activating* evolve will be quite necessary, but that's not the same as *using* it
tsaka__ has joined #pypy
<gracinet>
without evolve, on a repo that has had amended changesets, you get these:
<gracinet>
Dejan: by word of mouth, early adopters, and we're posting to several communication channels, including an issue on GitLab that says "please support mercurial" is that's what your asking.
<Dejan>
mattip, i went to foss.heptapod.net
<Dejan>
and tried to register as "dejan"
<mattip>
weird. There is no user with that name as far as I can see
<gracinet>
Dejan: maybe try the password reset sequence with the intended email address?
<Dejan>
Idk what was wrong... now it works...
<Dejan>
This is great as I am most familiar with GitLab :)
<gracinet>
Dejan: yes I can see you
<Dejan>
so foss. is for opensource stuff, right?
<gracinet>
Dejan: makes sense. The auth parts, issues etc are really as in GitLab
<gracinet>
we are also working with our hosting partner, Clever Cloud, to launch a commercial service
<Dejan>
that is great - I am a Mercurial newbie
<Dejan>
I only use it to grab PyPy stuff :)
<gracinet>
mattip: I have a meeting at 11. I suggest you try and import one of the secondary repos (but not all of them concurrently while the main one is running)
<gracinet>
btw mattip (Cc arigo): I have nothing against importing the experimental repos, even if they're dead. It can be justified by consistency in your case (and they are dwarfed by the main repo)
<mattip>
ok. where is the workflow for the import?
<gracinet>
mhmh maybe I have to merge that, let's see.
<mattip>
we have begun the import of pypy to foss.heptapod.net/pypy/pypy
<arigato>
for those who don't follow irc, maybe send a mail to pypy-dev reminding us that the "12 Feb" deadline is passed, which explains why we got a mail "your access level to pypy/pypy has been changed from write to read" :-)
<cfbolz>
(we generally do quite good at emulators/hardwary stuff, it seems)
<gracinet>
mattip: the import is done, I'm switching the project to private while we are setting up clone bundles
<flok>
mattip: I did once do a un-scientific comparison of several languages and their compilers though. i had written a brainfuck-to-* compiler (brainfuck to python, perl, c, c++ and so on) and then converted a mandelbrot in brainfuck (yes!) with it and ran the results. see https://vanheusden.com/misc/blog/2016-05-19_brainfuck_compilers_compared.php
<Dejan>
I hope I will have something soon to put there
<Dejan>
flok, I remember someone from the D community implemented BF in just few lines of code... on top of that it statically compiled BF into a D function that can be called and executed at runtime...
<Dejan>
some people have lots of time... I envy those
<flok>
cool :-)
<Dejan>
are you a chess player? :)
<flok>
no, i'm a chess-programs-writer. i know the rules but that's about it
<Dejan>
:D
<Dejan>
is your engine competing on CCRL?
<flok>
yeah most of my programs are. check for "embla" and "micah"
<Dejan>
2176 ELO is quite good
<Dejan>
I never had a chance to work on one myself...
<Dejan>
lots of theory needed to implement something that is serious enough
<flok>
oh I also wrote one in python using pychess but it was too slow to compete
<Dejan>
I may port it to D if I find time...
<Dejan>
I stopped using C++ 15+ years ago
<mattip>
flok: interesting that nuitka did so well on the compiler comparison
<mattip>
shedskin too
<gracinet>
mattip: pypy/pypy ready to be cloned on foss.heptapod.net
<Dejan>
woooohooooo
* mattip
trying
<flok>
mattip: yeah but older versions
<flok>
i've been trying to convince the writer of shedskin to compile py3 as well for years
<gracinet>
please take a look at issues and mrs and tell me if that's a go for public
<gracinet>
I'm correcting the default branch
<flok>
but he prefers playing Go and toying with his 3d-printers
<Dejan>
let me know when i can clone pypy from heptapod :)
<Dejan>
at the moment I do not see the repo
<mattip>
gracinet: issues look good. It would be nice to replace the "created by" with the "created originally" username, but I understand you can't do that
<gracinet>
Dejan: it's still private at the moment. I'm switching to "internal" so that all logged in people can see it
<gracinet>
Dejan: done, you can now see it.
<gracinet>
I'll make it public soon
<gracinet>
mattip: you mean in the list ? No, sorry that's something we can't do for the time being
<mattip>
yeah, understood :(
<mattip>
cloning is using the clone bundles - good
<gracinet>
yes, we're moving them from static.octobus.net to a sub-path within foss.heptapod.net as we speak
<gracinet>
PyPy has roughly x2 changesets compared to the Mercurial repo, but it's 10x times heavier
<gracinet>
we realised today that it wouldn't need that many concurrent clones to kill the server
<mattip>
seems as good as it will get. The download is quick, building the manifests and file changes is slow
<gracinet>
mattip: yes, that's client-side though, outside of our immediate control
<gracinet>
we're waiting for a full blown Mercurial client with both PyPy and Rust extensions though HPy ;-)
<gracinet>
ah, marmoute has been uploading new bundles that should apply faster
<mattip>
gracinet: a quick check of open PRs shows 15 on bitbucket, 14 on heptapod. I wonder which one went missing
<mattip>
the hashes of the commits match - nice job
<gracinet>
mattip: for the open PR, the topic changesets should have new hashes though. That's because they are obsoleting the ones from bitbucket
<gracinet>
mattip: the missing one is PR 561
<gracinet>
I can import it later once I've fixed the problem with it
<mattip>
gracinet: ok. pypy/pypy seems good to go.
<gracinet>
I happen to have some limited incremental reimport facilities
<mattip>
maybe not a big deal: it sat for two years. We can recreate it before shutdown in May.
<gracinet>
mattip: great!
<mattip>
cfbolz, arigato, ronan: want to take a look at foss.heptapod.net/pypy/pypy ?
<mattip>
gracinet: the lost commits in pypy/buildbot are next?
<cfbolz>
mattip: looks good, will dig around a bit
<gracinet>
yes, we're finalizing the clonebundles config right now
<gracinet>
moving from Octobus to Clever Cloud servers
<gracinet>
for better… herm… everything
Rhy0lite has joined #pypy
<Dejan>
gracinet, I am not a PyPy dev btw
<gracinet>
Dejan: nor am I
<gracinet>
;-(
<gracinet>
sorry, misstyped :-)
<Dejan>
i still can't seen the PyPy
<Dejan>
I mean PyPy/pypy
<mattip>
right, it is not public yet
<gracinet>
it's gonna be public in a few minutes, but normally all logged in users should see it
<Dejan>
ah sorry I thought you made it public
<cfbolz>
gracinet: hm, are the attachments of issues lost somehow?
<cfbolz>
gracinet: I am amazed that the issue numbers have remained stable, that's really fantastic
<Dejan>
pr0 Bitbucket import
dddddd has joined #pypy
tsaka__ has joined #pypy
<mattip>
keeping the issue numbers and commit hashes stable is really important for an easy transition, cudos to the team
<mattip>
kudos too
Smigwell has joined #pypy
<gracinet>
cfbolz mattip: thanks guy
<gracinet>
currently running a final housekeeping task to help performance
<gracinet>
about issue numbers, that's the GitLab importer's doing: it just creates them with the same number (which kills incrementality of course). Same with MRs
<gracinet>
the kind of incrementality I have for MRs is reimporting the failed ones
Alphare has joined #pypy
<mattip>
gracinet: still waiting for news about pypy/buildbot to know if I should start the other imports
<gracinet>
yes mattip: I don't forget about you. Was checking the RST thing while the housekeeping was running
<gracinet>
mais pour cette issue précise, Laurent et Quentin m'ont dit que Clever comptait nepas utiliser le registry de GitLab qui serait, je cite « pourri »
<marmoute>
gracinet: wrong channel
<tos9>
awesome, well done guys
<arigato>
yes, thanks for all the work!
<gracinet>
oops as well :-)
<tos9>
gracinet: where's a good place to put completely-not-important-wish-list-y feature requests (and/or pull requests for them :) -- is there an issue tracker for heptapod itself there or is it basically gitlab (which I know very little about)
<gracinet>
mattip: that's what I've got in the logs abort: HTTP Error 410: Gone
<marmoute>
tos9: you can file issue agains heptapod
<gracinet>
mattip: that's the same Bitbucket hickup I had in a test run 3 days ago
<gracinet>
yes, that means it'll have the n+1 number, with n=biggest one imported from Bitbucket
<cfbolz>
mattip: cool
<arigato>
mattip: is there a way to stop people from writing more comments into existing issues on bitbucket?
<Dejan>
what I really like on GitLab is the CI/CD stuff, do you guys provide that on foss.heptapod.net ?
<mattip>
arigato: I didn't find a way other than to make the issues private, which means they would dissapear
<gracinet>
Dejan: yes, but we are currently quite limited on resources.
<gracinet>
For the time being, we've been granted two servers by osuosl.org, and are following some serious leads
<Dejan>
So if I can't create fork/personal-project - how do I make a PR?
<mattip>
arigato: dissapearing them means anyone's links to bitbucket will break
<gracinet>
Dejan: you ask for Developer role, then you can push a topic and make a MR out of it. I believe mattip has written that down
<gracinet>
guys, I'll be afk for an hour
<mattip>
see ya
<Dejan>
okidoki, thanks
<arigato>
mattip: indeed, thanks
<gracinet>
forgot to say, I'm not usually in #pypy, the best way ask questions on a chat about Heptapod is on mattermost.heptapod.net
<Dejan>
you should port that Ruby to Python...
<Dejan>
:P
<gracinet>
hear, hear
<gracinet>
and that Go to Rust ;-)
<Alphare>
gracinet: go eat
<Alphare>
:D
<Dejan>
nah, Go to D ;)
<mattip>
arigato: the problem will go away May 31, in 3 1/2 months
<arigato>
yeah
<tos9>
that clone was so much faster I almost want to rm -rf pypy to do it again a few times
<mattip>
:)
<Alphare>
heh
<arigato>
finally a clone that runs at the full speed of my home's limited 1MB/s :-)
xcm has quit [Remote host closed the connection]
<Alphare>
mhhh I wonder how much that takes for me
xcm has joined #pypy
* cfbolz
tries to push something to heptapod
<cfbolz>
worked, it seems
jvesely has joined #pypy
<mattip>
cfbolz: are you using ssh push?
<cfbolz>
yep
jacob22_ has quit [Ping timeout: 272 seconds]
<mattip>
changing my default url and pulling caused a bunch of disk churn and now I have a "default" head for each of the topic branches
<cfbolz>
are there any topic branches?
<mattip>
all the PRs became topic branches
<cfbolz>
ah
<cfbolz>
I don't know how to use that part of hg yet
<marmoute>
mattip: beware of publishing (I am looking into adding more security around this right now)
<mattip>
cfbolz: yeah, me either. I did get your commit though, on default
<cfbolz>
ok
<cfbolz>
fix on py3.6 coming in a few min
<mattip>
marmoute: is "publish" what we used to call "merge"?
<marmoute>
mattip: they are independant.
<mattip>
marmoute: so I don't know what publish is.
<mattip>
my tortoisehg knows about push, merge, pull, and from the command line I know branch, --close-branch
<mattip>
publish is a new one
<marmoute>
publish is just turning things in the public phases.
<mattip>
I thought once you push to another repository it becomes public
<marmoute>
with heptapod, topic stay draft, and changeset without a topic are publish on push.
<cfbolz>
marmoute: after a topic is merged, it's no longer visible in the history which topic the commits came from, correct?
<cfbolz>
mattip: I managed to push to py3.6, the crash should be fixed
<mattip>
when I pull I get an error:
<mattip>
mercurial.error.ProgrammingError: stablesortcache must be warmed before use (98695 < 98696)
<mattip>
ahh, that seems to be tortoisehg without the evolve extension. Pulling from a version of hg with the extension works
<mattip>
no, it's the same hg. What?
tsaka__ has quit [Ping timeout: 246 seconds]
<mattip>
gracinet: could you check the pypy/extradoc import? It has been more than an hour ...
<mattip>
i guess it is a rather large repo
bbot2 has quit [Quit: buildmaster reconfigured: bot disconnecting]
bbot2 has joined #pypy
tsaka__ has joined #pypy
<marmoute>
cfbolz: Yes, the information is no longer visible. (but still exists stored somewhere)
<mattip>
but but but, that is why we didn't move to git!
<mattip>
so for pypy, we should disable the topic stuff
<cfbolz>
if it's still showable, that's already better
<mattip>
or design a workflow that converts topic branches to true branches before merging
<marmoute>
You can work with branch
<gracinet>
mattip: topic changesets also have branch information. It's very possible to push a topic for a branch that doesn't have any publicj changeset yet
<cfbolz>
yes, I think that's the plan. we keep our branch-based workflow
<mattip>
arigato: do you remember how to restart the bbhook/run.py on buildbot.pypy.org?
<mattip>
I changed it to look at foss.heptapod.py (in pypy/buildbot), but forgot how to restart it
<mattip>
ahh, I think I found the relevant README
<mattip>
nope. When I try to run screen -xd bbhook as buildmaster, it tells me there is no such screen
<arigato>
mattip: yes, I think for us we keep the branch-based workflow, and consider topics as mostly an implementation detail for pull requests
<mattip>
+1.
jvesely has quit [Read error: Connection reset by peer]
<mattip>
arigato: could you restart bbhook?
<arigato>
for small pull requests, it's not that bad if we don't end up with an artificial long branch name like bitbucket PRs do
xcm has quit [Killed (orwell.freenode.net (Nickname regained by services))]
<arigato>
and for big pull requests we can always make a branch in the pull request, and then the PR adds this branch too (using branches *and* temporarily topics as long as it's not merged)
xcm has joined #pypy
<arigato>
mattip: I think it's antocuni that runs it
<arigato>
ah or no, likely it's the 'buildmaster' user
<mattip>
README-cobra mentions a screen, but I couldn't find one
<arigato>
ah, just "screen -r" found the screen for me
<arigato>
just restarted now
<mattip>
thanks
<mattip>
it seems buildbot found the correct repo, nice
<arigato>
:-)
<arigato>
if you feel like, maybe we could replace the README.rst on bitbucket (only) saying that we moved to heptapod, with a link?
<gracinet>
arigato: that one has failed. I'm gonna retry it on my workstation, but it could be due to large files in there
<marmoute>
cffi seems pretty small, having that take time is suspicious
<gracinet>
yes
<gracinet>
3292 changesets / 5.1MB
<gracinet>
I've never tried that one previously.
<gracinet>
not sure that's the cause but extradoc would really need LFS
<antocuni>
mattip: this is used by check_for_local_repo
<marmoute>
gracinet: Is do not see anything hughe in extradoc
<antocuni>
basically, the current bbhook keeps a clone of each repo locally, so it needs to know where to clone it from
<marmoute>
largest revlog is 15MB
<gracinet>
I've seen something in the logs about a file 266MB but could have been mistaken
<marmoute>
maybe uncompressed ?
* marmoute
dig in thatr direction
<marmoute>
not even
<marmoute>
largest things seems to be extradoc/talk/pepm2011/benchmarks/html5lib.trace.bz2 (8.2Mb)
<marmoute>
(MB)
<gracinet>
ok, found the log line for extradoc, [hg] abort: Connection reset by peer
<gracinet>
I say we let cffi finish, and then we retry extradoc
* marmoute
is scrubbing bitbucket for archival and he often get Connection Reset
<gracinet>
one thing's for sure, they aren't going to pour lots of resources in making Mercurial reliable
<antocuni>
mattip: but the details of the new bbhook depends on the format of gitlab's webhooks, of course
<mattip>
antocuni: yeah. I guess I should test it locally with a throw-away repo
<gracinet>
I'm removing the 'testpypy' group now. It's of no use anymore (that's a former test of mine)
tsaka_ has joined #pypy
marky1991 has joined #pypy
YannickJadoul has quit [Quit: Leaving]
<mattip>
gracinet: I don't know how much longer you will be around today, but cffi is still importing
<gracinet>
I think it shoudl be almost done, let me getback to those logs
marky1991 has quit [Ping timeout: 268 seconds]
Ai9zO5AP has joined #pypy
<gracinet>
the hg part is done, it is now pushing to the inner git repo. It looks perfectly normal in Mercurial, but somehow translation to Git is painful for that one.
<gracinet>
can't wait till we don't need to cheat at all
marky1991 has joined #pypy
jacob22_ has quit [Quit: Konversation terminated!]
benbangert has joined #pypy
<mattip>
gracinet: still importing all three. Is there some way to start over?
<gracinet>
cffi is still pushing to Git, I don't think starting over would change much
<gracinet>
but there's been some progress since last time I looked
xcm has quit [Remote host closed the connection]
xcm has joined #pypy
jacob22 has joined #pypy
plan_rich has quit [Ping timeout: 260 seconds]
plan_rich has joined #pypy
<gracinet>
mattip: I'm restarting over extradoc, cffi is still of the middle of its inner push
<mattip>
Let's leave extradoc to the end. Could you restart benchmarks?
<gracinet>
ok then
<mattip>
extradoc is going to be painful, alot of commits and some big payloads
<gracinet>
I see that benchmarks is still running as well
<mattip>
is it running successfully?
<mattip>
maybe running two imports at once swamps the machine?
<gracinet>
unclear
<gracinet>
normally, it should not be a problem unless we hit some limits
<gracinet>
I'm killing thebenchmarks import and retry it
<mattip>
ok. It's fine if you want to leave it till cffi finishes and we will do them one at a time
<mattip>
the really critical repos are pypy, cffi
<mattip>
the other ones are more for internal use or archival
<gracinet>
Ok, then I'll come back in a few hours, see where cffi stands.
<gracinet>
I've killed the benchmarks import anyway
<mattip>
thanks
i9zO5AP has joined #pypy
Ai9zO5AP has quit [Ping timeout: 268 seconds]
xcm has quit [Remote host closed the connection]
xcm has joined #pypy
plan_rich has quit [Ping timeout: 272 seconds]
xcm has quit [Remote host closed the connection]
xcm has joined #pypy
i9zO5AP has quit [Ping timeout: 240 seconds]
<cfbolz>
The two commits I just pushed are a 15% warmup improvement
oberstet has quit [Remote host closed the connection]
CrazyPython has joined #pypy
i9zO5AP has joined #pypy
CrazyPython has quit [Read error: Connection reset by peer]
<gracinet>
mattip: to let you know, I've investigated problems with extradoc, it turns out I've got systematic clone failures (even completely outside of Heptapod context) on https://bitbucket.org/pypy/extradoc Tryiing over SSH to see if I get more luck