alyssa changed the topic of #panfrost to: Panfrost - FLOSS Mali Midgard & Bifrost - https://gitlab.freedesktop.org/panfrost - Logs https://freenode.irclog.whitequark.org/panfrost - <daniels> avoiding X is a huge feature
vstehle has quit [Ping timeout: 246 seconds]
jernej has quit [Remote host closed the connection]
afaerber has quit [Ping timeout: 252 seconds]
afaerber has joined #panfrost
jstultz has quit [Ping timeout: 268 seconds]
jstultz has joined #panfrost
vstehle has joined #panfrost
tomeu has joined #panfrost
jernej has joined #panfrost
<chewitt> @alyssa I bumped mesa to current head .. spotted the files moving from gallium folders (broke our sed for 32-bit)
<chewitt> lots of this showing now
<chewitt> Jul 12 05:10:33 VIM2 kernel: panfrost d00c0000.gpu: js fault, js=0, status=DATA_INVALID_FAULT, head=0x2020080, tail=0x2020080
<chewitt> Jul 12 05:10:33 VIM2 kernel: panfrost d00c0000.gpu: gpu sched timeout, js=0, status=0x58, head=0x2020080, tail=0x2020080, sched_job=000000000c7090f3
<chewitt> using the same combination of reverts and changes that had Kodi working (with black text) before
davidlt has joined #panfrost
guillaume_g has joined #panfrost
davidlt has quit [Ping timeout: 248 seconds]
guillaume_g has quit [Quit: Konversation terminated!]
guillaume_g has joined #panfrost
chewitt has quit [Ping timeout: 245 seconds]
guillaume_g has quit [Quit: Konversation terminated!]
guillaume_g has joined #panfrost
raster has joined #panfrost
raster has quit [Remote host closed the connection]
<guillaume_g> While trying kmscube on exynos, it fails with "MESA-LOADER: failed to open exynos (search paths /usr/lib/dri)" but I guess it should try to load panfrost, no?
<daniels> guillaume_g: panfrost (for the GPU rendering) needs integration with exynos (for the display output) through the mesa 'kmsro' driver
<daniels> atm this requires patching mesa so that the kmsro driver also exposes panfrost; you will probably also need to port changes from a downstream vendor tree so the kernel driver knows how to power on and reset the GPU
<daniels> ah wait, exynos already has kmsro support. did you build Mesa with -Dgallium-drivers=panfrost,kmsro?
<guillaume_g> daniels: I do not have kmsro part
<daniels> guillaume_g: ok, you need to enable kmsro in your build
raster has joined #panfrost
<guillaume_g> daniels: added it now, it is rebuilding. Thanks for this info :)
raster has quit [Quit: Leaving.]
raster has joined #panfrost
raster has quit [Quit: Leaving.]
guillaume_g has quit [Ping timeout: 245 seconds]
guillaume_g has joined #panfrost
raster has joined #panfrost
davidlt has joined #panfrost
davidlt has quit [Read error: Connection reset by peer]
davidlt has joined #panfrost
raster- has joined #panfrost
raster has quit [Quit: Leaving.]
raster- has quit [Remote host closed the connection]
raster has joined #panfrost
afaerber has quit [Quit: Leaving]
afaerber has joined #panfrost
jernej has quit [Quit: Free ZNC ~ Powered by LunarBNC: https://LunarBNC.net]
herbmillerjr has quit [Ping timeout: 245 seconds]
Elpaulo has quit [Quit: Elpaulo]
Elpaulo has joined #panfrost
yann has joined #panfrost
tgall_foo has joined #panfrost
guillaume_g has quit [Quit: Konversation terminated!]
raster has quit [Remote host closed the connection]
<alyssa> chewitt: Oh, dear. THat's... ugh.
yann has quit [Ping timeout: 245 seconds]
jernej has joined #panfrost
chewitt has joined #panfrost
<chewitt> alyssa: might have been me getting the sed(s) wrong
<chewitt> up to bb483a91663f663fac33879491dfb62b87fce3b1 is working so far .. I'll step through more commits this evening
<chewitt> working = Kodi runs .. so far everything still shows the black text
<tomeu> chewitt: may be worth trying with this branch: https://gitlab.freedesktop.org/tomeu/mesa/commits/always-64
<tomeu> I think you need some of the stuff that was guarded by is_t6xx, because you are running on 32bit
<tomeu> that branch should take care of that
<alyssa> chewitt: I'm building latest Kodi master now
<chewitt> tomeu: so that branch should run 32-bit without our hacky sed(s)?
<chewitt> or we need to config something?
<alyssa> Yeah
<alyssa> chewitt: With that branch, you can just drop the panfrost patching entirely
<alyssa> And itought to just work
<chewitt> ohh .. shiny
<chewitt> i'll test this evening
<alyssa> ...What time is it where you again? :p
<chewitt> i'm in the UK at the moment so 4.16pm
<chewitt> back to the sandpit in ~8 days
<alyssa> :+1:
chewitt has quit [Quit: Zzz..]
chewitt has joined #panfrost
<alyssa> Kodi takes a long time to build, huh
<chewitt> depends what you build it on :)
<alyssa> My RK3399 laptop that I use to build Mesa round the clock ;p
<chewitt> I can normally re-spin an LE image with a Kodi change in ~60 seconds
<alyssa> Yeah, but that's a partial rebuild
<alyssa> I'm going from clean
<chewitt> full OS image (240 packages) takes 28 mins
<chewitt> clean build
<chewitt> clean Kodi is ~2 mins
<jernej> chewitt: which CPU you have?
<chewitt> AMD TR2 with 16x Cores (32 Threads)
<chewitt> plus fast I/O
<jernej> ok, then I will certainly look into buying AMD 3900X in a month or two + NVMe
<chewitt> cross-compile is the way to go
* alyssa rathers like working all on-board
<alyssa> Except for building Kodi, I guess
<alyssa> 63% blues
<alyssa> 74%, I guess this might be speeding up? :P
raster has joined #panfrost
<chewitt> i'm definitely not as patient as you :)
<alyssa> chewitt: Build finished (I had to patch a tiny GBM thing but no rendering changes)
<alyssa> Latest Kodi master works fine on RK3399 with latest Panfrost master
<chewitt> white text?
<alyssa> (Text is the correct colour)
<chewitt> okay, thanks
<alyssa> So that rules out Kodi changes as the cause
<alyssa> And means it's specifically something related to T820 or 32-bit
<alyssa> So I think tomeu/always-64 may well fix things for you
<chewitt> I'm picking the patches to prep a test image .. it will be a few hours before I can unpack a board and connect it to a TV though
<alyssa> OK
<alyssa> I'll be here :)
<alyssa> I will note, Kodi 19 / master / GBM is *substantially* slower than Kodi 18 / Debian / X11
<alyssa> (Latest Panfrost for both)
<alyssa> Not sure what that's about
chewitt has quit [Quit: Zzz..]
chewitt has joined #panfrost
<chewitt> @alyssa: I can't see whether white text is working (Federer vs. Nadal at Wimbledon is monopolising the single TV) but I booted an image with the 32-bit changes and Kodi runs .. so that's a good sign
<chewitt> I'll be allowed to switch sources on the TV once one of them loses :)
<alyssa> Priorities ! :D
<chewitt> current mesa HEAD (or thereabouts) + 3x commits in the always-64 branch + my hacked kernel driver
<chewitt> Q: What's the difference between your mother and a terrorist?
<chewitt> A: You can negotiate with terrorists
<chewitt> "we will not be switching the telly over"
<alyssa> Your mum?
<chewitt> indeed
<alyssa> "We all have mothers" -robclark
<alyssa> I never had much luck with perf for various reasons,
<alyssa> but daniels suggested sysprof, and that is working quite well
<alyssa> (hello from GNOME, blah blah)
<alyssa> Findings from Supertuxkart on master:
<alyssa> - We're spending an obnoxious amount of time computing the scoreboard
<alyssa> - That's a good bottleneck to have since it's well-understood CPU code; algorithmic improvements are possible to get it down to linear time, I think
<alyssa> ( It's accidentally O(N^2) right now)
<alyssa> - Creating and releasing bo's seems to happen a lot. That's slow. Don't do it
<alyssa> - Easy enough to handle since memory usage is sane. We can pool/cache BOs and save a lot of grief.
<alyssa> - Often times these are transient internal BOs. A huge chunk of time is spent creating BOs for u_vbuf_draw_vbo
<alyssa> - Does STK use user vbufs and then Gallium is uploading? We might benefit from doing the upload ourselves since transient memory is specially optimized
<alyssa> Also, given how helpful that was, let's try sysprof on the memory management branch (which is a step forward but regresses perf hence why it's not merged)
<alyssa> Gotta say, this is a super exciting capability to have
<alyssa> If we can tackle CPU side bottlenecks and CPU-GPU shared bottlenecks, that's... 80% of the battle
<chewitt> @alyssa: black text :(
<alyssa> chewitt: Fudge.
<chewitt> (they left the room to make tea)
<alyssa> chewitt: What am I missing here.... Hrm
<alyssa> RK3399 64-bit is perfectly happy on identical Kodi
<alyssa> chewitt: Well, if it doesn't outright regress everything, that's still a good sign
<alyssa> Text colour aside, dropping the LE patches is a win for both of us
<chewitt> ^ that's what it looks like
<chewitt> dropping the sed stuff is a nice win :)
<alyssa> Let's see if my graphics nerd eyes see something you don't uhhh
<alyssa> Hum
<alyssa> This is the fix-64 branch hh
<chewitt> it's mesa master + the last three commits from that branch (which look like the only changes)
<alyssa> Okay, hm
<chewitt> I'm also missing a load of kernel driver commits .. I'm basically running the initial 5.2 merge commit and the WIP "soft reset" patch from narmstrong
<alyssa> chewitt: This is almost certainly a userspace bug, so old kernel is ok :)
<chewitt> but I don't think those are relevant as this issue was present on 5.1 before
<alyssa> I just don't understand how this could've worked fine before and regressed without regressing T860
<chewitt> http://ix.io/1OgY <= midgard shader debug
<alyssa> chewitt: It wouldn't be shaders, since shaders are identical across T8xx and the same Kodi is working on T860
<alyssa> panfrost_find_free_slab is the bottleneck on the memory management branch. That's.... weird... but ok
<chewitt> I'll start an aarch64 build running .. just to eliminate the idea from suspicion
<alyssa> chewitt: I'd definitely appreicate that :)
<chewitt> http://ix.io/1Oh3 <= the other debug thingy (partial again)
<alyssa> chewitt: That snippet, at least, looks totally normal to me
stikonas has joined #panfrost
<alyssa> As for errata, the only difference between T820 and T860 is T820 being affected by issue T76X_1909
<alyssa> But I don't know what that issue is
<alyssa> _mesa_set_next_entry is now a bottleneck, hm. Just moving the problem
<chewitt> raster: what's errata T76X_1909 ???
<alyssa> chewitt: I rather doubt it's relevant to our bug :)
<alyssa> Also, raster most likely neither knows nor could disclose that :)
<raster> chewitt: i have no idea... :)
<alyssa> Told ya
<raster> and if i did... unless the existing kbase driver has the info... i cant talk about it :)
<alyssa> Told ya x2
<raster> so kernel stuff only :)
<raster> if that changes in future... you'll know :)
<alyssa> raster: You know, older kbase had comments listing all the errata... wonder why kbase stopped ;) :P
<raster> possibly TMI? :)
<alyssa> "If it's good for Alyssa, it must be bad for us"
<raster> hahahahaha
afaerber has quit [Quit: Leaving]
<chewitt> alyssa: aarch64 is the same (black text) so it's not a 64/32 thing
<alyssa> chewitt: Okay, that's helpful!
<alyssa> Rules out a whole class of bugs
<alyssa> So we're left to, honest-to-goodness, T820 vs T860 differing somehow
<alyssa> I know how to debug this but.. you won't like it :p
* alyssa is *this* close to grabbing a devboard
<Lyude> alyssa: I've got a T820 system that I can give you access to this weekend if you'd like
<alyssa> Lyude: I would appreciate that :)
<Lyude> (also, that's one of the ones I actually wrote detailed instructions for fedora installs :)
<Lyude> alyssa: sure thing, I'll look at it this weekend. I think you should still be able to login to lyude.net
<alyssa> chewitt: Anyway, I'd like to run a full dEQP-GLES2 run
<alyssa> Inevitably, some tests will fail that pass on T860, and that'll tell us what to zero in on
<chewitt> what's needed to do that?
<alyssa> Clone/build dEQP, run it, send the results
<alyssa> I'm assuming LE is too limited to do that within the system image
<chewitt> depends .. if it's just another package to add, it's probably not impossible
<chewitt> where's the sourceS?
<alyssa> Not sure if dEQP works on raw GBM (otherwise, also add weston or X/i3 into the mix)
<chewitt> @Lyude .. what's your T820 board running at the moment?
<chewitt> (as it's the same board I have)
<Lyude> chewitt: an old kernel, I'm probably going to go update/maybe redo the install on it when I set it up this weekend
<Lyude> note it's not turned on right now
davidlt has quit [Ping timeout: 246 seconds]
<Lyude> chewitt: cool, thanks!
<chewitt> it's a bit frankenstein with the reverts and such, but it works
<alyssa> chewitt: Anyway, I'll set it up on Lyude's board once that's alive and work from there :)
<chewitt> feel free to find/fix the kernel driver issue :)
<chewitt> https://github.com/chewitt/linux/commits/panfrost-broken <= same branch, but different "Add initial panfrost driver" commit
<chewitt> https://github.com/chewitt/LibreELEC.tv/blob/amlogic-master/projects/Amlogic/linux/linux.aarch64.conf <= defconfig to match .. but might be a bit minimal for you
MistahDarcy has joined #panfrost
chewitt has quit [Ping timeout: 245 seconds]
afaerber has joined #panfrost
chewitt has joined #panfrost
yann has joined #panfrost
<alyssa> Alright, my memory management series is now in a place I'm happy with (after the third rewrite..)
<robher> chewitt: If I understood the diff yesterday correctly, this should fix the issue:
<robher> From Mali Allwinner H6 thread...
<alyssa> robher: Not sure if you saw / if this affects you, but tomeu got two things working today (kinda tied)
<alyssa> - T720 support (good enough to run Weston, FBOs are missing)
<alyssa> - 64-bit descriptors on T7xx
<alyssa> The upshot is that in the next few weeks (before the 19.2 branch point in early August), we'll be switching to 64-bit job descriptors everywhere
<robher> alyssa: nice. Though I only have rk3399.
<alyssa> Same (well, and RK3288 but out of service right now)
<chewitt> rebuilding..
<chewitt> [ 9.536160] panfrost d00c0000.gpu: Fatal error during GPU init
<chewitt> [ 9.536309] panfrost: probe of d00c0000.gpu failed with error -12
<chewitt> ^ robher
<jcureton> chewitt: shot in the dark without knowing your full setup off my head, but the line "if (cfg-ias != 48 || cfg-oas > 40)" in io-pgtable-arm.c also has to be changed to "if (cfg-ias > 48 || cfg-oas > 40)" for devices with a smaller mmu bus width
<jcureton> there was a patch with it on the mailing list at one point but I never saw it merge
<robher> jcureton: the one I just pasted above...
<jcureton> robher: the one you posted above ensures a full 4-level pgtable is always returned, it doesn't change the sanity check in allocating a pgtable
<robher> jcureton: ah, right. Both may be needed.
<jcureton> relevant dri-devel patch - https://patchwork.kernel.org/patch/10954241/
<chewitt> rebuilding..
MistahDarcy has quit [Quit: Leaving]
<robher> The quick fix on both issues is just hard code ias to 48 in the panfrost driver instead of reading the register value.
<chewitt> jcureton robher .. we have a winner :)
<chewitt> I'm now running 5.2 with all panfrost commits and mesa HEAD on T820
stikonas has quit [Read error: Connection reset by peer]
stikonas_ has joined #panfrost
stikonas_ is now known as stikonas
paulk-leonov has quit [Ping timeout: 252 seconds]
<chewitt> only the black text issue to go then
<alyssa> Ugh, why am I always the one causing issues
<Lyude> because you're always the one writing a lot of code :)
<alyssa> Lyude: This is your fault, too, somehow!!! :p
<Lyude> hehe
<alyssa> Good news - I just did some optimization on panfrost_scoreboard_link_batch, so that function is now faster by more than an order of magnitude
<alyssa> Which means moar fps on apps that do a lot of draw calls (3D games..)
paulk-leonov has joined #panfrost
<alyssa> We're now spending nearly 1/3 of the time either allocating or freeing BOs.
<alyssa> So next up, I think, should be a BO cache.
<alyssa> (..Actually, nearly 1/2 T_T)
<robher> alyssa: maybe first, get rid of some of the mmap'ing.
<alyssa> robher: The mmaps aren't much of a bottleneck by contrast
<robher> alyssa: perhaps they are mapped on demand... Enabling hugepages help much?
<alyssa> Oh! Let me try, a moment
stikonas has quit [Ping timeout: 245 seconds]
stikonas has joined #panfrost
<alyssa> robher: Just did a branch skipping the userspace-side mmap opportunistically
<alyssa> At a minimum it saves mapping 256MB (!) on startup for each app
<robher> alyssa: the heap support should help there too.
<alyssa> GROWABLE, you mean?
<robher> alyssa: if you insist on keeping the kbase name, yes. ;)
<alyssa> robher: huge pages don't seem to be a huge win, although I don't see any regressions either (functional or performance)
<alyssa> robher: oh, this is interesting... anything in practicular that would cause us to hit the wrath of kcompatd?
<alyssa> compact
<alyssa> (and kswapd)
<alyssa> Is that just, generally, low memory conditions, or something specific to frost?
<robher> alyssa: do you actually have a swap file/partition?
<alyssa> Definitely no swap partition
<alyssa> I don't think a swap file either (I never set one up)
<robher> Then all pages are always in memory and if you run out I guess random processes are killed...
<robher> But we pin all pages on alloc, so with swap all BO pages are still always in memory.
<robher> IOW, we don't support swapping BOs.
<alyssa> Fun
<alyssa> Alright... BO cache um
<alyssa> Oh, should probably cleanup non-transient memory generally first
herbmillerjr has joined #panfrost
<alyssa> "Don't do this in production, kids"
<alyssa> That's not a line I want to read in my production driver source.
* alyssa curses her self from a year ago
<HdkR> Past self always is your worst enemy :D
<alyssa> We do legitimately need memory management for shaders, but... meh, our strategy so far works well enough, ahem
herbmillerjr has quit [Quit: Konversation terminated!]
<alyssa> Just cleaned up all of that nonsense
<alyssa> We'll see what CI thinks
<alyssa> For now, weekend \o
<alyssa> On monday, we'll continue this... adding a BO cache really is the next up
<alyssa> I'm excited to see how much of a gain it'll be!