05:55 <scanakci> Did anyone recently try Vexriscv on Genesys board with a frequency lower than 125 MhZ?

05:56 <scanakci> Memory tests fail if I use 75MhZ rather than default 125 MhZ.

06:05 <scanakci> Memory tests for Rocket also fails if I use 75 MhZ clock frequency.

10:53 <lkcl> scanakci: interestingly i have a similar isssue on a versa ecp5 - below 55-60 mhz the memory tests fail

10:54 <lkcl> however in discussing with daveshah yesterday he mentioned that some boards can go as low as 48mhz no problem

10:55 <lkcl> meaning: the particular DRAM ICs or something to do with the way they're soldered is causing problems.

10:56 <lkcl> really this needs fixing by having DRAM on its own PLL.

10:56 <lkcl> on the versa ecp5 there's 2 spare to do that

10:56 <daveshah> It's more complex than that, as you then need CDC on the DRAM interfaces too

10:56 <lkcl> CDC?

10:57 <lkcl> cross-domain clock synchronisation?

10:57 <daveshah> yes

10:58 <daveshah> for both the CSR control bus and the main memory Wishbone bus

10:58 <lkcl> between wishbone and the DRAM... ah the CSR as well.

14:42 <_florent_> lkcl: the sys_clk is currently coupled to the dram_clk to avoid CDC that can be costly

14:43 <_florent_> in term of resources

14:44 <_florent_> lkcl: this is generally working fine with CPUs that are really targeting FPGAs, but indeed does not work well with CPUs targeting ASICs

14:45 <_florent_> that's something we could work on in the next months

14:49 <_florent_> some automatic benchs have been already created to check calibration at various frequencies (with a main PLL reconfigured dynamically), this need to be extended to more boards and after that we could see if it's interesting to add CDC support directly with LiteX, or if this should be done with a standalone LiteDRAM controller (with a SERV CPU doing the initialization) and do the CDC on the user ports. The latter is

14:49 <_florent_> already possible, but still require some time to create/test the design.

14:49 <_florent_> the bench on Arty is here: https://github.com/enjoy-digital/litedram/blob/master/bench/arty.py

14:49 <tpb> Title: litedram/arty.py at master · enjoy-digital/litedram · GitHub (at github.com)

16:23 <scanakci> thanks @lkcl

16:58 <lkcl> _florent_: appreciated

16:58 <lkcl> well, fortunately, after 2 *days* of arseing about, i've tracked down a bunch of critical path issues

16:59 <lkcl> none of them being important in simulation of course

16:59 <lkcl> one of which was that the instruction decoder (which is enormous in POWER9) was combinatorially linked/chained to register read.

16:59 <lkcl> looovely

17:00 <lkcl> daveshah: Info: 2.9 ns logic, 14.8 ns routing

17:00 <lkcl> Info: Max frequency for clock '$glbnet$sys_clk': 56.72 MHz

17:00 <lkcl> yaaaaaa it's above 55 mhz, w00t.

17:03 <lkcl> but.. boy did i make a dog's dinner of the RTL finding out that it was the decoder...

17:05 <lkcl> picorv32 runs @ 55mhz on this LFE5UM... libresoc shouuuld therefore also run

17:08 <lkcl> _florent_: just looking at nextpnr critical path

17:08 <lkcl> now that i've fixed the decoder combinatorial issues in libresoc

17:09 <lkcl> the 32-to-64 bit wishbone converter is a major combinatorial path

17:09 <lkcl> Info: 0.2 13.3 Source testsoc_converter1_counter_converter1_next_value_LUT4_Z_D_LUT4_Z_SLICE.F1

17:09 <lkcl> Info: 0.0 17.5 Setup csr_bankarray_interface1_bank_bus_dat_r_LUT4_C_Z_LUT4_D_Z_LUT4_Z_SLICE.CE

17:10 <lkcl> 3ns logic, 5ns routing

17:11 <lkcl> which means, any core with a 64 bit bus is inherently going to have to run slower

17:40 <_florent_> lkcl: great if you managed to improved fmax, for the critical path, can you try with upstream LiteX?

17:41 <_florent_> https://github.com/enjoy-digital/litex/commit/e4f5dd987eb8d2a98c714d4e2130fe015e1df244 and https://github.com/enjoy-digital/litex/commit/35929c0f8a8f1cc098a6b6ebb569caca8df8c08d could help

17:41 <tpb> Title: interconnect/wishbone/Wishbone2CSR: add registered version and use it… · enjoy-digital/litex@e4f5dd9 · GitHub (at github.com)

18:46 <lkcl> _florent_: yes currently on commit bb7f33434ec1dc6493b4a25d58cd8aecce9694a5

18:47 <lkcl> _florent_: ahh, more recent. excellent will give it a shot

18:56 <lkcl> _florent_: yep, one of the pipelines is back to being "the" critical path :)

18:58 <_florent_> it's your turn then :)

18:58 <lkcl> haha funny man :)

18:59 <_florent_> btw have you been able to test it on hardware with the DDR3 at 55MHz?

18:59 <lkcl> not yet: i have a missing instruction (hrfid) which previously was ignored

18:59 <_florent_> ah ok

19:00 <lkcl> some changes to the POWER9 decoder accidentally "un-ignored" it...

19:00 <lkcl> and it's now being interpreted as an illegal instruction.. trap... faillll..

19:01 <lkcl> currently trying to track down why, when i know i've added support for it, it is being ignored

20:33 <lkcl> _florent_: fixed in sim (changes to FSM forgot to copy PC, MSR)

20:33 <lkcl> ecp5 should work, will find out in... 2-3 mins

20:40 <lkcl> _florent_: libresoc core back up and running in ecp5, however those bus changes miiight have broken something

20:40 <lkcl> i'm checking again with picorv32 except that's 32-bit

20:41 <lkcl> ah: yes, broken with picorv32 as well

20:41 <lkcl> 1 sec will checkout commit bb7f33434ec1dc6493b4a25d58cd8aecce9694a5

20:41 <daveshah> how is it failing? totally dead, or DDR issues?

20:42 <lkcl> DDR

20:42 <lkcl> - bus errors: 4/256

20:42 <lkcl> - addr errors: 8192/8192

20:42 <lkcl> - data errors: 524288/524288

20:42 <lkcl> where on bb7f3343, picorv32 was going "yep i'm happy with that"

20:43 <lkcl> errr... and now it isn't

20:43 <lkcl> err

20:44 <lkcl> ah 1 sec

20:44 <lkcl> i think i wrote "+6" to the ecp5dram.py

20:44 <lkcl> after we experimented yesterday

20:44 <lkcl> 1 sexc

20:44 <lkcl> sec

20:46 <lkcl> ok yep working with litedram changes reverted (picorv32)

20:46 <daveshah> cool

20:49 <lkcl> ha, all good there too: picorv32, git master litex

20:49 <lkcl> litex/soc/interconnect/wishbone.py updates

20:53 <lkcl> interesting - libresoc, bb7f33434 still fails @ 55mhz

20:55 <lkcl> i should try microwatt, next.

20:59 <lkcl> nope on libresoc git master, 64-bit 55mhz.

21:00 <lkcl> off to get coffee and watch a bit of season 5 episode 5 andromeda :)

21:52 <lkcl> holy cow microwatt takes a long time on nextpnr-ecp5

21:53 <lkcl> 45 minutes and it's only 85% complete

21:53 <daveshah> Yes, I think there is an issue with bram inference that results in a large and congested design

21:53 <daveshah> Last I looked anyway

21:54 <lkcl> ahh raptor engineering mentioned this