15:34 <somlo> _florent_, daveshah: over the weekend I ran a quick experiment on how performance would be affected by the data width of a future direct link between Rocket's cached-ram 'mem-axi' port and LiteDRAM: https://github.com/enjoy-digital/litex/issues/299

15:39 <daveshah> somlo: very interesting

15:40 <daveshah> impressed how big the linpack difference is

15:40 <somlo> should more or less match the "integer" performance portion of NBench

15:41 <somlo> since fp is emulated in bbl (can't fit a real FPU on ecp5)

15:48 <somlo> so, in conclusion, I'm tempted to add a new Rocket variant to litex, one with a 256-bit mem_axi port, which would access main-ram in bursts of 2 x 256bit beats, instead of 8 x 64bit, and which would perfectly match the trellisboard

15:49 <daveshah> 2 beats seems fairly low for ddr3

15:50 <somlo> during early experimentation I noticed that over the 64bit mem_axi there's always 8 accesses at a time, which tells me the cache line is 512 bits

15:51 <somlo> * L1 cache line (only cache there is on Rocket, atm)

15:53 <daveshah> oh nvm, I realise that that's 2 system clock cycles

15:53 <daveshah> which is 8 memory cycles which is the burst length of the dram

15:53 <daveshah> all makes sense

15:54 <somlo> oh, was this an "axi burst" vs. "ddr3 burst" terminology overload thing ? :)

15:54 <daveshah> yeah

15:59 <somlo> So given that Rocket has 512-bit cache lines internally, my instincts tell me it'll always be more efficient to dump as much of it out per system clock cycle as LiteDRAM is capable of accepting, as opposed to chopping it into smaller slices internally, only to have some data-width conversion RTL reassemble it into 256-bit slices downstream, before passing it to LiteDRAM

16:00 <somlo> admittedly, the performance delta between my 128-bit measurements and the 256-bit ones is less spectacular than I would have expected :)

16:01 <daveshah> Guess it is possible that memory just isn't the bottleneck for those

16:01 <somlo> but then I also think having *some* L1 cache alleviates that difference, and if I rebuilt everything with less (or no) L1 cache on the rocket side, I'd see more of an impact

16:01 <daveshah> Rocket isn't awfully high IPC anyway

16:02 <daveshah> would be much bigger than an ecp5, but might be interesting to repeat the experiment with boom...

16:14 <somlo> _florent_, daveshah: LiteX code I used in the experiment is here: https://github.com/enjoy-digital/litex/pull/300

16:14 <tpb> Title: RFC: Direct link between Rocket/mem_axi <--> LiteDRAM dataport by gsomlo · Pull Request #300 · enjoy-digital/litex · GitHub (at github.com)

16:15 <somlo> (minus the additional Rocket variants themselves, which are simply more pre-built verilog variants in rocket-litex-verilog, with additional width options for axi_mem)

16:29 <_florent_> somlo: thanks interesting, i'll have a closer look at it a bit later