sb0 changed the topic of #m-labs to: https://m-labs.hk :: Logs http://irclog.whitequark.org/m-labs :: Due to spam bots, only registered users can talk. See: https://freenode.net/kb/answer/registration
Maczilla-Ob has joined #m-labs
Maczilla-Ob has quit [Remote host closed the connection]
mac_marioHa has joined #m-labs
mac_marioHa has quit [Remote host closed the connection]
salasrodzR has joined #m-labs
salasrodzR has quit [Remote host closed the connection]
cr1901_modern has quit [Quit: Leaving.]
<sb0> hartytp: there is necessarily some data lost or some bogus data inserted
cr1901_modern has joined #m-labs
<sb0> rjo: do you have the latex sources for the poster? https://quartiq.de/talks/ARTIQ_Sinara.pdf
cr1901_modern has quit [Quit: Leaving.]
cr1901_modern has joined #m-labs
mattlet has joined #m-labs
mattlet has quit [Remote host closed the connection]
sb0 has quit [Quit: Leaving]
rohitksingh_work has joined #m-labs
gruetzkopf has quit [Quit: quit]
sb0 has joined #m-labs
hmmhesaysmi has joined #m-labs
hmmhesaysmi has quit [Read error: Connection reset by peer]
m4ssi has joined #m-labs
massi_ has joined #m-labs
ohsix_ has joined #m-labs
ohsix has quit [*.net *.split]
esden has quit [*.net *.split]
esden has joined #m-labs
puiterwijkrG has joined #m-labs
puiterwijkrG has quit [Ping timeout: 252 seconds]
hartytp has joined #m-labs
<hartytp> sb0: isn't that an issue? If we rearm the sync machine during setup a non-deterministic number of times (as done in the current ARTIQ master) then won't that lead to a non-constant number of samples being discarded and hence to non-determinism?
bmueller has joined #m-labs
bmueller has quit [Remote host closed the connection]
<sb0> hartytp: I meant, the dac output will glitch...
<hartytp> but, if the dac looses some samples and the jesd core isn't reset can't that lead to a time offset between the rtio clock and the dac output?
sb0 has quit [Quit: Leaving]
<hartytp> sb0: anyway, that wasn't the issue. altering my code so that we do not allow multiple armings of the sysref machine does not fix the issue
<hartytp> I'm out of ideas
hartytp has quit [Quit: Page closed]
hartytp has joined #m-labs
<hartytp> sb0: is this correct
<hartytp> artiq_flash -f routing_table file
<hartytp> or do you mean artiq_mkfs -f routing_table file image; artiq_flash image
gruetzkopf has joined #m-labs
sb0 has joined #m-labs
<sb0> hartytp: mkfs
<sb0> hartytp: I don't know what is going on with sayma again either :(
<hartytp> thanks.
<hartytp> over about 100 restarts, I see only three phases of the rf
<sb0> hartytp: if you believe this is an issue with the jesd core "losing" data, note that it is not supposed to have deterministic latency across boots
<sb0> there's an elastic buffer inside
<hartytp> one at -8.3ns and one at + 3.3 ns
<sb0> how often does each of them happen?
<hartytp> not sure what that means :( but I don't know enough about JESD to begin to debug
<hartytp> hard to tell because I don't have a better diagnostic than looking on a scope
<hartytp> not that often
<hartytp> 1 in 5, 1 in 10 at a guess
<sb0> what are your DAC and RTIO frequencies again?
<sb0> and JESD line rate
<hartytp> same as master
<hartytp> jesd line rate is 6GHz
<sb0> but you have interpolation?
<sb0> and 2.4GHz clock?
<hartytp> data rate is 600MHz
<sb0> *2GHz
<hartytp> but, yes, I have interpolation, so DAC clock is 2.4GHz
<hartytp> rtio is 150MHz
<hartytp> clocking is as follows:
<hartytp> refclk driven by synth but split three ways
<hartytp> 1. to retiming FF
<hartytp> 2. to HMC830 reference
<hartytp> 3. to feed FPGA rtio_clk via the SYSCLK1 UFL connectors
<sb0> so 3.3ns is 2 samples of 600MHz and 8.3ns is 5 samples
<hartytp> HMC830 produces 2.4GHz, no output divider so deterministic latency
<hartytp> HMC7043 passes DAC clocks straight through so deterministic as well
<sb0> those aren't multiples of the 150MHz clock of the FPGA
<sb0> there isn't anything in the FPGA that is clocked at 600MHz
<sb0> so this doesn't make much sense
<hartytp> SYSREF at 9.375 produced by FPGA, no S/H issues since I'm now clocking the FPGA from the same clock as the FF/DAC
<hartytp> transceivers clocked from gtp1 via the HMC7043 so random phase
<hartytp> sb0: the 8.33ns looks like 5 periods of 600MHz
<hartytp> or 10 DAC clock cycles
Ed0Op has joined #m-labs
<hartytp> 3.3 is 2 cycles of 600MHz or 4 cycles of DAC clock
<hartytp> I assume it's a framing issue in the JESD core, an issue with the SAWG reset, etc
<hartytp> I've checked that the SYSREF is deterministic w.r.t. the rtio-TTL
Ed0Op has quit [Remote host closed the connection]
<sb0> what happens without SAWG?
<sb0> and the pattern generator instead
<hartytp> maybe some issue with the DAC sync not doing what it's supposed to, given the weird behaviour at this clock freq. but, that will be easy to test in the next hardware rev once we have deterministic latencies for all dac clock frequencies
<hartytp> sb0: to test that, I would need an output from the pattern generator to run at 9.375MHz
<hartytp> do any of them do that, or can you send me a diff to do that (that code is dense and I never got my head around how it works)
<sb0> why would you need that?
<hartytp> true, I guess I don't, it's just nice to be able to check the stability of the RF w.r.t. the sysref
<sb0> you can use a longer sinusoid on ch1 to get 9.375, if needed
<hartytp> the requirement is that my rtio TTL should toggle at a sub-multiple of the pattern frequency. That's easiest if the pattern frequency is a power-of-two division of the rtio clock
<sb0> what about the ramp?
<hartytp> what frequency do those ramps run at?
<sb0> rtio clock divided by 16, as I see from the code
<sb0> so, same as sysref
sonne_ has joined #m-labs
sonne_ has quit [Remote host closed the connection]
<hartytp> okay, I'll rebuild and reflash that
<hartytp> sb0: can you think of any way to distinguish between bugs in the dac and gateware bugs?
<hartytp> the only other test I can think of would be to try running without interpolation
<sb0> that's an annoying proposition, but does your fancy scope support monitoring the jesd lanes?
<hartytp> so, synth at 600MHz (no hmc830)
<hartytp> dac clocked at 600Mhz by the 7043
<hartytp> ff also clocked at 600MHz which should be fine
<hartytp> fpga would then be clocked by the hmc7043 so its phase would be non-deterministic, which isn't idea
<hartytp> ideal
<sb0> for the jesd data transceiver it doesn't matter
<hartytp> hmm or maybe I just use a second synth to clock the fpga and phase lock the two synths at 10MHz
<hartytp> that's quite easy actually
<hartytp> I'll try that
<hartytp> sb0: this 8.33ns number is 20 cycles of the sample clock. I feel like that's related to the jesd frame size
rohitksingh_wor1 has joined #m-labs
<hartytp> sb0: nope
<hartytp> the ramp generator output looks v. glitchy
<sb0> hmm that's not normal
<sb0> why is it like that?
rohitksingh_work has quit [Ping timeout: 252 seconds]
<sb0> is it that board that had unexplainable jesd init errors?
<hartytp> nope
<hartytp> but consistent with some kind of jesd framing issue
<hartytp> the only thing I can think of is the interpolation
<hartytp> can you try interpolation on your board and see if the ramp gen works?
<hartytp> sb0: nope it's still there witout interpolation
<sb0> well it's not there on my board...
<sb0> is it there with sync/sysref disabled?
<hartytp> can you see any changes I've made which could cause this?
<hartytp> sb0: are you sure it's not there
<rjo> sb0: nope.
<sb0> rjo: why?
<rjo> sb0: it's not latex.
<hartytp> some channels look ok (Just a tiny glitch) and some are crap, so you'd need to check all channels...
<hartytp> sb0: can you send me a bitstream which you think works?
<sb0> hartytp: i don't have sayma set up right now
<hartytp> ok
<hartytp> well, if the ramp gen is broken without interpolation then there is no point worrying about sync
<hartytp> sounds like it would be best for you to try to get my code running on your board and see if you can reproduce this
<hartytp> okay, I'll build master now and see what it looks like
rohitksingh_wor1 has quit [Read error: Connection reset by peer]
mrq has joined #m-labs
mrq has quit [Remote host closed the connection]
xiaoguangLL has joined #m-labs
xiaoguangLL has quit [K-Lined]
<hartytp> sb0: testing switching, I have a kasli master and a kasli slave
rohitksingh has joined #m-labs
<hartytp> I've programmed in my routing table so that destination 1 is the first slave
<hartytp> but I'm getting get_rtio_destination_status(1) == 0 constantly
<hartytp> what diagnostics can I do
<hartytp> >
<hartytp> ?
<sb0> hartytp: what is your routing table? what is in the log of each board?
<sb0> hartytp: did you reboot the master after programming the routing table?
<hartytp> link light off on master sfp1
<hartytp> sb0: still see crap on the ramp gen with artiq at origin/master
FletcherWa has joined #m-labs
FletcherWa has quit [Remote host closed the connection]
<hartytp> sb0: any idea about the drtio link?
<hartytp> don't see anything on the slave uart
danmackay has joined #m-labs
danmackay has quit [Remote host closed the connection]
<sb0> hartytp: well basically your transceiver is not working. was it working with those boards and master?
<hartytp> sorry, my fault
<hartytp> I got the wrong hw rev in my build script (moved hw around for switching tests and forgot to upgrade build scripts)
<hartytp> sb0: I'm rebuilding sayma with a completely fresh conda env and origin/master to re-check the ramp gen
<sb0> hartytp: okay. could be your board. we should try on other boards, uTCA bug festival not helping (and my ATX power supply seems flaky...)
<hartytp> sb0: ack
<hartytp> well, if the ramp gen is still flaky on my board with a fresh conda env then it makes sense for me to stop for now
<hartytp> I can give you remote access if that helps
<sb0> rjo: ^
api has joined #m-labs
<hartytp> sb0: rebuilt with correct hwrev, but still nothing on the satellite uart
<sb0> hartytp: are you using the correct uart channel?
<hartytp> aah, no nevermind
<hartytp> an I2C error has appeared
<hartytp> SDA stuck low on bus 0
<sb0> hartytp: did you power cycle the board after flashing gateware for the wrong hardware rev?
<hartytp> yes
<sb0> hartytp: no gateware/firmware mismatch?
<hartytp> sorry, yes, there is
api has quit [Remote host closed the connection]
<hartytp> aah, fixing the build didn't trigger a rebuild of the firmware. okay, I'll rebuild and reflash
<sb0> fixing the build?
shabomB has joined #m-labs
shabomB has quit [Remote host closed the connection]
massi_ has quit [Ping timeout: 268 seconds]
m4ssi has quit [Ping timeout: 268 seconds]
InFerNo_ has joined #m-labs
InFerNo_ has quit [Killed (Sigyn (Spam is off topic on freenode.))]
<hartytp> so it's flashing the wrong firmware to my satellites
m4ssi has joined #m-labs
massi_ has joined #m-labs
<hartytp> sb0: okay, fixed that
<hartytp> master log stops after dest 0 is up
<hartytp> satellite stops after si5324 locked
m4ssi has quit [Quit: Leaving]
<hartytp> what should I see?
Conjectureaa has joined #m-labs
<hartytp> hmm...my switching master paniced with an alignment exception
<hartytp> timing was met when it built
Conjectureaa has quit [Ping timeout: 268 seconds]
rohitksingh has quit [Quit: Leaving.]
<sb0> hartytp: try increasing the cpu clock frequency, could be #1149
<hartytp> hmm not sure if it will meet timing if I do that
<hartytp> sb0: can you rebase switching from the current master, please?
<sb0> why?
<sb0> there isn't any fix for the bug you're seeing now, afaict
<hartytp> there are a few useful commits there, like the flexible rtio frequency which I want to use in my testing. atm I'm testing of my own branch with these merged
rohitksingh has joined #m-labs
<sb0> well it would be simpler to test the same code as me..
<sb0> why don't you just test the normal switching branch?
<hartytp> because I need to test at 125Mhz
<sb0> as you know, FPGA compilation is also quite unpredictable and shitbugs like those crashes can pop out out of nowhere
<hartytp> I need to share hardware for these tests as I can't get enough Kasli atm
<hartytp> the setup I'm sharing with requires 125MHz rtio clock
<sb0> hartytp: if you flash the original code, it'll configure the si5324 and generate a 150MHz clock from the xtal without requiring any external clock. what am I missing?
<hartytp> i.e. these boards are actively being used for experiments which require f_rtio=125MHz.
<sb0> if you reflash them with another gateware, the experiments will stop anyway, no?
<hartytp> right. i want to use the same gateware for experiments and switching testing
<hartytp> i can switch gateware when people aren't around and then run tests at the same time as experiments
<hartytp> anyway, switching needs to work at 125MHz as that's the only rtio frequency we use, so will have to get working there sooner or later...
<GitHub-m-labs> [artiq] hartytp opened issue #1166: sayma ramp gen issues https://github.com/m-labs/artiq/issues/1166
hartytp has quit [Quit: Page closed]
<GitHub-m-labs> [artiq] sbourdeauducq commented on issue #1166: @marmeladapk / @jbqubit are you able to reproduce this? https://github.com/m-labs/artiq/issues/1166#issuecomment-426695993
massi_ has quit [Remote host closed the connection]
<rjo> sb0: what's not clear?
<rjo> sb0: the comment four lines above that?
ZackW has joined #m-labs
t^RP has joined #m-labs
ZackW has quit [Remote host closed the connection]
t^RP has quit [Remote host closed the connection]
trewqgq has joined #m-labs
trewqgq has quit [Remote host closed the connection]
X-Scale has quit [Ping timeout: 244 seconds]
X-Scale has joined #m-labs
rohitksingh has quit [Remote host closed the connection]
Guest20358 has joined #m-labs
Guest20358 has quit [Ping timeout: 260 seconds]
KeitaroWh has joined #m-labs
KeitaroWh has quit [K-Lined]
[X-Scale] has joined #m-labs
X-Scale has quit [Ping timeout: 268 seconds]
[X-Scale] is now known as X-Scale
futarisIRCcloud has joined #m-labs
<mithro> Has anyone at m-labs used mor1k with Yosys at all?
<sb0> hartytp: no crash here with the original switching branch and latest migen/misoc
<sb0> rjo: what this does; is it reversing the ramp using two's complements tricks? the comment should be more explicit