whitequark changed the topic of #glasgow to: glasgow debug tool · code https://github.com/GlasgowEmbedded/Glasgow · logs https://freenode.irclog.whitequark.org/glasgow
<marcan> whitequark: haven't added the discharge resistor yet?
_whitelogger has joined #glasgow
mehar has quit [Ping timeout: 246 seconds]
mehar has joined #glasgow
mehar has quit [Remote host closed the connection]
<_whitenotifier-1> [Glasgow] whitequark commented on issue #118: Improve ESD immunity - https://git.io/fjIqf
<whitequark> marcan: it stays unmodified I think
mehar has joined #glasgow
<_whitenotifier-1> [Glasgow] whitequark commented on issue #68: Expose on-board I2C for accessories - https://git.io/fjIqt
<_whitenotifier-1> [Glasgow] whitequark commented on issue #68: Expose on-board I2C for accessories - https://git.io/fjIqO
<whitequark> tnt: ok, I think I know how to reproduce it without that applet
<_whitenotifier-1> [Glasgow] marcan commented on issue #118: Improve ESD immunity - https://git.io/fjImv
puck has quit [Quit: nya]
puck has joined #glasgow
<marcan> whitequark: thoughts?
<whitequark> marcan: seems about right
<whitequark> marcan: hm
<whitequark> so I poked around the rgb grabber applet
<whitequark> I tried aligning the USB packets with video rows
<whitequark> if the pixel data I'm transferring is, 1 1 1 2 2 2 3 3 3 ... 1f 1f 1f 0 0 0, and so on
<whitequark> oh wait
futarisIRCcloud has quit [Quit: Connection closed for inactivity]
<whitequark> this is worse
<whitequark> yeahhh
<whitequark> i'm looking at a red herring
<whitequark> sigh
<whitequark> okay, i have an idea
<marcan> whitequark: do you have a self contained repro? I can try to take a look at this later maybe
<tnt> whitequark: so no repro ?
<whitequark> no self contained repro yet :S
<whitequark> marcan: tnt: I think I conclusively narrowed it down to one specific condition
<whitequark> there are three transitions out of the state in the state machine that writes a byte to FIFO
<whitequark> 1, it can go write another byte
<whitequark> 2, it can explicitly assert PKTEND (I changed the code to never rely on flags *during* writes at all) and send the packet
<whitequark> 3, it can decide to "poll other FIFOs"
<whitequark> I have reduced the state machine to the bare minimum, so both states 2 and 3 add a few cycles of latency and go check the flag again
<whitequark> (482 is the size of a row of video data)
<whitequark> marcan: tnt: so, if I remove the state 3, everything works very well
<whitequark> the transition 3*
<whitequark> but if I add this transition (which fires roughly every 1-2 pixels, i.e. 3-6 bytes) i get this noise
<whitequark> the noise appears to happen if I deassert SLWR when SLWR was asserted but PKTEND was not
<whitequark> and it manifests as, for example, 1c becoming 00, or 14 becoming 1d
<whitequark> the next byte in the stream seems to be fine
<whitequark> I've fixed all other lines
setrofim has quit [Killed (barjavel.freenode.net (Nickname regained by services))]
<whitequark> so, FIFOADR is constant, FDOE is constant
<whitequark> SLOE is constant
<whitequark> SLRD is never asserted, etc
Guest39051 has joined #glasgow
<whitequark> now here is the weirdest part
<whitequark> this does NOT appear to be a setup/hold violation BETWEEN SLWR and FD
<whitequark> because i can make sure that when SLWR is deasserted, FD does not change
<whitequark> and that doesn't change anything
<whitequark> some more mutations:
<whitequark> 18->01 00->09 08->11 16->1f 0a->13 1e->07
<whitequark> this doesn't look to me like any kind of PCB noise at all
<whitequark> it does, however, look to me like a CDC violation
<whitequark> I've also tried simultaneously setting FD=0 when deasserting SLWR
<whitequark> that did not change anything
<_whitenotifier-1> [Glasgow] whitequark reopened issue #89: Use SB_GB_IO instead of SB_IO+SB_GB - https://git.io/fjvFD
<whitequark> I went even further and completely disabled any feedback from the FX2
<tnt> mmm, weird, there should definitely not be any relations between slwr and pktend
<whitequark> tnt: I think PKTEND is a red herring, or rather, there is nothing wrong with PKTEND
<whitequark> I am currently able to observe glitches that are not at all related to asserting PKTEND
<whitequark> and before, I was able to observe glitches even when PKTEND is not asserted at all
<whitequark> marcan: tnt: SUCCESS
<whitequark> I've made a standalone repro
<_whitenotifier-1> [Glasgow] whitequark commented on issue #89: Use SB_GB_IO instead of SB_IO+SB_GB - https://git.io/fjIsr
<gruetzkopf> That looks surprisingly regular
<whitequark> yes, it is quite regular
<whitequark> there appears to be a kind of "beat frequency" phenomenon with the FIFO filling and emptying
<whitequark> tnt: marcan: I've confirmed that the corrupted byte is always the byte immediately following a restart
<tnt> oh, so it's the first byte after the fifo has emptied ?
<whitequark> looks so, yes
<tnt> Thanks for the test case, I'll try to have a detailled look tonight.
<whitequark> tnt: note this patch: https://paste.debian.net/1075607/
<whitequark> it is very dirty, but it simplifies the FSM radically
<whitequark> well, let me clean it up a bit
<whitequark> still bad, but you see the idea
<whitequark> you can try disabling ANY feedback from FX2 to FPGA, just remove any mention of flags
<whitequark> this is not metastability
<tnt> whitequark: do you see the same on both glasgow rev btw ?
<whitequark> tnt: very similar
<whitequark> let me recheck with updated gateware
<_whitenotifier-1> [Glasgow] whitequark commented on issue #89: Use SB_GB_IO instead of SB_IO+SB_GB - https://git.io/fjIGL
<_whitenotifier-1> [Glasgow] whitequark commented on issue #89: Use SB_GB_IO instead of SB_IO+SB_GB - https://git.io/fjIGZ
thaytan has joined #glasgow
<tnt> whitequark: in the cypress post, it's not really clear what you do when the red transition isn't there and readable == 0.
<whitequark> tnt: yes. i just noticed that.
<whitequark> tnt: so, this doesn't make any sense.
<whitequark> tnt: let's say I comment out the Else() in the XFER state
<whitequark> this essentially brings slwr down
<whitequark> and then back up again
<whitequark> and that's fine
<whitequark> but if I add another state, say "XXX" and transition to this state from the Else()
<whitequark> and then right back to "XFER"
<whitequark> then I see glitches.
<whitequark> I have a sneaking suspicion.
<whitequark> and the suspicion is that something fucky is going on with the SB_IO pin
<_whitenotifier-1> [Glasgow] enjoy-digital commented on issue #89: Use SB_GB_IO instead of SB_IO+SB_GB - https://git.io/fjIca
<_whitenotifier-1> [Glasgow] whitequark commented on issue #89: Use SB_GB_IO instead of SB_IO+SB_GB - https://git.io/fjIcP
<Hellsenberg> which is the simplest applet glasgow has?
<whitequark> Hellsenberg: for what purpose?
<whitequark> the UART applet is p simple
<Hellsenberg> trying to understand how applets work
<_whitenotifier-1> [Glasgow] enjoy-digital commented on issue #89: Use SB_GB_IO instead of SB_IO+SB_GB - https://git.io/fjICY
<_whitenotifier-1> [Glasgow] whitequark commented on issue #89: Use SB_GB_IO instead of SB_IO+SB_GB - https://git.io/fjIC3
<_whitenotifier-1> [Glasgow] daveshah1 commented on issue #89: Use SB_GB_IO instead of SB_IO+SB_GB - https://git.io/fjICG
<tnt> daveshah: it's all in the same clock domain here though, so does that patch matter ?
<daveshah> No
<_whitenotifier-1> [Glasgow] whitequark commented on issue #89: Use SB_GB_IO instead of SB_IO+SB_GB - https://git.io/fjIC8
<daveshah> In any case, it won't fix anything on iCE40, but it will not infer BRAM instead of creating broken logic at least pointing out the problem
<_whitenotifier-1> [Glasgow] whitequark commented on issue #89: Use SB_GB_IO instead of SB_IO+SB_GB - https://git.io/fjICB
<_whitenotifier-1> [Glasgow] daveshah1 commented on issue #89: Use SB_GB_IO instead of SB_IO+SB_GB - https://git.io/fjIC1
<tnt> whitequark: AttributeError: 'FX2Arbiter' object has no attribute 'xsync'
<tnt> (trying to run the patch you posted above)
<whitequark> tnt: remove the xsync line
<whitequark> or, add: self.fx2_arbiter.xsync = self.platform.request("sync")
<whitequark> in target.hardware
<whitequark> doesn't matter
<_whitenotifier-1> [GlasgowEmbedded/Glasgow] whitequark pushed 1 commit to master [+0/-0/±1] https://git.io/fjICF
<_whitenotifier-1> [GlasgowEmbedded/Glasgow] whitequark f746966 - cli: suggest valid revisions if --rev fails parsing.
<_whitenotifier-1> [Glasgow] whitequark commented on issue #89: Use SB_GB_IO instead of SB_IO+SB_GB - https://git.io/fjICb
<whitequark> tnt: try doing what it suggests
<whitequark> i mean i could just give you a binary
<tnt> whitequark: yeah, no hope to run that, my rust is too old I guess and it's tied to my firefox version and definitely won't be upgrading that whole thing right now.
<whitequark> tnt: do you want me to just give you a binary?
<tnt> whitequark: sure we can give that a shot
<tnt> whitequark: "thread '<unnamed>' panicked at 'cannot assert reset: Pipe', src/libcore/result.rs:997:5"
<whitequark> tnt: okay, give me a sec
<whitequark> trying to use yosys master fucked up my install...
<tnt> Oh do I need to put another .bin .. Seems it's reloading the ICE40.
<whitequark> ahh
<whitequark> sorry
<whitequark> comment the bitstream = parts in config.toml
<tnt> Oh, yeah removed the "bitstream=" from config and it works.
<tnt> And I definitely see the bug.
<_whitenotifier-1> [Glasgow] whitequark commented on issue #89: Use SB_GB_IO instead of SB_IO+SB_GB - https://git.io/fjIWG
<_whitenotifier-1> [GlasgowEmbedded/Glasgow] marcan pushed 1 commit to master [+0/-0/±3] https://git.io/fjI8L
<_whitenotifier-1> [GlasgowEmbedded/Glasgow] marcan cd921fd - revC1: add inline resistors to AUX/SYNC, add testpoints to rear
<marcan> whitequark: took care of the easy changes
<marcan> I need some time to look at that ESD stuff...
<marcan> are we okay with putting it on the rear?
<whitequark> i thiiiink so? we already have passives on the rear, nothing huge
<whitequark> so ESD stuff is just more of the same, more or less
<marcan> also, er, it's definitely going to run up against the wave soldering keepout :(
<whitequark> shiii
<marcan> like there is no way in hell we can save that
<whitequark> i guess
<marcan> so I can juust about move all the shifters two clicks to the left, drag those resistor packs up like that and slightly into the 1.27" headers (doesn't actually clash mechanically)
<marcan> and free up about yay much space between that and the IO connector
<marcan> so the question is can I fit the ESD protection in there.
<whitequark> nodnod
<tnt> whitequark: so ... migen's fifo is buggy ?
<whitequark> i don't think it is buggy
<whitequark> SyncFIFOBuffered is used in most migen designs, probably
<whitequark> it's been tested to death, including formal verification in nmigen
<whitequark> you can take another look there but i'm not sure if that's useful
<whitequark> what IS useful is looking at two things
<whitequark> 1. verilog generated by migen for BRAMs
<whitequark> 2. the processing of this verilog by yosys
<whitequark> do you think you can try to compile this bitstream with icecube? that would exonerate yosys if it also breaks
<tnt> I can give it a shot.
<tnt> I need to find where "reg [7:0] storage[0:15359];" comes from.
<tnt> icecube only works with power of 2 ...
<whitequark> rgb_input/__init__.py:156
<whitequark> 512*30
<marcan> whitequark: so one thing that is *definitely* not happening with the diodes on top is layout-correct protection of the pulls
<marcan> IOW the signal path is going to be IO -> diodes -> R -> shifters, and IO -> R -> pulls
<marcan> there is no way in hell I can tee off after the diodes
<marcan> of course it's the same net anyway, but I mean impedance-wise and such
<whitequark> marcan: is that actually important?
<marcan> it is, because we're talking very fast rise times
<whitequark> hmmm
<whitequark> what if we put them on the bottom?
<marcan> now, that said, the pulls are behind 10K or something
<marcan> whitequark: bottom breaks the wave soldering
<marcan> so.
<marcan> it's possible on top.
<marcan> https://mrcn.st/t/Screenshot_20190403_020743.png using NUP4202, which is something random that was in the kicad library (same package as our shifters)
<marcan> oh, slightly larger even
<marcan> note that that's just a pcb scribble, schematic isn't even done yet
<marcan> just proving I can fit the traces in
<whitequark> christ, that's dense
<whitequark> are you absolutely sure they wouldn't interfere with shrouds?
<marcan> not *absolutely* yet, but close
<tnt> whitequark: mmm ... bitstream built with icecube does ... nothing
<whitequark> that's... encouraging
<whitequark> can you give me the bitstream?
<marcan> whitequark: so I clear the shroud by like 0.05mm, using worst case tolerance lol.
<marcan> if I can find the same pinout diodes in the same package as our shifters (which is smaller) then I think I'm fine
<whitequark> marcan: that seems kinda bad
<marcan> I could probably also move the pullup resistor packs & connectors slightly to the right and make more space
<marcan> wait no, our shifter package won't work
<whitequark> tnt: wow, osmocom :D
<marcan> not enough clearance inside
<whitequark> marcan: do they make any ESD diodes in long thin packages?
<marcan> XBP14E5UFN is like that, 10 pins, arranged with 4 NCs in line so you can run wires through them
<marcan> but I need to draw up that package/layout, it's funky
<marcan> I guess I'll do that and see
<whitequark> tnt: do you get error Pipe ?
<tnt> whitequark: yeah, I have to unplug/replug after loading that bitstream ...
<whitequark> tnt: that's just i2c being fucked
<tnt> Oh ... that might be me screwing up. I had to manually patch the verilog to use SB_IO_OD for the i2c pins.
<whitequark> tnt: ok, I removed the i2c parts from the grabber
<whitequark> it still doesn't send anything
<whitequark> all the BULK IN requests return with -ENOENT
<daveshah> Do you have any non-zero inits?
<whitequark> nope
<whitequark> wait
<whitequark> for BRAM?
<whitequark> or in general?
<daveshah> In general
<whitequark> a ton of them
<daveshah> inits other than $readmemh are ignored, afaik
<whitequark> that seems uh, bad
<whitequark> so, yeah, there are inits in i2c code
<tnt> whitequark: I pushed another bitstream that removes the i2c IO buffers all together ...
<whitequark> tnt: try replacing ce and de registers with just 1
<whitequark> and you also need to replace reg_1 with 0
<whitequark> oh and the reset generator relies on init values too
<daveshah> well, bad but standard-compliant
<whitequark> it's not like anyone cares about verilog standards in the first place
<marcan> okay, that one is obviously going to work
<marcan> tiny package though
<marcan> whitequark: can you validate XBP14E5UFN for availability etc?
<marcan> oh wait, it doesn't even have a vcc connection
<marcan> it's just a gnd side clamp
<marcan> which is fine I think?
<whitequark> sure
<marcan> it clamps to 13V, but I'm not sure what that implies wrt actual ESD protection
<marcan> max operational is 5V (if I'm reading this right) so it seems OK
<marcan> I guess 13V through our protection resistors for a few microseconds should be survivable for whatever is downstream
<whitequark> yeah
<marcan> ok, I need sleep now, got work and stuff to do tomorrow
<whitequark> ack
<marcan> but please validate that part because it feels a bit unicorn
<marcan> if you see nothing wrong with it I'll implement this later this week
<marcan> I guess the good side of this is that if we run into availability issues we can just leave it off
<whitequark> ack
<marcan> as for the pulls, I can probably at least dump a bunch of vias in there and reduce the inductance with a different routing
<tnt> whitequark: got the icecube bitstream running. Had to rewrite the reset generator.
<tnt> whitequark: no noise ...
<whitequark> yep, sounds about right
<whitequark> ugh :S
<tnt> I updated the bitstream you can check.
<daveshah> Happy to take this one on
<daveshah> I've dealt with a few BRAM-related bugs over the last few months
<whitequark> daveshah: please do
<whitequark> this has been going on for almost a -year- at this point
<whitequark> it's the single worst bug i have ever seen
<whitequark> in my entire life
<daveshah> How much poking would be needed to get it working in sim?
<daveshah> *get it not working
<whitequark> what do you mean?
<daveshah> I guess it's a bit more than just give it a clock and it will start producing data
<whitequark> give it a clock and it will start producing data
<whitequark> literally zero inputs
<daveshah> perfect
<whitequark> i've reduced it that much, at least
<daveshah> that makes life a lot easier
<whitequark> ok, you'll need to deassert the reset that's normally asserted via i2c
<whitequark> let me hack that off for you
<whitequark> and just generally remove the i2c parts
<daveshah> thanks
<whitequark> i could try reducing it further, but it's a lot of work, and if it stops breaking, even more
<tnt> whitequark: if I use 512 * 4 as fifo size, works with yosys. just a data point.
<whitequark> it's not a very large design as it is
<whitequark> tnt: iiiiiiinteresting
<tnt> Bigger and you need to use multiple BRAM with external muxes.
<tnt> for 512x4 8bit I guess it uses 4 BRAMs in 2-bit-wide mode.
<whitequark> marcan: that part is on mouser and digikey
<_whitenotifier-1> [Glasgow] marcan commented on issue #118: Improve ESD immunity - https://git.io/fjI4x
<marcan> I can at least add vias to reduce the inductance by teeing off closer to the ESD protection
<whitequark> tnt: i cannot confirm that your updated bitstream works
<marcan> tight, but it fits.
<whitequark> are you sure you uploaded the right file?
<whitequark> oh wait
<marcan> whitequark: look sensible?
<whitequark> tnt: nop. doesn't work.
<whitequark> tnt: sha1 9243cb571c91be1304bedecd5433b16dbde3d572
<tnt> huh ... yeah, that's the right file.
<whitequark> marcan: yeah
<tnt> but it works here :/
<whitequark> tnt: hang on
<whitequark> tnt: i didn't realize you fix i2c too
<whitequark> yes, i confirm it works
<marcan> whitequark: I'm probably going to ditch all these changes and do everything from scratch fresh, because this involves slightly shifting *all* the shifters and other nonsense, so I'm not going to push any of this for now
<whitequark> what FIFO depth is that? 16x512?
<marcan> when I get the chance to do it in one go I will, later this week
<whitequark> marcan: ack
<whitequark> thank you!
<tnt> whitequark: yes, 16*512
<whitequark> tnt: have confirmed it doesn't appear at 4*512, and becomes kinda progressively worse at higher depths
<whitequark> that's exceedingly useful
<whitequark> thank you, you've already helped a lot
<daveshah> very interesting
<whitequark> i need to make an acknowledgements section or something
<whitequark> daveshah: would you prefer more or less depth?
<daveshah> doesn't really matter
<whitequark> ok, will be 30 then
<marcan> whitequark: https://www.onsemi.com/pub/Collateral/ESD8008-D.PDF might also work
<whitequark> oooh, that's a cute chip
<marcan> I guess if we're doing this I need to figure out how to shoehorn a single channel chip for SYNC too
<marcan> can probably make it happen if I sacrifice or at least downsize the Sync label
<marcan> we'll see
<tnt> daveshah: I think the register that delays the additional address bits for the mux between the different BRAM on the read side is missing the read enable signal.
<tnt> (was that understandable ?)
<daveshah> aha
<daveshah> that sounds like a problem
<tnt> That's from reading the .blif ... so ... I might be wrong :p
<whitequark> minimized as much as possible
<daveshah> whitequark: not getting much life
<daveshah> what state should fx2_flag be?
<whitequark> daveshah: 0bx1xx
<whitequark> sorry, forgot to cut that out
<tnt> damn, I can't make heads or tails of bram mapping pass :/
<whitequark> daveshah: does that work for you?
<daveshah> No, still no life :(
<daveshah> Going to look at the BRAM seperately
<whitequark> let me take another look...
<whitequark> it does work in hardware
<whitequark> wait
<whitequark> you know there is a 2048 cycle long reset right?
<daveshah> pretty sure it's run for a few 10k cycles
<whitequark> ok, let me see
<daveshah> I had to add CLOCK_ENABLE(1'b1) to the IOs btw
<daveshah> otherwise it is undefined in sim
<whitequark> ugh
<whitequark> the lattice docs specifically say to NOT put a 1'b1 there
<whitequark> daveshah: how are you running the sim?
<daveshah> Icarus
<daveshah> sounds like the Yosys primitives are broken then
<daveshah> hmm, actually wait a minute
<_whitenotifier-1> [Glasgow] electroniceel commented on issue #118: Improve ESD immunity - https://git.io/fjIBQ
<daveshah> looks like the simulator gets stuck shortly after the reset timeout expires
<whitequark> oh.
<whitequark> is your icarus recent enough?
<whitequark> it uhhhh does that
<whitequark> which is why migen has an internal simulator
<whitequark> but you shouldn't trust that simulator on block RAMs, it lies
<whitequark> this is all fixed in nmigen :S
<daveshah> :)
<daveshah> Let me just look at the BRAM mux issue for now
<daveshah> think I have a fix for this issue
<daveshah> just rebuilding Yosys, updated LLVM so clean rebuild time
<whitequark> wow, that was fast
<whitequark> patch?
<daveshah> totally untested
<daveshah> this is the issue tnt pointed out
<whitequark> daveshah: YES
<whitequark> THAT WAS IT
<daveshah> mostly tnt
* Hellsenberg applauds in excitement
<daveshah> will make a PR
<tnt> \o/
<daveshah> The vast majority of my missynthesis bugs have been in memory_bram tbh
<daveshah> I think it might need some rework
<whitequark> yeah
<_whitenotifier-1> [Glasgow] electroniceel commented on issue #68: Expose on-board I2C for accessories - https://git.io/fjIRs
<tnt> whitequark: however ... how does that explain gruetzkopf issue ?
<whitequark> tnt: it doesn't
<whitequark> that's separate, probably.
<_whitenotifier-1> [Glasgow] electroniceel commented on issue #68: Expose on-board I2C for accessories - https://git.io/fjIR4
<_whitenotifier-1> [Glasgow] whitequark commented on issue #68: Expose on-board I2C for accessories - https://git.io/fjIRw
<_whitenotifier-1> [Glasgow] electroniceel commented on issue #68: Expose on-board I2C for accessories - https://git.io/fjI0l
<_whitenotifier-1> [Glasgow] whitequark commented on issue #68: Expose on-board I2C for accessories - https://git.io/fjI0E
<_whitenotifier-1> [Glasgow] electroniceel commented on issue #68: Expose on-board I2C for accessories - https://git.io/fjI0u
<_whitenotifier-1> [Glasgow] whitequark commented on issue #68: Expose on-board I2C for accessories - https://git.io/fjI0a
<_whitenotifier-1> [Glasgow] electroniceel commented on issue #68: Expose on-board I2C for accessories - https://git.io/fjI0y
<_whitenotifier-1> [Glasgow] electroniceel opened issue #119: Print IO bank pin mappings on silkscreen - https://git.io/fjI0h
<_whitenotifier-1> [Glasgow] whitequark commented on issue #119: Print IO bank pin mappings on silkscreen - https://git.io/fjIEJ
bgamari_ has quit [Ping timeout: 246 seconds]
bgamari has joined #glasgow
futarisIRCcloud has joined #glasgow