electronic_eel_ has quit [Ping timeout: 264 seconds]
PyroPeter_ has joined #glasgow
PyroPeter_ is now known as PyroPeter
PyroPeter has quit [Ping timeout: 256 seconds]
<lukego>
electronic_eel: same result with USB power i.e. no response. I'm doing an inspection now. So far I see that U34 is backwards.
<lukego>
I'm not convinced that U1 is really soldered. I think that I have quite a lot of solder on the ground pad and it might be lifting the QFN pads too high for context. I'll start by adding more solder around the sides to make good visible connections.
<lukego>
Could also be that I was misreading the silkscreen on U34... not sure, accidentally sent that chip flying across the room never to be found again while trying to handle it with reverse-action tweezers. had a replacement on hand, fortunately!
<lukego>
I did very briefly see the PWR LED come on while probing test points 3&4 above (U15 to the big cap) but can't reproduce
<lukego>
otherwise on the bottom side 5V test point is reading 5V and 3V3 test point is reading 0V
<lukego>
I'm poking around in KiCAD now though I don't really know what I'm doing especially. I can see 5V across the big cap and also 5V between pins 2&4 of U15.
<d1b2>
<edbordin> how confident are you that there's no bridging under the bga? I've seen people looking at them from the side to see light shining between the balls, but it seems to work best if you added solder so that it sits above the board
<lukego>
I'm not especially confident about that. I soldered it straight onto a fluxed PCB using hot air.
<lukego>
I see 5V between ENA&GND (1&2) on U15 too
<d1b2>
<edbordin> fair enough, that is definitely the trickiest part on there!
<lukego>
and 5V between OUT&GND (4&2) so I guess the juice isn't getting stopped at that point
<lukego>
I have some more FPGAs so it'll be easy to remove that and try again if needed but I'd like to do a bit more detective work first
<whitequark>
you can probably reball it too, it's coarse pitch
<whitequark>
even without a stencil
<d1b2>
<edbordin> you'd think it's harder to get bridges if you have bare minimum solder there, but I admit this is mostly me in my armchair
<lukego>
just placing balls manually? yeah guess so, though I have found that JLCPCB can produce plausible looking stencils, meaning to order for this part. but being lazy until I run out of fresh parts. I got a few extra of this FPGA because I wanted to meet minimum order value on Digi-key (having ordered the rest of the BOM from Mouser)
<whitequark>
lukego: yeah, it's kind of a pain (ok it's a real pain) but i've done it before
<lukego>
naively it seems like a bridge is most likely to cause a short (overcurrent) than my situation (undercurrent) right? but I guess that's very approximate
<whitequark>
the key part seems to be using a tacky flux so that the balls don't get blown away
<lukego>
I've tried manually reballing an ECP5 that way but I got bored half way. trick there seems to be that I was using el cheapo solder balls that had oxidized a lot and didn't show when they melted
<lukego>
so I ended up putting a looot of heat waiting for visual feedback that wasn't coming
<lukego>
but I also have some generic stencils of various pitches that might work here, just having to mask a rectangle with tape then remove the balls that shouldn't be there. haven't actually tried using the BGA reballing jig for rolling balls into place yet.
<lukego>
lessee where power goes to next after U15...
<lukego>
okay so it feeds that big cap which I already know gets 5V. next up..
<whitequark>
this sounds vaguely like a reset problem, having 5V but not 3V3
<lukego>
which parts are involved in that?
<lukego>
I'm n00bing around in the schematic trying to work out where 3V3 is sourced from
<whitequark>
sec
<whitequark>
U7 and U8
<lukego>
I wish that I were not too lazy to fix the problem that my magnification and task lighting is on the opposite side of the bench to my power supply... not making it easy for myself here...
<lukego>
thanks, finding and probing those chips
<lukego>
oh I added some solder to those parts just now and I see that I didn't clean up the flux very well. a bit of IPA now. it's a "no clean" gel but I think that only means it's non-conductive after being properly baked, not just poked a bit with an iron
<lukego>
U8 is reading 5V on both the pins labeled 5V and 0V everywhere else.
<lukego>
so that means it's reading 0V on 3V3EN which sounds relevant
<lukego>
3V3EN is just a resistor connecting to 5V so it's interesting that this isn't going high
<whitequark>
3V3EN is not just a resistor!
<whitequark>
it's connected to U7
<lukego>
hm
<lukego>
U7 is reading 5V on #1 and 0.42V on #3 (PGOOD) and 0V elsewhere
<lukego>
finding datasheet
<whitequark>
0.42V on PGOOD seems low
<whitequark>
should be around 1.47 or so
<whitequark>
is it your conductive flux maybe? if its resistance is as low as kOhms it might mess that up
<whitequark>
so, check R5 and R6
<lukego>
I dunno if I'm probing wrong but it seems like R5 is short and R6 is open
<lukego>
nah I dunno, I'm not getting sensible readings from any of the resistors I'm trying now, maybe I need to RTFM on the multimeter
<lukego>
I don't get it actually. trying a whole bunch of other 0402 resistors and not seeing continuity anywhere, just open circuit. the 33ohm resistor network I probe is fine though.
<whitequark>
weird
<lukego>
maybe I accidentally disabled the auto mode on the DMM and it can only detect really low resistance atm
<lukego>
the 10Kohm RN is also reading as open so yeah let's look closer at the multimeter settings...
<lukego>
okay yeah I'm a doofus, I thought I was measuring resistance but really I was checking continuity
<lukego>
Good catch! I'm reading 2kOhm on R6 which should be 10k. I'm having a hard time getting a stable reading on R5. Maybe replace both to be safe.
<lukego>
Is it complicated to check resistance in circuit? I soldered on a fresh 24k3 resistor and it's reading back more like 7k to me. Checked another from the same tape out of circuit and it reads right.
<whitequark>
yes
<whitequark>
your meter is back-powering the entire device
<lukego>
oh hey! I hooked it up again and thought there was no action - no LEDs - but it's drawing 20uA now!
<lukego>
mA
<lukego>
or, hm, did the supply just go into constant current mode somehow...
<lukego>
(voltage)
<lukego>
oh okay I guess constant voltage mode is normal and that LED just means it's turned on. So yeah I believe the device is actually drawing 20mA now.
<lukego>
The LEDs on the supply keep bouncing back and forth between constant voltage and constant current, worryingly, and between about 4V and 5V while maintaining 20mA of current. maybe something up actually.
<whitequark>
the cypress chip draws something like 100 mA while booting iirc
<whitequark>
what's your current limit set to?
<whitequark>
it should be at least 100 mA@5V
<lukego>
ok I've limited the supply to 50mA so I'll crank that a bit
<whitequark>
i'd probably put it at 250 or so
<lukego>
done. now seems to be cycling from 80mA to 130mA at about 1Hz
<whitequark>
hrm
<whitequark>
does it enumerate?
<lukego>
oh this is just bench supply. I'll try usb
<lukego>
though hm I should perhaps find an expendable machine for such a test
<whitequark>
well you already know it won't trigger overcurrent after plugging in
<whitequark>
(most USB hosts have overcurrent protection, though the absolute rock bottom ones sometimes don't or it doesn't work; when in doubt, you might want to use a hub)
<lukego>
you only live once I guess. ok, nothing in dmesg when I connect it to my laptpo
<d1b2>
<daveshah> nothing at all, not even an error?
<whitequark>
hm, so there's no pullup turned on
<lukego>
nada
<whitequark>
what's on U1 pin 42? is there 24 MHz on pin 5? do pins 8 and 9 connect to D+/D-?
<lukego>
oh hey awesome a chance to fire up the oscilloscope :-) I'll take the dog for a walk and come back to follow these threads!
<whitequark>
also might want to poke the 3V3 line with a scope
midnight has quit [Ping timeout: 272 seconds]
tf68 has quit [Quit: Ping timeout (120 seconds)]
tf68 has joined #glasgow
midnight has joined #glasgow
Getorix has joined #glasgow
Getorix__ has quit [Ping timeout: 272 seconds]
<lukego>
okay let's see where was I
<d1b2>
<Attie> wb 🙂
<lukego>
5V test pad on the bottom is cycling from about 3V2 to 4V2
<d1b2>
<Attie> not a good sign - got the scope handy?
<lukego>
and 3V3 cycling between 0.25V and 0.7V. yep firing up the scope
<tnt>
When it reaches 4v2 it's probably when the 'power good' signal rises, enabling all the regulators and power rails ... triggering whatever is causing it to fall back to 3v.
<lukego>
crap I pulled the pad that GND was connected too when I connected the scope and supply using aligator clips
<lukego>
or maybe the solder just hadn't bonded well and fell off, lessee if we can just resolder it
<tnt>
The 2 port A/B connectors are full of very solid ground pins.
<lukego>
oh thanks for the tip!
<lukego>
I really need to solder some banana plugs onto a USB cable
<lukego>
I'm a bit limited at the moment with a poorly setup bench and perhaps suboptimal scope probes, not having much luck reliably probing U1 pin 5 because it's so small
<lukego>
let's see what we can check...
<d1b2>
<Attie> start at the top
<d1b2>
<Attie> if you can, disable each of the derrived supplies
<d1b2>
<Attie> (disable U8 and U36)
<lukego>
like, desolder the chips?
<d1b2>
<Attie> 1 sec...
<d1b2>
<Attie> the Enable pin (#4) has an internal pulldown, so you could just remove R7 / R8
<d1b2>
<Attie> or if you'd prefer, put a wire from the correct side of R7 / R8 to 0v / GND
<lukego>
I'll try to tombstone those resistors so they can be restored easily later
<lukego>
or just shift them to sit on only one pad maybe
<d1b2>
<Attie> oh, hang on... i mis-read (this is referring to the active pulldown on the output)
<lukego>
aha! mine are indeed swapped compared with yours. furthermore, my U2 and U3 are both populated with the same chip (like your U2)
<d1b2>
<Attie> ok - swap U8 / U36
<d1b2>
<Attie> remember about U2 / U3, but don't worry about it right now... they are used for the firmware / bitstream, and serial number
<whitequark>
uh, i'm not sure about U3
<whitequark>
i'd have to check that it doesn't result in n address conflict
<lukego>
oh those are the flash memories of different sizes? okay, filed mental note
<d1b2>
<Attie> yeah - one for the iCE40, one for the FX2
<whitequark>
they're on the same i2c bus so they can have address conflicts
<d1b2>
<Attie> I suspect it would result in an address conflict, but we can resolve later?
<d1b2>
<Attie> (i.e: get power stable first? U3 is effecively going to be "missing", which probably has the same effect as shorting R40?)
<whitequark>
ahh yeah
<d1b2>
<Attie> ... same effect on the FX2's boot
<whitequark>
i'm not entirely sure if FX2 would boot still
<whitequark>
but it's definitely sensible to get power sorted out first
<lukego>
oki letting the board cool after swapping those happy chappies. while I'm waiting I can check for another flash chip
<lukego>
ok located but I'll resist the temptation to do it now, only have about 45min left to play with this for the moment
<d1b2>
<Attie> @lukego for future reference - it's not always true that parts will survive getting the wrong supply voltage... iirc people have reported recitfying the U8 / U36 swap works out, so you should be okay in this instance
<lukego>
ok cool. I talk the precaution of ordering 4 x BOM since this is my first electronics project and I want to get at least one - preferably two - working
<lukego>
hm this is not ideal, it looked like it was somehow drawing 9V
<d1b2>
<Attie> "drawing" 9v?
_florent_ has quit [Quit: Connection closed for inactivity]
<lukego>
er, the bench supplywas showing 9V, I'm not sure why
<d1b2>
<Attie> sounds like you were supplying 9v ... 😦
<d1b2>
<Attie> turn it down to 5v again, and retry...
<d1b2>
<Attie> if you had the current limit in place still you may be lucky
<lukego>
which must be some kind of advanced user error because it's set to 5V and 250mA
<lukego>
it was showing 151mA at the time
<d1b2>
<Attie> hmm
<lukego>
oh, no, I think that what I perceived to be a 9 was actually a very rapidly flickering alternation between 4 and 5
<d1b2>
<Attie> ha, possible... crosses fingers
<lukego>
yeah it really seems so. I have it hooked up again now, it's drawing 150mA at 5V
<lukego>
let's try probing
<d1b2>
<Attie> 150mA is certainly high... the current limit is 250mA?
<lukego>
On 5V I'm reading only0.8V and on 3V3 only 2.8V
<lukego>
yep
<d1b2>
<Attie> how are you getting those figures? (because they sound wrong - remember 3v3 is derrived from 5v, so it can't really be higher)
<d1b2>
<Attie> look with a scope, it may not be a static level
<lukego>
Sorry, 0.28V on 3V3
<lukego>
it's with DMM, I'll try scope
<d1b2>
<Attie> oh, ok
<d1b2>
<Attie> this sounds like you're running into the current limit then (once the current limit is reached, the voltage will start to drop to maintain that current setpoint)
<d1b2>
<Attie> check for shorts in your rework, or step back and disable the 3v3 supply again
<lukego>
on the scope I'm seeing what looks like a spike at 8.333kHz
<lukego>
ah
<lukego>
cleaning up with IPA and checking for shorts didn't help, although I did at some point enable also 1V2 again (pulled the bodge wire loose), so I'll turn that off again first
<lukego>
(probably happened while I swapped the chips with a big hot air nozzle)
<d1b2>
<Attie> ah, yep - disable that again... is 3v3 stable now?
<lukego>
Same deal. I'll try disabling 3V3 again too
<lukego>
Same still. okay, something fishy with my rework eh
<lukego>
oh yeah, orientation of U8 is wrong.
<lukego>
not for long...
<lukego>
okay, now it's pulling a steady 26mA with 5V and 3V3 both at expected levels (and 1V2 disabled.) Enabling 1V2...
<d1b2>
<Attie> did you see the PWR LED light?
<lukego>
oh I didn't check that side of the board
<lukego>
I'm checking the power rails from the test points on the bottom, maybe can do on the top too?
<d1b2>
<Attie> oh ok
<lukego>
OK seeing power bright now and FX2 and ICE dimmer
<d1b2>
<Attie> good stuff
<d1b2>
<Attie> turn on 1v2 again... and i suspect you'll find there is an issue / short there
<lukego>
1V2 is reading 0.55V
<d1b2>
<Attie> yip
<lukego>
(turned on now)
<d1b2>
<Attie> take some easy visual inspection on the caps under the FPGA
<d1b2>
<Attie> look for any grub
<d1b2>
<Attie> otherwise, it's likely there's an issue under the FPGA itself
<lukego>
is this now the kind or problem that is likely from a bridge under the FPGA or FX2?
<lukego>
right
<lukego>
thanks for all the help! I'm out of time now but eagerly looking forward to diving into this again, monday at the latest :)
<sorear>
has anyone invented a tool that can read out magnetic fields and show you, in physical space, exactly where 50mA is flowing?
<d1b2>
<Attie> 1v2 only supplys the FPGA
<d1b2>
<Attie> np!
<d1b2>
<Attie> @sorear i've not used it for ~50mA, but thermal imaging can do good stuff for finding shorts
<d1b2>
<Attie> @lukego - i'd prepare yourself for reworking the bga... reballing if you feel up to it, or straigth swap if not
<sorear>
ooh, that's a good idea too
<GregNGM>
I've not used one, but I've seen hall effect probes marketed for measuring current in PCB traces
<vup>
whitequark: I think the main problem left with it was, that AsyncFIFOBuffered counts the output register towards the depth of the fifo, but the {r,w}_level logic does not take this into account, so it can never reach {r,w}_level = depth
<whitequark>
hmm
<whitequark>
never?
<whitequark>
even when the fifo is never read from?
<whitequark>
wait, we're talking about *Buffered.
<vup>
yes Buffered
<vup>
iirc it can never reach depth, because {r,w}_level are just taken from the underlying non-buffered fifo, that has a depth of (depth - 1)
<whitequark>
honestly, i don't know how to deal with this
<whitequark>
it hinges on the question of "what r_level/w_level useful for"
<whitequark>
it seems that there is no way to technically implement ideal behavior
<whitequark>
which means we can either keep slightly broken _level attributes, or remove them entirely
<whitequark>
i, of course, lean towards the latter
<vup>
hmm, the current behaviour in quite unintuitive, but if it is documented properly I think it would be fine this way.
<vup>
I am less sure about counting the output register towards the fifo depth for the *Buffered FIFOs, but there is probably a good reason for it
<vup>
anuejn and me are currently using {r,w}_level, so removing them would mean we would have to bring them back for our stuff
<whitequark>
hmm
* whitequark
sighs
<whitequark>
there's another option, which is to change the semantics of depth for AsyncFIFOBuffered
<whitequark>
ah, hang on
<whitequark>
for r_level we can fix it, right?
<vup>
yes
<whitequark>
it's just the w_level which can never quite reach depth
<vup>
yeah, I don't see how one could fix w_level for the async case
<whitequark>
well
<whitequark>
we could resynchronize the state of the output buffer back to the write side
<vup>
but then if would have a couple of cycles latency, right?
<whitequark>
yes, but that is the same as for w_level itself, no?
<whitequark>
the gray counters already use FFSynchronizers
<whitequark>
if you use FFSynchronizer for the output register, the latency would be the same
<whitequark>
so it would be no more wrong than the current impl, in principle
<whitequark>
this solution is kinda meh but I think it's less meh than the others we have
<vup>
ah honestly I never looked at the implementation of {r,w}_level for AsyncFIFO in depth, I just though it would have no latency, because w_rdy has to have no latency
<whitequark>
it's pessimistic
<vup>
I see
<whitequark>
it can't have no latency because, well, CDC inherently has latency
<whitequark>
do you think you (or anuejn) could implement that? it would be very helpful going towards the 0.3 release
<whitequark>
should be straightforward I think, and you have a use case too
<vup>
well now I see, why I always failed to understand how the AsyncFIFO works, I always assumed it had no latency and going in with that premise, the logic made no sense :)
<whitequark>
heh :)
<whitequark>
there's a Cliff Cummings paper linked in the source
<vup>
yeah, I think anuejn has a implementation for r_level already done, adding w_level with this new knowledge should be doable
<whitequark>
very good
<whitequark>
wtf, this is #glasgow not #nmigen
<whitequark>
can irssi color-code channels or something
<electronic_eel>
I have one of them. it is quite sensitive to positioning (earth mag field an so on). so it really helps if you put some signature on your current flow. for example modulate the voltage a bit
<whitequark>
still it's super cool and i see how it can turn a 10 hour debugging session into 10 minute one
<electronic_eel>
if you have a short it is easy to find with it
<whitequark>
i actually have a board with a short i cannot find at all
<whitequark>
not even with flir, seems too low impedance
<electronic_eel>
yeah, the lower the impedance, the easier to find with the iprober
<electronic_eel>
but you should have something being able to modulate the voltage. I have a programmable power supply and use that for it
<d1b2>
<Attie> intersting - i was wondering about a near-field probe... presumably with power/ground planes, the short shows up quite well - and i guess with a power trace you'd be able to follow it to the problem?
<electronic_eel>
you could probably also hack something with a resistor+fet and some 555 or small mcu
<whitequark>
electronic_eel: the board has some PMBus crap on it
<whitequark>
it's a kcu105
<electronic_eel>
quite expensive
<whitequark>
got it for free because it's dead
<whitequark>
and... it sits uselessly because i don't really have anything to diagnose a fault with except for guesswork
<whitequark>
i know which rail it is
<electronic_eel>
is the short on one of the power rails reachable from the outside? or on some rail generated on the board itself
<whitequark>
i *suspect* a specific transistor which i can buy
<whitequark>
the short is on a derived rail
<whitequark>
3V3aux or something? I'll have to check
<whitequark>
but the rail is rated for over ten amps
<whitequark>
so naturally everything related to it doesn't get any hotter before protection kicks in
<electronic_eel>
what does the regulator for this rail do? does it hiccup?
<whitequark>
yeah, a few times and then goes dark
<electronic_eel>
oh, just a few times and not permanently makes it hard to debug
<whitequark>
yes
<whitequark>
if i recover this board i will gladly donate it to any open-source effort
<whitequark>
i don't have any use for it and i don't even like xcu
<electronic_eel>
hehe, not a good motivation for deeper debugging and investing in special tools...
<whitequark>
i can contract out the rework but i'd need to know *what* to rework first
<whitequark>
(the phone repair folks at the nearest electronics mall do *magic*, this should be cheap and easy for them)
<whitequark>
(i have absolutely zero desire to touch it myself with hot air, too many layers and too many obnoxious expensive components)
<electronic_eel>
if the rail is rated for 10A, there could be some seriously thick traces leaking the heat away
<whitequark>
yep
<whitequark>
if you have the schematic... i *think* the problem is probably U135
<electronic_eel>
now the iprober may be sensitive, but not sensitive enough for me to probe a board on your bench ;)
<whitequark>
i mean, do you want a kcu105?
<whitequark>
i literally don't have enough patience to deal with xilinx tools on a daily basis. if you do, be my guest
<electronic_eel>
I have no use for it. if I wanted a more beefy fpga board, it should be something with open tools...
<electronic_eel>
but thanks for the offer
<electronic_eel>
if you still want to debug it, just for the fun of it, have you tried resetting the regulator? so that it restarts it's hiccup and does not abort it after a few times
<electronic_eel>
if it keeps doing hiccup, you increase the chance to see something on the thermal cam
<electronic_eel>
also if you want to compare voltages on different points on the rail it really helps if it keeps going
<whitequark>
mhmm
<whitequark>
it is probably doable... by using a proprietary pmbus dongle and a windows vm
<whitequark>
does not exactly provide more motivation
<electronic_eel>
so the regulator is switched on via pmbus?
<electronic_eel>
or you mean reconfiguring the hiccup cycle can be done via pmbus?
<whitequark>
it's configured via pmbus and i think it has NVM
<whitequark>
so you can *probably* make it not stop after some attenpts
_florent_ has joined #glasgow
<tnt>
Not sure about what other use that kind of stuff for but for me, a configurable afull/aempty is usually enough. I just want to be sure I'll be able to write/read a burst without interruptions.
<tnt>
Damnit ... my history was scrolled up, forget that.
<whitequark>
the idea is that afull/aempty would be derived from level signals
<whitequark>
since doing it any other way would typically not be any more efficient (i think in case of asyncfifo it would never be)
<tnt>
yeah, sure that was what my idea of what levels are useful for. But that also means in most case a slight brokenness / lag is fine.
<whitequark>
ahh
<whitequark>
the problem is aempty
<whitequark>
wait, no, nevermind
samlittlewood has quit [Ping timeout: 256 seconds]
samlittlewood has joined #glasgow
samlittlewood has quit [Ping timeout: 260 seconds]
samlittlewood has joined #glasgow
bvernoux has joined #glasgow
samlittlewood has quit [Ping timeout: 265 seconds]
samlittlewood_ has joined #glasgow
Stormwind_mobile has quit [Ping timeout: 260 seconds]