<u1138>
Hi! I wanted to ask some questions about support of github.com/GlasgowEmbedded/Glasgow on Windows 10. Is this channel adequate?
<sorear>
yes
<hellsenberg>
u1138: sure, what's your inquiry?
<u1138>
Context: I've been recently trying to configure/program an icestick from a Docker container running on Docker Desktop (which runs on top of a Hyper-V VM with Alpine Linux). Since Hyper-V VMs do not allow to share USB devices from the host, I used USB/IP. Fortunately, kernel modules and userspace tools for GNU/Linux are open source; hence, I could b
<u1138>
uild/use them. Unfortunately, I found no open source USB/IP server for Windows which is straightforward to install. The workaround for now is to use virtualhere; neither the server nor the client are open source, although it uses Linux kernel modules. See https://github.com/ghdl/docker/tree/master/usbip
<u1138>
Question: how does Glasgow handle communication with embedded devices on Windows? Are libusbK/Zadig used?
pie_ has quit [Ping timeout: 250 seconds]
<electronic_eel>
u1138: Glasgow uses libusb1 to communicate
<u1138>
electronic_eel, then, I think it is the same context I explained. I can install Python3 + libusb either on the container or on the host (be it MINGW/MSYS2 or Cygwin). Still, a driver needs to be installed. Which driver(s) does Glasgow use?
<electronic_eel>
hmm, sorry, I don't know enough about libusb on windows to answer this
<u1138>
Anyway, thanks a lot for your attention.
<ZirconiumX>
u1138: You definitely want libusbk
<ZirconiumX>
USB on Windows is a mess
<ZirconiumX>
"Beware: it is impossible to simultaneously understand and appreciate USB."
pie_ has joined #glasgow
<electronic_eel>
ZirconiumX: but that knowledge isn't bound to Windows alone
<ZirconiumX>
Of course
<hellsenberg>
the only thing I like about USB is that the older standards work over just 4 wires, including power
<electronic_eel>
but the usage of 4 wires is only sane for lowspeed and fullspeed (12mbit)
<electronic_eel>
once highspeed (480mbit) comes into play it becomes insane
<u1138>
ZirconiumX, thanks for stepping by! I understand that using libusbk and Zadig is the 'default' and supported procedure. However, I've been watching users having problems with FTDI/ice40 based FPGAs for almost four years now; and I feel pain when I see their frustration because the user experience with libusbK/Zadig is awful (it is not their fault,
<u1138>
tho; they do their best given how a mess Windows is). When I tried virtualhere, I was surprised at how easily it found the icestick on the host and how it is capable of 'making it disappear' from the device manager, which means that virtualhere changes/replaces/removes the driver. Moreover, in the virtualhere license the following note is found:
<u1138>
ify this driver simply Stop the VirtualHere Server, install updated version in windows and restart the VirtualHere Server"
<u1138>
A bulb went on: what if it was possible to provide some 'glue' between VBoxUSB and libusb so that all the free/open source project that now rely on libusbK/Zadig can have a better UX? Having a documented procedure to do so would allow to use https://github.com/jwise/pyusbip straightaway (which connects this with the context I explained above).
<u1138>
So, since I know that most of the users involved in Glasgow are technically brilliant and have a background on 'hacking' embedded devices, here I am. Does any of this make any sense to you?
<gruetzkopf>
for windows 10 at least you can add a magic usb descriptor which adds instand winUSB support
<gruetzkopf>
quite useful
<ZirconiumX>
It does make sense, but I'm not the one for the job
<u1138>
gruetzkopf, would that allow to use the icestick with libusb-1.0 from Cygwin/MSYS2 (let's ignore containers) without having to manually set the driver each time the icestick is connected to a new port?
<u1138>
ZirconiumX, me neither ATM. But I'd like to gather information about it, in case I become the one for the job in the future. Do you know of any open source project which is based on or uses VBoxUSB.sys? All the references I find are related to the whole VirtualBox product.
<ZirconiumX>
Unfortunately I only know of projects which use libusb or libusbk
u1138 has quit [Remote host closed the connection]
krbtgt|jp has joined #glasgow
krbtgt has quit [Quit: Decommisioning]
pie_ has quit [Ping timeout: 250 seconds]
pie_ has joined #glasgow
<kc8apf>
Pretty sure Zadig just makes overly specific driver matching rules. VirtualBox uses a much more relaxed filter-based scheme
pie_ has quit [Excess Flood]
pie_ has joined #glasgow
u1138 has joined #glasgow
pie_ has quit [Ping timeout: 250 seconds]
pie_ has joined #glasgow
<whitequark>
u1138: ok, so
<whitequark>
Glasgow uses libusb1's WinUSB support
<whitequark>
this requires the device to export a special descriptor
<whitequark>
moreover, this special descriptor *has* to be present the *first* time you plug in the deviec
<whitequark>
WinUSB has more restrictions than that as well... it's also slow... but this results in essentially zero configuration
<u1138>
whitequark, does that mean that Glasgow works with the hardware device you design because you ensure that the cypress chip identifies itself with some special descriptor?
<whitequark>
u1138: essentially yes
<whitequark>
I think it's possible to do this for FTDI as well but I'm not sure how offhand
<whitequark>
they have a EEPROM and I think there's some software that can inject custom descriptors into it
<u1138>
that's the next question i was going to ask, and your reply is very interesting, indeed.
<u1138>
i'll try to follow that trail. IIRC, marcan found the issue with the FTDI driver update a few years ago; he might know off the top of his head.
<electronic_eel>
hmm, can you really add custom device descriptors via EEPROM in the ftdi chips?
<electronic_eel>
I don't have detailed knowledge of the newer ones, but I played around a lot with FT232R and FT2232D and they all couldn't do this
<electronic_eel>
I think you can just set the default modes, pull strength, CBUS usage and so on via EEPROM, only stuff that FTDI designed to be in there
<electronic_eel>
you can also use the unused space on the eeprom yourself, but you can only read and write with the methods they provide. It does not influence the usb descriptors
<whitequark>
electronic_eel: ah I didn't realize
<whitequark>
u1138: they you're probably out of luck yeah.
<u1138>
overall, I think the better approach is not to use FTDI. however, there is no <50€ board available with an ice40 and without FTDI I am aware of.
<daveshah>
u1138: there is also the tinyfpga stuff (but imo more cursèd than ftdi)
<u1138>
daveshah, I know. I think it is a very brave approach, but keeping good support in that bootloader must be nightmare. I hope they can make it stable at some point.
<whitequark>
u1138: somewhere around $150-$200
<whitequark>
Glasgow is certainly not an extreme low-cost board
<whitequark>
and was never intended to be that
<whitequark>
(well, not since revA)
<u1138>
@white
<u1138>
whitequark, I am aware of the difficulties to make small badges of hardware not very costly. I expected a minimum of 100€, so it is not suprising to me. plus, it has some specific features that are not for lower cost uses.
<whitequark>
u1138: apart from that, it was not designed to be low-cost in the first place
<whitequark>
there's probably at least $5 in capacitors alone if not more
<whitequark>
it has a ton of fine pitch SMT components and so on... the BOM is rather massive
<whitequark>
the FX2 is not cheap
<u1138>
yes, with "lower cost uses" I meant "uses that do not rely on the quality of the components".
<whitequark>
right. it's meant to be the tool you can rely on, not the tool you throw out when it breaks
<u1138>
I'm sorry if my wording was not clear. I brought the question because the elements are similar to the blackice, but the great difference relies precisely on the number and type of connectors. I am aware that connectors are very expensive.
pie_ has quit [Ping timeout: 250 seconds]
<whitequark>
a lot of the cost comes purely from the huge number of BOM lines
<whitequark>
pnp isn't free
<u1138>
may i ask why did you decide to put the cypress in the board, instead of using a header to interact with the FPGA from any Arduino/ARM board?
<electronic_eel>
u1138: it's an integrated solution, the cypress fx2 is needed not only to transfer data to the fpga, but also control the internal i2c bus with voltage regulators and so on
<whitequark>
^
<whitequark>
it also makes it self-contained and unbrickable
<whitequark>
as long as you have USB, you *cannot* kill this board other than electrically
<electronic_eel>
supporting configuring the fpga from a lot of different arduino/arm boards would also be a support nightmare
<whitequark>
yeah, it's a completely different school of design
<whitequark>
i'm essentially making a product, not a module
<whitequark>
glasgow uses vertical integration in hardware, firmware, gateware and software
<electronic_eel>
whitequark: I looked at controlling stuff on the internal i2c, where the stuff on the test jig board will be connected
<electronic_eel>
currently there is a relatively high level interface, like REQ_PULL or REQ_IO_VOLT
<whitequark>
yes
<whitequark>
there should be something like REQ_DEBUG_I2C
<electronic_eel>
do we want to access the test jig board through a similar high level interface?
<whitequark>
no
<whitequark>
waste of firmware space
<whitequark>
the high level interface exists so you could e.g. write a linux kernel driver
<electronic_eel>
I mean future revs will also have the addon boards connected to the internal i2c
<whitequark>
or a rust driver
<whitequark>
(which i did)
<whitequark>
addon boards will be more complicated due to i2c switches
<whitequark>
there will be a special request that switches to (and protects the internal i2c bus from) addon i2c bus
<whitequark>
so it's separate
<electronic_eel>
let's put aside i2c switches, just the basic interface
<electronic_eel>
do we want the same interface that is also used by the i2c applets?
<electronic_eel>
and parse it in firmware on the fx2?
<whitequark>
absolutely not
<whitequark>
it should be a nice idiomatic USB control request
<whitequark>
I *think* `index` should contain the address, and `value` should contain the write/read split
<whitequark>
or something like that
<whitequark>
to handle "register read" type transactions
<whitequark>
alternatively it might be better to send a linked list of requests essentially, a la linux
<electronic_eel>
hmm, you said you wanted to make the applets portable to also support your nrf24 radio interface
<electronic_eel>
so you'd need some code to decode i2c control messages on the nrf24 module
<electronic_eel>
wouldn't it make sense to use a similar interface for the fx2?
<electronic_eel>
so you just have one applet, but it can be used on different i2c buses?
<whitequark>
electronic_eel: but that's not how applets are structured!
<whitequark>
the important part is the *interface* of the I2C applet
<whitequark>
so you have write and read commands basically
<whitequark>
[read|write](addr, data, stop=False)
<whitequark>
the specific bytes that translates into in the FIFO are a pure implementation detail and subject to change at any time
<electronic_eel>
yes, I understand
<whitequark>
so for example it wouldn't make sense to have a start command, stop command, etc for the FX2
mubes has joined #glasgow
<whitequark>
because the FX2 *cannot* leave the i2c bus in the middle of a transaction between 2 USB requests
<electronic_eel>
you'd have to send a somewhat self contained string of commands to the fx2
<whitequark>
yeah
<electronic_eel>
do you see a way to implement this command grouping for the interface of the i2c applet?
<electronic_eel>
or do you think it is too much trouble to reuse the applets and we should reimplement the stuff for accesses to the internal i2c bus?
<mubes>
whitequark Sorry, new over here, pointed in this direction by @esden. Been working on some h/w for parallel trace capture from CORTEX-M CPUs as an input for Orbuculum. I've done my own with ft600 and ECP5 but I understand we share some love for FTDI. The ICE40 isn't quick enough for this job in the general case. Are you folks working on ECP5?
<whitequark>
electronic_eel: I think the USB interface should use an idiomatic way to access I2C via USB
<whitequark>
and that one, unlike the I2C applet, will be actually stable
<u1138>
whitequark, electronic_eel, thanks for clarifying! Is there any documentation available about how is the interaction with the FPGA managed. I'm interested on having a look to how you used the cold boot feature, in case you did.
<whitequark>
mubes: so, the roadmap for Glasgow is: revC production run, revD design, revE design
<whitequark>
revD is essentially just revC but wider.
<whitequark>
revE will be based on ECP5 and feature PCIe, on-board memory, GigE/USB3 interfaces and TCP implemented in gateware
<electronic_eel>
mubes: current revC is using ice40
<mubes>
I just begged esden for a couple of revC's when he gets his finger out to build some :-)
<whitequark>
mubes: I have fairly strong opinions on the basic design of revE, so we should really discuss those to see if we can collaborate
<mubes>
electronic_eel yeah, I just took a look at your CDs....there's a lot of work been done! Silly for me to be repeating it.
* esden
hides behind a rock...
<whitequark>
specifically, revE will use an FX3 ($$$, sadly) as a "board management controller"
<whitequark>
most of the communication will be done by the ECP5 either directly via Ethernet or via USB-Ethernet via FX3 (3 Gbps vs 1 Gbps)
<whitequark>
however, when it's time to reconfigure, the host sends a magic knock packet (the PHY I will use can detect a long sequence) and takes over the entire board
<whitequark>
notably, this works even if the FPGA is totally b0rked
<whitequark>
then it uploads a known good GigE passthrough bitstream and takes over with a software TCP stack, to accept the configuration
<mubes>
Frankly, if you're not using ftdi then it's a big plus. I only have a chunk of gates that need to mangle the TRACE bits (4 bits, DDR) into something sane and throw away the rubbish. It's all tested and working on ice40 but tops out at about 70MHz clock....needs to be at least 102MHz (DDR) to cope with CPUs like LPC43xx...and ideally faster.
<whitequark>
hmmm
<mubes>
...but it would need to be _streamed_ in the general case, because we need extended sequences of it (e.g. for code coverage).
<whitequark>
mubes: so. revE doesn't really exist right now other than some vague plans.
<whitequark>
I do know which connector standard I will use (a derivative of SYZYGY, identical electrically but not logically)
<whitequark>
I do know it will be the largest ECP5-5G, so not a cheap board
<ZirconiumX>
Presumably future-proofing to at least some extent
<electronic_eel>
and before revE will be released, revD will be developed
<whitequark>
the diffpairs will be broken out as one of those SFF-* connectors, not sure exactly which but the idea is that you could use it for PCIe, SATA, other things
<mubes>
Yeah, but it's probably closer than the TRACE h/w, and since if it ever got built it would probably be done by esden then it would make more sense to focus on the software stack and onward develop Orbuculum while waiting for it (and hopefully contributing).
<whitequark>
the FX3 board support package will need to be written
<whitequark>
the gateware TCP stack is no joke either
<whitequark>
what timespan are you looking at?
<mubes>
The ECP5's are pretty pin-interchangable if you select carefully, but they're spendy. Cheapest 85K version is about 33 GBP, which will be about $33 if we keep heading in the current direction :-(
<mubes>
No deffo timescale. Was looking at UDP-only on local subnet for a network connection, 'cos that get things off the ground and can be upgraded later.
<whitequark>
of course that will be the start.
<whitequark>
I don't think it is remotely realistic to expect revE prototypes before ~1y from now
<whitequark>
(working prototypes, anyway)
<mubes>
:-)
<whitequark>
1-1.5y seems somewhere in the right range. so if that's fine with you, welcome.
<whitequark>
I believe in making things work well over making things quickly, although, ideally both :p
<mubes>
If I was looking for fast, I'd be paying someone ....
<whitequark>
ha!
<mubes>
So...how about I carry on banging on the stuff I'm playing with right now, with a view to folding it into yours if it makes sense? Means I don't have to stress about production, only getting rid of the nasties, and we'll maintain contact. What timezones are you guys generally in?
<TD-Linux>
u1138, for flashing an ice40, I think this has a lot of promise, it's like $2 BOM cost to put on a board https://github.com/adamgreig/ffp
<TD-Linux>
no off the shelf boards using it yet tho
<whitequark>
yeah ffp is neat
<esden>
mubes: I think we could build some of the software stack for orbuculum using the existing glasgow hardware already... it won't be as fast as you want it, but the framework would be at least partly there and effort would at least partly carry over to future hardware versions.
<hellsenberg>
huh, pretty neat
<whitequark>
what does orbuculum do again?
<electronic_eel>
hmm, couldn't you use the 2$ stlink-clones instead of the ffp hardware?
<mubes>
Parallel trace from a ARM, so you can do things like this with _no_ instrumentation in your code and _no_ slowdown;
<whitequark>
is there a reason this can't/shouldn't be a glasgow applet btw?
<whitequark>
Glasgow doesn't yet have a great story for integrating with 3rd party tools, that's true... hm
<hellsenberg>
that paste got split in five lines in this 1366x768 display, and I have a pretty tiny font :P
<u1138>
TD-Linux, something like ffp is precisely what I meant. However, I concerned with the other part, the FPGA board. There is no off the shelf board which provides the necessary pins to connect the ffp, is it?
<whitequark>
one thing that would be very easy is to build your gateware as a (possibly out-of-tree) applet and use your own software
<whitequark>
the USB interface is stable and well documented
<TD-Linux>
u1138, most ice40 boards provide the pins, however in a different connector
<esden>
and it has a few backends for different hardware already... it might be a good idea to have glasgow be another backend... not sure
<mubes>
yes, that would make sense
<whitequark>
one problem i haven't yet solved is, what's the appropriate layer for other software to interact with glasgow at?
<mubes>
I just need some hardware esden 8)
<whitequark>
should that software open the USB device itself? should it talk to glasgow over a socket?
<whitequark>
you could certainly provide prebuilt bitstreams for specific glasgow revisions, at the cost of losing pin agility
<whitequark>
(and you'd need to pull in the FX2 crossbar)
<esden>
all of those solutions have their dis-/advantages
<whitequark>
indeed
<mubes>
orbuculum has a fairly significant amount of data to scrunch in realtime, so its written in C, but it could certainly be librar-fied.
<whitequark>
mubes: in general I prefer exporting a socket from the Glasgow Python stack
<whitequark>
with the expectation that some day, one could open a socket to Glasgow directly
<whitequark>
(on revE)
<esden>
from the user perspective I just want to open a software tool and plug in glasgow and use it. I don't want to have several terminals open with processes like I do with openocd... that is not fun :)
<whitequark>
that is indeed true
<whitequark>
so that means that you probably don't want to use the Glasgow software stack
<esden>
but having server client architecture has certain very specific advantages. But those are probably should not be a default, they should be an option if needed.
<mubes>
At the moment O can connect via things like JLink via a socket, so that's not a big stretch, provided the stack were fast enough.
<whitequark>
mubes: the stack is pretty fast... not fast enough to saturate USB on a raspberry pi 3 though
<whitequark>
(i forgot to measure if it was due to USB or CPU though)
<whitequark>
on a modern laptop (which is what I use) 40+ MB/s with Python is entirely doable
<whitequark>
even as it manages the USB transfers itself
pie_ has joined #glasgow
<mubes>
Would need about 30 MB/sec of end-to-end data, but with a chunk of processing on top.
<mubes>
so sounds like it would fit.
<u1138>
TD-Linux, I'm not sure about that. Neither the icestick, alhambra or alhambra II allow it. I think that blackice's don't either: https://github.com/mystorm-org/BlackIce-II/wiki/The-USB-Ports-of-the-BlackIce-II-Board. In general, all of them expect the only master to be either the FTDI or the STM32, and connecting a different master is not supported
<u1138>
. Please, correct me if I'm wrong.
<whitequark>
mubes: I like to do the costly processing in gateware, if possible
<mubes>
Do you have a chunk of RAM available alongside the FPGA?
<daveshah>
u1138: on something like the blackice you can just hold the STM32 in reset to free the ice40 pins
<whitequark>
mubes: not on revC
<whitequark>
we use ~every single pin on the FPGA
<whitequark>
and there's no board space anyway
<whitequark>
it's probably one of the densest boards I've ever seen, so much crammed into it
<whitequark>
mubes: also
<whitequark>
the only RAM that would fit is an SPI PSRAM, but refresh will probably screw up your timings anyway
<u1138>
daveshah, thanks! I was missing that point!
<daveshah>
I think there's something like this for FTDIs too
<mubes>
Hmm...then we can't do it on the FPGA, at least not in the general case. We need to account for each instruction as it executes....would need a hunk of RAM to do on the FPGA I'm afraid.
<whitequark>
ah I see
<whitequark>
it'd be quite hard to do this on CPython
<electronic_eel>
mubes: so you hold the firmware of your dut in ram and lookup what it is doing?
<mubes>
Yes, the addresses come in raw and then are reverse mapped into lines and functions in the PC side.
<mubes>
Depends on what instrumentation your cortex has. In general though they only instrument program....but you can 'live watch' data to see when it's changing, for example (DMA is a different bus master, so there are potentially cache issues if you're on a M7)
pie_ has quit [Ping timeout: 250 seconds]
u1138 has quit [Ping timeout: 260 seconds]
u1138 has joined #glasgow
uovo has joined #glasgow
pie_ has joined #glasgow
u1138 has quit [Remote host closed the connection]