<GitHub57>
nu-servo/master 82da85e Robert Jordens: dds_sim: fix
<sb0>
"This may need update of Exar firmware - i.e. phase margin may be too low or some other issue may happen.
<sb0>
I had such issues with other boards but fixed it by increasing phase margin."
Gurty has joined #m-labs
Gurty has quit [Changing host]
Gurty has joined #m-labs
Gurty has quit [Excess Flood]
Gurty has joined #m-labs
<sb0>
I'm trying to come up with a good design for the new artiq build system
<sb0>
first I think the descriptions should be in static data files (e.g. JSON) so that we don't execute arbitrary code if we offer "builds as a service"
<sb0>
there should be "node type" descriptions, where a "node" is one FPGA board, and a "system" description that defines how the nodes are connected in a drtio tree
<sb0>
the node types descriptions should include all details, including things like what devices are on the SPI buses, so that a full device database can be derived from them, and because some gateware needs to be specific to certain SPI devices, e.g. for zotino DAC monitoring
<sb0>
the system should build one bitstream per node type (not per node)
<sb0>
there maybe should be some sort of caching mechanism to avoid bitstream rebuilds when something minor has changed (e.g. add one SPI device to a bus without any sort of moninj gateware support)
<sb0>
the node type descriptions reference a "board" and "I/O modules" which are part of the ARTIQ libraries
<sb0>
the "IO module" should both affect the bitstream, and create an entry with the appropriate driver in the device database
<sb0>
maybe the caching can be done by having an intermediate file format that contains the bitstream instructions created by the IO modules, and if those instructions are not changed, the bitstream is not rebuilt
<sb0>
there needs to be some standardized way to reference RTIO channels in DDB entries, so that the DDB created by a node type description can be reused with offsets added depending on the position of the node in the DRTIO tree, when building the global DDB
<rjo>
sb0: ack the static format. we might still be facing problems from DOS though.
<sb0>
there is also the problem of naming the devices in the DDB
<rjo>
denial of service someone just hammering the builds...
<sb0>
we can authenticate users
<rjo>
with the "nodes of a system graph" you seem to be hardcoding the drtio tree layout?
<sb0>
that's just used for generating the ddb
<rjo>
if we trust them we could also just do arbitrary code. but yeah. it's a technical detail.
<rjo>
do we unify the ddb and the board-build definition?
<sb0>
if you swap things around without changing the contents of the nodes, then all bitstreams are "cached"
<sb0>
yes. the ddb contains other things like controllers, but that can be handled by using python, since the ddb is a python script
<sb0>
in fact the DDB could read some JSON file
<rjo>
ok. so the board definition is part of the ddb syntax (which is fine, i.e. board definition is strictly json while ddb is pyon/python)?
<sb0>
the DDB is a Python script already, the master and artiq_run import it then access one of its global variables that is the device->driver dictionary
<rjo>
the bitstreams could be just cached by hash(standardized_json(board_definition)).
<sb0>
yes, or even without the hash
<rjo>
you have to name them uniquely somehow
<sb0>
yes, name of the node type
<sb0>
we can have 1 JSON file per node type
<sb0>
then all other things named after it
<rjo>
what if the content of the node definition changes and the node name stays? that becomes impossible to reconstruct.
<rjo>
but also a smaller detail.
<sb0>
as I said above, there are also some changes to the board definition that do not impact the bitstream and should not trigger a rebuild
<sb0>
what I was proposing is the IO modules create another JSON file that contains what should be done to the bitstream
<sb0>
then the contents of that bitstream JSON file is used for the caching
<rjo>
what about the actual board definiton format (beyond it being json)?
<sb0>
board type (kasli, sayma), parameters of the board (specific to the board type), list of IO modules, specific parameters of each IO module
<sb0>
the parameters of each IO module would include nice names for I/O pins, e.g. referencing EEM connectors or EEMs directly
<rjo>
re: the caching. i would split the format so that when the bitstream-definiton changes, a new bitstream *has* to be built. but when mapping-definition changes (what's on the SPI channels, what the latency offsets are, calibration factors, global drtio channel offsets) then it won't.
<rjo>
ack that format.
<sb0>
yes, that's what I'm proposing re. caching
<rjo>
ok.
<sb0>
or you mean have two user-edited input files?
<rjo>
i'd derive a template for the mapping-definition from the baord-definition and have the user edit that (as well).
<sb0>
the problem with those two files is, the content of the SPI channels can actually impact the bitstream, e.g. with DAC moninj
<sb0>
hmm patching machine-generated files is messy
<sb0>
if you regenerate the file you lose your changes
<rjo>
just generate it with the bitstream and ship it.
<rjo>
for the user to copy-paste it.
<sb0>
I think it's better if the software detects what bitstreams need rebuilding, from a single source of data edited by the user
<sb0>
also this way, the user knows what boards need reflashing
<rjo>
re DAC moninj. isn't the design broken? on the RTIO layer we have raw SPI commands, but on the moninj layer we have the (more abstract) DAC values? i seem to think that we half-hartedly agreed that this was needed for initial state (?) but we may want to revisit.
<sb0>
including in non-obvious cases like adding a DAC on a SPI channel that requires a bitstream rebuild
<sb0>
the DDS monitoring does that and I don't think it's broken, in the DDS case it's just that the RTIO layer can do more things than the monitoring layer can see (the latter only captures FTW)
<rjo>
ok. if the software/builder "prunes" the node definiton and drops all the mapping-definition, that's fine as well.
<rjo>
it seems likely that every possible spi device (novo, urukul, zotino, etc) will need a custom moninj layer as well. or we make moninj-for-SPI programmable... nightmare.
<sb0>
I don't see a good programmable moninj-for-SPI
<rjo>
agreed.
<sb0>
so yes, it should be a bit of gateware for each device. this is what is killing the "multi-purpose configurable RTIO PHY" idea, in favor of using the FPGA reprogrammability
<rjo>
also agreed.
<rjo>
i gave up on that.
<rjo>
so how do you want to proceed? prototype the board-definition and its extraction on the sayma/zotino system you have?
<rjo>
i could also use it (at least dry-test it) for nu-servo plus kasli right now.
<sb0>
all systems should migrate to the new build infrastructure, sayma, kasli, the nist kc705 stuff
<rjo>
ultimately. but we need to develop/test the format and the builder first, no?
<sb0>
yes. i think we should first convince joe or others that this is a good idea.