<rqou>
i was checking if i could get access to ieee standards without using the university's library proxy that i shouldn't have access to anymore but still do
<rqou>
and the answer is no
<rqou>
because a) my membership is expired and b) even if it wasn't, access to documents needs a separate subscription
amclain has quit [Quit: Leaving]
pointfree has quit [Remote host closed the connection]
pointfree has joined ##openfpga
SpaceCoaster has joined ##openfpga
landley_ has quit [Ping timeout: 250 seconds]
Bike has joined ##openfpga
C47 has joined ##openfpga
C47 has quit [Ping timeout: 240 seconds]
C47 has joined ##openfpga
<whitequark>
hmm
<whitequark>
link?
<cr1901_modern>
I haven't been an IEEE member for years. I have no incentive to be.
<whitequark>
who needs ieee membership if you have libgen membership
<cr1901_modern>
That too (or sci-hub) :P
<rqou>
i mean, i have multiple not-super-legit ways to still get access to documents
<rqou>
i have ssh access to a number of servers that are on campus
<rqou>
so I can just proxy through them
<whitequark>
yeah but libgen is just so... convenient
<rqou>
heh, i just downloaded ieee 1076 (VHDL LRM) from libgen and the watermark is at least removed
<rqou>
or more precisely replaced with a chinese website :P
DocScrutinizer05 has quit [Disconnected by services]
DocScrutinizer05 has joined ##openfpga
digshadow has quit [Ping timeout: 264 seconds]
digshadow has joined ##openfpga
<rqou>
azonenberg: I wrote up some opinions about splash that you are free to ignore: https://gist.github.com/rqou/42978048b5a56911d1d6bf6c7eeb726c . Unfortunately, it's mostly pointing out problems without offering any useful solutions to them. It's not intended to be criticism.
<cr1901_modern>
I wish scons would get their shit together and support Python 3...
<rqou>
eh, whatever
<rqou>
I don't mind having multiple copies of Python installed
<whitequark>
>Can a build target generate new targets and/or otherwise mutate the build DAG? Can build scripts contain computation? Is there a safe way this can even be done that doesn't force serialization, cause nondeterminism/race conditions, or make the build script accidentally Turing-complete?
<whitequark>
don't do it.
<whitequark>
nothing good will come out of it
<whitequark>
ocamlbuild did it and see what happened
<rqou>
hrm, not familiar with ocamlbuild
<rqou>
what did they do?
<whitequark>
dependency discovery
<whitequark>
you give it a target and it applies various rules until it can do it no longer
<whitequark>
but if somehting's missing, you get super obscure errors
<rqou>
seems to be a common problem among languages with powerful type systems
<rqou>
but yes, I don't want that in my build system
<cr1901_modern>
Is there a way to make the errors not-super-obscure, or was it a side effect of "ocamlbuild doing something it was not meant to do"?
<cr1901_modern>
I mean, even most C projects would prob be okay with dependency discovery with hints
doomlord has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
digshadow has quit [Read error: Connection reset by peer]
doomlord has joined ##openfpga
<azonenberg>
rqou: Just got home
<azonenberg>
Reading...
<azonenberg>
So first off, it is intended to be a build system but none of the design constraints inherently prevent it from being used for other purposes at this point
<azonenberg>
The only assumption I make right now is that individual operations are idempotent
<azonenberg>
IOW is it safe to run the same build operation multiple times (if the output was flushed from the cache due to capacity limits)
<azonenberg>
and it is also safe to reuse a cached value and not re-run it
<azonenberg>
Multiple inputs are a key feature so obviously they'll be supported
<azonenberg>
Multiple outputs will also be supported although i havent figured out exactly how to do that
<azonenberg>
for FPGA stuff it's a critical feature (for example, netlist and placement report file)
<azonenberg>
The two required outputs for every step are console output (mixed stdout/stderr) and the output data file
<azonenberg>
each unique DAG node is identified by a sha256 hash and the cache contains a directory for that hash
<azonenberg>
Re integration with tools, the "splash" client can be used just like "make" although probably with slightly different argument syntax
<azonenberg>
So it should be easy to integrate with IDEs
<azonenberg>
i havent implemented the client yet so details are TBD
<azonenberg>
When the control server pushes a build job to a build server, the Toolchain object is responsible for actually executing that operation
<azonenberg>
This can involve invoking arbitrary external compilers/executables as needed for that toolchain
<azonenberg>
I haven't reached that point in the implementation so details are TBD
<azonenberg>
i'm still in the "generate the DAG" phase
<azonenberg>
The graph is immutable once created, any time a source file or build script changes the relevant subgraph is regenerated and new nodes with different hashes are created
<azonenberg>
I use a mark-and-sweep garbage collector to clean up old nodes as needed, right now i GC all the time but in the future i will probably be less aggressive to allow caching if you revert a file
<azonenberg>
What i may do is make the GC delete only a subset of unreferenced nodes based on some LRU heuristic etc
<azonenberg>
(this is in-memory dependency graph nodes, there's a separate cache for command outputs and that is much less aggressive at flushing since a compile is way more work than a dependency scan)
<azonenberg>
The build language is intentionally declarative
<azonenberg>
There will be support for automatic code generation
<azonenberg>
Most likely the way this will be handled is by having a special node in the "sources" section of a script that specifies a code generator plugin to invoke
<azonenberg>
if the output of that node exists, it's used as a source
<azonenberg>
if it does not, the generator is invoked first
<azonenberg>
There is no "native" vs "cross" compilation
<azonenberg>
A given directory has a set of default architectures that a particular toolchain compiles for, you can override or supplement this list on a per target basis
<azonenberg>
If you have a compiler that only runs on, say, ARM
<azonenberg>
just spin up a splashbuild daemon on an ARM box
<azonenberg>
whether emulated or native is irrelevant
<azonenberg>
that daemon will then advertise it has that compiler to the control server, which will schedule jobs appropriately
<azonenberg>
As far as the controller is concerned, it just has to tell one of the nodes with compiler X to build the following sources with the following flags
<azonenberg>
Ditto for containerization etc
<azonenberg>
Skipping most of the "issues from other build systems" for a moment and going to handling of bugs
<azonenberg>
Individual tasks will specify estimated memory usage and the scheduler will do its best to not OOM the machine
<azonenberg>
The toolchain driver will pick sane defaults for typical configurations and you can override these if you have a particularly complex executable etc
<azonenberg>
If i can detect that the process was killed due to OOM (not sure how easy this is) I'll re-run the job on a node with more memory available, perhaps by oversubscribing less jobs
<azonenberg>
I use TCP so loss of a few packets is no big deal
<azonenberg>
If the entire connection drops, right now the client disconnects and never comes back
<azonenberg>
I will probably add a reconnect loop of some sort
<azonenberg>
Re handling of attached hardware, most likely what will happen is (I haven't implemented the handling for hardware-in-loop for this splash version yet) the test job will hang waiting to be scheduled
<azonenberg>
because there are no available nodes pulling that work unit
<azonenberg>
ditto for if all build servers with a particular compile drop offline
<azonenberg>
If this happens i'll probably have the client print a message
<azonenberg>
saying that there is nothing that can run that job right now, and give you the option of canceling the build or blocking until the host comes back
C47 has quit [Quit: Leaving]
<azonenberg>
Re absolute paths, that was a known issue in splash v1 (one of the reasons i couldn't publish binaries for libjtaghal etc before)
<azonenberg>
and one i plan to solve in this version
<rqou>
hmm it seems to me that what you've got in mind is great for monolithic projects where everything is under similar management (e.g. Google and blaze)
<azonenberg>
That is my primary use case, yes
<rqou>
but not as great for a project like OpenWRT
<azonenberg>
Antikernel
<rqou>
OpenWRT-style embedded linux seems to be the cases where you hit the most spectacular build failure
<rqou>
*failures
<azonenberg>
Where you have one giant project with a lot of subprojects (jtag library, unit tests, code generation tools, softcore CPUs, firmware for FPGAs and MCUs on multiple boards, etc)
<azonenberg>
If it ends up being useful for other stuff, great
<azonenberg>
But i'm making it to solve my problem
<rqou>
I was thinking I might want it to handle OpenWRT-style builds
<azonenberg>
If it solves other problems i'm not going to complain, and i'm willing to put reasonable effort into making it generic where it doesn't break with my core requirements
<azonenberg>
But i'm not trying to replace make for everything
<rqou>
i see
<rqou>
i'm not sure what you really can do about the embedded linux distro use case
<azonenberg>
Yeah
<azonenberg>
That is not something i am trying to solve right now
<azonenberg>
My problem is that i have well over a hundred targets for over a dozen architectures
<azonenberg>
if i build them all serially on one node it takes hours
<azonenberg>
maybe most of a day by now
<rqou>
many programs that are broken in embedded linux are broken for dumb build system reasons rather than nonportable code
<azonenberg>
now imagine throwing a couple of developers at that project
<azonenberg>
Which is where i am trying to take it
<azonenberg>
Down the road I plan to make splash run over TLS with client cert authentication so you can have collaborating developers from multiple locations sharing one cache and build cluster
<azonenberg>
I'll probably still VPN it to be extra safe
<azonenberg>
and WAN stuff is not necessarily the primary goal (compared to multiple devs at one facility sharing a cluster)
<rqou>
my experience with TLS client certs is that you will be very frustrated at people that never realized they existed and "did it wrong"
<azonenberg>
I'll support pre-shared key auth too for small deployments
<azonenberg>
But i've deployed client certs for OpenVPN in the past with decent results
<rqou>
openvpn works great
<azonenberg>
I have my own private CA
<azonenberg>
i created one intermediate CA for HTTPS and one for VPN
<rqou>
client certs in the browser-related stack is lots of fun
<azonenberg>
then signed node certs from there on
<azonenberg>
I dont currently use client certs over HTTPS
<azonenberg>
only for VPN
<rqou>
client certs in things that got TLS duct-taped on (SMTP, IMAP) work awful
<azonenberg>
my dream is that one day we can murder the evil that is passwords
<azonenberg>
and use 100% client certs for everything
<azonenberg>
each user account will map to an array of public keys
<azonenberg>
you can have as many as you want for different devices, VMs, whatever
<rqou>
yeah, that's how my personal server stuff is set up
<azonenberg>
one or more of those keys would be marked as admin keys for the account, allowed to add/remove other keys
<azonenberg>
(not necessarily all, for example you might not want your phone to be able to add new keys to your online banking)
<azonenberg>
I've already eliminated passwords from my LAN (need to make one final pass soon next time I do an internal pentest and make sure I got everything)
<azonenberg>
i have two cisco switches that dont seem to support client cert auth on SSH so those still use passwords sadly
<azonenberg>
but everything else is SSH client certs exclusively
<rqou>
for SSH i also dropped passwords a long time ago
Bike has quit [Quit: wide-eyed terror]
<rqou>
i don't really know why people don't use client certs more often
<rqou>
it always seems to be treated as an "enterprise" thing to do
<azonenberg>
Yeah i know
<azonenberg>
I'd love to switch the wifi to wpa2-enterprise with client certs eventually
<azonenberg>
but that isnt happening for a while as game consoles etc dont usually support it
<rqou>
thanks nintendo
<azonenberg>
Not that it really matters a ton, my wifi is on the DMZ vlan :P
<azonenberg>
basically gets internet access and nothing else
<azonenberg>
everything with anything sensitive on it is either copper or fiber gig-e
<rqou>
it's hilarious how nintendo is really great at using weird wifi features that nobody else knows about but at the same time can't get e.g. wpa2-eap working
<azonenberg>
lol
<rqou>
e.g. they can do streetpass
<azonenberg>
is that wifi? i thought it was BT
<azonenberg>
never looked at the implementation
<rqou>
streetpass is wifi
<rqou>
there's no BT on 3DS
<rqou>
(as far as anyone can tell)
<rqou>
wiimotes are BT
<rqou>
but Wii U gamepads are wifi
<azonenberg>
One day i will be replacing those ciscos with custom switches
<azonenberg>
well that plus the controllers for the 1000base-X and 1000base-T interfaces
<rqou>
which chip is the ram? the one below the fpga?
<azonenberg>
I do not yet have a 10gig controller core
<azonenberg>
Yes
<azonenberg>
I need to redo it
<azonenberg>
the layout you see there is impossible to length match, not enough space
<azonenberg>
going to rotate 90 deg and switch io bank assignments a bit
<rqou>
what type of ram is that that doesn't have empty columns in the middle?
<azonenberg>
QDR-II+
<azonenberg>
that one chip has the I/O bandwidth of an entire DIMM of DDR3 1066
<azonenberg>
Considering it's SRAM and doesn't need refreshes, it's probably closer to 1333
<rqou>
it's not as commodity though
<azonenberg>
the latency is also way lower, 2.5 clocks vs however many tens the CAS latency is up to now :p
<azonenberg>
Oh and also, the controller is way simpler to write
<azonenberg>
I may not even need it
<azonenberg>
it was more of a hedge against running out of block ram
<azonenberg>
my proposed block diagram does not even require it
<azonenberg>
But if i decide to go all out and implement layer-3 switching functionality i may need it for routing tables etc
<azonenberg>
This thing is a bit overkill :P
<azonenberg>
the chip in the top left is a zynq
<azonenberg>
the chip in the middle is the largest artix7 (biggest gate count and biggest package)
<azonenberg>
The entire packet forwarding datapath will be in the artix
<rqou>
no ecc on the ram?
<azonenberg>
The zynq is intended as a testbed to see if i can get antikernel running on a cortex-a9
<azonenberg>
The RAM is 72 bits wide
<azonenberg>
I intend to use it as 64 + ECC
<rqou>
ah
<azonenberg>
it's only i think 4 MB capacity
<azonenberg>
but it's 72 bit wide simple dual port
<azonenberg>
so one read and one write bus at i think 800 MT/s
<azonenberg>
4-word bursts, one shared address bus alternating between R and W addresses
<azonenberg>
I take that back actually, i think the ram is 36 bits
<azonenberg>
but four word bursts
<azonenberg>
My plan was to take every 2 36-bit words and do a standard 64->72 hamming code
<azonenberg>
Then concatenate the 64 bit blocks to a 128-bit cache line
<azonenberg>
But again thats a long way out
<azonenberg>
I tabled the board for a bunch of reasons, one of which is that I got started on openfpga
<azonenberg>
right now it has a coolrunner as the PMIC
<azonenberg>
plus some discrete comparators etc
<azonenberg>
i plan to replace all of that with a slg46620
<azonenberg>
that will let me axe one of the jtag headers and LDOs and save a bunch of board space
<azonenberg>
(plus the chip package itself is like 1/4 the size)
<rqou>
is the board 6 layers or more than that?
<azonenberg>
8, controlled impedance with filled via-in-pad
<azonenberg>
Which is the other reason i tabled it, budgetary :p
<azonenberg>
Don't have the quote in front of me but i recall it will cost me something like $900 to make a minimum order of it at my usual chinese fab
<azonenberg>
that's bare PCBs
<azonenberg>
the FPGA alone is another $250 per board
<azonenberg>
the zynq is $50ish
<azonenberg>
the RAM is $50ish
<azonenberg>
another $50 probably in assorted support stuff (buck converters, passives, etc)
<rqou>
heh, ~10 years ago my father was getting poor yield with crappy chinese fabs and via in pad
<azonenberg>
1g SFPs are commodity, i can get them secondhand for less than a dollar, but the 10gig SFP has to be bought new for $80 or so
<azonenberg>
This is not a crappy fab
<azonenberg>
I've used them before for designs down to 0.35mm pitch WLCSP
<rqou>
for 10g can't you just use direct attach?
<azonenberg>
i broke out a 176-ball 0.4mm BGA with stacked microvias in pad (layer 1-2 blind and 2-3 buried)
<azonenberg>
I could, but the cables are $$$
<azonenberg>
and i have OM3 fiber between the house and garage already
<azonenberg>
(servers are in garage, desktop is in house)
<azonenberg>
right now it terminates at an old cisco 2970g
<azonenberg>
on each end
<azonenberg>
then breaks out to baseT for endpoint devices, plus a few short run fibers to prototype boards
<azonenberg>
i explicitly laid fiber so that i could go 10g in the future and have an upgrade path without a new cable plant
<rqou>
oh you're doing long distances
<rqou>
a 1m direct attach cable is $20
<azonenberg>
this is probably more like 10m
<azonenberg>
if you include all of the up and down from the rack into the ceiling, through the crawl space, then up to the second floor of the house
<azonenberg>
Here's a few pics of boards i made with that fab btw
<rqou>
hmm this site claims to sell a 10g sfp for $25
<azonenberg>
there is a barely perceptible divot in the center of the filled vias that you can see if you hold the board at an angle with good lighting under a microscope
<azonenberg>
The biggest screwup i got is when they sent me a solder test sample from the wrong board
<azonenberg>
but the QA report reflected my specs, not those of the specimen they mailed me
<azonenberg>
probably just somebody grabbed the wrong one off the bench
<azonenberg>
(it has green mask, my actual board had - and the QA report said - blue)
<azonenberg>
I got the right boards so i was happy lol
<rqou>
regarding the crappy end: one of my father's friends tried to get a pcb assembled in china and was asked by the assembly house what the warranty period was before getting a quote