<azonenberg>
monochroma: there should be a scopehal issue for more expanded trigger APIs
<azonenberg>
a scopehal issue for making the lecroy driver implement them
<azonenberg>
and a scopehal-apps issue for the ui to call those apis
<azonenberg>
defining the APIs is likely to be the hardest part
<azonenberg>
in a way that makes the API generic and not lecroy specific, while still exposing the advanced features of the lecroy firmware in particular
<azonenberg>
we may want to consider, say, a polymorphic Trigger class that has EdgeTrigger, GlitchTrigger, etc derived from it
<azonenberg>
and then an Oscilloscope::SetTrigger(Trigger* trig)
<azonenberg>
or similar
<azonenberg>
not saying thats the way to go, just an idea. Should be a topic of discussion when i get back from doing some sar administrative stuff i'm about to head out to
<azonenberg>
what i want to avoid is a combinatorial explosion of apis that every scope has to implement default "unimplemented" overrides for
<azonenberg>
and given that i'm about to implement ILA trigger stuff this is a good time
<azonenberg>
i figure if we do a polymorphic trigger object (and blow away all of the old trigger APIs in the process) you can just use RTTI to figure out what kind of trigger you got
<azonenberg>
and error out on anything you dont understand, without having to special case every trigger mode
<apo>
azonenberg: can't you just have the base class for the scope implement that?
<_whitenotifier-3>
[scopehal-apps] lainy opened issue #79: Add busy indicator in UI. - https://git.io/JvoLA
<azonenberg>
apo: I could. It's more a question of what would be the cleanest API
_whitelogger has joined #scopehal
m4ssi has joined #scopehal
<monochroma>
azonenberg: tested your 4.3 core profile code with my version check code, works! will clean my code up and make it more useful, and merge it into master tomorrow probably
<monochroma>
hmmm
<monochroma>
though
<monochroma>
i suspect on anything that doesn't have the 4.3 profile, it will fail when attempting to create the context and not even get to my code :P
<monochroma>
probably just redo how i do the version check to be based on that failure mode
electronic_eel has quit [Ping timeout: 256 seconds]
electronic_eel_ has joined #scopehal
electronic_eel_ has quit [Ping timeout: 260 seconds]
electronic_eel has joined #scopehal
m4ssi has quit [Remote host closed the connection]
<_whitenotifier-3>
[scopehal] azonenberg pushed 2 commits to master [+2/-0/±4] https://git.io/JvoKX
<_whitenotifier-3>
[scopehal] azonenberg 9e5b961 - LeCroyOscilloscope: fixes to digital channel handling
<bvernoux>
and VNA does not help to find where does come the problem with location
<bvernoux>
which can be done with SA + RF Probe
<azonenberg>
this is the probe calibration board i will be sending out to multech shortly, i need to miter the junctions where the CPWs meet the SMAs/probe headers still
<bvernoux>
to locate accurately where is the issue in different spots
<bvernoux>
or issues
<azonenberg>
and add stitching vias to the lower ground plane
<azonenberg>
but otherwise i think it's done
<bvernoux>
ha great
<azonenberg>
Look good? plan is you'll stick a short machine pin in the probe socket and connect directly to the probe
<azonenberg>
then just have the ground blade touch that exposed silver area
<azonenberg>
i have one with a 50 ohm termination at the point of probing and one without
<bvernoux>
personally depending on freq required I will change the SMA with the one I use up to 26.5GHz
<bvernoux>
they are really better for about 4.1USD per unit
<azonenberg>
My probe was only simulated out to 10 GHz and i'm expecting usable freq after tips etc is going to be closer to 2-5 GHz. but we'll see
<bvernoux>
if you have caracterized the SMA anyway it shall be good
<azonenberg>
anyway the point of the integrated standards here is to de-embed the test board and get s-parameters from the probe itself
<bvernoux>
I can check it for you
<bvernoux>
as I plan to characterize different SMA connectors
<azonenberg>
Ok
<azonenberg>
My plan was to send you one assembled board plus one probe, and maybe throw in a blank board if you want to test with other connectors?
<azonenberg>
gonna send the same to the harmon instruments guy so he can do some data as well
<azonenberg>
When i send you think i can throw in the pile of other stuff i've been meaning to send you too :p
<bvernoux>
I need to send them a quotation to check the price ;)
zigggggy has joined #scopehal
<azonenberg>
o/ zigggggy
<zigggggy>
hi
<azonenberg>
Continuing our conversation from earlier: xdev, fastwaveport, and com apis are all things i am starting to look into
<azonenberg>
the ultimate goal being to get higher wfm/s from lecroy instruments than is possible via the scpi api
<azonenberg>
I have not had the time to actually do any kind of benchmarking of the various possibilities
<zigggggy>
yeah
<azonenberg>
i am not looking to replace the scpi stack entirely and will probably still use it for configuration and control, as well as a fallback if the scope doesn't have my custom thing on it
<azonenberg>
i just want something that i can use to grab waveforms
<azonenberg>
ideally, in fact, i'd want a push-based system where i set up a socket and subscribe to notifications on a channel
<azonenberg>
then every time the scope triggers it will push the waveform for all active channels to me without any polling being needed
<azonenberg>
this would greatly improve responsiveness especially when using a remote instrument over VPN, saving a round trip time would help a lot
<azonenberg>
it also would allow control-plane traffic to take place while downloading a large waveform
<zigggggy>
yes this can be done fairly easily in C#
<zigggggy>
subscription model
<zigggggy>
pub/sub
<azonenberg>
Awesome. We'll need to discuss a bit to figure out what is the best way to integrate it into my software stack, as well as what the over-the-wire protocol should look like
<azonenberg>
But you're definitely better positioned to write such a bridge than me
<zigggggy>
it should be completely decoupled.. just a server that adapts COM to something else
<azonenberg>
i meant at the other end
<zigggggy>
im not sure what the server API looks like
<azonenberg>
figuring out the protocol, encapsulation, etc
<zigggggy>
you can provide suggestions
<azonenberg>
Yeah i'll think about it tonight, my lunch break is almost over and i should get back to billable work doon :p
<azonenberg>
soon*
<zigggggy>
C# should be pretty easy write this
<azonenberg>
lain/monochroma: fyi zigggggy is the friend at lecroy i mentioned, he's possibly interested in writing a C# bridge app that runs on the scope and provides more efficient waveform access via a pub/sub model instead of polling over scpi
<azonenberg>
zigggggy: yeah. again the ultimate goal is to get higher wfm/s for remote operation
<zigggggy>
there are cross platform serializer libraries
<azonenberg>
yes, like protobufs
<zigggggy>
i forgot the name
<zigggggy>
so we can serialize objects
<zigggggy>
right?
<azonenberg>
it's more a question of, given how simple the data structure is (basically an array of samples) is that even needed?
<azonenberg>
or just raw binary with a header
<azonenberg>
I'm thinking at a high level, you'd be writing a standalone tool that lives in the scopehal-apps repository. It would connect to the scope via com and then listen on a TCP socket server
<azonenberg>
at any time the client can send a command requesting to subscribe or unsubscribe from a channel
<azonenberg>
every time the scope triggers the server loops over all subscribed analog or digital channels for all connected clients and sends them the waveform data
<azonenberg>
For a first test, let's say the client-to-server protocol consists of messages structured as follows
<zigggggy>
using a library to make it more transparent
<azonenberg>
(protobufs still need external length framing )
<zigggggy>
let me find the library i've seen that is very popular.. dont think it is protobuf
<azonenberg>
capnproto?
<azonenberg>
basically the main thing i want to avoid is requiring a heavy parser to run on the incoming MB of waveform data
<azonenberg>
i dont mind more complex framing on the control link but i want the stream of waveforms to require as little processing as possible
<azonenberg>
because if you have 16-bit samples * 32M points * 4 channels that's 256 MB of data per trigger that has to be parsed
<azonenberg>
so if i can just have a minimal header (serialized using any framing you want) followed by a length and then raw waveform data that would be great
<electronic_eel>
couldn't the control channel be something very common, where lots of libraries exist for client & server? I think of HTTP+REST or something like that
<azonenberg>
electronic_eel: well i was thinking it should be over the same socket
<electronic_eel>
the actual data sent from the scope should of course be packed tightly
<azonenberg>
since this is for pub/sub subscription itself
<azonenberg>
i would be using the existing SCPI commands for things like changing trigger config or offsets
<azonenberg>
this is an experimental idea for bypassing the heavyweight scpi stack and vicp protocol for waveform data only
<electronic_eel>
if it is the same socket it will not be easy to tell the scope to stop sending data or send other control commands
<azonenberg>
the "control plane" is literally just sub/unsub requests going one way
<azonenberg>
and waveforms going the other
<azonenberg>
via a single socket
<azonenberg>
then a second scpi socket for everything else
<electronic_eel>
is sub/unsub really the only thing that needs to be sent to the c# agent on the scope?
<azonenberg>
i cant think of anything else
<azonenberg>
like i said this is just a performance accelerator
<azonenberg>
it's not a replacement for the existing control link to the scope
<azonenberg>
what i want to avoid is two things
<electronic_eel>
maybe you'll need to verify if the c# agent has the same idea of data sources as the scopehal driver
<azonenberg>
first, scpi link is monopolized when downloading a waveform and prevents you from e.g. querying channel offset until the download is done
<azonenberg>
second, lots of number crunching and software overhead when pulling waveforms causing WFM/s performance to suffer
<azonenberg>
I don't know how big a boost this will be, or if it's even worth doing. Just experimenting right now
<electronic_eel>
yes, I understand the reasoning behind it
<azonenberg>
If successful i'd love to write similar bridges for other scope firmware if possible
<azonenberg>
lecroy being the highest priority because we have both a friendly guy working there and a lot of folks in the channel using their products
<electronic_eel>
not easy if the scope isn't designed to have programs installed on it
<zigggggy>
over time we can adapt more commands in the server app
<azonenberg>
electronic_eel: yes, thats another advantage of lecroys wrt customization
<azonenberg>
at least the higher end ones not running winCE etc
<azonenberg>
its just a windows OS you can install custom tools on
<zigggggy>
its not that either but i found this just now
<electronic_eel>
one thing I'd look for in the protocol is if you can abort a transfer
<electronic_eel>
if you connect through a vpn and the connection is completely stalled due to all the data being sent, I think it would be good to be able to send an abort command and have the agent abort immediately
<azonenberg>
electronic_eel: hmm yeah probably a good idea
<azonenberg>
zigggggy: ok well do some research and we'll figure things out. To start, do you have access to a waverunner 8k or hdo 9k to test on?
<azonenberg>
So our results are comparable between you and me
<electronic_eel>
if the protocol goes like "length of blob"+blob it won't be easy to abort
<azonenberg>
(i assume you've got to have some floating around the lab)
<azonenberg>
electronic_eel: maybe divide blob into smaller sub-blobs which can't be aborted, but you can abort between them?
<zigggggy>
this is kind of cool but may have a lot of overhead
<electronic_eel>
yes, that would be one way to do it
<monochroma>
azonenberg: oh cool!
<azonenberg>
zigggggy: i figure the first step would be to write an app that pulls the waveform from say channel 1 of the scope only and sends it out
<azonenberg>
and benchmark how many WFM/s you can get on, say, 1M points @ 8 bit precision
<azonenberg>
then compare that to what i get now with scpi
<azonenberg>
then decide at that point if it's even worth the effort to go further
<azonenberg>
dont worry about framing the data, just spit out a raw uint8_t[1000000]
<azonenberg>
nothing will be on the other end :p
<zigggggy>
azonenberg ok
<zigggggy>
azonenberg one nice thing is we can maybe optimize it before we send it
<azonenberg>
Yeah some level of compression or delta coding may help
<azonenberg>
but right now my assumption is that, at least on a LAN, it's the scope getting cpu bound
<azonenberg>
not the network as the limiting factor
<zigggggy>
could be
<azonenberg>
so i think less processing is better than more, I have not got close to saturating gigabit with any of my scopes yet
<azonenberg>
But again, we'll need to run some experiments and see. I don't know where the bottlenecks are yet
<azonenberg>
i just know i want faster :p
<zigggggy>
when accessing COM objects, i should get a reference to the data instead of a copy?
<zigggggy>
a copy would be pretty inefficient
<azonenberg>
That would probably be faster. But I have very little experience with COM in general
<azonenberg>
i haven't done windows dev since vista was the new hotness
<zigggggy>
i could keep a buffer in C# for doing processing on it and keep it alive so it there is less pressure on the GC
<zigggggy>
and do memory level copy from the com object to the buffer
<azonenberg>
yeah that sounds like the sort of stuff you'd want to be doing
<zigggggy>
i havent done that before because i never cared too much about speed
<azonenberg>
On my end, i am looking at seeing how little overhead i can get, with the ultimate goal of being able to DMA direct from the ethernet card into GPU memory
<zigggggy>
in C# by default memory management is handled by the run time
<azonenberg>
that's a long ways out and at least for the moment i will be doing two copies from nic-main ram and ram-gpu
<zigggggy>
so if you copy data from say one array to another, it will create new space on the heap and then GC it
<azonenberg>
but down the road i will be doing protocol decodes on the GPU too
<azonenberg>
at least for things like math functions that don't depend on earlier results
<azonenberg>
obviously something like a CDR PLL will be hard to parallelize
<zigggggy>
but you can instead ask for a buffer to be created and use a lower level of copying
<azonenberg>
Yeah
<azonenberg>
But what i might be able to do, if waveforms are coming in fast enough, is run CDR on several different waveforms on different GPU/CPU threads in parallel
<azonenberg>
then integrate those all into say an eye pattern
<zigggggy>
but we can also benchmark the C# stuff to see where to optimize..
<azonenberg>
Yeah
<zigggggy>
if i've hit the limit of C#, then we can even move some of the processing to C++ and marshall
<azonenberg>
BTW, you probably know maui better than i do...
<zigggggy>
C# does interop with C++ very well
<azonenberg>
Does SDA have a feature that lets you cross-probe from an eye pattern back to a time domain waveform?
<azonenberg>
i.e. given a mask violation or even just an interesting region of an eye, can i find what the time domain view of the signal producing that was?
<zigggggy>
no idea
<zigggggy>
i know very little about sda
<azonenberg>
that's a feature i have planned for my software that i havent seen elsewhere
<zigggggy>
i would think not
<azonenberg>
each pixel in the eye pattern will store a timestamp
<zigggggy>
because i dont think the data is stored
<zigggggy>
its a lot of data
<zigggggy>
just the bitmap
<azonenberg>
so if you select the pixel you'll be able to jump back to the history and see the waveform in question
<zigggggy>
or maybe a bin of the data is stored
<zigggggy>
and the bitmap is updated each time
<electronic_eel>
azonenberg: that looks like a very useful feature to me
<azonenberg>
electronic_eel: yes exactly
<zigggggy>
azonenberg i think you will run out of memory very quickly if you did that
<azonenberg>
it will use a lot of ram though, at least if you want deep eye captures
<azonenberg>
zigggggy: my workstation has 192GB of ram
<azonenberg>
i think i'll be ok for a while :p
<electronic_eel>
ram on the pc isn't that much of an issue anymore, with the current ryzen/epic systems you can upgrade into the tb regions
<zigggggy>
but arent processes limited?
<azonenberg>
zigggggy: on linux you can use basically all of the ram in one process, you might have to poke some setting to enable it
<zigggggy>
you're also gonna make a mess of the heap :P
<azonenberg>
and it's not going to be one huge buffer
<zigggggy>
and it will prob take a while to eve allocate that much space
<azonenberg>
it's one buffer per waveform
<zigggggy>
you're going to allocae on each waveform?
<azonenberg>
i've run glscopeclient into the tens of GB ram usage with deep history already
<zigggggy>
that's a lot of calls to the kernel
<zigggggy>
and will fragment memory badly
<azonenberg>
Reducing overhead per waveform on my end is definitely a TODO
<azonenberg>
i am considering having a pool of some sort preallocated once you set the max history depth
<zigggggy>
i would think its better to allocate in chunks
<zigggggy>
maybe 500M chunks
<azonenberg>
if anything i'd allocate waveform-sized chunks and then have a pool of however big a waveform is
<zigggggy>
yeah
<azonenberg>
but so far my performance bottleneck has been getting data from the scope fast enough
<zigggggy>
can also have a feature where if you run out of memory you start overwriting the oldest wfm
<zigggggy>
since most will not have nearly that much ram
<azonenberg>
Well i already have a cap to history depth
<azonenberg>
but right now it's not "smart" and you have to specify the cap yourself
<azonenberg>
i have no "cap below 100% of ram" feature yet, although that would be good to have
<azonenberg>
if you set the history depth to thousands of waveforms glscopeclient will happily use all of your ram
<monochroma>
i'm guessing getting this working on my ancient DDA-5005 scopes might be a no-go? :P
<azonenberg>
monochroma: i think you can install current .net framework on XP
<azonenberg>
or maybe just recompile the source for an older framework
<azonenberg>
the com objects shouldnt have changed too much wrt accessing a waveform
<monochroma>
yeah, worst case i could do the Windows 7 upgrade path that is uhhhh kinda crazy
<azonenberg>
win7 on a DDA-5005? um
<azonenberg>
that sounds a bit wrong :p
<monochroma>
it requires a motherboard replacement (with something more modern but still has a PCI slot)
<azonenberg>
lol
<azonenberg>
what is the pci connection, a cable or something?
<azonenberg>
or does it assume the card is in a specific plaec?
<azonenberg>
do you need an AGP slot too?
<azonenberg>
monochroma: anyway i think the bigger issue would be the maui version you're running being too old, not the OS
<monochroma>
yeah that too
<electronic_eel>
did lecroy always route outputting the graphics through the pc and windows to the graphics card or did they also have a setup with an fpga writing into the graphics card?
<electronic_eel>
with "always" I mean the scopes running windows
<azonenberg>
does windows ce count? i think, not certain, the wavesurfer 3000 series use fpga accelerated graphics
<azonenberg>
zigggggy would know
<azonenberg>
oh speaking of accelerating graphics
<azonenberg>
Anybody want to look into implementing antialiasing in the new compute shader line renderer?
<monochroma>
oh that reminds me, what's the status of the eye diagram with the horizontal lines through it?
<azonenberg>
i mostly see them vertical not horizontal. I think it's a rounding/quantization issue
<azonenberg>
but i havent had time to work on it
<azonenberg>
Been busy trying to get MSO stuff up and running
<monochroma>
azonenberg: uhh i might have to send you a screenshot in a bit
<azonenberg>
but i know the issue you're talking about
<zigggggy>
maybe i'll use this to create/manage the socket
<lain>
zigggggy: you could target .net core or .net standard
<zigggggy>
.net standard is for libraries.. for executables you have to target .net core or .net framework
<zigggggy>
.net standard libraries can then be used in .net framework or .net core
<lain>
ah right
<zigggggy>
.net core is preferred since .net framework dev is over
<lain>
yep
<zigggggy>
but there are things .net core does not yet support
<zigggggy>
COM could be one of them
<zigggggy>
although with .net core 3 a lot of stuff is there now
<lain>
the way I've typically handled that in cross-platform code is to have windows-specific code target .net framework as a separate assembly, loaded by the .net core main program
<zigggggy>
the supersocket library looks pretty nice though.. very high level control of socket comm
<lain>
ah
<zigggggy>
that is typically how you structure it, yes.. you create library projects and put most of your code there
<zigggggy>
the exe project is pretty bare bones.. just does bootstrapping for the most part
<zigggggy>
maximizes code reuse and decoupling
<azonenberg>
lain: keep in mind we are talking about something that runs on the scope itself, not a client app
<lain>
yep. I only point it out because, at least from the landing page, it looks like SuperSocket targets Mono, not .NET Core?
<zigggggy>
i like to have a core library with no dependencies and then if i need a 3rd party dependency i might make a separate library.. this way the core could be used for other things without dragging in a bunch of 3rd party dependencies
<zigggggy>
but if its a small project, i wont do that
<lain>
azonenberg: ahh ok, I haven't read all the scrollback
<azonenberg>
lain: we're talking about a com app that runs on the scope and bridges directly from the internal com objects to a lightweight protocol bypassing all of the maui scpi stack
<azonenberg>
to see if we can get higher wfm/s that way
<lain>
aha
<azonenberg>
keeping the normal scpi interface for control plane stuff