anuejn has quit [Remote host closed the connection]
anuejn has joined #scopehal
electronic_eel_ is now known as electronic_eel
<azonenberg>
OK so let's see, what's next?
<azonenberg>
Bird|otherbox: did your new GPU come in yet?
azonenberg_work has quit [Ping timeout: 240 seconds]
azonenberg_work has joined #scopehal
_whitelogger has joined #scopehal
juli965 has joined #scopehal
<azonenberg>
Unless anyone has something more major my next priority is going to be implementing enum arguments to filters (probably tomorrow) then generic-ifying the spi flash decoder so it works with stuff other than W25N family
<azonenberg>
I have a Zybo with a S25FL128S already hooked up to my scope as a source of test data
<azonenberg>
And yeah i think the first step re streaming will simply be to create devices / a mode for a device that once triggered retriggers instantly with no delay
<azonenberg>
so waveforms occur back to back and can be displayed as consecutive waveforms
<azonenberg>
better presentation of them as a single seamless stream can come later
<azonenberg>
e.g. support for filter executions that span across waveforms
<azonenberg>
and maintain state
<tnt>
So just make it appear as N consecutive waveforms of a fixed length (and N is set by the max waveform history config)
<azonenberg>
That's the easy part
<azonenberg>
You dont need to do anything special for that
<azonenberg>
Just have a device that spits out buffers as fast as it can without waiting for triggers between them
<azonenberg>
The hard part is making filters and the UI maintain state
<azonenberg>
so a spi byte with 5 clock edges in one waveform and 3 in the next comes out correctly
<azonenberg>
etc
<azonenberg>
this will be critical with complex traces that span seconds or more of activity
<azonenberg>
oh actually i think the other high priority is adding compression to the lecroy LA driver
<azonenberg>
or... there has to be a better name for that process
<azonenberg>
sparsification?
<NeroTHz>
fml RDP shit the bed, now having to edit >1 GB photoshop PSD files on CPU graphics
<Bird|otherbox>
azonenberg: will be a little while still
<azonenberg>
I think from now on the official term for this process will be "de-duplication"
<azonenberg>
it's not compression per se, it's making better use of the RLE inherent in the existing data structure
<tnt>
Do you keep it in memory as de-duplicated ?
<azonenberg>
tnt: The in memory and on disk formats are substantially the same
<azonenberg>
a Waveform consists of a bit of metadata then int64 samplerate, vector<int64> offsets, vector<int64> durations, vector<T> data
<azonenberg>
sorry samplerate is called timescale
<azonenberg>
and is the basic time step in picoseconds for that waveform, offsets/durations are measured in that unit
<azonenberg>
Each sample represents a defined region in time which begins at (waveform start timestamp) + timescale_per_ps*offsets[i]
<azonenberg>
and ends at that time plus timescale_per_ps*durations[i]
<sorear>
are you doing anything to compress the offsets and durations or just the data
NeroTHz has quit [Read error: Connection reset by peer]
<azonenberg>
As of now there is no compression of any kind
<azonenberg>
It's too slow when you need random access for decodes and rendering etc
<azonenberg>
Anyway, i'm not done explaining :)
<azonenberg>
The in memory representation typically uses more memory than the on disk, because filters and decodes are not stored to disk
<azonenberg>
they're only stored as metadata of what filter is connected how with what settings
<azonenberg>
the output never gets serialized
<azonenberg>
so my current "big" dump is 7 channels * 11 waveforms * 64M points per channel = about 5 billion data points
<azonenberg>
it's 80GB on dis kand uses about 110 GB of RAM when loaded
<azonenberg>
What i'm proposing doing is, when a scope driver is reading waveforms from a LA that does not output RLE natively
<azonenberg>
that i should detect two consecutive samples with the same value
<azonenberg>
and store {off=N, len=2, value=x} rather than {off=N, len=1, value=x} and {off=N+1, len=1, value=x} like i do now
<azonenberg>
basically right now whjen you have repeating data on the input, you get degenerate RLE with len=1 framing overhead on every single sample
<azonenberg>
even those that actually repeat
<azonenberg>
MAXWELL and other instrumentation we build will output RLE natively, so this won't be a problem
<azonenberg_work>
What do you folks think of some kind of scripted "sweep powerup trace" utility for glscopeclient?
<azonenberg_work>
the basic idea is, set up a capture of a given length with trigger on Vdd, POR, or similar
<azonenberg_work>
set trigger to the very start of the waveform, connect to a managed power supply, turn it on, grab a trace
<azonenberg_work>
power down, shift trigger by one capture length
<azonenberg_work>
power up and grab another
<azonenberg_work>
basically what i did for this boot trace, but fully autonomous
<azonenberg_work>
there may be a little bit of gap/overlap at the boundaries due to jitter depending on how reproducible boot timing is
<azonenberg_work>
but for general tracing purposes it should be a good way to get a lot of data fast