<sb0>
kernel compilation does all the getattr()'s at compile time, so parameters are not updated while a kernel is running
<sb0>
though the kernel can RPC a function that does the getattr, so yes, it's already possible in this way
<sb0>
and this is only for parameters, not arguments.
nicksydney has joined #m-labs
<rjo>
ack. imho we should expose the pdb get/set explicitly to the experiment. can that be proxied as self.pdb[param_name] or self.pdb.param_name?
<rjo>
(lets move over here for the UI discussion)
<rjo>
that autocompleting (based on previously targeted pipeline names) drop down box would be somewhere near the submit button i presume.
<rjo>
"remote text editor" would be based on the remote import backend?
<sb0>
it would be where the current text field is, which is indeed just above the submit button
<rjo>
ack
<sb0>
remote filesystem access, yes. we don't need the import part.
<sb0>
and potentially with some git integration
<rjo>
ok. yet your recent email outlines a path for remote import?
<rjo>
with the executor.
<sb0>
yes
<rjo>
so remote import for GUI code/plugin triggering but not for remote editor?
<sb0>
why would the remote editor need any special imports?
<rjo>
no import but the file retrieval machinery.
<sb0>
yes, that would be the same code in the master for both remote imports and edits
<rjo>
ack. yes. the editor would not need the importlib stuff but the remote filesystem implementation.
<rjo>
about shared (real) filesystem vs. artiq-network-filesystem: in virtually all cases the filesystem where the experiment code tree lives would be shared anyway.
nicksydney_ has joined #m-labs
nicksydney has quit [Ping timeout: 244 seconds]
<rjo>
the only advantages of a custom artiq-fs that i can think of are: well defined and stable namespace and somewhat easier setup (in the rare case where the underlying fs is not already shared).
<sb0>
and retrieval of files from older git revisions
<rjo>
"support plotting data from parameters": can't you just make the reference-histogram-builder experiment emit a result and plot that in addition to changing the pdb?
<rjo>
yes. git.
<sb0>
yes, that would work, but it obviously duplicates data
<rjo>
and couldn't the "clear reference histogram" button be just an input type to the ref-hist-builder (once inputs can change for running experiments)?
<sb0>
why does this have to do with changing inputs for a running experiment?
<rjo>
if the ref-hist-builder is running all the time.
<sb0>
ah. you mean the ref histogram builder reschedules itself periodically
<sb0>
yes
<rjo>
yes
<rjo>
rescheduling itself vs pausing/suspending itself. the former would see the input change. the later would require realtime input updates.
<sb0>
it can be, but then you have to wait for the histogram builder to run again, vs. getting instant feedback when you click the button
<rjo>
true.
<rjo>
"split on underscore + CamelCase": i guess there are few better alternatives....
<rjo>
ok. now i have to do a content diff of your list to ours in my head...
<sb0>
maybe the file sharing protocol can be HTTP. then we can reuse git-over-HTTP code for pushing commits etc.
<rjo>
but it basically sounds good to me if there is nothing missing.
<rjo>
it would not be so much git-over-http as git-aware-fs-over-http. you need single file retrieval, listdir() etc, you would not submit the commit data but just trigger it based on the data already on the other side.
<rjo>
TIL: pandoc can convert .rst to .docx ;)
<rjo>
unfortunately not the other way round...
<rjo>
sb0: ah. the GUI state persistence for the experiment input/output items.
nicksydney has joined #m-labs
nicksydney_ has quit [Ping timeout: 244 seconds]
<rjo>
and exposing TTL overrides DDS overrides in the UI.
<sb0>
"* TTL and DDS monitoring and injection"
<rjo>
ok.
<sb0>
and "* interface to associate experiment inputs to widgets, ***association stored in the GUI state***"
<sb0>
is that what you mean by "experiment input/output items"?
<sb0>
also "* outputs fully defined in the GUI state (what to show, how)"
nicksydney_ has joined #m-labs
<rjo>
ok. do you agree with the basic idea of the initialization precedence for these UI widgets?
nicksydney has quit [Ping timeout: 244 seconds]
<rjo>
you introduced that term input output items. i had called them "widgets" before.
<rjo>
but yes.
<rjo>
ok. let me mentally diff it again...
<rjo>
artiq-fs has another problem: if you use it to expose files for editing through it, people will want to rename (save as), delete, copy things as well...
<rjo>
and then you have to ship the UI for doing that.
<sb0>
ok, so maybe make the shared-FS an optional feature that is only required for editing
<sb0>
s/feature/dependency
<rjo>
then artiq-fs would be readonly and only used for remote import and remote git gymnastics.
<rjo>
not even git gymnastics if the master/worker trigger the commits and tag stuff with the commit hash.
<rjo>
and artiq-fs would be used for submit-by-content (test submission without git commit).
<sb0>
do we need remote import at all?
<sb0>
on controllers, mostly?
<rjo>
if the shared filesystem is optional, we need remote import for datacrunching-on-controllers and experiment-specific-qt-code
<rjo>
do you agree that the "log" and all the filtering and routing can be done with "logging"? logging.Record can have metadata etc
<rjo>
re remote-import: i am not a big fan of it. i think we can live without and make datacrunching-on-controllers et al. depend on a shared filesystem.
nicksydney has joined #m-labs
nicksydney_ has quit [Ping timeout: 244 seconds]
<sb0>
rjo, I'm not sure if it'll work that easily with logging
<sb0>
you need post-mortem filtering and on a different machine, whereas the logging.LogRecord mechanism is meant to be used in the same Python interpreter and formats the messages immediately based on predetermined parameters
nicksydney_ has joined #m-labs
nicksydney has quit [Ping timeout: 244 seconds]
nicksydney has joined #m-labs
nicksydney_ has quit [Ping timeout: 244 seconds]
<mithro>
Has anyone tried using misoc/migen stuff with the Project IceStorm workflow at all?
<mithro>
I saw discussion about Yosys and stuff the other day
<sb0>
aren't those fpgas too small for even lm32?
<mithro>
sb0: I think so - you'd only really be able to do very simple things
<mithro>
sb0: so it wouldn't be super practical, but you should be able to do a light blinking project
<mithro>
I guess it's more migen then misoc
nicksydney has quit [Ping timeout: 244 seconds]
nicksydney has joined #m-labs
nicksydney_ has joined #m-labs
nicksydney has quit [Ping timeout: 244 seconds]
nicksydney has joined #m-labs
nicksydney_ has quit [Ping timeout: 244 seconds]
<mithro>
hey nicksydney, it looks like you are in Sydney, NSW?
<nicksydney>
mithro: yup
nicksydney has quit [Read error: Connection reset by peer]
nicksydney has joined #m-labs
<mithro>
nicksydney: I'm in Sydney, NSW too - what are you doing with m-labs / misoc / migen?
<nicksydney>
mithro: learning about the project and hopefully can contribute something in future :)
<mithro>
nicksydney: Any idea what you want to work on?
<nicksydney>
not too sure...just poking around at the moment...need to get the hardware too
<nicksydney>
what you working on ?
_florent_ has joined #m-labs
nicksydney has quit [Ping timeout: 244 seconds]
nicksydney has joined #m-labs
nicksydney_ has joined #m-labs
nicksydney has quit [Ping timeout: 272 seconds]
nicksydney has joined #m-labs
nicksydney_ has quit [Ping timeout: 272 seconds]
nicksydney_ has joined #m-labs
nicksydney has quit [Ping timeout: 272 seconds]
nicksydney has joined #m-labs
nicksydney has quit [Remote host closed the connection]
nicksydney_ has quit [Ping timeout: 272 seconds]
<GitHub104>
[artiq] fallen pushed 1 new commit to master: http://git.io/vkFFA
<GitHub104>
artiq/master c7953da Yann Sionneau: test: add unittest for sync_struct
<GitHub84>
[artiq] fallen pushed 1 new commit to master: http://git.io/vkFNf
<GitHub84>
artiq/master e5f16b2 Yann Sionneau: sync_struct: fix test case name
<ysionneau>
sb0: I propose to modify the other tests (like worker) so that they do asyncio.new_event_loop() instead of get_event_loop
<sb0>
sure, if that works
<sb0>
it should, but asyncio is a buggy part of python. last time i tried doing things like that, things broke.
<sb0>
maybe they fixed it - they eliminated a few annoying asyncio problems in 3.4.3
_florent_ has quit [Ping timeout: 258 seconds]
<ysionneau>
(I can reproduce the same error you reported though, let's see if I can fix that by using a new event loop...)
<sb0>
ysionneau, and btw the artiq master will crash because of those problems on < 3.4.3; are setup.py and conda checking that at least that version is present?
<ysionneau>
ah that's why I have issues on my 3.4.2 then
<ysionneau>
sb0: they don't at the moment, I'll do it, it's easy in conda
<ysionneau>
as for setup.py I propose to just check sys.version_info and throw an error if it's not >= 3.4.3
<sb0>
isn't there a standard way of doing things for this supposedly common problem?
<ysionneau>
you also seem to have platform.python_version()
<sb0>
something setuptools-specific maybe
<ysionneau>
the only info I can find about setup tools and python version is their "2to3" support, and in their documentation page they seem to use sys.version_info for some checking
<ysionneau>
hummmm with python 3.4.3 (anaconda) I get this while running the current worker test: http://pastebin.com/N93v9xBw
<sb0>
the signal stuff is a python bug afaict
<sb0>
the rest, well, you can investigate
<ysionneau>
the new_event_loop() + set_event_loop() does not seem to work at all in worker.py (just no output and nothing happens)
<ysionneau>
very weird
fengling has quit [Quit: WeeChat 1.1.1]
<sb0>
ysionneau, is the gui correctly packaged by conda now?
<sb0>
on both windows and linux
<ysionneau>
it's packaged yes, I tested only on linux though
<ysionneau>
let me check the windows package
<ysionneau>
ah there's no recent windows package
<ysionneau>
let's run a conda build then
<ysionneau>
but then the pyqtgraph bug will prevent the gui from running
<ysionneau>
maybe we should just temporarily fork pyqtgraph and build our conda package from this fork
_florent_ has joined #m-labs
<sb0>
ysionneau, i was hoping this problem was going to be resolved soon.
<sb0>
can you email the maintainer?
<ysionneau>
ok
<ysionneau>
doing now
<ysionneau>
done
<ysionneau>
I updated the win32 conda package
<ysionneau>
If we don't get any response from Luke (the maintainer) by the end of the week, I propose to fork temporarily so that we can package correctly
_florent_ has quit [Quit: Leaving]
<ysionneau>
sb0: guys at #pypa told me there's no smart way to check python micro (3rd digit) version in setuptools :/
<ysionneau>
hum the blocking I get when using the new_event_loop() happens in asyncio_wait_or_cancel, when doing the asyncio.wait() on the canceled task
<sb0>
figure out which canceled task, and what it is doing...
<ysionneau>
it's blocked on waiting the closed.wait()
<ysionneau>
I tried to replicate this issue in a small program just doing an Event.wait() and doing a cancel on it and then a wait() but it works on my small test case
<ysionneau>
s/replicate/reproduce/
<ysionneau>
got it, that's because I did the new_event_loop/set_event_loop() _after_ instanciating Worker() ... therefore the Event() was using another event loop I guess
<GitHub89>
[artiq] fallen pushed 1 new commit to master: http://git.io/vkNCV
<GitHub89>
artiq/master 21d88d8 Yann Sionneau: tests: use a different event loop for each test
<GitHub132>
[artiq] fallen pushed 1 new commit to master: http://git.io/vkNuo
<kristianpaul>
or just straight from hdl to the fpga, still a big gain when you are using floss and a single development tool like migen or myhdl for instance
<rjo>
sb0: where do you get that info re logging's limited use from? logging can easily span several processes and machines.
<rjo>
sb0: the Logger needs to have a pretty noisy level, then collect from module-level loggers into the root logger, then pass the unformatted LogRecords with a DatagramHandler (or SocketHandler) to another machine, filter at whatever level and based on whatever metadata (pipeline, experiment, controller) you want and emit (file, log dock).
<rjo>
reinventing distributed logging, filtering etc would be a bad idea.
_florent_ has quit [Quit: Leaving]
<rjo>
ysionneau: if it is just the tests failing i woule like to mark them unittest.expectedFailure for lower python versions instead of requiring python 3.4.3.
<ysionneau>
hummm I think it's not just the test sb0 was talking about, but better ask him for confirmation
<rjo>
sure. but if it is only because of new_event_loop(), forcing 3.4.3 is not needed.
<ysionneau>
I understood that quite a few things have been fixed in 3.4.3 in asyncio
<ysionneau>
and reading the documentation of asyncio you can see quite a few "changed in 3.4.3"
<ysionneau>
I guess he's referring to that
<ysionneau>
but indeed let's clarify exactly why we need 3.4.3
<ysionneau>
rjo: about ventilator, I've read about OSERDESE2 (the 7 series equivalent of the S6's OSERDES2), and they seem to have almost 1 "sys_clk" latency
<ysionneau>
(one clkdiv latency)
<ysionneau>
how do you compensate for that?
<ysionneau>
I mean, if you trigger the OSERDESE2 when the timestamp == counter_value and then you want to match the fine_timestamp using the OSERDESE2, ... if you have 7 high-speed-clock periods of latency ... you almost miss the entire "fine timestamp" window, don't you?
<ysionneau>
(for 8:1 data_width, it's written OSERDESE2 has 7 CLK cycles of latency)
<rjo>
re 3.4.3 ok. fair enough.
<rjo>
the latency does not matter (for tdc/dtc).
<rjo>
it is the same as a bit of wire which we have to compensate for anyway where it matters.
<ysionneau>
right, so as long as we can calibrate the latency we don't care I guess
<rjo>
yes. but you don't have to think about the latency at all.
<rjo>
in the end we will have to think a bit about where best to adjust for latency if such an adjustment is needed for a channel.
<rjo>
the nice thing is that since the ttl channels have independent fifos the potential reordering is not a problem.
<rjo>
so it an be done pretty high up, probably with another LatencyCompensatedTTL{In,Out} in artiq.coredevice.tll
<rjo>
let's define the reference plane of "time" as the rtio_clk end of the fifos, ignoring all tdc/dtc/serdes/driver/level shifter/cable latency. this statement might go into the manual.
<sb0>
on < 3.4.3, the StreamWriter.drain() coroutine sometimes does not even return a Future