dangole has joined #openwrt-devel
dangole has quit [Remote host closed the connection]
dangole has joined #openwrt-devel
<rsalvaterra> Oh, my…! How old is the "add lzma support to jffs2" monster patch…? :|
<rsalvaterra> I mean, it surely must predate native kernel xz support (which is basically lzma)…
adrianschmutzler has quit [Quit: ~ Trillian - www.trillian.im ~]
mangix has quit [Read error: Connection reset by peer]
<dangole> rsalvaterra: it is very old...
<rsalvaterra> dangole: We really should just implement xz support in jffs2, but then there's that pesky backwards compatibility thing…
<dangole> rsalvaterra: i don't think we need to bother with jffs2 filesystems created with anything else than the kernel we are running
<dangole> would be nice to have tooling on the buildhost, but binwalk does the trick i guess
<rsalvaterra> dangole: Hm… ok, that's a big plus.
<rsalvaterra> Adding support for an existing compressor to jffs2 shouldn't be too hard, from what I can tell: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/fs/jffs2?id=c799aca31bfe61ba3a91133acf5a13a0773087d4
<rsalvaterra> Of course, it also depends on the codec itself.
<Namidairo> does anything use libzstd in packages yet
<rsalvaterra> I may take a stab at it, one of these days.
<rsalvaterra> Namidairo: I don't think so, but we'll probably need it if we move squashfs from xz to zstd.
<aparcar[m]> rsalvaterra: sorry could you please summarize your plan?
<rsalvaterra> aparcar[m]: do I look like a guy with a plan? XD
<rsalvaterra> In all seriousness, though… of all native kernel compressors, I believe zstd provides the best speed/ratio trade-off.
<rsalvaterra> I think we could implement zstd support for jffs2 (which ubifs already supports) and get rid of that crazy lzma jffs2 patch.
<rsalvaterra> If people *really* want lzma, implementing both xz and zstd support for jffs2 would still be a much smaller patch than the existing one.
<aparcar[m]> we could add it for squashfs first can't we?
<rsalvaterra> aparcar[m]: Sure, I know you're already on it.
<aparcar[m]> rsalvaterra: *I was
<rsalvaterra> Oh.
<aparcar[m]> I thought you're the better candidate
<rsalvaterra> Well, this is long-term stuff, of course.
<rsalvaterra> aparcar[m]: I don't know if I'm the better candidate, but I do have a personal interest in it, for sure. ;)
* rsalvaterra wants to zstd all the things
<goliath> If someone's interested, 2 weeks ago I did a SquashFS serial *decompression* benchmark on a Debian image: https://github.com/AgentD/squashfs-tools-ng/blob/master/doc/benchmark.txt#L197
<goliath> Shiny graphs with compression ratio for comparison: https://github.com/AgentD/squashfs-tools-ng/blob/master/doc/benchmark.ods
<rsalvaterra> If these numbers still hold… https://lore.kernel.org/patchwork/patch/802867/
<rsalvaterra> … zstd will do wonders for boot times.
<rsalvaterra> At what xz level are we currently compressing the squashfs images?
<rsalvaterra> Decompression memory requirements are higher at higher levels, that's why I'm asking.
<goliath> rsalvaterra, AFAIK the highest with some micro management extra settings: https://github.com/openwrt/openwrt/pull/2916
<rsalvaterra> Oh, but we're using small blocks, so memory shouldn't be an issue.
<rsalvaterra> Off to bed now, almost 2 AM here. Bye!
elektroman has quit [Ping timeout: 260 seconds]
elektroman has joined #openwrt-devel
dansan has joined #openwrt-devel
hbug___ has joined #openwrt-devel
hbug__ has quit [Ping timeout: 240 seconds]
danitool has quit [Quit: Cubum autem in duos cubos, aut quadratoquadratum in duos quadratoquadratos]
victhor has quit [Ping timeout: 260 seconds]
tobleminer-tSYS has quit [Quit: AS4242423214]
tobleminer-tSYS has joined #openwrt-devel
<aparcar[m]> goliath: Oh hi, my plan is still to migrate over to your implementation
dangole has quit [Quit: Leaving]
mangix has joined #openwrt-devel
luke-jr has quit [Quit: ZNC - http://znc.sourceforge.net]
luke-jr has joined #openwrt-devel
philipp64 has quit [Read error: Connection reset by peer]
philipp64 has joined #openwrt-devel
wvdakker has joined #openwrt-devel
wowaname has joined #openwrt-devel
opal has quit [Remote host closed the connection]
wowaname is now known as opal
nitroshift has joined #openwrt-devel
valku has quit [Quit: valku]
dedeckeh has joined #openwrt-devel
_whitelogger has joined #openwrt-devel
<ynezz> aparcar[m]: AFAIK it was disabled by jow on the old server, but it seems to be running on the new server as https://firmware-selector.openwrt.org/?version=SNAPSHOT&id=tplink_archer-c7-v5 says 22.11.2020
<ynezz> aparcar[m]: ideally it should be moved to CI, or at least some locking implemented (this probably might as well go in to the cron job)
feriman has joined #openwrt-devel
Night-Shade has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<ynezz> aparcar[m]: that script is now running every 8 hours, which is probably bearable
<ynezz> aparcar[m]: that CI approach would mean scraping of the json files over HTTPS which would probably cause more load on the server then the local file scraping
Ycarus has joined #openwrt-devel
glyph has quit [Quit: End of line.]
glyph has joined #openwrt-devel
gch98121334 has joined #openwrt-devel
gch9812133 has quit [Ping timeout: 264 seconds]
gch98121334 is now known as gch9812133
thagabe has joined #openwrt-devel
<ynezz> aparcar[m]: http://sprunge.us/focA4Q doesn't looks that bad to me
<ynezz> the new server simply rocks :)
<ynezz> /home/mirror/downloads is 912G currently
Borromini has joined #openwrt-devel
<rsalvaterra> 'morning!
<thagabe> good midnight
<rsalvaterra> :)
xback has quit [Read error: Connection reset by peer]
pkgadd has quit [Ping timeout: 264 seconds]
pkgadd has joined #openwrt-devel
<Borromini> meurning
<ynezz> jow: aparcar[m]: I assume, that it should be fine for some time with this command: flock -n /var/run/collect-py.flock ionice -c 3 nice -n 19 /usr/local/bin/collect.py /home/mirror/downloads 'https://downloads.openwrt.org/{base}/{target}' /opt/firmware-selector/www
thagabe has quit [Ping timeout: 260 seconds]
<aparcar[m]> very polite collector script
<aparcar[m]> Tha ks ynezz for checking and the update
black_ant has joined #openwrt-devel
black_ant has joined #openwrt-devel
black_ant has quit [Changing host]
Borromini has quit [Ping timeout: 260 seconds]
Nick_Lowe has joined #openwrt-devel
Borromini has joined #openwrt-devel
thagabe has joined #openwrt-devel
thagabe has quit [Ping timeout: 240 seconds]
<karlp> rsalvaterra: (annnd dangole, we should certainly deal with jffs2 created by _older_ kernels of ours too! not just the kernel we are running!
<rsalvaterra> karlp: Rats. ¬_¬
<karlp> I mean, how else is upgrade from older owrt versions meant to work? :)
<rsalvaterra> I thought files were backed-up at sysupgrade, the jffs2 partition was initialised, and the files were restored afterwards.
<rsalvaterra> But… yeah. If it's done this way, we're still using the old kernel.
<karlp> hrm, I didn't think so, because I thought I only saw the long init time delays at first install, not upgrades
<karlp> but I could very easily be wrong on that.
<PaulFertser> karlp: on upgrade the tar file with configs is written as a new and only file of a new jffs2 system.
<rsalvaterra> However, that patch is a monstrosity. I'd love to put it out of its misery.
<PaulFertser> It's put on the erase marker embedded last in squashfs sysupgrade image.
<karlp> ok, don't me then, dangole was right, only current kernel matters :) sorry for the noise rsalvaterra
<rsalvaterra> And (this is mostly to aparcar[m]) after sleeping on it, I think the squashfs zstd compression should be optional, since zstd's compression ratio is lower than xz's. People with small flash will most likely still prefer xz.
<rsalvaterra> Fast boot times are nice, but I wager normal users aren't rebooting their routers so often it makes a difference.
<aparcar[m]> rsalvaterra: ack
<PaulFertser> My router just rebooted (short power outage) and it took about 7 minutes to get connected to it. That kinda sucks.
<rsalvaterra> PaulFertser: 7 minutes?! O_o
<rsalvaterra> My AirGrid M2 boots in around two minutes. With a 390 MHz 24Kc CPU.
<PaulFertser> rsalvaterra: yes. It seems like netifd , jsonsh etc scripts are very slow. So ifup events for wan and br-lan take plenty of time.
<russell--> has anyone played with the ubiquiti bullet ac? or know anything about them?
<rsalvaterra> What hardware is that?
<PaulFertser> rsalvaterra: dir-615-e4 , same thing I was trying zram on
<rsalvaterra> Oh…! Right.
<rsalvaterra> You have 4 MiB flash, so you're definitely compiling everything with -Os. That makes *a lot* of difference on MIPS, especially 24Kc.
<PaulFertser> rsalvaterra: I have 16 MiB flash now.
wvdakker has quit [Quit: wvdakker]
gch98121338 has joined #openwrt-devel
gch9812133 has quit [Ping timeout: 246 seconds]
gch98121338 is now known as gch9812133
cepxuo has quit [Quit: Bridge terminating on SIGTERM]
aparcar[m] has quit [Quit: Bridge terminating on SIGTERM]
olmari has quit [Quit: Bridge terminating on SIGTERM]
nick[m] has quit [Quit: Bridge terminating on SIGTERM]
pavlix has quit [Quit: Bridge terminating on SIGTERM]
agb[m] has quit [Quit: Bridge terminating on SIGTERM]
svanheule[m] has quit [Quit: Bridge terminating on SIGTERM]
voltagex has quit [Quit: Bridge terminating on SIGTERM]
decke[m] has quit [Quit: Bridge terminating on SIGTERM]
akanouras has quit [Quit: Bridge terminating on SIGTERM]
pgwipeout[m] has quit [Quit: Bridge terminating on SIGTERM]
fblaese has quit [Quit: Bridge terminating on SIGTERM]
MatMaul has quit [Quit: Bridge terminating on SIGTERM]
MatrixTravelerbo has quit [Quit: Bridge terminating on SIGTERM]
agb[m] has joined #openwrt-devel
anon145 has joined #openwrt-devel
Night-Shade has joined #openwrt-devel
muhaha has joined #openwrt-devel
Night-Shade has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
akanouras has joined #openwrt-devel
fblaese has joined #openwrt-devel
aparcar[m] has joined #openwrt-devel
pavlix has joined #openwrt-devel
pgwipeout[m] has joined #openwrt-devel
voltagex has joined #openwrt-devel
decke[m] has joined #openwrt-devel
olmari has joined #openwrt-devel
svanheule[m] has joined #openwrt-devel
nick[m] has joined #openwrt-devel
cepxuo has joined #openwrt-devel
MatMaul has joined #openwrt-devel
caravel has joined #openwrt-devel
victhor has joined #openwrt-devel
feriman has quit [Ping timeout: 256 seconds]
<nitroshift> anyone can tell me how i can set the board config to 1.1? (swconfig -> dsa)
<nitroshift> somehow it didn't get updated although i'm using dsa
<Borromini> nitroshift: just an -F would be sufficient afaik
<Borromini> with sysupgrade
<nitroshift> Borromini, did that, seems it didn't have any effect
anon145 has quit [Remote host closed the connection]
Nick_Lowe has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<Borromini> oh :-/
<ynezz> nitroshift: what effect it should have?
<ynezz> you need to setup DSA manually
<nitroshift> ynezz, when sysupgrading through luci, it doesn't perform the sysupgrade
<nitroshift> i'm actually running 5.9.10 atm
<nitroshift> switched to dsa since target got 5.4 support
<Borromini> ynezz: it's a check to make sure people wipe configs from swconfig to dsa
<ynezz> I still don't follow, what is wrong then?
adrianschmutzler has joined #openwrt-devel
<nitroshift> spolier: multiple lines
<nitroshift> sysupgrade -k -F openwrt-mvebu-cortexa9-linksys_wrt3200acm-squashfs-s
<nitroshift> ysupgrade.bin
<nitroshift> The device is supported, but the config is incompatible to the new image (1.0->1.1). Please upgrade without keeping config (sysupgrade -n).
<nitroshift> Config cannot be migrated from swconfig to DSA
<Borromini> nitroshift: did you tick 'wipe config'?
<Borromini> if you're using luci
<Borromini> oh
<Borromini> you need to use -n
<Borromini> that's what that warning tells you.
<nitroshift> Borromini, when first moved to dsa i sysupgraded without keeping settings
<Borromini> the check was implemented later.
<Borromini> hence why you're bumping into it.
<nitroshift> i redid all configs after
<adrianschmutzler> then you need to update the compat version manually
<adrianschmutzler> wait a sec, I once wrote a manual somewhere
<nitroshift> adrianschmutzler, that's what i was looking for
<Borromini> adrianschmutzler: /etc/board.json?
<Borromini> nitroshift: it's in there ^^
<Borromini> and i think you can just edit it
<Borromini> compat_version
<adrianschmutzler> no, you need to change the uci setting in /etc/config/system
<rsalvaterra> Hmm… I'm having the exact same problem, so I'm interested in this. All my master builds since a few days ago complain about DSA on sysupgrade.
<nitroshift> Borromini, i have 2 identical devices, the other one doesn't complain
<adrianschmutzler> the uci config is generated from the board.json. But once you have the uci config, the board.json is not looked at anymore
<adrianschmutzler> so uci set system.@system[0].compat_version="1.1"
<nitroshift> heh... compat_version in board.json *is* 1.1
* nitroshift scratches hos head
<nitroshift> *his
<adrianschmutzler> but in /etc/config/system as well?
* Borromini has no compat_version in /etc/config/system
<nitroshift> adrianschmutzler, there's nothing there about compat_version
<Borromini> oh nevermind me, i do.
<Borromini> was grepping for the wrong keys
<adrianschmutzler> nitroshift: obviously not, because /etc/config/system has been created before the update.
<adrianschmutzler> and once you add it there, it will work.
<nitroshift> right, then there's the fault
<nitroshift> shoud i add it under system paragraph?
<adrianschmutzler> Yes, or just run the command as stated
<nitroshift> ran the command, still nothing in /etc/config/system
<adrianschmutzler> you also need uci commit system after that
<adrianschmutzler> as in the article I linked
<nitroshift> adrianschmutzler, thanks, skipped that part, obviously not enough coffee :|
<nitroshift> it's there now
<nitroshift> thanks!
<adrianschmutzler> And from now on it should just work
<nitroshift> yep
<nitroshift> thanks again
black_ant has quit [Ping timeout: 240 seconds]
black_ant has joined #openwrt-devel
black_ant has quit [Changing host]
black_ant has joined #openwrt-devel
* russell-- cracked case on a ubnt unifi ac mesh, aka uap-ac-pico
Nick_Lowe has joined #openwrt-devel
Borromini has quit [Ping timeout: 260 seconds]
dedeckeh has quit [Remote host closed the connection]
Dracos-Carazza has quit [Read error: Connection reset by peer]
Dracos-Carazza has joined #openwrt-devel
black_ant has quit [Ping timeout: 265 seconds]
caravel has quit [Ping timeout: 256 seconds]
caravel has joined #openwrt-devel
black_ant has joined #openwrt-devel
black_ant has quit [Changing host]
black_ant has joined #openwrt-devel
Borromini has joined #openwrt-devel
Borromini has quit [Ping timeout: 240 seconds]
nitroshift has quit [Quit: Gone that way --->]
feriman has joined #openwrt-devel
muhaha has quit [Quit: Connection closed]
caravel has quit [Ping timeout: 272 seconds]
Borromini has joined #openwrt-devel
caravel has joined #openwrt-devel
<rsalvaterra> The more I read about jffs2, the more perplexed I get. Now I was wondering why it doesn't just use the crypto API for compression, like all good citizens…
<rsalvaterra> … turns out, someone actually started implementing it: http://www.inf.u-szeged.hu/projectdirs/jffs2/compression.php
<rsalvaterra> *15 years ago*.
<rsalvaterra> Nobody cared, I guess.
<karlp> rsalvaterra: did that api exist then?
<rsalvaterra> karlp: If the patch exists, I assume the answer is "yes".
<rsalvaterra> We're talking Linux *2.5*.
<rsalvaterra> 30th October 2002. It's old enough to drive a car.
Tapper has quit [Quit: Instantbird 1.6a1pre -- http://www.instantbird.com]
<karlp> pretty sure crypto of jffs partitions wasn't exactly a roadmap item for jffs implementors, at all, let alone using brand new kernel features...
<karlp> jow: I've got some luci controller actions() like https://paste.jvnv.net/view/nWNOs that return json blobs directly, what's the right way of putting them into a menu.d json file?
<karlp> I've made as a "view" and it gets called, but just returing the raw json doesn't make things happy, it just renders a blank inside the normal luci page: https://paste.jvnv.net/view/FlVEG
<karlp> is there some other "type" I can put in the action, or some overrides I can call or some other trick?
<karlp> the view wrapper always does this ui.instantiateView(path) thing
<karlp> template requires it to be a .htm file as well.
<rsalvaterra> karlp: I'm not talking about data encryption. The Linux crypto API is used for all kinds of data transformation, encryption and compression being two examples.
<karlp> was it used for compression _then_ ? or just reused for that later?
<karlp> it doesn't surprise me _at all_ that someone making a novel out of tree filesystem wasn't using brand new crypto api :)
<rsalvaterra> karlp: Yes, it was. In 2005, deflate was already implemented as a crypto API algorithm.
<karlp> wellll, if jffs2 was added to the kernel in 2.4.10, wasn't it the failing of the crypto api to not update the jffs2 compression layer? :)
blocktrron has quit [Ping timeout: 272 seconds]
blocktrron has joined #openwrt-devel
<rsalvaterra> karlp: In my opinion, I don't think so. Nothing was removed, just added, so all previous users still work, albeit with lots of (nowadays) unnecessary boilerplate.
Borromini has quit [Quit: Lost terminal]
<rsalvaterra> (And without the benefit of getting new algorithms "for free", when they're added to the crypto API.)
Tapper has joined #openwrt-devel
Misanthropos has quit [Ping timeout: 272 seconds]
<karlp> jow: just dropping the menu.d for now, I've converted my cbi to js view, but leaving the controller for dispatching for now, if there's a new path intended for the future, I'll try and move to it.
<karlp> do I still need an rcp.d/acl.d file if I'm not using menu.d files?
<KGB-0> https://tests.reproducible-builds.org/openwrt/openwrt_ath79.html has been updated. (99.1% images and 98.5% packages reproducible in our current test framework.)
dedeckeh has joined #openwrt-devel
gch9812133 has quit [Quit: The Lounge - https://thelounge.chat]
gch98121338 has joined #openwrt-devel
valku has joined #openwrt-devel
pgjermshus has joined #openwrt-devel
Nick_Lowe has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
Nick_Lowe has joined #openwrt-devel
luke-jr has quit [Read error: Connection reset by peer]
luke-jr has joined #openwrt-devel
muhaha has joined #openwrt-devel
jg_ has joined #openwrt-devel
Namidairo has quit [Ping timeout: 240 seconds]
muhaha has quit [Quit: Connection closed]
Namidairo has joined #openwrt-devel
luke-jr has quit [Excess Flood]
luke-jr has joined #openwrt-devel
luke-jr has quit [Read error: Connection reset by peer]
luke-jr has joined #openwrt-devel
dedeckeh has quit [Ping timeout: 245 seconds]
feriman has quit [Quit: WeeChat 3.0]
mangix has quit [Read error: Connection reset by peer]
Logsebvl has joined #openwrt-devel
Logsebvl has quit [Remote host closed the connection]
dedeckeh has joined #openwrt-devel
opal has quit [Ping timeout: 240 seconds]
opal has joined #openwrt-devel
feriman has joined #openwrt-devel
Ycarus has quit [Quit: Ycarus]
madwoota has quit [Ping timeout: 246 seconds]
madwoota has joined #openwrt-devel
madwoota has joined #openwrt-devel
rsalvaterra1 has joined #openwrt-devel
rsalvaterra has quit [Ping timeout: 272 seconds]
Borromini has joined #openwrt-devel
Nick_Lowe has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
Nick_Lowe has joined #openwrt-devel
Darkmatter66 has joined #openwrt-devel
indy has quit [Max SendQ exceeded]
bame has joined #openwrt-devel
dedeckeh has quit [Remote host closed the connection]
indy has joined #openwrt-devel
Acinonyx has joined #openwrt-devel
Nick_Lowe has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
Nick_Lowe has joined #openwrt-devel
Nick_Lowe has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
Nick_Lowe has joined #openwrt-devel
goliath has quit [Quit: SIGSEGV]
Darkmatter66 has quit [Quit: ZNC 1.7.5 - https://znc.in]
Darkmatter66 has joined #openwrt-devel
Darkmatter66 has quit [Max SendQ exceeded]
danitool has joined #openwrt-devel
Darkmatter66 has joined #openwrt-devel
Darkmatter66 has quit [Max SendQ exceeded]
Darkmatter66 has joined #openwrt-devel
rsalvaterra has joined #openwrt-devel
<xdarklight> adrianschmutzler: you have previously reviewed https://github.com/openwrt/openwrt/pull/3380 - your comments have been addressed so it would be great if you could put this on your TODO-list and merge this PR if you agree that everything is fine now. thank you!
Darkmatter66 has quit [Max SendQ exceeded]
Darkmatter66 has joined #openwrt-devel
rsalvaterra1 has quit [Ping timeout: 260 seconds]
Darkmatter66 has quit [Max SendQ exceeded]
Darkmatter66 has joined #openwrt-devel
Darkmatter66 has quit [Max SendQ exceeded]
Nick_Lowe has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
Darkmatter66 has joined #openwrt-devel
Nick_Lowe has joined #openwrt-devel
Nick_Lowe has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
Nick_Lowe has joined #openwrt-devel
opal has quit [Ping timeout: 240 seconds]
opal has joined #openwrt-devel
opal has quit [Remote host closed the connection]
opal has joined #openwrt-devel
feriman has quit [Ping timeout: 260 seconds]
ivanich has quit [Read error: Connection reset by peer]
ivanich has joined #openwrt-devel
Nick_Lowe has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
Borromini has quit [Quit: leaving]
black_ant has quit [Ping timeout: 260 seconds]
ivanich has quit [Remote host closed the connection]