<kiedtl>
KipIngram: There is an interactive repl, but input is 'compiled' and stuffed into a temporary word (anagolous to `main') before being executed.
<kiedtl>
analogous
<kiedtl>
*
<kiedtl>
I need to sleep.
veltas has quit [*.net *.split]
ovf has quit [*.net *.split]
dnm has quit [*.net *.split]
veltas has joined #forth
dnm has joined #forth
ovf has joined #forth
lispmacs has quit [Ping timeout: 240 seconds]
dave0 has joined #forth
<dave0>
maw
<KipIngram>
Ah, ok. I've seen a Forth system that does that - executes line by line, but complles the whole line before starting execution.
<KipIngram>
One advantage it offers is that control constructs like IF and so on can be used interactively.
<KipIngram>
Evening dave0.
Zarutian_HTC1 has joined #forth
Zarutian_HTC has quit [Read error: Connection reset by peer]
<dave0>
maw KipIngram
<KipIngram>
Hey dave0 - refresh my memory on that return-based direct threading? I know you "return through" the definitions, so each cell of a definition is an address of code. What is the code pointed to? For a regular definition, would it be "docol"? How do your (:) and (;) operations work?
<KipIngram>
I'm thinking there might be something you could do with the "load effective address" instruction that would help out with (:).
<KipIngram>
I suppose for primitives the addresses in a definition just point right at the primitive, right?
<dave0>
yes i keep the address of the Enter primitive in %rbp
<KipIngram>
Ah, so the code pointed to is jmp %rbp?
<dave0>
at the start of a definition it's jmp %rbp which is only 2 bytes
<dave0>
yep!
<KipIngram>
Nice. How does the (:) code recover the address of the new definition?
<dave0>
it's only 2 bytes so it works all the way down to 16 bit cells
<dave0>
ah it grabs the address from %rsp-8
<KipIngram>
Got it.
<KipIngram>
Make sense.
<dave0>
it makes execute a bit trickier
<KipIngram>
Yes, but figure-out-able.
<dave0>
but everything else is the same... and ret is only 1 byte! so it's really tiny
<KipIngram>
Right.
<KipIngram>
It feels like the best way to do a direct threaded implementation to me.
<dave0>
i felt clever to figure it out :-)
<dave0>
but i just reinvented it
<KipIngram>
So (:) grabs [sp-8], increments it, and puts it on the return stack.
<dave0>
exactly
<KipIngram>
Oh, wait - no I'm sorry - it puts rsp on the return stack. The gets [rsp-8], increments it by 2, and puts puts that in rsp.
<dave0>
hang on i'll check
<dave0>
yes, except instead of 2, i increment it a whole cell (64 bits)
<KipIngram>
Hmmm. But your new definition starts right after the jmp rbp?
<dave0>
just to make everything the same size as a cell
<KipIngram>
I see.
<KipIngram>
You have some pad.
<dave0>
well it's little endian, so the jmp %rbp is at the low address
<KipIngram>
Right.
<dave0>
yes, i waste 6 bytes, but who cares about that :-)
<KipIngram>
Well, you might be able to do something useful there.
<KipIngram>
I am thinking there's probably an lea instruction you could put there that would prevent you from having to look back at where you came from to get the new word address.
<KipIngram>
As far as I can tell, the lea instruction is the only one that can operate "PC relative."
<KipIngram>
So you could lea into rsp; you might get the whole (:) operation in those eight bytes.
<dave0>
you could use a call rbp .. oh wait no because that means writing to the stack which doesn't work
<KipIngram>
No, that writes over...
<KipIngram>
yeah.
<KipIngram>
Lemme go look at the details of LEA.
<KipIngram>
Brb...
<dave0>
it would be a neat trick to pack docol into the start of every word :-)
<KipIngram>
Ugh. Looks like LEA takes a lot of space.
<KipIngram>
That address prim is just past the ret.
<KipIngram>
I just punched a few lines into my own source and ran nasm on it.
<dave0>
i'm gonna try
<KipIngram>
Good luck, man.
<KipIngram>
Bet it'll work.
<dave0>
i got exactly what you got for that assembly
<dave0>
very interesting
<dave0>
i never thought of inlining docol
<dave0>
it wouldn't work on i386 because it doesn't have pc-relative addressing
<KipIngram>
How's this:
<KipIngram>
Oh wait - missed something.
<KipIngram>
One sec.
<KipIngram>
Nah, the LEA instruction is just too big.
<KipIngram>
It's 7 bytes.
<KipIngram>
So you really can't do anything here unless you spend two cells.
pointfree has joined #forth
<KipIngram>
That's right - i386 only has that mode for a handful of control transfer instructions. But the x64 changes made it available to many.
<KipIngram>
Since you're already spending 8 bytes, even though you're only using 2, this is a question of "is it worth 8 bytes per definition to inline (:)"
<dave0>
it's the classic computer trade-off :-)
<dave0>
speed vs memory
<KipIngram>
It also avoids that memory referennce that you have to do.
<KipIngram>
Where you read out [rsp-8]
<dave0>
oh true true
<KipIngram>
So, you could avoid a jump, and a full cell read.
<KipIngram>
Anyway, it bears thinking on. Let me know if you try it.
<KipIngram>
I'll do it once more, with aligns before and after.
<KipIngram>
Here you go - looks like it's 13 bytes to do the full inlining:
<KipIngram>
I do seem to recall xchg as being a tight instruction.
cmtptr has quit [Ping timeout: 240 seconds]
<KipIngram>
Yes - fiddle. That still wouldn't get you to 8 bytes, so it would be a matter of choosing the fastest one.
<KipIngram>
It's a damn shame it uses so many bytes in the LEA instruction even when your offset is so small.
cmtptr has joined #forth
<KipIngram>
Hmmm. Let me try something else. ;-)
<dave0>
ok :-)
<KipIngram>
No, no go. Even if you arrange the offset in lea to be zero, it still consumes all those bytes.
<KipIngram>
I was thinking it might be smaller to lea a zero offset addr and then just add.
<KipIngram>
I think you got the best one, though.
<KipIngram>
Just between you, me, and the lamppost, I'd absolutely do this. :-)
<KipIngram>
If I ever decide to try a direct threaded system, I will do it.
<dave0>
lol
<dave0>
cool :-)
<KipIngram>
Ok, that was fun.
<dave0>
i love how easy it is to experiment with forth
<KipIngram>
Yes indeed.
<KipIngram>
Maybe you've already implemented this so you know, but it won't surprise me if you find implementing CREATE / DOES> rather trying.
<KipIngram>
I've always felt like that indirect threading really makes that easier.
<dave0>
i'm not far along
<KipIngram>
So my system has a CFA/PFA pair for : definitions, each of which is a pointer.
<KipIngram>
The CFA points to code (docol), and the PFA *points to* the new definition list.
<KipIngram>
That CFA/PFA pair is up in higher memory right by the name string.
<KipIngram>
So, primitives don't need the PFA - and my primitive header macro doesn't provide one.
<KipIngram>
So that saves a bit of wasted space.
<KipIngram>
Right now my constants actually have the PFA point to a cell elseshere with the constant value in it, but I might just have the value be IN the PFA pointer cell. It's only a four-byte cell, though, so that would make it impossible (with that mechanism) to have 64-bit constants.
gravicappa has joined #forth
<KipIngram>
One thing I can do that you can't is have multiple entry points in a definition.
<KipIngram>
You can't define an entry point without having your code snip there.
<KipIngram>
Since my CFA/PFA pointers are elsewhere, I can have as many names as I wish pointing into the same list of cells.
<KipIngram>
So right now, I've got this new one to the point where it has a QUIT loop that just lets me type in lines of code - it applies BL WORD iteratively to those lines and prints individual words back at me, one per line, until it runs out, and then loops.
<KipIngram>
So next stage is to implement the dictionary lookup of the words.
<KipIngram>
And finish that QUIT loop out into an actual Forth outer interpreter.
<KipIngram>
It's already snapshotting my system before calling INTERPRET, so I'm all set to implement error detection and recovery as well (error recovery will copy the snapshot back into the working image).
<KipIngram>
So if a line I type encouters an error, it resets everything back to how it was at the start of the line.
<KipIngram>
I've got the dictionary lookup code written in a text file - just need to enter it into the assembly file.
<KipIngram>
So, I thought of a simple way to approach profiling. Might nnot be the most memory efficient, but it's easy to implement annd provides a lot of data.
f-a has joined #forth
<KipIngram>
So the idea is to allocate a memory buffer the same size as the part of my system where the code and definition lists reside. Fill it with zero data and point a spare register at it. Then overwrite the first few bytes of NEXT with a jump instruction to a "new NEXT," which does an "inc qword [<buffer_reg>+<ip_reg>]
<KipIngram>
Run my application for a while and then restore the original NEXT.
<KipIngram>
Now that buffer will be full of count values, and they'll tell me exactly what regions in my image received the most attention.
<KipIngram>
A post-processor could associate word name information and so on with all of that data.
f-a has left #forth [#forth]
<KipIngram>
If I wanted to I could reduce the buffer size by only covering a portion of the whole image, but that would mean a new NEXT with more in it. But that might be preferable; don't know yet.
pointfree has quit [Quit: Connection closed for inactivity]
<proteusguy>
KipIngram, McCabe's "Forth Fundamentals Vol I" seems to be made of unobtainium. I've never been able to get hold of a copy. Someone should scan it.
<ecraven>
there's two on amazon.de for a thousand bucks each ....
<proteusguy>
veltas, I've noted yours down for future inclusion should a future arise where I have such a page. It actually may happen this month. :-)
<proteusguy>
kiedtl, read the welcome message to #forth. If you have 2+ stacks and speak RPN then you're welcome here. We might even be a bit more flexible than that.
shmorgle has quit [Ping timeout: 265 seconds]
tech_exorcist has joined #forth
tech_exorcist has quit [Remote host closed the connection]
tech_exorcist has joined #forth
tech_exorcist has quit [Remote host closed the connection]
tech_exorcist has joined #forth
tech_exorcist has quit [Remote host closed the connection]
tech_exorcist has joined #forth
tech_exorcist has quit [Remote host closed the connection]
<KipIngram>
Well, I'd scan mine, but I can't bear to dismantle it. It's a fairly low quality paperback, and I'm pretty sure it wouldn't surfive being opened onto a flat bed to every page. :-(
<KipIngram>
Maybe I can figure out a wa to take pictures of the pages, with the book held more gently open.
<KipIngram>
Maybe with one of those camera apps that do page strightening.
<KipIngram>
It reall is an excellent book.
<proteusguy>
I have one of those scanners that flattens bent pages actually... :-)
<proteusguy>
It also does OCR.
<proteusguy>
I can't even find a reference to the author to find out if he's still alive.
<ecraven>
KipIngram: way back when I just took photos from above a book with a normal camera, still better than *not* having it at all
<nihilazo>
I've heard things about using forth as a macroassembler, how does that work? Could I use a forth as a cross-macro-assembler?
<nihilazo>
(wanting to experiment with uxn but the official assembler is insanely ugly)
<nihilazo>
(wanting to experiment with uxn but the official assembler is insanely ugly and forth is incer)
tech_exorcist has joined #forth
<KipIngram>
ecraven: I've got an app that straightens pages on my android - I'll take a look into it.
<ecraven>
KipIngram: that'd be great. but anything is better than nothing at all, so if you can share just plain pictures, someone else can look into prettying things up ;)
<KipIngram>
Yep - remind me in a few days if I haven't said anything more about it.
rpcope has quit [Ping timeout: 260 seconds]
rpcope has joined #forth
f-a has joined #forth
<proteusguy>
nihilazo, what cpus would you be targeting. For ones less complex than the x86 then yes assemblers in forth can be quite practical. Beyond that, an assembler for complex CPUs is as big a project as the forth itself sometimes.
<nihilazo>
the CPU is the UXN virtual machine, which is very simple
<nihilazo>
I feel like maybe building a forth assembler for uxn might be less easy becasue uxn is itself a stack machine? Although that might make writing a forth assembler easier
<nihilazo>
actually it shouldn't be hard
<nihilazo>
I'm just not sure how to deal with uxn's callback things and stuff
<nihilazo>
(to be honest, I don't understand how the normal assembler works lol)
tech_exorcist has quit [Quit: tech_exorcist]
tech_exorcist has joined #forth
<veltas>
I think it would be easier
<veltas>
proteusguy: nice
<nihilazo>
I mean, I think it might be easier?
<nihilazo>
idk tbh
<veltas>
I don't see how it could be harder
<veltas>
No operands to specify will simplify things
<veltas>
Even simple assembly like Z80 is a lot of work
<veltas>
Well Z80 is one of the more complicated 8-bit architectures
<nihilazo>
the thing I'm least sure how to do is that the uxn expects certain things in certain places
<KipIngram>
You can write a "restricted" x64 assembler though, without it becoming a massive headache. I've done up the moves, conditional and unconditional jumps, and arithmetic / logic instructions.
<KipIngram>
You really don't need that crazy big instruction set for 99.9% of what you might want to do.
<KipIngram>
In my case I want just enough assembly to write my "virtual instruction" layer.
<nihilazo>
like, it seems to have three locations (0200, 0204, 0208) which are called RESET, ERROR, and FRAME in the demo code
<nihilazo>
and I think the uxn VM executed specifically what is at those locations
<nihilazo>
but idk how to do that with forth given that labels are harder
<nihilazo>
(the ROM file starts at 0200)
<KipIngram>
It took about 2 4k disk blocks, used inefficient like old Forth, to hold that partial assembler.
<KipIngram>
Maybe 2 and a half.
<KipIngram>
If you really think on the addressing modes, you can render them in an algorithm fairly cleanly.
gravicappa has joined #forth
<nihilazo>
hmm, yeha
<nihilazo>
I think the main thing idk about right now is how to deal with those things that eeds to be at specific addresses. In all of the demo code they're jumps to a few different labels
<nihilazo>
(in the existing uxn assembler, |0200 means "at 0200")
<nihilazo>
^whatever is the realative address of a label ,whatever is an absolute address of a label
<nihilazo>
it's a very odd asm
Zarutian_HTC1 has quit [Read error: Connection reset by peer]
Zarutian_HTC has joined #forth
<nihilazo>
like, I read that thing about building an assembler in forth that f-a sent but I'm not sure how to deal with having to have certain things be at certain addresses in a forth assembler
<nihilazo>
maybe I'm overthinking it or missing an obvious trick
<KipIngram>
That doesn't really come up - generally speaking the instructions you actually use (to write primitives) just operate on the stack and things obtained from the stack.
<KipIngram>
Everywhere in memory you need to get to you will have a pointer to laying around somewhere.
<KipIngram>
I think a appropos comment would be to say that you really don't need to deal with labels in Forth assembly.
<KipIngram>
The Forth dictionary supplies all your "memory roadmap" needs.
<KipIngram>
Well, you might think of using a label for the short jumps in conditional primitives, but I implemented that with something similar to how IF THENN is handled.
<KipIngram>
leave the backpatch address on the stack, and fill in its content when you know what it needs to be.
<KipIngram>
Honestly, for primitives, though, you could just count bytes and code jump distances manually.
<KipIngram>
You don't wind up writing some big sprawling thing in assembly - it's just a whole flock of short little snips.
<KipIngram>
Lemme see if I can get some syntax examples for you.
<KipIngram>
Here are the two main blocks of my assembler - there's some usage of it near the end.
<KipIngram>
If you want it you can scratch my code generator algorithm out of that.
<KipIngram>
I might use some words you're not familiar with, that I cooked up myself - feel free to ask questions.
<KipIngram>
I'm about to go on the road for the bulk of the day, though, so it will probably be tonight before I can answer.
<nihilazo>
thanks
<nihilazo>
I understand "The Forth dictionary supplies all your "memory roadmap" needs." for things i'm defining myself but I'm not sure for things the machine itself needs to be at specific addresses
<KipIngram>
Like what, for example?
<KipIngram>
I do some system calls in mine, for keyboard / disk/ display service.
<KipIngram>
And I allocate a couple of blocks with syscalls, right at the beginning, which I then use for the system.
<nihilazo>
on uxn, the machine has specific addresses it looks for to start, so it starts executing at 0200, and then executes starting at 0208 every frame it draws
<KipIngram>
Oh, well, those are just numbers, right?
<KipIngram>
You have to code them into your startup code.
<KipIngram>
In Forth variables leave an address on the stack, but nothing stops you from creating a constant and then treating it like a variable.
<nihilazo>
I'm confused
<KipIngram>
It's one of the great things - you can just poke at anything you want to.
<nihilazo>
this assembler isn't a forth running on the machine, it's a forth running on x86 that I am using to write an assembler for uxn
<KipIngram>
Ah, you're cross-compiling.
<KipIngram>
Ok, well, the same applies - you have to hard code target resource access. Only downside is you just can't test it incrementally very well
<nihilazo>
I want to write an assembler for it, and I wanted to try writing a forth assembler
<KipIngram>
Oh, cool. Ok. Well, like I said I'm hitting the road here any minute, but I'll look that over later. I'd be glad to give you some counsel if you want it.
<nihilazo>
thanks
<KipIngram>
I think it'll turn out to be easier than you think - something will go click and then you'll have it down.
<KipIngram>
Should get back here arounnd 5pm (seven hours).
<nihilazo>
ok
<nihilazo>
I might just write a header into the file that has those pointers in it? idk
<nihilazo>
it says that defining works analogous to those on the dictionary for the target machine is good to do
<nihilazo>
but do I need to have a dictionary then?
<nihilazo>
or do I need to make a seperate dictionary for the target stuff?
<MrMobius>
nihilazo, what kind of assembler? you could use a format that looks like traditional assembly in your source file and write a forth program to assemble that
<MrMobius>
or make forth words that output binary data so the assembly source is read into the forth prompt
<nihilazo>
I was initially going to go for the first but then somebody said about the second
<nihilazo>
the second seems more powerful but it would be hard to do some specific things i want to do?
<nihilazo>
idk
<MrMobius>
the second is less flexible and potentially harder to read the source but easier to implement
<nihilazo>
hmm, yeah
<nihilazo>
I would like to have some higher level constructs like an IF ... ELSE ... THEN and some other stuff
<nihilazo>
idk
<nihilazo>
and being able to use all of forth to create macros in the second type seems very powerful
<nihilazo>
really I'm not sure what i want to do exactly
<nihilazo>
the one described on that page seems to be the second type
<veltas>
Easier to implement is usually the best choice. Forth programs empower users more when they're shorter and easier to understand
<veltas>
If there is a point to Forth that is one of the points
<nihilazo>
true
<veltas>
It's got a very open source spirit in a way
<nihilazo>
the thing with doing it in the way where the assembler is inside forth is cool
<nihilazo>
but I'm not sure how to put stuff at very specific addresses without doing jumps
<nihilazo>
s/jumps/labels/
<veltas>
You can have labels
<veltas>
Just don't centre the universe around them
<nihilazo>
I'm not sure how to do that in an assembler that's inside forth
<nihilazo>
I guess?
<nihilazo>
idk
<nihilazo>
really I only need one thing (being able to have a few jumps at specific places that go to parts of the code I actually write)
<nihilazo>
just not sure the best way to do that
<nihilazo>
I'm probably overthinking it
dave0 has quit [Quit: dave's not here]
Zarutian_HTC1 has joined #forth
Zarutian_HTC has quit [Read error: Connection reset by peer]
Zarutian_HTC1 has quit [Ping timeout: 260 seconds]
gravicappa has quit [Ping timeout: 260 seconds]
gravicappa has joined #forth
<nihilazo>
OK I think building a good forth assembler is beyond my abilties rn
<nihilazo>
just like, have a huge CREATE [whatever] and allot 64k to it because that's the max program size?
<nihilazo>
I'm confused
tech_exorcist has quit [Ping timeout: 265 seconds]
tech_exorcist has joined #forth
tech_exorcist has quit [Remote host closed the connection]
Zarutian_HTC has joined #forth
tech_exorcist has joined #forth
tech_exorcist has quit [Remote host closed the connection]
tech_exorcist has joined #forth
gravicappa has quit [Ping timeout: 240 seconds]
tech_exorcist has quit [Read error: Connection reset by peer]
f-a has joined #forth
tech_exorcist has joined #forth
shmorgle has joined #forth
f-a has quit [Read error: Connection reset by peer]
f-a has joined #forth
f-a has quit [Quit: leaving]
<KipIngram>
Hi guys. I'm home.
<KipIngram>
nihilazo - going to look at that website.
<KipIngram>
That's a rather strange looking assembly language.
<KipIngram>
nihilazo: What your were told before, about the 64k buffer and so on. When you cross compile for another system, you must distinguish the stuff you mean to go to your target vs. the stuff you mean to act on your host. The 64k buffer would be intended to be a memory image of what your target's memory will look like when you load and run your program.
<KipIngram>
Somehow you have to specify a starting place, where your compiled code will begin. In normal Forth systems that piece of information lives in a variable DP and the word HERE does DP @.
<KipIngram>
For cross compiling you would have something like target_dp.
<KipIngram>
That's a variable in the host system, that tells you where to put the next byte of machine code in the target system image.
<KipIngram>
You create Forth words (that run on your host) that produce machine code bytes for the target. Whenever you execute one of those, it pokes each byte it produces into the target image location at the next location.
<KipIngram>
So, say you have the next byte of target assembl on the stack. You'd have something like
<KipIngram>
If you are going to need to jump somewhere in your image, you have to note down somewhere what the target is (just save the value in target_dp when that byte is generated), and then you can put that saved address into the jump instruction as the target.
<KipIngram>
This is really as straightforward as it can be, once you get the hang of it.
<KipIngram>
The hard part is understanding your target instruction set well enough to craft a set of words that will be your assembly opcodes that produce the correct bytes.
<KipIngram>
Usually the opcode itself is easy. In your target it looks like that produce 5 bits of your instruction byte. Then you have to also craft a lexicon for expressing the addressing mode that gives you the other three bits. You'll then type some set of words like <opcode> <mode_info> and it will put the byte together and then it will assemble it.
<KipIngram>
I can try to answer questions if you have any.
<KipIngram>
nihilazo: In some of the earlier assembles I used way back in the day there would be an ORG directive - for origin. If your processor started executing code at address 0x200, then you'd just stick an "org 0x200" in there. The processor starts up the way it does because the hardware is designed that way, and you just "accommodate it." Give it what it wants.
<KipIngram>
The assembler can't figure out where to start, but it can move the pointer along correctly as you generate bytes of content.
<KipIngram>
It looks like the least significant 5 bits of your opcode bytes are the opcode. There's a table of them here:
<KipIngram>
So I'd start by defining words that take their names from that table near the start of that web page - for instance,
<KipIngram>
: SWP 5 ;
<KipIngram>
So for each instruction you'll start by giving the opcode. Then you'll follow that with words that specify the address mode in some convenient way (yet to be worked out) and OR those bits into the top three bit locations of the 5-bit value you start with.
<KipIngram>
I haven't sussed out the address modes yet.
<KipIngram>
Hmmm. Looks like it's got a rather interesting way of managing the return stack.
<KipIngram>
Ok, so opcode bits are bits [04:00]. Bit 05 is a long/short specifier - looks like if it's set you use the top two bytes of the stack as a 16-bit address. If bit 05 is clear, you use the top one byte of the stack as a *relative* address. I guess that's relative to the instruction pointer.
<KipIngram>
Looks like you specify that in their suggested assembly langage by appending 2 to the opcode. So LIT or LIT2,
<KipIngram>
Not sure yet what the meaning of that would be for all opcodes.
<KipIngram>
Maybe SWP is swap th top two bytes of the stack, SWP is swap ht top two 16-bit pairs?
<KipIngram>
Bit 06 specifies the return stack as the target of the operation.
<KipIngram>
Looks like you append r to the mnemonic for that.
<KipIngram>
So now we can have LIT, LIT2, LITr, and LIT2r
<KipIngram>
nihilazo: This is rather different from Forth, even though it is a stack machine. These code bytes have an "encoded structure," where bits in the opcode bytes mean specific things. In Forth the number you lay down is an address related to where that generating word is in memory - it's just the address that particular routine happens to sit at. So there's no correllation whatsoever between the code bytes and
<KipIngram>
what the code bytes do.
<KipIngram>
Forth words that are written in machine code are usually called "primitives," and they can do absolutely anything you want them to.