evocation/amd64.e

~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~ ~~ Assembly language for the AMD64 architecture ~~
~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~
~   This is also often called the x86-64 architecture, but Intel didn't
~ invent it (they had their chance) and there's no reason to name it after
~ their product line. We have a bunch of assembler words that, taken as a
~ whole, form a sort of assembly language inside of the Forth-style language.
~
~   It's all backwards and stuff.
~
~   Okay, but seriously, the convention is: target on the top of the stack,
~ source behind it. This is similar to how the Forth "!" and "@" words work.
~
~   These routines use the binary packing routines such as pack64, defined in
~ core.e. They're called in the same way: an output address which we call the
~ "output point", followed by data items specific to what's being output. They
~ also chain together in the same way, returning the updated output point.
~
~   TODO cite the Intel reference manual here and explain the notation used
~ for the section citations below
~
~   TODO define instructions, assembly code, machine code, opcodes. if we ever
~ also want to recommend a childrens' introduction to binary, this might be
~ the place to do it.
~
~
~ Hex-transformed output
~ ~~~~~~~~~~~~~~~~~~~~~~
~
~   In addition to the actual semantics of the assembly words, there's an
~ important layer of meaning atop them. The hex transform, described in detail
~ in transform.e, modifies a compilation process to produce a commented hex
~ dump instead of a binary. It includes a magic-comment feature which allows
~ words that have opinions on what their output's hex dump should look like to
~ express them.
~
~   The code in this file makes extensive use of the magic-comment feature.
~ Any comment which starts with : as its first word is a magic comment meant
~ for the hex transform. When the hex transform is not in use, these are
~ ordinary comments and are ignored. When it is in use, they create and modify
~ metadata annotations in a complex way.
~
~   The magic comments in this file follows two important conventions: First,
~ entries are created by whichever word directly corresponds to a top-level
~ assembly instruction from the perspective of someone programming with it,
~ at some point prior to calling any word which might pack output. Second,
~ output bytes are accounted for by whichever word directly calls a pack* word
~ for them.
~
~   Thus, for example, the word "syscall" has a comment which creates an entry
~ describing its length as 2 bytes, which never varies.
~
~   The word "mov-reg64-imm32", on the other hand, has a comment which
~ creates an entry describing is length as 5 bytes - the opcode and the
~ immediate value, which are output with calls to pack8 and pack32 in that
~ word. However, the actual length of the instruction is 5 bytes. When
~ mov-reg64-imm32 calls rex-w, a magic comment there modifies the most recent
~ entry in-place, marking it as being one byte longer. Subsequently,
~ mov-reg64-imm32 calls addressing-reg64, which calls modrm, which again
~ modifies the same entry. Notice that addressing-reg64 didn't need to do
~ anything, because it isn't directly responsible for any packing.
~
~   Notice that when an entry is modified in place, it needs to have been
~ created first. Attempting to do it in the wrong order will have an
~ undesired effect, most likely editing the entry of an unrelated instruction.
~ The hex transform doesn't have detailed knowledge of assembly, and isn't in
~ a position to know that the edits that are happening are sensible.
~
~   Hopefully these examples show why an accounting convention is needed. The
~ good news is that the convention is quite robust and needs essentially no
~ play-by-play commentary, just this explainer at the top.


~ Keywords
~ ~~~~~~~~
~
~   We define a bunch of keywords, which evaluate to their own codeword
~ addresses. We use these to refer to registers and condition codes by name.
~
~
~ On registers
~ ~~~~~~~~~~~~
~
~   The x86 architecture has been around a while, it has been through
~ several transitions from smaller word sizes to larger ones. Therefore it
~ has different names for the "same" registers, depending on how much of
~ them you're using.
~
~ TODO there's more to write here

~   The names of the 64-bit registers. The second half of these are considered
~ "extended" registers because they don't correspond to 32-bit registers in
~ the way the first eight do.
s" :rax" keyword
s" :rcx" keyword
s" :rdx" keyword
s" :rbx" keyword
s" :rsp" keyword
s" :rbp" keyword
s" :rsi" keyword
s" :rdi" keyword
s" :r8" keyword
s" :r9" keyword
s" :r10" keyword
s" :r11" keyword
s" :r12" keyword
s" :r13" keyword
s" :r14" keyword
s" :r15" keyword

~   The names of the 32-bit registers. The processor treats these as being
~ alternate names for the low halves of the 64-bit registers. There is a
~ very finicky distinction about what that means in different settings: Some
~ instructions operate on a 32-bit source or target, while others merely
~ accept a 32-bit value that gets sign-extended to 64 bits. We've taken pains
~ to clarify these cases in the instruction-specific notes, as they come up.
s" :eax" keyword
s" :ecx" keyword
s" :edx" keyword
s" :ebx" keyword
s" :esp" keyword
s" :ebp" keyword
s" :esi" keyword
s" :edi" keyword

~   The names of the 16-bit registers. Similarly, the processor treats these
~ as being alternate names for the low halves of the 32-bit registers.
s" :ax" keyword
s" :cx" keyword
s" :dx" keyword
s" :bx" keyword
s" :sp" keyword
s" :bp" keyword
s" :si" keyword
s" :di" keyword

~   The names of the 8-bit registers. The pattern here is a little bit
~ different; these come in "low" and "high" pairs, where for example :al is
~ the low half of :ax and :ah is the high half. Yes, this architecture grows
~ like a tree, with all the old things being still present, surrounded in the
~ new ones.
s" :al" keyword
s" :cl" keyword
s" :dl" keyword
s" :bl" keyword
s" :ah" keyword
s" :ch" keyword
s" :dh" keyword
s" :bh" keyword

~   The condition codes. Yes, there sure is a lot of duplication in these
~ names. The names are based on Intel's documented mnemonics...
~
~   "Above" and "below" are for unsigned comparisons. "Greater" and "less" are
~ for signed comparisons.
~
~   This is documented on the individual opcode pages, and also in B.1.4.7.
s" :cc-overflow" keyword
s" :cc-no-overflow" keyword
s" :cc-below" keyword
s" :cc-above-equal" keyword
s" :cc-equal" keyword
s" :cc-not-equal" keyword
s" :cc-below-equal" keyword
s" :cc-above" keyword
s" :cc-sign" keyword
s" :cc-not-sign" keyword
s" :cc-even" keyword
s" :cc-odd" keyword
s" :cc-less" keyword
s" :cc-greater-equal" keyword
s" :cc-less-equal" keyword
s" :cc-greater" keyword


~ Bits and pieces
~ ~~~~~~~~~~~~~~~
~
~   Here, we have a bunch of helpers which generate specific encoded
~ representations that are part of many instructions. We start with the
~ trivial ones that handle individual fields, then work up to combinations of
~ fields.
~
~   When we say that a word accepts a register as a parameter, what we mean
~ is it accepts the name keyword for that register. When we say that a word
~ accepts a scale factor, what we mean is that it accepts a byte count for
~ that scale factor. In the cases where we mean the encoded form, we'll say
~ "encoded value" or "value".
~
~ (register -- 3-bit encoded value for register)
: reg64
  ~ : provide-keyword
  dup :rax = { drop 0 exit } if
  dup :rcx = { drop 1 exit } if
  dup :rdx = { drop 2 exit } if
  dup :rbx = { drop 3 exit } if
  dup :rsp = { drop 4 exit } if
  dup :rbp = { drop 5 exit } if
  dup :rsi = { drop 6 exit } if
  dup :rdi = { drop 7 exit } if
  ." Parameter to reg64 is not a reg64." 1 sys-exit ;

~ (register -- 3-bit encoded value for register)
: extrareg64
  ~ : provide-keyword
  dup :r8 = { drop 0 exit } if
  dup :r9 = { drop 1 exit } if
  dup :r10 = { drop 2 exit } if
  dup :r11 = { drop 3 exit } if
  dup :r12 = { drop 4 exit } if
  dup :r13 = { drop 5 exit } if
  dup :r14 = { drop 6 exit } if
  dup :r15 = { drop 7 exit } if
  ." Parameter to extrareg64 is not an extrareg64." 1 sys-exit ;

~ (register -- 3-bit encoded value for register)
: reg32
  ~ : provide-keyword
  dup :eax = { drop 0 exit } if
  dup :ecx = { drop 1 exit } if
  dup :edx = { drop 2 exit } if
  dup :ebx = { drop 3 exit } if
  dup :esp = { drop 4 exit } if
  dup :ebp = { drop 5 exit } if
  dup :esi = { drop 6 exit } if
  dup :edi = { drop 7 exit } if
  ." Parameter to reg32 is not a reg32." 1 sys-exit ;

~ (register -- 3-bit encoded value for register)
: reg16
  ~ : provide-keyword
  dup :ax = { drop 0 exit } if
  dup :cx = { drop 1 exit } if
  dup :dx = { drop 2 exit } if
  dup :bx = { drop 3 exit } if
  dup :sp = { drop 4 exit } if
  dup :bp = { drop 5 exit } if
  dup :si = { drop 6 exit } if
  dup :di = { drop 7 exit } if
  ." Parameter to reg16 is not a reg16." 1 sys-exit ;

~ (register -- 3-bit encoded value for register)
: reg8
  ~ : provide-keyword
  dup :al = { drop 0 exit } if
  dup :cl = { drop 1 exit } if
  dup :dl = { drop 2 exit } if
  dup :bl = { drop 3 exit } if
  dup :ah = { drop 4 exit } if
  dup :ch = { drop 5 exit } if
  dup :dh = { drop 6 exit } if
  dup :bh = { drop 7 exit } if
  ." Parameter to reg8 is not a reg8." 1 sys-exit ;


~   There's a packed format called the SIB byte, which we'll get to in a
~ second. One of its bitfields is called the scale field. This word produces
~ an encoded value for that field.
~
~   The input value is a byte count; the output value is suitable for use in
~ the SIB byte.
~
~ (scale factor -- 2-bit encoded value)
: scalefield
  ~ : provide-decimal
  dup 1 = { drop 0 exit } if
  dup 2 = { drop 1 exit } if
  dup 4 = { drop 2 exit } if
  dup 8 = { drop 3 exit } if
  ." Parameter to scalefield is not 1, 2, 4, or 8." 1 sys-exit ;


~ [Intel] volume 2D, appendix B, section B-1.4.7, table B-10. Also see the
~ individual opcode pages.
~
~   Every instruction has an "opcode", a specific byte or sequence of bytes
~ which uniquely identifies the combination of operation, addressing mode,
~ and certain miscellaneous characteristics. This is not just another way of
~ referring to the entire sequence of bytes corresponding to the instruction;
~ the opcode is a specific part within that, as distinct from ie. the rex
~ byte, the SIB byte, the Mod/RM byte, and various immediate values and other
~ rare tidbits.
~
~   Some of these opcodes have bitfields within them, to specify condition
~ codes. This word produces an encoded value for that condition-code field.
~
~ (condition -- 4-bit encoded value)
: condition-code
  ~ : provide-keyword
  dup :cc-overflow = { drop 0 exit } if
  dup :cc-no-overflow = { drop 1 exit } if
  dup :cc-below = { drop 2 exit } if
  dup :cc-above-equal = { drop 3 exit } if
  dup :cc-equal = { drop 4 exit } if
  dup :cc-not-equal = { drop 5 exit } if
  dup :cc-below-equal = { drop 6 exit } if
  dup :cc-above = { drop 7 exit } if
  dup :cc-sign = { drop 8 exit } if
  dup :cc-not-sign = { drop 9 exit } if
  dup :cc-even = { drop 10 exit } if
  dup :cc-odd = { drop 11 exit } if
  dup :cc-less = { drop 12 exit } if
  dup :cc-greater-equal = { drop 13 exit } if
  dup :cc-less-equal = { drop 14 exit } if
  dup :cc-greater = { drop 15 exit } if
  ." Parameter to condition-code is not a condition code." 1 sys-exit ;


~   The "rex" byte appears before an opcode to modify its behavior in various
~ ways. It has four distinct bits within it, leading to sixteen variations,
~ as you can see.
~
~   The way these are all spelled out like this is slightly ridiculous, there
~ must be a better way. We only ever use rex-w and rex-wb, so it's tempting to
~ get rid of the rest, but they're worth having so that our future selves
~ don't have to revisit this topic.
~
~ (output point -- output point)
: rex-0
  ~ : 1 adjust-length
  0x40 pack8 ;
: rex-w
  ~ : 1 adjust-length
  0x48 pack8 ;
: rex-r
  ~ : 1 adjust-length
  0x44 pack8 ;
: rex-x
  ~ : 1 adjust-length
  0x42 pack8 ;
: rex-b
  ~ : 1 adjust-length
  0x41 pack8 ;
: rex-wr
  ~ : 1 adjust-length
  0x4C pack8 ;
: rex-wx
  ~ : 1 adjust-length
  0x4A pack8 ;
: rex-wb
  ~ : 1 adjust-length
  0x49 pack8 ;
: rex-rx
  ~ : 1 adjust-length
  0x46 pack8 ;
: rex-rb
  ~ : 1 adjust-length
  0x45 pack8 ;
: rex-xb
  ~ : 1 adjust-length
  0x43 pack8 ;
: rex-wrx
  ~ : 1 adjust-length
  0x4E pack8 ;
: rex-wrb
  ~ : 1 adjust-length
  0x4D pack8 ;
: rex-wxb
  ~ : 1 adjust-length
  0x4B pack8 ;
: rex-rxb
  ~ : 1 adjust-length
  0x47 pack8 ;
: rex-wrxb
  ~ : 1 adjust-length
  0x4F pack8 ;


~   Some opcodes use their low three bits as a field to give a register name.
~ This is usually in addition to a register name given in a Mod/RM byte,
~ serving a different role for the instruction.
~
~   This word accepts an opcode byte with those three bits clear, and combines
~ it with a register value, then outputs the resulting byte. Each opcode
~ accepts some specific kind of register; to allow different kinds, here we
~ expect the step of converting the register name to the encoded bits to have
~ already been done.
~
~ (output point, 3-bit encoded value for register, opcode byte
~  -- output point)
: opcodereg
  ~ : 1 adjust-length
  | pack8 ;


~   Some opcodes use their low four bits as a field to give a condition code.
~ This word accepts an opcode byte with those four bits clear, and combines it
~ with a condition code value, then outputs the resulting byte. For
~ consistency with opcodereg, we expect the step of converting the condition
~ code name to the encoded bits to have already been done.
~
~ (output point, 4-bit encoded value for condition code, opcode byte
~  -- output point)
: opcodecc
  ~ : 1 adjust-length
  | pack8 ;


~   A Mod/RM byte ("mode / register-or-memory") is part of the encoding of
~ many instructions. It's divided into three fields: "mod" (mode),
~ register/opcode, and register/memory ("RM").
~
~   This word outputs a Mod/RM byte given fully-processed, numeric values for
~ its fields. Most code will want to call one of the higher-level
~ addressing-* words, instead.
~
~ (output point, mod field, register/opcode field, register/memory field
~  -- output point)
: modrm
  ~ : 1 adjust-length
  swap 8 * | swap 64 * | pack8 ;

~   An SIB byte ("scale, index, base") is part of the encoding of many
~ instructions. It's divided into three fields, with the names you've already
~ guessed.
~
~   This word outputs an SIB byte given fully-processed, numeric values for
~ its fields.
~
~ (output point, scale field, index field, base field -- output point)
: sib
  ~ : 1 adjust-length
  swap 8 * | swap 64 * | pack8 ;


~ Addressing modes
~ ~~~~~~~~~~~~~~~~
~
~   These are higher-level words meant to be easier to use than the bits and
~ pieces above. Each corresponds to some specific addressing mode. When
~ applicable, they accept keywords rather than pre-encoded values.
~
~   That's not all the time, because there are cases, such as the reg/op
~ field, where the meaning is up to the individual instruction. In those
~ cases, these words do accept fully-processed, numeric values.
~
~   The general rule is that the responsibility of these addressing-mode words
~ is for the parts that are common to all instructions using that addressing
~ mode.


~   The simplest of the addressing modes: Direct register addressing. There
~ are no special cases to check.
~
~   It's important to notice that the R/M field may describe either a source,
~ or a target, depending on what the instruction is. So, this helper doesn't
~ get to know that. It also doesn't get to know whether the value in the
~ reg/op field describes a register, or if instead it's an extension of the
~ opcode. The caller is responsible for figuring that all out.
~
~ (output point, reg/op field value, reg/mem field register
~  -- output point)
: addressing-reg64 reg64 3 3unroll modrm ;
: addressing-reg8 reg8 3 3unroll modrm ;


~   This is a helper for assembly instructions that want to do a form of
~ addressing that requires a value of 1 in the modrm byte's mode field, and
~ do not want to do any indexing. That's the indirect mode, which takes a
~ 64-bit register, treats it as an address, and looks up the 64-bit value it
~ points to.
~
~   The helper's main responsibility is to deal with the scenario that
~ requires an SIB byte, which happens when the R/M field has a value of 4,
~ which would otherwise refer to the register rsp. In that situation, it also
~ generates an SIB byte which indicates a scale of 1, no indexing, and rsp as
~ the base register.
~
~   When the register is :rbp, the only modes available also have
~ displacement; we disallow that. For that case, use an instruction that
~ uses a disp8 mode, and set a displacement of 0.
~
~   In understanding this, pay close attention to the Op/En column in the
~ opcode table. The "RM" variant means the ModRM byte's R/M field (the third
~ one) is the source, while its reg field (the middle one) is the target. This
~ is what we want, because the R/M field is the one that gets indirection
~ applied to it. Opcode 0x8B with an REX.W prefix is the all-64-bit RM
~ variant. [Intel] volume 2B, chapter 4, section 4-3, "MOV".
~
~   For the indirection modes, don't be confused by the many similar tables.
~ 64-bit mode is encoded the same as 32-bit mode except for adding a REX.W
~ prefix, as per 2.2.1.1, so you want table 2-2 to understand the ModRM byte.
~ The presence or absence of an SIB byte is determined by where in that table
~ we fall, and we aren't using a mode that has one. [Intel] volume 2A,
~ chapter 2, section 2-1.5, table 2-2.
~
~ (output point, reg/op field value, reg/mem field register
~  -- output point)
: addressing-indirect-reg64
  ~ Exit with an error if the R/M register is :rbp.
  dup :rbp != {
    ~ Check whether the R/M register is :rsp. Save the test result for later.
    dup :rsp = 4 unroll
    ~ (equality result, output point, reg/op value, reg/mem name)
    reg64 0 3unroll modrm
    ~ (equality result, output point)
    ~ If the R/M register was rsp, we need an SIB byte; otherwise, skip it.
    swap {
      0 4 :rsp reg64
      ~ : drop-subitem
      sib
    } if
    exit
  } if
  ." R/M parameter to addressing-indirect-reg64 is :rbp." 1 sys-exit ;

~ (output point, reg/op field value, reg/mem field register,
~  displacement value -- output point)
: addressing-disp8-reg64
  ~ This mode can do :rbp fine, so no need to check for that.
  ~ Check whether the R/M register is :rsp. Save the test result for later.
  swap dup :rsp = 5 unroll swap
  ~ Stash the displacement value out of the way, too.
  4 unroll
  reg64 1 3unroll modrm
  ~ If the R/M register was rsp, we need an SIB byte; otherwise, skip it.
  3roll {
    0 4 :rsp reg64
    ~ : drop-subitem
    sib
  } if

  ~ The displacement byte.
  ~
  ~   When addressing-disp8-reg64 references a source, as in
  ~ lea-reg64-disp8-reg64, the first two parameters of the assembly word
  ~ calling it are the source register and the source displacement value. When
  ~ it references a target or destination, as in mov-disp8-reg64-reg64, the
  ~ LAST two parameters are the target registera and target displacement
  ~ value.
  ~
  ~   In both cases, the register comes first and the displacement value
  ~ comes second. When these are used in a comment template without any other
  ~ rearranging going on, whichever is on the top of the stack will be popped
  ~ off first. So, we swap them here, thus putting the register on the top.
  swap
  ~ : 1 adjust-length
  ~ : provide-hex8
  ~ : swap-subitems
  pack8 ;

~ (output point, reg/op field value, reg/mem field register,
~  displacement value -- output point)
: addressing-disp32-reg64
  ~ This mode can do :rbp fine, so no need to check for that.
  ~ Check whether the R/M register is :rsp. Save the test result for later.
  swap dup :rsp = 5 unroll swap
  ~ Stash the displacement value out of the way, too.
  4 unroll
  reg64 2 3unroll modrm
  ~ If the R/M register was rsp, we need an SIB byte; otherwise, skip it.
  3roll {
    0 4 :rsp reg64
    ~ : drop-subitem
    sib
  } if

  ~   The displacement value. This is subject to the same considerations as in
  ~ addressing-disp8-reg64.
  swap
  ~ : 4 adjust-length
  ~ : provide-hex32
  ~ : swap-subitems
  pack32 ;

~ (output point, reg/op field value,
~  scale factor, index register, base field register
~  -- output point)
: addressing-indexed-reg64
  ~ Exit with an error if the base register is :rbp.
  dup :rbp != {
    ~ Reg/mem value 4 means to use an SIB byte (at least, with this mode).
    5 roll 0 6 roll 4 modrm 4 unroll
    reg64 3unroll reg64 3unroll scalefield 3unroll
    ~   We want the scale factor to show after the registers, so we put it
    ~ last on the stack. We also want the base register to show before the
    ~ index register, so the base comes first.
    ~
    ~   Importantly, we have to do this before the sib so it'll count as being
    ~ inside the span the assembly word's description attaches to.
    ~
    ~ : -3 roll-subitems
    ~ : swap-subitems
    sib
    exit
  } if
  ." Base parameter to addressing-indexed-reg64 is :rbp." 1 sys-exit ;

~ (output point, reg/op field value,
~  scale factor, index register, base field register,
~  displacement value -- output point)
: addressing-disp8-indexed-reg64
  ~ This mode can do :rbp fine, so no need to check for that.
  ~ Reg/mem value 4 means to use an SIB byte (at least, with this mode).
  6 roll 1 7 roll 4 modrm 5 unroll
  5 unroll reg64 3unroll reg64 3unroll scalefield 3unroll sib
  swap
  ~ : 1 adjust-length
  ~ : provide-hex8
  ~
  ~   We want the parameters to be shown in the order:
  ~
  ~ base register, index register, index scale factor, displacement
  ~
  ~   So, we put them on the stack in the opposite of that order.
  ~
  ~   Importantly, we have to do this before the pack8 so it'll count as being
  ~ inside the span the assembly word's description attaches to.
  ~
  ~ : -4 roll-subitems
  ~ : -3 roll-subitems
  ~ : swap-subitems
  pack8
  ;


~ Easy instructions
~ ~~~~~~~~~~~~~~~~~
~
~   It's not worth pretending there's a coherent category behind this
~ grouping. These are the ones that were easy to deal with.

~ (output point -- output point)
: cld
  ~ : 1 cld
  0xFC pack8 ;
: std
  ~ : 1 std
  0xFD pack8 ;
: syscall
  ~ : 2 syscall
  0x0F pack8 0x05 pack8 ;
: hlt
  ~ : 1 hlt
  0xF4 pack8 ;

~ (output point, source register -- output point)
: push-reg64
  ~ : 0 # push-reg64
  reg64 0x50 opcodereg ;
: push-extrareg64
  ~ : 0 # push-extrareg64
  swap rex-b swap extrareg64 0x50 opcodereg ;
  ~   Note the use of the B rex bit here; this instruction puts the register
  ~ number in the opcode field, so it uses Table 3-1.

~ (output point, target register -- output point)
: pop-reg64
  ~ : 0 # pop-reg64
  reg64 0x58 opcodereg ;
: pop-extrareg64
  ~ : 0 # pop-extrareg64
  swap rex-b swap extrareg64 0x58 opcodereg ;
  ~   Note the use of the B rex bit here; this instruction puts the register
  ~ number in the opcode field, so it uses Table 3-1.

~ (output point, immediate value -- output point)
: push-imm32-extended64
  ~ : 5 # push-imm32-extended64
  swap 0x68 pack8 swap
  ~ : provide-hex
  pack32 ;

~ (output point, source register, source displacement value, target register
~  -- output point)
: lea-reg64-disp8-reg64
  ~ : 1 # # # lea-reg64-disp8-reg64
  4 roll rex-w 0x8D pack8 4 unroll
  reg64 3unroll addressing-disp8-reg64 ;

~ (output point, source register, source displacement value, target register
~  -- output point)
: lea-reg64-disp32-reg64
  ~ : 1 # # # lea-reg64-disp32-reg64
  4 roll rex-w 0x8D pack8 4 unroll
  reg64 3unroll addressing-disp32-reg64 ;

~ (output point,
~  source base register, source index register, source index scale factor,
~  target register -- output point)
: lea-reg64-indexed-reg64
  ~ : 1 # # # # lea-reg64-indexed-reg64
  5 roll rex-w 0x8D pack8 5 unroll
  reg64 4 unroll 3unroll swap addressing-indexed-reg64 ;

~ (output point,
~  source base register, source index register, source index scale factor,
~  source displacement value,
~  target register -- output point)
: lea-reg64-disp8-indexed-reg64
  ~ : 1 # # # # # lea-reg64-disp8-indexed-reg64
  6 roll rex-w 0x8D pack8 6 unroll
  reg64 5 unroll 3 roll 4 roll 3 roll addressing-disp8-indexed-reg64 ;


~ Move instructions
~ ~~~~~~~~~~~~~~~~~
~
~   These are, like, MOST of what we care about, so they get their own
~ section. Although it's very much the case that almost every two-operand
~ instruction offers this many distinct modes, we don't care about most of
~ those and don't yet implement them. We do care about all the modes for move
~ instructions.
~
~   Someday perhaps we'll have extra-high-level features which generate all
~ the distinct versions of each instruction in a concise way, but that is not
~ this day.

~ (output point, immediate value, register -- output point)
: mov-reg64-imm32
  ~ : 5 # # mov-reg64-imm32
  3roll
  rex-w 0xC7 pack8 swap
  0 swap addressing-reg64
  swap
  ~ : provide-hex
  pack32 ;
: mov-reg64-imm64
  ~ : 8 # # mov-reg64-imm64
  3roll rex-w swap reg64 0xB8 opcodereg swap
  ~ : provide-hex
  pack64 ;
: mov-extrareg64-imm64
  ~ : 8 # # mov-extrareg64-imm64
  ~   Note the use of the B rex bit here; this instruction puts the register
  ~ number in the opcode field, so it uses Table 3-1.
  3roll rex-wb swap extrareg64 0xB8 opcodereg swap
  ~ : provide-hex
  pack64 ;

~ (output point, source register, target register -- output point)
: mov-reg64-reg64
  ~ : 1 # # mov-reg64-reg64
  3roll rex-w 0x89 pack8 3unroll
  swap reg64 swap addressing-reg64
  ~ : swap-subitems
  ;
: mov-indirect-reg64-reg64
  ~ : 1 # # mov-indirect-reg64-reg64
  3roll rex-w 0x89 pack8 3unroll
  swap reg64 swap addressing-indirect-reg64
  ~ : swap-subitems
  ;

~ (output point, source register, target register, target displacement value
~  -- output point)
: mov-disp8-reg64-reg64
  ~ : 1 # # # mov-disp8-reg64-reg64
  4 roll rex-w 0x89 pack8 4 unroll
  3roll reg64 3unroll addressing-disp8-reg64
  ~ : 3 roll-subitems
  ;

~ (output point, source register, target register -- output point)
: mov-reg64-indirect-reg64
  ~ : 1 # # mov-reg64-indirect-reg64
  3roll rex-w 0x8B pack8 3unroll
  reg64 swap addressing-indirect-reg64 ;

~ (output point, source register, source displacement value, target register
~  -- output point)
: mov-reg64-disp8-reg64
  ~ : 1 # # # mov-reg64-disp8-reg64
  4 roll rex-w 0x8B pack8 4 unroll
  reg64 3unroll addressing-disp8-reg64 ;
: mov-reg64-disp32-reg64
  ~ : 1 # # # mov-reg64-disp32-reg64
  4 roll rex-w 0x89 pack8 4 unroll
  3roll reg64 swap 3roll addressing-disp32-reg64
  ~ : swap-subitems
  ~ : 3 roll-subitems
  ;

~ (output point,
~  source base register, source index register, source index scale factor,
~  target register -- output point)
: mov-reg64-indexed-reg64
  ~ : 1 # # # # mov-reg64-indexed-reg64
  5 roll rex-w 0x8B pack8 5 unroll
  reg64 4 unroll 3unroll swap addressing-indexed-reg64 ;

~ (output point, source register,
~  target base register, target index register, target index scale factor
~  -- output point)
: mov-indexed-reg64-reg64
  ~ : 1 # # # # mov-indexed-reg64-reg64
  5 roll rex-w 0x89 pack8 5 unroll
  4 roll reg64 4 unroll
  3unroll swap addressing-indexed-reg64
  ~ : 4 roll-subitems
  ;

~ (output point, source register, target register -- output point)
: mov-indirect-reg64-reg32
  ~ : 1 # # mov-indirect-reg64-reg32
  3roll 0x89 pack8 3unroll
  swap reg32 swap addressing-indirect-reg64
  ~ : swap-subitems
  ;

~ (output point, source register, target register, target displacement value
~  -- output point)
: mov-disp8-reg64-reg32
  ~ : 1 # # # mov-disp8-reg64-reg32
  4 roll 0x89 pack8 4 unroll
  3roll reg32 3unroll addressing-disp8-reg64
  ~ : 3 roll-subitems
  ;

~ (output point, source register, target register -- output point)
: mov-reg32-indirect-reg64
  ~ : 1 # # mov-reg32-indirect-reg64
  3roll 0x8B pack8 3unroll
  reg32 swap addressing-indirect-reg64 ;

~ (output point, source register, source displacement value, target register
~ -- output point)
: mov-reg32-disp8-reg64
  ~ : 1 # # # mov-reg32-disp8-reg64
  4 roll 0x8B pack8 4 unroll
  reg32 3unroll addressing-disp8-reg64 ;

~ (output point, source register, target register -- output point)
: mov-indirect-reg64-reg16
  ~ : 2 # # mov-indirect-reg64-reg16
  3roll 0x66 pack8 0x89 pack8 3unroll
  swap reg16 swap addressing-indirect-reg64
  ~ : swap-subitems
  ;

~ (output point, source register, target register, target displacement value
~  -- output point)
: mov-disp8-reg64-reg16
  ~ : 2 # # # mov-disp8-reg64-reg16
  4 roll 0x66 pack8 0x89 pack8 4 unroll
  3roll reg16 3unroll addressing-disp8-reg64
  ~ : 3 roll-subitems
  ;

~ (output point, source register, target register -- output point)
: mov-reg16-indirect-reg64
  ~ : 2 # # mov-reg16-indirect-reg64
  3roll 0x66 pack8 0x8B pack8 3unroll
  reg16 swap addressing-indirect-reg64 ;

~ (output point, source register, target displacement value, target register
~  -- output point)
: mov-reg16-disp8-reg64
  ~ : 2 # # # mov-reg16-disp8-reg64
  4 roll 0x66 pack8 0x8B pack8 4 unroll
  reg16 3unroll addressing-disp8-reg64 ;

~ (output point, source register, target register -- output point)
: mov-indirect-reg64-reg8
  ~ : 1 # # mov-indirect-reg64-reg8
  3roll 0x88 pack8 3unroll
  swap reg8 swap addressing-indirect-reg64
  ~ : swap-subitems
  ;

~ (output point, source register, target register, target displacement value
~  -- output point)
: mov-disp8-reg64-reg8
  ~ : 1 # # # mov-disp8-reg64-reg8
  4 roll 0x88 pack8 4 unroll
  3roll reg8 3unroll addressing-disp8-reg64
  ~ : 3 roll-subitems
  ;

~ (output point, source register, target register -- output point)
: mov-reg8-indirect-reg64
  ~ : 1 # # mov-reg8-indirect-reg64
  3roll 0x8A pack8 3unroll
  reg8 swap addressing-indirect-reg64 ;

~ (output point, source register, source displacement value, target register
~  -- output point)
: mov-reg8-disp8-reg64
  ~ : 2 # # # mov-reg8-disp8-reg64
  4 roll 0x8A pack8 4 unroll
  reg8 3unroll addressing-disp8-reg64 ;

~ (output point, source register, target register -- output point)
: mov-reg8-reg8
  ~ : 1 # # mov-reg8-reg8
  3roll 0x88 pack8 3unroll
  swap reg8 swap addressing-reg8
  ~ : swap-subitems
  ;


~ String instructions
~ ~~~~~~~~~~~~~~~~~~~
~
~   These are in their own section because there's an awful lot of
~ combinations, and fortunately they are very uniform in structure.
~
~   What makes these useful is that they take their parameters from certain
~ fixed registers, which are chosen such that the operations chain into each
~ other well. Thus you can use them to build various block-memory and string
~ operations, and even if you need unusual forms of loop unrolling or
~ alignment tweaking, the code will end up uniform in structure. On modern
~ processors, this is even the high-performance approach, due to highly
~ optimized microcode, though these operations were inefficient when they
~ were first invented.
~
~   We break with the Intel mnemonics, which follow the pattern
~ movsb/movsw/movsd/movsq, because this would otherwise be the only place we
~ use the b/w/d/q thing instead of 8/16/32/64. Tradition and pronounceability
~ are both nice things, but approachability to newcomers is important, too.
~
~   Some of these are repeatable; whether you view the repeatable variants
~ as different instructions is up to you. At any rate the machine code
~ representation of the repeatable variants is the same as for the regular
~ variants with an extra prefix, so we define them together.
~
~   This is a proper superset of the flatassembler implementations of string
~ instructions. The wisdom of that is questionable, but at least it's noted
~ here...

~ (output point -- output point)
: movs8
  ~ : 1 movs8
  0xA4 pack8 ;
: movs16
  ~ : 2 movs16
  0x66 pack8 0xA5 pack8 ;
: movs32
  ~ : 1 movs32
  0xA5 pack8 ;
: movs64
  ~ : 1 movs64
  rex-w 0xA5 pack8 ;
: rep-movs8
  ~ : 2 rep-movs8
  0xF3 pack8 0xA4 pack8 ;
: rep-movs16
  ~ : 3 rep-movs16
  0xF3 pack8 0x66 pack8 0xA5 pack8 ;
: rep-movs32
  ~ : 2 rep-movs32
  0xF3 pack8 0xA5 pack8 ;
: rep-movs64
  ~ : 2 rep-movs64
  0xF3 pack8 rex-w 0xA5 pack8 ;

~ (output point -- output point)
: lods8
  ~ : 1 lods8
  0xAC pack8 ;
: lods16
  ~ : 2 lods16
  0x66 pack8 0xAd pack8 ;
: lods32
  ~ : 1 lods32
  0xAD pack8 ;
: lods64
  ~ : 1 lods64
  rex-w 0xAD pack8 ;
: rep-lods8
  ~ : 2 rep-lods8
  0xF3 pack8 0xAC pack8 ;
: rep-lods16
  ~ : 3 rep-lods16
  0xF3 pack8 0x66 pack8 0xAD pack8 ;
: rep-lods32
  ~ : 2 rep-lods32
  0xF3 pack8 0xAD pack8 ;
: rep-lods64
  ~ : 2 rep-lods64
  0xF3 pack8 rex-w 0xAD pack8 ;

~ (output point -- output point)
: stos8
  ~ : 1 stos8
  0xAA pack8 ;
: stos16
  ~ : 2 stos16
  0x66 pack8 0xAB pack8 ;
: stos32
  ~ : 1 stos32
  0xAB pack8 ;
: stos64
  ~ : 1 stos64
  rex-w 0xAB pack8 ;
: rep-stos8
  ~ : 2 rep-stos8
  0xF3 pack8 0xAA pack8 ;
: rep-stos16
  ~ : 3 rep-stos16
  0xF3 pack8 0x66 pack8 0xAB pack8 ;
: rep-stos32
  ~ : 2 rep-stos32
  0xF3 pack8 0xAB pack8 ;
: rep-stos64
  ~ : 2 rep-stos64
  0xF3 pack8 rex-w 0xAB pack8 ;

~ (output point -- output point)
: cmps8
  ~ : 1 cmps8
  0xA6 pack8 ;
: cmps16
  ~ : 2 cmps16
  0x66 pack8 0xA7 pack8 ;
: cmps32
  ~ : 1 cmps32
  0xA7 pack8 ;
: cmps64
  ~ : 1 cmps64
  rex-w 0xA7 pack8 ;
: repz-cmps8
  ~ : 2 repz-cmps8
  0xF3 pack8 0xA6 pack8 ;
: repz-cmps16
  ~ : 3 repz-cmps16
  0xF3 pack8 0x66 pack8 0xA7 pack8 ;
: repz-cmps32
  ~ : 2 repz-cmps32
  0xF3 pack8 0xA7 pack8 ;
: repz-cmps64
  ~ : 2 repz-cmps64
  0xF3 pack8 rex-w 0xA7 pack8 ;
: repnz-cmps8
  ~ : 2 repnz-cmps8
  0xF2 pack8 0xA6 pack8 ;
: repnz-cmps16
  ~ : 3 repnz-cmps16
  0xF2 pack8 0x66 pack8 0xA7 pack8 ;
: repnz-cmps32
  ~ : 2 repnz-cmps32
  0xF2 pack8 0xA7 pack8 ;
: repnz-cmps64
  ~ : 2 repnz-cmps64
  0xF2 pack8 rex-w 0xA7 pack8 ;

~ (output point -- output point)
: scas8
  ~ : 1 scas8
  0xA8 pack8 ;
: scas16
  ~ : 2 scas16
  0x66 pack8 0xAF pack8 ;
: scas32
  ~ : 1 scas32
  0xAF pack8 ;
: scas64
  ~ : 1 scas64
  rex-w 0xAF pack8 ;
: repz-scas8
  ~ : 2 repz-scas8
  0xF3 pack8 0xAE pack8 ;
: repz-scas16
  ~ : 3 repz-scas16
  0xF3 pack8 0x66 pack8 0xAF pack8 ;
: repz-scas32
  ~ : 2 repz-scas32
  0xF3 pack8 0xAF pack8 ;
: repz-scas64
  ~ : 2 repz-scas64
  0xF3 pack8 rex-w 0xAF pack8 ;
: repnz-scas8
  ~ : 2 repnz-scas8
  0xF2 pack8 0xAE pack8 ;
: repnz-scas16
  ~ : 3 repnz-scas16
  0xF2 pack8 0x66 pack8 0xAF pack8 ;
: repnz-scas32
  ~ : 2 repnz-scas32
  0xF2 pack8 0xAF pack8 ;
: repnz-scas64
  ~ : 2 repnz-scas64
  0xF2 pack8 rex-w 0xAF pack8 ;


~ Arithmetic instructions
~ ~~~~~~~~~~~~~~~~~~~~~~~
~
~   Operations between two registers always have the target register as the
~ last parameter. Operations between a register and an immediate ALSO have the
~ target register as the last parameter, which is obvious for add-reg64-imm8
~ but can be confusing for sub-reg64-imm8 or cmp-reg64-imm8. Also note what
~ this means for sub-reg64-reg64 and cmp-reg64-reg64.
~
~   Every possible convention here leads to counterintuitive behavior in some
~ cases. This one has proven to be more memorable than the other obvious
~ choices, and that'll have to be enough.

~ (output point, source register, target register -- output point)
: add-reg64-reg64
  ~ : 1 # # add-reg64-reg64
  3roll rex-w 0x01 pack8 3unroll
  swap reg64 swap addressing-reg64
  ~ : swap-subitems
  ;

~ (output point, source register, target register -- output point)
: add-indirect-reg64-reg64
  ~ : 1 # # add-indirect-reg64-reg64
  3roll rex-w 0x01 pack8 3unroll
  swap reg64 swap addressing-indirect-reg64
  ~ : swap-subitems
  ;

~ (output point, source register, target register -- output point)
: add-reg64-indirect-reg64
  ~ : 1 # # add-reg64-indirect-reg64
  3roll rex-w 0x03 pack8 3unroll
  reg64 swap addressing-indirect-reg64 ;

~ (output point, immediate value, target register -- output point)
: add-reg64-imm8
  ~ : 2 # # add-reg64-imm8
  3roll rex-w 0x83 pack8 swap 0 swap addressing-reg64
  swap
  ~ : provide-hex
  pack8 ;

~ See above re: parameter order.
~
~ (output point, source register, target register -- output point)
: sub-reg64-reg64
  ~ : 1 # # sub-reg64-reg64
  3roll rex-w 0x2B pack8 3unroll
  reg64 swap addressing-reg64 ;

~ (output point, source register, target register -- output point)
: sub-indirect-reg64-reg64
  ~ : 1 # # sub-indirect-reg64-reg64
  3roll rex-w 0x2B pack8 3unroll
  swap reg64 swap addressing-indirect-reg64
  ~ : swap-subitems
  ;

~ See above re: parameter order.
~
~ (output point, immediate value, target register -- output point)
: sub-reg64-imm8
  ~ : 2 # # sub-reg64-imm8
  3roll rex-w 0x83 pack8 swap 5 swap addressing-reg64
  swap
  ~ : provide-hex
  pack8 ;

~ See above re: parameter order.
~
~ (output point, source register, target register -- output point)
: sbb-reg64-imm8
  ~ : 2 # # sbb-reg64-imm8
  3roll rex-w 0x83 pack8 swap 3 swap addressing-reg64
  swap
  ~ : provide-hex
  pack8 ;

~ (output point, target register -- output point)
: neg-reg64
  ~ : 1 # neg-reg64
  swap rex-w 0xF7 pack8
  swap 3 swap addressing-reg64 ;

~  The target register is always rax.
~
~ (output point, source register -- output point)
: mul-reg64
  ~ : 1 # mul-reg64
  swap rex-w 0xF7 pack8 swap
  4 swap addressing-reg64 ;

~   The dividend is 128 bits, and is formed from rdx as the high half and rax
~ as the low half. The divisor is a specified register. The quotient is
~ returned in rax, truncated towards zero. The remainder is in rdx. This
~ entire process is unsigned.
~
~   The official mnemonic for this is "div", but divmod is what it does.
~
~ (output point, divisor register -- output point)
: divmod-reg64
  ~ : 1 # divmod-reg64
  swap rex-w 0xF7 pack8 swap
  6 swap addressing-reg64 ;

~ Same as divmod, but signed.
~
~ (output point, divisor register -- output point)
: idivmod-reg64
  ~ : 1 # idivmod-reg64
  swap rex-w 0xF7 pack8 swap
  7 swap addressing-reg64 ;

~ (output point, target register -- output point)
: inc-reg64
  ~ : 1 # inc-reg64
  swap rex-w 0xFF pack8 swap 0 swap addressing-reg64 ;

~ (output point, target register -- output point)
: dec-reg64
  ~ : 1 # dec-reg64
  swap rex-w 0xFF pack8 swap 1 swap addressing-reg64 ;

~ (output point, source register, target register -- output point)
: and-reg64-reg64
  ~ : 1 # # and-reg64-reg64
  3roll rex-w 0x23 pack8 3unroll
  reg64 swap addressing-reg64 ;

~ (output point, source value, target register -- output point)
: and-reg64-imm8
  ~ : 2 # # and-reg64-imm8
  3roll rex-w 0x83 pack8 swap
  4 swap addressing-reg64
  swap
  ~ : provide-hex
  pack8 ;

~ (output point, source register, target register -- output point)
: or-reg64-reg64
  ~ : 1 # # or-reg64-reg64
  3roll rex-w 0x0B pack8 3unroll
  reg64 swap addressing-reg64 ;

~ (output point, source value, target register -- output point)
: or-reg64-imm8
  ~ : 2 # # or-reg64-imm8
  3roll rex-w 0x83 pack8 swap
  1 swap addressing-reg64
  swap
  ~ : provide-hex
  pack8 ;

~ (output point, source register, target register -- output point)
: xor-reg64-reg64
  ~ : 1 # # xor-reg64-reg64
  3roll rex-w 0x33 pack8 3unroll
  reg64 swap addressing-reg64 ;

~ (output point, target register -- output point)
: not-reg64
  ~ : 1 # not-reg64
  swap rex-w 0xF7 pack8
  swap 2 swap addressing-reg64 ;

~ (output point, bit count, target register -- output point)
: rol-reg64-imm8
  ~ : 2 # # rol-reg64-imm8
  3roll rex-w 0xC1 pack8 swap
  0 swap addressing-reg64
  swap
  ~ : provide-hex
  pack8 ;

~ (output point, bit count, target register -- output point)
: rol-reg8-imm8
  ~ : 2 # # rol-reg8-imm8
  3roll 0xC0 pack8 swap
  0 swap addressing-reg8
  swap
  ~ : provide-hex
  pack8 ;

~ (output point, bit count, target register -- output point)
: ror-reg64-imm8
  ~ : 1 # # ror-reg64-imm8
  3roll rex-w 0xC1 pack8 swap
  1 swap addressing-reg64
  swap
  ~ : provide-hex
  pack8 ;

~ (output point, bit count, target register -- output point)
: ror-reg8-imm8
  ~ : 2 # # ror-reg8-imm8
  3roll 0xC0 pack8 swap
  1 swap addressing-reg64
  swap
  ~ : provide-hex
  pack8 ;


~ Control flow instructions
~ ~~~~~~~~~~~~~~~~~~~~~~~~~
~
~   The comparator instructions cmp and test follow the same argument-order
~ conventions as the arithmetic instructions: The target register always comes
~ last. See the arithmetic section for more discussion.

~   Pretend to subtract right from left, and set the flags the same way as if
~ we actually had.
~
~ (output point, left register, right register -- output point)
: cmp-reg64-reg64
  ~ : 1 # # cmp-reg64-reg64
  3roll rex-w 0x3B pack8 3unroll
  reg64 swap addressing-reg64 ;

~ See above re: parameter order.
~
~ (output point, immediate value, target register -- output point)
: cmp-reg64-imm8
  ~ : 2 # # cmp-reg64-imm8
  3roll rex-w 0x83 pack8
  ~ (immediate value, register, output point)
  swap 7 swap addressing-reg64
  ~ ( output point, immediate value)
  swap
  ~ : provide-hex
  pack8 ;

~   Pretend to bitwise-and left with right, and set the flags the same way as
~ if we actually had.
~
~   The names of the condition codes can be a little confusing when using them
~ after "test", because they're really premised on the idea that you did
~ "cmp".
~
~ (output point, left register, right register -- output point)
: test-reg64-reg64
  ~ : 1 # # test-reg64-reg64
  3roll rex-w 0x85 pack8 3unroll
  swap reg64 swap addressing-reg64
  ~ : swap-subitems
  ;

~ (output point, condition code, target register -- output point)
: set-reg8-cc
  ~ : 1 # # set-reg8-cc
  3roll 0x0F pack8
  3roll condition-code 0x90 opcodecc
  swap reg8 3 0 3roll modrm
  ~ : swap-subitems
  ;

~ (output point, address offset value, condition code -- output point)
: jmp-cc-rel-imm8
  ~ : 1 # # jmp-cc-rel-imm8
  3roll swap condition-code 0x70 opcodecc
  swap
  ~ : provide-hex
  pack8 ;

~ (output point, address offset value, condition code -- output point)
: jmp-cc-rel-imm32
  ~ : 5 # # jmp-cc-rel-imm32
  3roll 0x0F pack8
  swap condition-code 0x70 opcodecc
  swap
  ~ : provide-hex
  pack32 ;

~ (output point, register -- output point)
: jmp-abs-indirect-reg64
  ~ : 1 # jmp-abs-indirect-reg64
  swap 0xFF pack8 swap
  4 swap addressing-indirect-reg64 ;

~ (output point, address offset value -- output point)
: jmp-rel-imm8
  ~ : 2 # jmp-rel-imm8
  swap 0xEB pack8
  swap
  ~ : provide-hex
  pack8 ;

~ (output point, address offset value -- output point)
: jmp-rel-imm32
  ~ : 5 # jmp-rel-imm32
  swap 0xE9 pack8
  swap
  ~ : provide-hex
  pack32 ;

~ This is technically a "near" call. The name intentionally doesn't say so.
~
~ (output point, address offset value -- output point)
: call-rel-imm32
  ~ : 5 # call-rel-imm32
  swap 0xE8 pack8
  swap
  ~ : provide-hex
  pack32 ;

~ Similarly, this is a "near" return.
~
~ (output point)
: ret
  ~ : 1 ret
  0xC3 pack8 ;