Become a cyber pro by building your skills in the National Cyber League!
To get the most out of this blog post I highly recommend that you use the topics as points to do further research on because each of the sub topics in this blog could have multiple blogs written about them. I will be providing a surface level explanation of some concepts but I highly recommend doing further research.
It is recommend that you have basic programming knowledge (assembly is a large bonus) and some basic Linux command line skills.
If you are unfamiliar with assembly I highly recommend reading the following article:
https://www.cs.virginia.edu/~evans/cs216/guides/x86.html
This blog post is going to be organized into the following three segments
- From source code to something run-able
- Compiling
- Linking
- Starting a program
- A running program
Before we dive into the article here are a few useful definitions
- Machine code/Byte code: Sequences of instructions a computer interprets
- Assembly: Human readable mappings of machine/byte code
- Source code: Higher level source texts for a program
- Pointer: The address of a portion of memory
At the end of each section I will link to one or two resources that explain the concepts in a lot more detail.
Staring with source code
Before we can talk about how a program runs we need to lay a simple foundation as to what needs to happen before it can run. This section of the blog is going to be a lot more theoretical and the knowledge can be transferred to most modern platforms (Linux, Mac, Windows)
Compiling
To begin with, all programs start as source code, but your computer does not directly run source code. You first need to compile it with a program called a compiler. The compiler takes a programmers source code as an input then goes trough a number of steps to covert it into machine code.
Compilers by themselves are a very complex topic, but a simple distilled version of the process they go through is as follows:
- Tokenization
- Parse into an Abstract Syntax Tree
- Generate byte code
This (simplified) process is what converts your source code into something that the computer understands. However it's very likely that your program is relying on various libraries and you may be wondering how your program is able to use said libraries. Which brings us to our next topic of linking.
Here is a link for how to implement a full blown programming language: https://lisperator.net/pltut/parser/
Linking
Linking is what allows for a program to be able to use code from varying libraries. There are two types of linking that can occur: static and dynamic linking.
Static linking is by far the easiest to understand, the libraries get compiled with the program all into the same binary. The largest draw back of this is that a lot of common libraries, for example libc, the standard library for C programs, would be stored multiple times over for each binary. Compiled out, libc is around 2MB in size which, when you do the math, having to store libc again for each program is a waste of storage.
Because of this issue with static linking, dynamic linking was created.
Dynamic Linking allows for your program to use compiled copies of libraries that are stored in a separate file, drastically reducing the amount of storage needed to install all the programs a fully complete system needs. In Linux, these compiled library files are called shared objects (DLLs for those who are on Windows). We will get more into the details of what dynamic linking looks like during runtime later on.
This link is OSx platform specific but still very well written: https://opensource.apple.com/source/ld64/ld64-136/doc/design/linker.html
Starting a Program
At this point in the blog we are going to shift towards looking at specific examples of how programs are run on Linux.
Asking the Operating System
On Linux when you run a command, for example:
$ lsYou are asking the operating system to go through a set of tasks to run your program. Assuming you have proper permissions to run the program, Linux will first read some meta data related to the program, and then allocate a space in memory for the program to run, load it into memory, and then start to run it. (It does the same with any shared objects the program needs to run)
ELF (and not the Tolkien kind)
Linux programs are stored with a header called an ELF header. According to the Linux programmers, manual ELF stands for Executable and Linker Format. An ELF file stores information that allows for the operating system to know how to load the binary into memory and what shared objects it needs to load. ELF headers also contain metadata like what processor the binary is meant to run on and whether or not the file is a shared object or an executable. It also contains the information as to the entry point of the file so the operating system knows where to start running the program.
Here is an excellent write up on the details of an ELF file: https://linuxhint.com/understanding_elf_file_format/
The entry point
The entry point of the program is a segment of code that does some initializations that are not handled by the operating systems itself before running the programmers code. On Linux, this mainly consists of calling a function to help setup libc. Here is a snippet of an entry point in a Linux binary.
0x00001073 4c8d05660100. lea r8, qword sym.__libc_csu_fini ; 0x11e0
0x0000107a 488d0def0000. lea rcx, qword sym.__libc_csu_init ; 0x1170
0x00001081 488d3dc10000. lea rdi, qword main ; 0x1149
0x00001088 ff15522f0000 call qword [reloc.__libc_start_main]The PLT
When a program wants to access Dynamically linked functions, it references a structure called the procedural linkage table. Put very simply the procedural linkage table allows for programs to resolve address for dynamically loaded libraries. It is often times abbreviated to PLT.
Here is an excellent writeup on how PLT works:
https://www.linkedin.com/pulse/elf-linux-executable-plt-got-tables-mohammad-alhyari/
A Running Program
Lastly, we can talk about what a program looks like when it is running.
The stack
Data for programs is usually stored in an area of memory called the stack (More specifically it tends to be things such as local variables). The stack is a First In Last Out (FILO) data structure that works by pushing (adding) and popping (removing) values off the top of the stack.
Function calling and stack frames
When our program goes to call a function, two important things of note happen: the callee passes arguments to the function using that platforms calling convention (In the case of 64 bit Linux arguments are passed by register) and then the called function will create a stack frame for its local variables.
Stack frames are generally kept track of by using two registers referred to as the Stack pointer and the Base pointer (RSP and RBP on x64 systems respectively). The Base pointer marks where in memory the stack frame begins and the Stack pointer marks were it ends.
A stack frame is created by first pushing the previous base pointer address onto the stack (effectively saving a copy of it in memory to restore to the previous stack frame) and then copying the value of the current stack pointer into the base pointer. Then the stack pointer is changed depending on how small or large of a stack frame is needed.
This tutorial is ARM specifically but does an amazing job at explaining how the stack and functions work at a low level:
https://azeria-labs.com/functions-and-the-stack-part-7/
I hope that you have gained value from this post. I strongly suggest also reading the articles I linked to for each section. Code execution is a vast subject that I hope I have shined a little bit of light onto or, at the very least, have given you a good direction to look towards.
This post was not affiliated, associated, authorized, endorsed by, or in any way officially connected with ARM. The ARM name and marks are registered trademarks of ARM and are only used for informational purposes.
Published by wolfshirtz