How to run a program without an operating system?

Ask Question

How do you run a program all by itself without an operating system running? Can you create assembly programs that the computer can load and run at startup, e.g. boot the computer from a flash drive and it runs the program that is on the cpu?

assembly x86 operating-system bootloader osdev

edited yesterday Peter Cordes 116k 16 176 302

asked Feb 26 '14 at 22:13 user2320609 461 3 5 6

2

On which architecture? x86? ARM? – Kissiel Feb 26 '14 at 22:14

I was speaking generaly, but most likely x86 or x64 – user2320609 Feb 26 '14 at 22:19

yes that is exactly how processors boot. doesnt have to be assembly, C is often used with a little bit of asm for a bootstrap and perhaps some other support. – old_timer Feb 26 '14 at 22:26

13

Think of it: if there was no such capability, how would the OS itself start and run? :) – Seva Alekseyev Dec 4 '15 at 19:21

2 Answers

How do you run a program all by itself without an operating system running?

You place your binary code to a place where processor looks for after rebooting (e.g. address 0 on ARM).

Can you create assembly programs that the computer can load and run at startup ( e.g. boot the computer from a flash drive and it runs the program that is on the drive)?

General answer to the question: it can be done. It's often reffered to as "bare metal programming". To read from flash drive, you want to know what's USB, and you want to have some driver to work with this USB. The program on this drive would also have to be in some particular format. On some particular filesystem... This is something that usually boot loaders do. Many ARM boards let you do some of those things. Some have boot loader to help you with basic setup.

Here you may find great tutorial of how to do basic operating system on Raspberry PI.

Edit: This article, and the whole wiki.osdev.org will anwer most of your questions http://wiki.osdev.org/Introduction

Also, if you don't want to experiment directly on hardware, you can run it as a virtual machine using hypervisors like qemu. See how to run "hello world" directly on virtualized ARM hardware here.

edited Feb 27 '14 at 5:19 Isa A 849 9 27

answered Feb 26 '14 at 22:23 Kissiel 1,115 1 8 9

Runnable examples

Technically, a program that runs without an OS, is an OS. So let's see how to create and run some minuscule hello world OSes.

The code of all examples below and more is present on this GitHub repo.

Boot sector

On x86, the simplest and lowest level thing you can do is to create a Master Boot Sector (MBR), which is a type of boot sector, and then install it to a disk.

Here we create one with a single printf call:

printf '\364%509s\125\252' > main.img sudo apt-get install qemu-system-x86 qemu-system-x86_64 -hda main.img

Outcome:

Tested on Ubuntu 18.04, QEMU 2.11.1.

main.img contains the following:

\364 in octal == 0xf4 in hex: the encoding for a hlt instruction, which tells the CPU to stop working.

Therefore our program will not do anything: only start and stop.

We use octal because \x hex numbers are not specified by POSIX.

We could obtain this encoding easily with:

echo hlt > a.asm nasm -f bin a.asm hd a

but the 0xf4 encoding is also documented on the Intel manual of course.

%509s produce 509 spaces. Needed to fill in the file until byte 510.

\125\252 in octal == 0x55 followed by 0xaa : magic bytes required by the hardware. They must be bytes 511 and 512.

If not present, the hardware will not treat this as a bootable disk.

Note that even without doing anything, a few characters are already printed on the screen. Those are printed by the firmware, and serve to identify the system.

Run on real hardware

Emulators are fun, but hardware is the real deal.

Note however that this is dangerous, and you could wipe your disk by mistake: only do this on old machines that don't contain critical data! Or even better, devboards such as the Raspberry Pi, see the ARM example below.

For a typical laptop, you have to do something like:

Burn the image to an USB stick (will destroy your data!):

sudo dd if=main.img of=/dev/sdX

plug the USB on a computer turn it on tell it to boot from the USB.

This means making the firmware pick USB before hard disk.

If that is not the default behavior of your machine, keep hitting Enter, F12, ESC or other such weird keys after power-on until you get a boot menu where you can select to boot from the USB.

It is often possible to configure the search order in those menus.

For example, on my old Lenovo Thinkpad T430, UEFI BIOS 1.16, I can see:

Hello world

Now that we have made a minimal program, let's move to a hello world.

The obvious question is: how to do IO? A few options:

ask the firmware, e.g. BIOS or UEFI, to do if for us VGA: special memory region that gets printed to the screen if written to. Can be used on Protected mode. write a driver and talk directly to the display hardware. This is the "proper" way to do it: more powerful, but more complex. serial port. This is a very simple standardized protocol that sends and retrieves characters from a host terminal.

Source.

It is unfortunately not exposed on most modern laptops, but is the common way to go for development boards, see the ARM examples below.

This is really a shame, since such interfaces are really useful to debug the Linux kernel for example. use debug features of chips. ARM calls theirs semihosting for example. On real hardware, it requires some extra hardware and software support, but on emulators it can be a free convenient option. Example.

Here we will do a BIOS example as it is simpler on x86. But note that it is not the most robust method.

main.S

.code16 mov $msg, %si mov $0x0e, %ah loop: lodsb or %al, %al jz halt int $0x10 jmp loop halt: hlt msg: .asciz "hello world"

link.ld

SECTIONS { . = 0x7c00; .text : { __start = .; *(.text) . = 0x1FE; SHORT(0xAA55) } }

Assemble and link with:

gcc -c -g -o main.o main.S ld --oformat binary -o main.img -T linker.ld main.o

Outcome:

Tested on: Lenovo Thinkpad T430, UEFI BIOS 1.16. Disk generated on an Ubuntu 18.04 host.

Besides the standard userland assembly instructions, we have:

.code16 : tells GAS to output 16-bit code

cli : disable software interrupts. Those could make the processor start running again after the hlt

int $0x10 : does a BIOS call. This is what prints the characters one by one.

The important link flags are:

--oformat binary : output raw binary assembly code, don't warp it inside an ELF file as is the case for regular userland executables.

Cooler x86 bare metal programs

Here are a few cooler baremetal setups that I've achieved:

multicore: What does multicore assembly language look like? paging: How does x86 paging work?

Use C instead of assembly

Since C compiles to assembly, using C without the standard library is pretty simple, you basically just need:

a linker script to put things in memory at the right place flags that tell GCC not to use the standard library

a tiny assembly entry point that sets required C state for main , notably: the stack zero out BSS

TODO: link so some x86 example on GitHub. Here is an ARM one I've created.

Things get more fun if you want to use the standard library however, since we don't have the Linux kernel, which implements much of the C standard library functionality through POSIX.

A few possibilities, without going to a full-blown OS like Linux, include:

Newlib

Detailed example at: https://electronics.stackexchange.com/questions/223929/c-standard- libraries-on-bare-metal/223931

In Newlib, you have to implement the syscalls yourself, but you get a very minimal system, and it is very easy to implement them.

For example, you could redirect printf to the UART or ARM systems, or implement exit() with semihosting. embedded operating systems like FreeRTOS and Zephyr.

Such operating systems typically allows you to turn off pre-emptive scheduling, therefore giving you full control over the runtime of the program.

They can be seen as a sort of pre-implemented Newlib.

ARM

In ARM, the general ideas are the same. I have uploaded:

a few simple QEMU baremetal examples here on GitHub. The prompt.c example takes input from your host terminal and gives back output all through the simulated UART:

enter a character got: a new alloc of 1 bytes at address 0x0x4000a1c0 enter a character got: b new alloc of 2 bytes at address 0x0x4000a1c0 enter a character

See also: How to make bare metal ARM programs and run them on QEMU? a fully automated Raspberry Pi blinker setup at: https://github.com/cirosantilli/raspberry-pi- bare-metal-blinker

See also: How to run a C program with no OS on the Raspberry Pi?

For the Raspberry Pi, https://github.com/dwelch67/raspberrypi looks like the most popular tutorial available today.

Some differences from x86 include:

IO is done by writing to magic addresses directly, there is no in and out instructions.

This is called memory mapped IO. for some real hardware, like the Raspberry Pi, you can add the firmware (BIOS) yourself to the disk image.

That is a good thing, as it makes updating that firmware more transparent.

Firmware

In truth, your boot sector is not the first software that runs on the system's CPU.

What actually runs first is the so-called firmware, which is a software:

made by the hardware manufacturers typically closed source but likely C-based stored in read-only memory, and therefore harder / impossible to modify without the vendor's consent.

Well known firmwares include:

BIOS: old all-present x86 firmware. SeaBIOS is the default open source implementation used by QEMU. UEFI: BIOS successor, better standardized, but more capable, and incredibly bloated. Coreboot: the noble cross arch open source attempt

The firmware does things like:

loop over each hard disk, USB, network, etc. until you find something bootable.

When we run QEMU, -hda says that main.img is a hard disk connected to the hardware, and

hda is the first one to be tried, and it is used.

load the first 512 bytes to RAM memory address 0x7c00 , put the CPU's RIP there, and let it run show things like the boot menu or BIOS print calls on the display

Firmware offers OS-like functionality on which most OS-es depend. E.g. a Python subset has been ported to run on BIOS / UEFI: https://www.youtube.com/watch?v=bYQ_lq5dcvM

It can be argued that firmwares are indistinguishable from OSes, and that firmware is the only "true" bare metal programming one can do.

As this CoreOS dev puts it:

The hard part

When you power up a PC, the chips that make up the chipset (northbridge, southbridge and SuperIO) are not yet initialized properly. Even though the BIOS ROM is as far removed from the CPU as it could be, this is accessible by the CPU, because it has to be, otherwise the CPU would have no instructions to execute. This does not mean that BIOS ROM is completely mapped, usually not. But just enough is mapped to get the boot process going. Any other devices, just forget it.

When you run Coreboot under QEMU, you can experiment with the higher layers of Coreboot and with payloads, but QEMU offers little opportunity to experiment with the low level startup code. For one thing, RAM just works right from the start.

Post BIOS initial state

Like many things in hardware, standardization is weak, and one of the things you should not rely on is the initial state of registers when your code starts running after BIOS.

So do yourself a favor and use some initialization code like the following: https://stackoverflow.com/a/32509555/895245

Registers like %ds and %es have important side effects, so you should zero them out even if you are not using them explicitly.

Note that some emulators are nicer than real hardware and give you a nice initial state. Then when you go run on real hardware, everything breaks.

GNU GRUB Multiboot

Boot sectors are simple, but they are not very convenient:

you can only have one OS per disk the load code has to be really small and fit into 512 bytes. This could be solved with the int 0x13 BIOS call. you have to do a lot of startup yourself, like moving into protected mode

It is for those reasons that GNU GRUB created a more convenient file format called multiboot.

Minimal working example: https://github.com/cirosantilli/x86-bare-metal- examples/tree/d217b180be4220a0b4a453f31275d38e697a99e0/multiboot/hello-world

I also use it on my GitHub examples repo to be able to easily run all examples on real hardware without burning the USB a million times. On QEMU it looks like this:

If you prepare your OS as a multiboot file, GRUB is then able to find it inside a regular filesystem.

This is what most distros do, putting OS images under /boot .

Multiboot files are basically an ELF file with a special header. They are specified by GRUB at: https://www.gnu.org/software/grub/manual/multiboot/multiboot.html

You can turn a multiboot file into a bootable disk with grub-mkrescue .

El Torito

Format that can be burnt to CDs: https://en.wikipedia.org/wiki/El_Torito_%28CD- ROM_standard%29

It is also possible to produce a hybrid image that works on either ISO or USB. This is can be done with grub-mkrescue (example), and is also done by the Linux kernel on make isoimage using isohybrid .

Resources

http://wiki.osdev.org is a great source for those matters. https://github.com/scanlime/metalkit is a more automated / general bare metal compilation system, that provides a tiny custom API

edited yesterday

answered Sep 9 '15 at 15:23 Ciro Santilli 新疆改造中 心 六四事件 法轮功 131k 29 513 441

2

Unikernels are an alternative for people that cannot/do not want to go so low level and still want to benefit from their very low footprint. – AndreLDM yesterday

1

@AndreLDM I was on the verge of adding that Linux based Unikernel news, but felt too edgy yet: next.redhat.com/2018/11/14/ukl-a-unikernel-based-on-linux – Ciro Santilli 新疆改造中心 六四事件 法轮功 yesterday

10

Amazing answer. I learned something interesting today ! – cosmin_popescu yesterday

8

Really detailed answer but "a program that runs without an OS, is an OS" isn't true. You can write a program that just flashes an LED on/off but that doesn't make it an OS. Some firmware code that runs the microcontroller on your flash drive doesn't make it an OS. An OS is at a minimum an abstraction layer to write other software more easily. At a bare minimum these days I'd say if there isn't a scheduler it's likely not an OS. – Vitali yesterday

2

This answer has created an HN thread. Wow. – Manoj R 22 hours ago

protected by Raghav Sood yesterday Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

By using our site, you acknowledge that you have read and understand our Cookie Policy, Privacy Policy, and our Terms of Service.