/r/osdev
I am following osdev.org for making my first kernel, and it uses VGA buffer as output device. But i am not really understanding what vga buffer mode is? and how is it exactly, manipulating it?
I have been exploring the “Raspberry Pi Bare Bones” tutorial on wiki.osdev.org. From what I understand, the proprietary firmware/bootloader initializes the hardware and then loads and executes kernel8.img.
I am looking for a detailed list of the initializations performed by the firmware/bootloader, such as setting secondary cores in a spin loop or partitioning the RAM. In my opinion, a kernel developer needs precise information about the state of the Raspberry Pi hardware before the kernel starts. However, I have not been able to find official documentation that provides these details.
I have read the boot sequence documentation on the Raspberry Pi site, which offers some insights, but it does not provide specific details about the hardware's final state as configured by default.
Hi! I was looking to better my understanding of Rust and OS kernels in general, and I stumbled upon the great Writing an OS in Rust series by Philipp Oppermann. I worked my way through until he got to interrupts where he used the x86 crate more heavily, so instead I made my way to the older version of the handler posts as it was a bit more in-depth. Now I am trying to implement the GDT with the x86 crate as I would like to get to some sort of interactivity with this kernel sooner, however I am running into the issue where I am (seemingly) loading the Interrupt Stack Table into memory, specifically with a stack for the double-fault exception (pointing to a static mut
byte array) however my handler never seems to switch to it in the event of a double fault and instead the system triple faults and resets. I am just wondering if I am missing a step in my double fault handler? Do I need to manually switch the stack over to the double-fault stack?
lazy_static! {
pub static ref IDT: idt::Idt = {
let mut idt = idt::Idt::new();
idt.set_handler(0, handler!(zero_div_handler), None);
idt.set_handler(3, handler!(breakpt_handler), None);
idt.set_handler(6, handler!(invalid_op_handler), None);
// set double fault handler options (IST index)
let mut double_fault_options = EntryOptions::new();
double_fault_options.set_stack_idx(DOUBLE_FAULT_IST_IDX);
idt.set_handler(8, handler_with_errcode!(double_fault_handler), Some(double_fault_options));
idt.set_handler(14, handler_with_errcode!(pg_fault_handler), None);
idt
};
}
// initialize the TSS
// use lazy_static! again to allow for one time static assignment at runtime
lazy_static! {
static ref TSS: TaskStateSegment = {
let mut tss = TaskStateSegment::new();
// note: this double_fault_handler() stack as no guard page so if we do
// anything that uses the stack too much it could overflow and corrupt
// memory below it
tss.interrupt_stack_table[DOUBLE_FAULT_IST_IDX as usize] = {
// calculate size of the stack
const STACK_SIZE: usize = 4096 * 5;
// initialize stack memory to all zeroes
// currently don't have any memory management so need to use `static mut`
// must be `static mut` otherwise the compiler will map the memory to a
// read-only page
static mut STACK: [u8; STACK_SIZE] = [0; STACK_SIZE];
// calculate beginning and end of the stack and return a pointer
// to the end limit of the stack
#[allow(static_mut_refs)]
let stack_start = VirtAddr::from_ptr(unsafe {core::ptr::from_ref(&STACK)} );
stack_start + STACK_SIZE // top of the stack from where it can grow downward
};
tss
};
}
Note: I know that issues should be submitted through the blog_os github but I have been unsuccessful in getting any responses there.
Context:
I understand this might not be sufficient context so here is my code in it's current state: My Github Repo
If anyone could help it'd be greatly appreciated as I'd love to be able to keep progressing
What learning tools do you recommend for learning UEFI? I already know about the quesofuego tutorial, the specification, and the beyond bios book. What do you all recommend for learning?
Hello, I've created a new operating system that implements cryptographic verification of all system operations, written from scratch in Rust.
VEKOS (Verified Experimental Kernel OS) uses Merkle trees and operation proofs to ensure system integrity - something I have never seen implemented in other OSes so I gave it a try(that's why it's experimental).
It has a working shell with core utilities and I'd love feedback from the community, especially on the verification system. If you have any question on the innerworkings of the development, just ask and I will gladly answer all questions.
Some context:
This is a part of my Makefile that's concerned with building the kernel floppy image:
$(BUILD_DIR)/$(OS_IMG_FLP): $(BUILD_DIR)/boot.bin $(BUILD_DIR)/kernel.bin
cat $< > $(BUILD_DIR)/$(OS_IMG_BIN)
cat $(BUILD_DIR)/kernel.bin >> $(BUILD_DIR)/$(OS_IMG_BIN)
dd if=$(BUILD_DIR)/$(OS_IMG_BIN) of=$(BUILD_DIR)/$(OS_IMG_FLP) bs=512 conv=notrunc
Now, I want to somehow put this stuff on my flash drive which has 7.2 gb. I tried just emptying the flash drive and using "dd if=os-image.flp of=/dev/sdc1 conv=notrunc". It actually loads but I get an error when reading the disk. I also tried using LBA48, but I can't get it to work on qemu, let alone on the flash drive. Help would be appreciated, thanks.
EDIT: Obviously I am trying to read data from disk before going into protected mode.
Hello,
I was exploring ways to reduce the number of IPI's sent to cores in the TLB shootdown protocol. One of the optimizations done in Linux (according to Understanding the Linux Kernel) is that for every core, the kernel tracks the tlb state. The book says:
When a CPU starts executing a kernel thread, the kernel sets the state field of its cpu_tlbstate element to TLBSTATE_LAZY.
My first question is what is meant by a kernel thread in this context? I assume it means any execution context for a process that runs in kernel mode? So would the "starts executing a kernel thread" happen only on a system call, interrupt, or exception? However, it also says that "no kernel thread accesses the user mode address space" which isn't true (i.e reading a file into a userspace buffer)? So this made me think maybe it's just referring to a CPU running kernel code but not in the context of any process (i.e in the scheduler?).
My second question relates to how the book describes that when a thread initiates a TLB shootdown by sending an IPI to all cores in the cpu_vm_mask, the core checks if it's CPU state is lazy and if it is, doesn't perform the shootdown and removes itself from the cpu_vm_mask. Why does the CPU remove itself from cpu_vm_mask only after receiving the first IPI? Why not remove itself immediately when it goes into the TLBSTATE_LAZY thus removing all IPIs to begin with? Is it a tradeoff to reduce extra work of removing the CPU index from the cpu_vm_mask in case the TLB shootdown doesn't occur? Although I would think that even one IPI is more expensive than that.
My third question is about a reply in this post (https://forum.osdev.org/viewtopic.php?t=23569) which says Lazy TLB mode is a technique in which the kernel toggles some permission or present/not present flag in the PTE to induce a page fault in other threads that try to access the PTE and then the kernel invalidates the TLB entry in the page fault handler. However, this seems to differ from the books description of lazy tlb mode, so is this not a universal term? Also, this approach doesn't seem correct because if the other cores have the PTE cached in the TLB then modifying PTE bits doesn't really matter to prevent it's use.
It'd be great if anyone understands these and can share! Thank you.
I want to build an OS as a project. I followed this https://youtube.com/playlist?list=PLBlnK6fEyqRiVhbXDGLXDk_OQAeuVcp2O&feature=shared Neso academy's course for learning. As I don't have practical / lab experience with OS I don't think I am ready to build an OS. So could you please help me by mentioning practical resources of OS and prerequisites that are required so that I am ready to start my project.
A group of my friends and I have been working on an operating system for a few years now.
We’ve just finished work on a primitive windowing system and are looking for more contributors for things like drivers.
Check the project out here https://git.ablecorp.us/ableos/ableos
Folks working in "operating systems" and "distributed systems" field, what kinds of projects are you guys working on in the company or personally? Can you share what kind of problems you guys are solving? Feel free to share details if possible even though it may be highly technical. TYIA.
Some good projects in the field of operating systems in C or Rust. What are you guys working on in the company projects?
Hello, I am building an OS in mostly assembler and a bit C, I am now busy with the video setup / graphics driver. First I found VESA / VBE, but I think it's not that efficient, is it? I want any resolution display, 32-bit color depth.
Does anyone know what the most efficient way of driving to the screen is? It's not a big problem if it is really hard to understand, if I've a goal I can spend hours / weeks / months to archive it.
Syscalls cause invalid opcode exception
I need to create an diskette with the bootloader at the first disk sector and the kernel at the second disk sector. how can I do this? I already posted here how can I make this but nobody replies. I already tried to use DD but it doesn't worked. I also know an Windows application named Fergo Raw, but I don't know if it's safe. Can someone help me?
Which flags and sel for IDT i should use? And can i have an example? (legacy int syscalls not SYSCALL instruction!)
I am currently trying to build a basic operating system, and the biggest difficulties I am facing are understanding how addressing works, how segment registers relate to directives I am using like org, and how this relates to the 16-bit or 32-bit directive, and how this affects how calculations are done in real mode (segment * 16 + offset) and protected mode (based on the GDT).
Then I have other doubts about how the GDT works, because I saw that you define a base and a limit, but how does this work? After defining the GDT, what physical memory addresses become the data or code segments?
For example, I've been trying for two days to understand why my jmp CODE_OFFSET:func is giving an error on the virtual machine. From what I understand, it’s because this jump is going to an address outside the GDT or outside the code segment, but I don’t understand why.
It's been a while since a last posted here. However, after noticing some very tricky to reproduce bugs that certain users have, bugs that are almost certainly caused by their specific combination of hardware and software. I have made the decision to open up PatchworkOS for contribution.
There aren't any strict guidelines to contribute just yet, so if you are interested, feel to rummage around the code base. Bug fixes, improvements or other ideas are all welcome!
As of a few days ago I made a simple internal kernel debugger for PaybackOS it now has the command REG which allows the user to dump the registers.
Hi everyone, I am new to linux kernel os development.
I was learning the workqueue mechanism of linux.
I meet this codes:
When user want to queue a work to a workqueue, they call `__queue_work
` in the end after servera forwarding, at the beginning of this function, it first check if the workqueue is at destroying or draining state by reading a `flag` variable. But it doesn't use `mutex_lock
` to guard the read.
// linux-src-code/kernel/workqueue.c
static void __queue_work(int cpu, struct workqueue_struct *wq,
struct work_struct *work)
{
struct pool_workqueue *pwq;
struct worker_pool *last_pool, *pool;
unsigned int work_flags;
unsigned int req_cpu = cpu;
lockdep_assert_irqs_disabled();
if (unlikely(wq->flags & (__WQ_DESTROYING | __WQ_DRAINING) &&
WARN_ON_ONCE(!is_chained_work(wq))))
return;
...
}
But in the `drain_workqueue
` and `destroy_workqueue
`, they guard the `flags
` variable with mutex lock, this confuse me. I think there could be a race between reading and writing to the `flags
`:
// linux-src-code/kernel/workqueue.c
void drain_workqueue(struct workqueue_struct *wq)
{
unsigned int flush_cnt = 0;
struct pool_workqueue *pwq;
mutex_lock(&wq->mutex);
if (!wq->nr_drainers++)
wq->flags |= __WQ_DRAINING;
mutex_unlock(&wq->mutex);
reflush:
__flush_workqueue(wq);
...
}
void destroy_workqueue(struct workqueue_struct *wq)
{
struct pool_workqueue *pwq;
int cpu;
workqueue_sysfs_unregister(wq);
/* mark the workqueue destruction is in progress */
mutex_lock(&wq->mutex);
wq->flags |= __WQ_DESTROYING;
mutex_unlock(&wq->mutex);
...
}
My question is: why the read access of `wq->flags
` in `queue_work
` function is not guarded by mutex but the write access in `destroy_workqueue
` does.
In my college project using lazy allocation, so I decided to mark down the pages which have been allocated using one of my available permission bits but then I realized that I cant do so since malloc is called on the user side thus I have no access to page tables and permissions and I need to find the virtual address before using a system call to allocate the physical memory. How do I keep track of my previously allocated spaces? How do I check if I marked down a page while being in user side?
RoT: element within a system that is trusted and must always behave as expected because any misbehavior cannot be detected at runtime. It's part of the TCB.
TCB: the smallest set of hardware, firmware, software, and other resources (e.g., processes or people) that must be trusted. Any vulnerabilities within the TCB jeopardizes the system security.
What are the differences? They both need to be trusted because their misbehavior cannot be detected...
RoT is part of TCB. So can you tell me some element that is part of TCB but is NOT a RoT?
Can you give me a list of what is RoT and what is TCB?
I am a product designer currently looking/hoping to work on an open source/commercial operating system. If interested you can DM me.
Hello. I am writing a thesis on instructional OS and want to give OS/161 a shot, because it seems very promising.
The problem is, that the setup guide on the official site isn't much help in determining what kind of version of a Linux distro I should use, or if there are any Docker alternatives.
So far I tried setting up an Ubuntu VM. I tried version 24.04.1 LTS at first, but didn't have much luck. Next was 22.04, but I still had issues there and was unable to get it working. Mostly, there are issues around all the prerequisites for installing OS/161 and even gcc; this one gave me even more trouble, honestly.
I found some Docker solutions (like this for example), but so far haven't tried them. If the result is the same, I might reconsider even trying, because I've spent way too much time on this, since the official setup guide really doesn't exactly determine how it should be setup. There is even a "hint" in the guide, saying " I've had a report that gcc 4.8 doesn't build on the latest Ubuntu (16.10) but I haven't had a chance to investigate". This is really dissapointing, because apparently it is a requirement to be setup with version 4.8, but how am I supposed to "guess" the correct version then?
Anyway, I would really appreciate anyone helping me set this up. Currently, my goal is to have a fresh Linux VM (of a correct version, ofc) that can run OS/161 (and can finish the setup of all the prerequisites and so on).
THANK YOU!
EDIT: I decided that trying to set up my own VM with a working OS161 was too much work and I encountered way too many inconsistencies. In the end, I used this to get myself a Docker container with a prebuilt toolchain and it worked just fine. Also, the guide is very helpful. This is the repo: https://github.com/marcopalena/polito-os161-docker Thank you all for your help and support. And thank you to the author of the repo linked above.
Hello, I'm working on an ARMv7 machine (32 bit) using arm-none-eabi-gcc
.
I have a custom printf-like function but sometimes it prints trash when I try to print %lld
(unsigned long long aka uint64_t on this target). I'm using newlib so it's not my crappy code, but I've also tried another implementation and have the same issue.
Debugging I've discovered the issue happens when the printf calls va\_arg(va, unsigned long long)
since it gets trash instead of the actual value present on the stack.
Here's a commented trace of my gdb session
// At the start of my 'kprintf' function, which has just called 'va_start(ap, format)'
(gdb) p va
$11 = {__ap = 0xe14259a8}
// We're now about to pop a %llu value from the stack.
// The value is "32" and you can see it's there on the stack (0x00000020 0x00000000)
(gdb) p va.__ap
$13 = (void *) 0xe14259ac
(gdb) x/4x va.__ap
0xe14259ac: 0x00000020 0x00000000 0x20676e69 0x69207075
// This is immediately after va_arg(va, unsigned long long) was called, and the returned value stored in a "value" variable
(gdb) p va.__ap
$16 = (void *) 0xe14259b8
(gdb) p/x value
$17 = 0x20676e6900000000
Note how the va.__ap
pointer ended up being increased by 16 instead of just 8, and how value
ends up with the 8 bytes immediately after what it should have actually popped. The generated assembly first aligns the pointer to 8 byte, and only then reads the 8 bytes for the argument.
---
I think I gave enough context, so here's my question: why is this happening and how can I fix this?
Just to explain, both "X" and "T" are being printed to the screen, but the "L" is not, which indicates that the kernel is not being loaded.
teste
and teste2
are the same code for printing a character; the difference is that they use different words.
The carry flag has not been set at any point, which suggests that no error occurred during the int 13h
call.
[BITS 16]
[ORG 7C00H]
call loadKernel
call teste
jmp teste2
jmp 7e00h
ala db "Taa",0
veio db "Paa",0
loadKernel:
mov ah, 2h ; function to read sectors
mov al, 1 ; read only 1 sector
mov ch, 0 ; use the first cylinder
mov cl, 2 ; the sector to be read is sector 2
mov dh, 0 ; first head
mov dl, 80h ; 80h is the first disk in the boot order
mov bx, 7e00h ; address to load the data
mov es, bx ; set es to the data address
mov bx, 0 ; set bx to 0
int 13h ; read the disk
jc error_occurred ; check for errors
ret
kernel
[BITS 16]
[ORG 0]
call teste
jmp osMain
teste:
mov si, tesle
mov ah, 0eh
mov al, [si]
int 10h
jmp END
tesle db "Laa",0
I don't know where I went wrong; I presume it was when loading the kernel or jumping to its address, but I believe the problem is actually with loading it.
If anyone could give me some insight on this, I would appreciate it.
It seems that bootloaders need the directive
[ORG 7C00H]
From what I understand, this tells the assembler that, when the code is compiled into binary, the generated code should consider that it is located at address 7C00H and onwards, which means from byte 31744 onward. But this implies that if the RAM is smaller than this, for example, if it has only 31,000 bytes (31KB), the bootloader wouldn't work because the BIOS expects everything to be at 7C00H.
I have this OS course in college, we are using the intel 86 sytem according to my knowledge with 2 level paging, I am forced to implement a lazy allocation for my malloc(), so I am thinking of using the present bit as my marker, what are possible drawbacks?
When I tried to follow Higher Half x86 Bare Bones with my existing OS it failed, so I made a seperate branch and for some reason it worked, I am not sure why it failed on the main branch, is anyone willing to take a look?
Hello there!
I'm quite new to this forum and I hope that I can get help here:
I recently started developing a small operating system in UEFI with a C kernel. Now I wanted to add support for a filesystem, because an OS is unusable if it has no filesystem access. I used the EFI simple filesystem protocol, but I always get an error: Invalid Parameter
. I think the error occurs finding the block handle.
Here's my code on GitHub: https://github.com/CleverLemming1337/OS-Y/blob/main/src/filesystem.c
If anyone knows how to fix my error, I would be really happy!