A File Server From Scratch

As a developer, I'm always looking for new and useful projects to code that provide a solution to a problem. Recently, I've been running low on disk space in my desktop computer, and an idea came to me. I decided I would code a file server from scratch. The normal solution is to purchase some cloud storage, but that's no fun! If there's a learning opportunity, I'll take it.

Why Rust?

The question arose: What language should I use? Some quickly came to mind, such as Python, Java, and JavaScript. In the end, I decided on using Rust, for a couple of reasons.

A Learning Experience - I've used Rust before as a general-purpose programming language, but I've never dived too deeply into its low-level capabilities. I figured this project would be the perfect introduction to Rust's powerful arsenal.
Memory Safety - When working with volatile data, such as file transfers (especially asynchronously), knowing your pointers always point to valid memory at compile time is a must. Any dangerous errors could lead to permanently losing data or files if there's no backup.
Amazing Compiler - Rust has an exceptional compiler that catches memory leaks, data races, and more at compile time. It's also easy to read since it focuses on providing precise and user-friendly error messages that make debugging a breeze.
Speed & Efficiency - This was crucial because I need to move terabytes of data between the server and my computer. Rust was built to be efficient and robust, which is exactly what the server needs to perform well enough to compete with billion-dollar cloud-based services. As the size of the data grows and my needs become more complex, this means the server will scale exceptionally well. I was able to stream a 50MB image from the server to 100 different requests simultaneously without breaking a sweat. For now, this is more than enough.

How it works

The server is designed to provide a fast and efficient file management solution. It listens for HTTP requests, processes them concurrently, and manages file storage. By employing asynchronous programming, the server is capable of handling numerous operations simultaneously, delivering exceptional performance and scalability.

I know I said "from scratch", however there are 2 specific crates (libs/packages for non-rust readers) that I used to simplify the coding process.

Axum/Tokio- Tokio provides the Async I/O runtime required to process requests asynchronously, and Axum for the routing abstraction.
sysinfo - This crate fetches system info required to provide server monitoring and diagnostics

The server is separated into 3 different routes: /api/dirs/

/api/dirs/

the dirs route uses the dir_controller to handle any requests related to directories. This includes fetching a directory's metadata, deleting a directory, replacing a directory, editing a directory (e.g: renaming).

/api/files/

the files route uses the file_controller to handle any requests related to files. This includes streaming a file, fetching it's metadata, deleting a file, replacing a file, editing a file (e.g: renaming). It also allows fetching the server's home directory path and creating a relative path from the server.

/api/sys/

the sys route uses the sys_controller to handle any requests related to system information. This includes fetching system details, process details, hardware health and information, etc...

Responses

The server returns JSON responses based on the request and response. Example responses include:

Getting a directory's metadata:

Returns an array of the directory's content's metadata.

Streaming a file's content:

Returns a vec<u8> (array of bytes) along with a CONTENT_TYPE header to allow the client to know how to convert the bytes into the proper format.