A File Server From Scratch

In 2024, I was running low on disk space, and an idea came to me. The normal solution is to use some sort of cloud storage, but that's no fun! If there's a learning opportunity, I'll take it, so I decided I would code a file server from scratch.

Why Rust?

The question arose: What language should I use? Some quickly came to mind, such as Java or NodeJS. In the end, I decided on using Rust, for a couple of reasons.

1. It needs to pair well with the frontend - I want this file server and the client to be able to run natively on any computer. I also want the modelization and transfer of data to be as similar and efficient as possible. Having both the server and frontend written in the same language was important. Tauri was a perfect fit for this purpose.

2. Speed & Efficiency - This was crucial because I need to move terabytes of data between the server and my computer. Rust was built to be efficient and robust, which is exactly what the server needs to perform well enough to start competing with billion-dollar cloud-based services. As the size of the data grows and my needs become more complex, this means the server will scale exceptionally well. I was able to stream a 50MB image from the server to 100 different requests simultaneously without breaking a sweat. For now, this is more than enough.

3. Memory Safety - When working with volatile data, such as file transfers (especially asynchronously), knowing your pointers always point to valid memory at compile time is a must. Any dangerous errors could lead to permanently losing data or files if there's no backup.

4. Amazing Compiler - Rust has an exceptional compiler that catches memory leaks, data races, and more at compile time. It's also easy to read since it focuses on providing precise and user-friendly error messages that make debugging a breeze.

The Server

The server is designed to provide a fast and efficient file management solution. It listens for HTTP requests, processes them concurrently, and manages file storage. By employing asynchronous programming, the server is capable of handling numerous operations simultaneously, delivering exceptional performance and scalability.

Here are some of the libraries that help the server work asynchronously:

• Axum/Tokio- Tokio provides the Async I/O runtime and functions required to process requests asynchronously, and Axum for the routing abstraction.

• sysinfo - This crate fetches system info required to provide server monitoring and diagnostics

The server is separated into multiple routes:

/api/dirs/

the dirs route uses the dir_controller to handle any requests related to directories. This includes fetching a directory's metadata, deleting a directory, replacing a directory, editing a directory (e.g: renaming).

/api/files/

the files route uses the file_controller to handle any requests related to files. This includes streaming a file, fetching it's metadata, deleting a file, replacing a file, editing a file (e.g: renaming). It also allows fetching the server's home directory path and creating a relative path from the server.

/api/sys/

the sys route uses the sys_controller to handle any requests related to system information. This includes fetching system details, process details, hardware health and information, etc...

Responses

The server returns JSON responses based on the request and response. Example responses include:

Getting a directory's metadata:

Returns an array of the directory's content's metadata.

Streaming a file's content:

Returns a vec<u8> (array of bytes) along with a CONTENT_TYPE header to allow the client to know how to convert the bytes into the proper format.

Getting the server's info:

Returns an array of disks, the free memory available, host name, OS, total RAM and used RAM.

Client

The frontend client is made using Tauri, which is a framework for building desktop apps using Rust. The client is a web app that allows users to interact with the server. It provides a user-friendly interface for managing files and directories on the server.

The interface:

Viewing a file:

Server diagnostics at a glance:

Detailed server diagnostics: