Storage types
An overview of the different storage types available in a computer and the cloud.
Hi Friends,
Welcome to the 67th issue of the Polymathic Engineer newsletter.
This week, we will discuss the different types of storage available on a computer and in the cloud.
When designing a system, software engineers tend to focus on the bigger picture but ignore the underlying components. However, understanding such components becomes increasingly important as the system grows in complexity.
It’s difficult to predict where bottlenecks, failures, or outages can occur, and the better you understand a system's components, the better you can try to prevent such issues before they happen.
Working on embedded devices and safety-critical systems for many years helped me develop a particular interest in and sensibility toward these topics.
The outline will be as follows:
Different types of storage in a computer
Different types of storage in the cloud
Interesting posts
Different types of storage in a computer
A computer has a hierarchy of memory types determined by speed and volume.
Speed measures how fast you can retrieve data, while volume measures how much data you can store.
Registers are the fastest possible memory and store the CPU's current instructions.
However, the most relevant type of memory for software developers is the Random Access Memory (RAM).
The main benefit of this memory is that you can access any storage location directly without reading or writing data sequentially. The main drawback is that this kind of memory is volatile, and data gets lost when power is off.
There are two main types of RAM: SRAM and DRAM. SRAM is usually associated with CPU caches. It provides lower latency but is more expensive. DRAM is slower but cheaper and ideal for main memory.
Non-volatile storage makes it possible to avoid losing data while the power is turned off. However, data access is very slow compared to RAM devices. The most popular nonvolatile storage devices are HDDs and SSDs.
Offline storage like CDs, DVDs, and USB flash drives are portable and compatible with various systems.
Different types of storage in the cloud
Being familiar with how the different types of storage in the cloud work is critical to choosing the best solution for your needs.
There are three main storage types you need to know:
Block storage divides data into fixed-sized blocks and saves each as a separate unit. Each block has a unique address assigned with a logical addressing scheme and is usually available as a volume. Block volumes can be set up as a file system and used that way, or applications can handle them directly. Block is the most flexible and performant form of storage, but it can only be used on a single server and can't be shared.
File storage is a general-purpose solution based on block storage that abstracts files and directories at a higher level, making it easier to work with them. It is accessible via file-level network protocols such as SMB and NFS, which makes it great for sharing large files and folders within an organization. File-level access control allows permissions to be set up and access control lists (ACLs) to increase security.
Object storage is the more recently introduced form of storage. It is a large-scale, low-cost, durable option that can store and back up unstructured data. It stores data as flat objects, usually accessible through RESTful API. It is slower than other storage options. AWS S3 and Azure blob storage are well-known object storage implementations.
There are good and bad things about block, object, and file storage that you should keep in mind.
For example:
block storages have a tight server connection that limits scalability
file storage faces challenges with managing lots of small files
object storage's slower performance is not an excellent fit for real-time needs.
Interesting posts
I get the point of the dilemma because I have it as well. But thinking rationally, software engineers are in a lucky position. They can take the risk of building something on their own, and the worst thing that could happen if they fail is to return to a traditional job. Link
I found this report very interesting. We still broadly use programming languages and technologies introduced 20 years ago. There is no reason to run after every new framework or technology. Link
Many people working in tech see their job as only a way to make a lot of money and then retire. But there is more. You spend a lot of time working, and that time impacts what you are and what you leave behind in the world. Link
There is a fun or not-so-fun fact about computer programming in the 50s. Before it became a financially rewarding profession, women mostly held all programming activities.
You don't have to worry about fixing your workplace. Your responsibility is to protect your health, career, and the people you care about. Link
Thanks for the write up ! Silly question but where physically is data from the cloud stores? Are those what we call data centres which are kind of like giant warehouses with a bunch of severs?