An Introduction to DAOS- The overlooked side of AI [Math Mondays]
If you're looking to get into next generation computing, this might be a goldmine for you.
Hey, it’s your favorite cult leader here 🐱👤
Mondays are dedicated to theoretical concepts. We’ll cover ideas in Computer Science💻💻, math, software engineering, and much more. Use these days to strengthen your grip on the fundamentals and 10x your skills 🚀🚀.
To get access to all the articles and support my crippling chocolate milk addiction, consider subscribing if you haven’t already!
p.s. you can learn more about the paid plan here.
Before we get into this- in case you missed it, I recently wrote about Elon Musk’s new AI Bot Grok here. The article goes over the business relevance of Grok, how to improve your RAG systems, and how Big Tech Companies are using fear as a way to push government regulation to stifle competition. The first and third are applicable to everyone (even if you’re not a developer), so it’s important to raise your voice and fight back against monopolistic lobbying. Now on with today’s write-up.
Today, I was invited to attend the Super Compute ‘23 Conference in Denver. It was a very interesting event, and I got to see a completely different side of AI/Software. Today’s talks were dominated by people on the storage and platform sides, fields that I don’t typically interact with. However, I had a few very interesting conversations with the attendees/exhibitioners, and based on that, I wanted to let y’all know about DAOS (Distributed Asynchronous Object Storage). If you want to get into next-gen tech and work with big companies, here are a few reasons you should care about DAOS-
There is a lot of money being put into acquisitions/purchasing of hardware/DAOS enablement tools. Many of these initiatives are led by Cloud Providers and Computing Providers, making it very lucrative.
There is a serious lack of talent in developing DAOS products.
The learning curve is fairly steep, which means that you can develop your skills and people will value you pretty highly.
For example, below is a slide from Google’s presentation, where they talk about some challenges that they want to solve (and are actively looking for partners in)-
Given how new the space is, you can get some serious muscle and expertise by getting in early. This post will be an introduction to DAOS (I don’t really have the knowledge to provide more). Whether you’re a developer looking to specialize, a person looking to stay on top of tech developments, or an educator looking to modernize the curriculums, I would strongly recommend monitoring DAOS as a potential specialization.
Intel is taking a pivotal step forward, solidifying our dedication to open-source technology,” said Intel’s Hillarie Prestopine. “The DAOS Foundation will drive innovation in storage software, delivering what is essential for AI and high-performance computing.
- Intel is another big name that is investing into this project
Understanding DAOS
What is DAOS- Distributed Asynchronous Object Storage (DAOS) is integral to High-Performance Computing (HPC) due to its cutting-edge storage capabilities designed for next-generation computing needs. DAOS is a software-defined, scale-out object store that delivers high bandwidth, low latency, and supports high I/O operations per second (IOPS) for non-volatile memory.
DAOS and AI- DAOS has been tied deeply into AI platforms/infrastructure and will allow the development of stronger AI products. According to Intel documents, DAOS is oriented towards improving Reads at scale, which will allow more efficient pulls, queries, and joins- all mainstays in Data Engineering. “Storage is no longer driven by traditional workloads with large streaming writes like checkpoint/restart, but is increasingly driven by complex I/O patterns from new storage pillars. High-performance data-analytics workloads are generating vast quantities of random reads and writes. Artificial-intelligence (AI) workloads are reading far more than traditional high-performance computing (HPC) workloads. Data streaming from instruments into an HPC cluster require better quality of service (QoS) to avoid data loss. Data-access time is now becoming as critical as write bandwidth. New storage semantics are required to query, analyze, filter, and transform datasets.” IBM is another big name that has been very bullish on DAOS for the cloud.
Given all that I’ve seen, I’m very bullish on the space. That being said, I know next to nothing about this space. I always encourage skepticism, but in this case, I want to stress it because I don’t know enough to make educated opinions. Much of what I know is from preliminary reading and conversations. Given how new this space is, most people talking about it have vested interests (the person who invited me to the conference is a part of Google Cloud’s presenting team). The ideas discussed make sense to me so far, but I don’t have the ability to think critically here. I’ll keep you updated as I know more. If you have more insight into the field- please do reach out.
That being said, given the open-source nature of DAOS, it is possible to get involved at minimal costs. If you don’t already have something you want to specialize in, the ROI of trying it out is great.
That is it for this piece. I appreciate your time. As always, if you’re interested in working with me or checking out my other work, my links will be at the end of this email/post. If you found value in this write-up, I would appreciate you sharing it with more people. It is word-of-mouth referrals like yours that help me grow.
Save the time, energy, and money you would burn by going through all those videos, courses, products, and ‘coaches’ and easily find all your needs met in one place at ‘Tech Made Simple’! Stay ahead of the curve in AI, software engineering, and the tech industry with expert insights, tips, and resources. 20% off for new subscribers by clicking this link. Subscribe now and simplify your tech journey!
Using this discount will drop the prices-
800 INR (10 USD) → 640 INR (8 USD) per Month
8000 INR (100 USD) → 6400INR (80 USD) per year (533 INR /month)
Reach out to me
Use the links below to check out my other content, learn more about tutoring, reach out to me about projects, or just to say hi.
Small Snippets about Tech, AI and Machine Learning over here
AI Newsletter- https://artificialintelligencemadesimple.substack.com/
My grandma’s favorite Tech Newsletter- https://codinginterviewsmadesimple.substack.com/
Check out my other articles on Medium. : https://rb.gy/zn1aiu
My YouTube: https://rb.gy/88iwdd
Reach out to me on LinkedIn. Let’s connect: https://rb.gy/m5ok2y
My Instagram: https://rb.gy/gmvuy9
My Twitter: https://twitter.com/Machine01776819