- Dask DataFrame is not Pandas - Nov 22, 2021.
This article is the second article of an ongoing series on using Dask in practice. Each article in this series will be simple enough for beginners, but provide useful tips for real work. The next article in the series is about parallelizing for loops, and other embarrassingly parallel operations with dask.delayed.
Dask, Pandas, Python, Saturn Cloud
- Data Engineering Technologies 2021 - Sep 21, 2021.
Emerging technologies supporting the field of data engineering are growing at a rapid clip. This curated list includes the most important offerings available in 2021.
Abacus.ai, Dask, Data Engineering, Databricks, Dataiku, DataRobot, dbt, Fivetran, Pachyderm
- If You Can Write Functions, You Can Use Dask - Sep 21, 2021.
This article is the second article of an ongoing series on using Dask in practice. Each article in this series will be simple enough for beginners, but provide useful tips for real work. The first article in the series is about using LocalCluster.
Cloud, Dask, Python, Saturn Cloud
- Speeding up Neural Network Training With Multiple GPUs and Dask - Sep 14, 2021.
A common moment when training a neural network is when you realize the model isn’t training quickly enough on a CPU and you need to switch to using a GPU. It turns out multi-GPU model training across multiple machines is pretty easy with Dask. This blog post is about my first experiment in using multiple GPUs with Dask and the results.
Dask, GPU, Neural Networks, Training
- Building Machine Learning Pipelines using Snowflake and Dask - Jul 28, 2021.
In this post, I want to share some of the tools that I have been exploring recently and show you how I use them and how they helped improve the efficiency of my workflow. The two I will talk about in particular are Snowflake and Dask. Two very different tools but ones that complement each other well especially as part of the ML Lifecycle.
Dask, Machine Learning, Pipeline, Snowflake
Pandas not enough? Here are a few good alternatives to processing larger and faster data in Python - Jul 8, 2021.
While the Pandas library remains a crucial workhorse in data processing and management for data science, some limitations exist that can impact efficiencies, especially with very large data sets. Here, a few interesting alternatives to Pandas are introduced to improve your large data handling performance.
Dask, Modin, Pandas, Python, Scalability
- KDnuggets™ News 21:n10, Mar 10: More Resources for Women in AI, Data Science, and Machine Learning; Speeding up Scikit-Learn Model Training - Mar 10, 2021.
More Resources for Women in AI, Data Science, and Machine Learning; Speeding up Scikit-Learn Model Training; Dask and Pandas: No Such Thing as Too Much Data; 9 Skills You Need to Become a Data Engineer; 8 Women in AI Who Are Striving to Humanize the World
AI, Dask, Data Engineer, Data Science, Machine Learning, Modeling, Pandas, scikit-learn, Training, Women
- Dask and Pandas: No Such Thing as Too Much Data - Mar 4, 2021.
Do you love pandas, but don't love it when you reach the limits of your memory or compute resources? Dask provides you with the option to use the pandas API with distributed data and computing. Learn how it works, how to use it, and why it’s worth the switch when you need it most.
Dask, Distributed Computing, Pandas
- KDnuggets™ News 21:n09, Mar 3: Top YouTube Channels for Data Science; Data Science Learning Roadmap for 2021 - Mar 3, 2021.
The top YouTube channels for Data Science; they will help you with Data Science Learning Roadmap for 2021; Another great learning option is Machine Learning Systems Design: A Free Stanford Course; and if you are still using pandas to process large datasets, here are two better options.
Dask, Data Science, Roadmap, Stanford, Youtube
Are You Still Using Pandas to Process Big Data in 2021? Here are two better options - Mar 1, 2021.
When its time to handle a lot of data -- so much that you are in the realm of Big Data -- what tools can you use to wrangle the data, especially in a notebook environment? Pandas doesn’t handle really Big Data very well, but two other libraries do. So, which one is better and faster?
Big Data, Dask, Data Preparation, Pandas, Python, Vaex
- Distributed and Scalable Machine Learning [Webinar] - Feb 17, 2021.
Mike McCarty and Gil Forsyth work at the Capital One Center for Machine Learning, where they are building internal PyData libraries that scale with Dask and RAPIDS. For this webinar, Feb 23 @ 2 pm PST, 5pm EST, they’ll join Hugo Bowne-Anderson and Matthew Rocklin to discuss their journey to scale data science and machine learning in Python.
Capital One, Dask, Distributed, Machine Learning, Python, scikit-learn, XGBoost
- Computer Vision at Scale With Dask And PyTorch - Nov 23, 2020.
A tutorial on conducting image classification inference using the Resnet50 deep learning model at scale with using GPU clusters on Saturn Cloud. The results were: 40x faster computer vision that made a 3+ hour PyTorch model run in just 5 minutes.
Computer Vision, Dask, Python, PyTorch, Scalability
- KDnuggets™ News 20:n43, Nov 11: The Best Data Science Certification You’ve Never Heard Of; Essential data science skills that no one talks about - Nov 11, 2020.
The Best Data Science Certification You've Never Heard Of; Essential data science skills that no one talks about; Pandas on Steroids: End to End Data Science in Python with Dask; How to Build a Football Dataset with Web Scraping; 2 Coding-free Ways to Extract Content From Websites to Boost Web Traffic
Certification, Courses, Dask, Data Science, Data Science Skills, Football, Pandas, Python, Soccer, Web Scraping
Pandas on Steroids: End to End Data Science in Python with Dask - Nov 6, 2020.
End to end parallelized data science from reading big data to data manipulation to visualisation to machine learning.
Dask, Data Science, Pandas, Python
- Data Science in the Cloud with Dask - Oct 20, 2020.
Scaling large data analyses for data science and machine learning is growing in importance. Dask and Coiled are making it easy and fast for folks to do just that. Read on to find out how.
Cloud, Dask, Data Science, Python
- Machine Learning in Dask - Jun 22, 2020.
In this piece, we’ll see how we can use Dask to work with large datasets on our local machines.
Dask, Machine Learning, Pandas, Python
- KDnuggets™ News 20:n16, Apr 22: Scaling Pandas with Dask for Big Data; Dive Into Deep Learning: The Free eBook - Apr 22, 2020.
4 Steps to ensure your AI/Machine Learning system survives COVID-19; State of the Machine Learning and AI Industry; A Key Missing Part of the Machine Learning Stack; 5 Papers on CNNs Every Data Scientist Should Read
AI, Big Data, Coronavirus, COVID-19, Dask, Deep Learning, Free ebook, Machine Learning, Pandas
- Why and How to Use Dask with Big Data - Apr 15, 2020.
The Pandas library for Python is a game-changer for data preparation. But, when the data gets big, really big, then your computer needs more help to efficiency handle all that data. Learn more about how to use Dask and follow a demo to scale up your Pandas to work with Big Data.
Big Data, Dask, Data Engineering
- Five Interesting Data Engineering Projects - Mar 17, 2020.
As the role of the data engineer continues to grow in the field of data science, so are the many tools being developed to support wrangling all that data. Five of these tools are reviewed here (along with a few bonus tools) that you should pay attention to for your data pipeline work.
Dask, Data Engineering, dbt, DVC, Python
- Learn Quantum Computing with Python and Q#, Get Programming with Python, Data Science with Python and Dask - Sep 4, 2019.
Save 40% on Get Programming with Python, Data Science with Python and Dask, and Learn Quantum Computing with Python and Q# with code nlpython40.
Dask, Manning, Programming, Python, Quantum Computing
- K-means Clustering with Dask: Image Filters for Cat Pictures - Jun 18, 2019.
How to recreate an original cat image with least possible colors. An interesting use case of Unsupervised Machine Learning with K Means Clustering in Python.
Clustering, Dask, Image Classification, Image Recognition, K-means, Python, Unsupervised Learning
- Top KDnuggets tweets, Jan 30 – Feb 05: state-of-the-art in #AI, #MachineLearning - Feb 6, 2019.
Also Brilliant tour-de-force! Reinforcement Learning to solve Rubiks Cube; Dask, Pandas, and GPUs: first steps; Neural network AI is simple. So Stop pretending you are a genius.
Dask, GPU, Pandas, Python, Reinforcement Learning, Top tweets
- Introducing Dask-SearchCV: Distributed hyperparameter optimization with Scikit-Learn - May 12, 2017.
We introduce a new library for doing distributed hyperparameter optimization with Scikit-Learn estimators. We compare it to the existing Scikit-Learn implementations, and discuss when it may be useful compared to other approaches.
Dask, Distributed Computing, Distributed Systems, Machine Learning, Optimization, scikit-learn
- Dask and Pandas and XGBoost: Playing nicely between distributed systems - Apr 27, 2017.
This blogpost gives a quick example using Dask.dataframe to do distributed Pandas data wrangling, then using a new dask-xgboost package to setup an XGBoost cluster inside the Dask cluster and perform the handoff.
Dask, Distributed Systems, Pandas, Python, XGBoost
- Top KDnuggets tweets, Sep 07-13: Dask for #Parallel Programming; Computationally generated Average Face - Sep 14, 2016.
Computationally generated Average Face; Dask for #Parallel Programming; The (Not So) New #DataScientist Venn Diagram; Human in #AI loop - #DeepLearning lets you take an image of a dress and show...
Dask, Deep Learning, Face Recognition, Fashion, Top tweets
- Introducing Dask for Parallel Programming: An Interview with Project Lead Developer - Sep 7, 2016.
Introducing Dask, a flexible parallel computing library for analytics. Learn more about this project built with interactive data science in mind in an interview with its lead developer.
Analytics, Continuum Analytics, Dask, Data Science, Distributed Computing, Parallelism, Python, Scientific Computing
