- Building AI Models for High-Frequency Streaming Data – Part Two - Dec 10, 2020.
Many data scientists have implemented machine or deep learning algorithms on static data or in batch, but what considerations must you make when building models for a streaming environment? In this post, we will discuss these considerations.
- Building AI Models for High-Frequency Streaming Data - Dec 2, 2020.
This post is the first in a two-part series on AI for streaming data. Here, we’ll walk through strategies for aligning times and resampling the data.
- Toward a More Effective Disease Outbreak Alert System: A Symptoms Approach to Biosurveillance [Nov 19 webinar] - Nov 12, 2020.
Learn how the use of more granular symptoms-level data combined with innovative statistical techniques has the potential to identify disease outbreaks faster while limiting false positives.
- Do’s and Don’ts of Analyzing Time Series - Nov 12, 2020.
When handling time series data in your Data Science analysis work, a variety of common mistakes are made that are basic, but very important, to the processing of this type of data. Here, we review these issues and recommend the best practices.
- KDnuggets™ News 20:n42, Nov 4: Top Python Libraries for Data Science, Data Visualization & Machine Learning; Mastering Time Series Analysis - Nov 4, 2020.
Top Python Libraries for Data Science, Data Visualization, Machine Learning; Mastering Time Series Analysis with Help From the Experts; Explaining the Explainable AI: A 2-Stage Approach; The Missing Teams For Data Scientists; and more.
- Mastering Time Series Analysis with Help From the Experts - Oct 28, 2020.
Read this discussion with the “Time Series” Team at KNIME, answering such classic questions as "how much past is enough past?" others that any practitioner of time series analysis will find useful.
- 10 Underrated Python Skills - Oct 21, 2020.
Tips for feature analysis, hyperparameter tuning, data visualization and more.
- KDnuggets™ News 20:n37, Sep 30: Introduction to Time Series Analysis in Python; How To Improve Machine Learning Model Accuracy - Sep 30, 2020.
Learn how to work with time series in Python; Tips for improving Machine Learning model accuracy from 80% to over 90%; Geographical Plots with Python; Best methods for making Python programs blazingly fast; Read a complete guide to PyTorch; KDD Best Paper Awards and more.
- Introduction to Time Series Analysis in Python - Sep 24, 2020.
Data that is updated in real-time requires additional handling and special care to prepare it for machine learning models. The important Python library, Pandas, can be used for most of this work, and this tutorial guides you through this process for analyzing time-series data.
- Visualization Of COVID-19 New Cases Over Time In Python - Sep 15, 2020.
Inspired by another concise data visualization, the author of this article has crafted and shared the code for a heatmap which visualizes the COVID-19 pandemic in the United States over time.
- Understanding Time Series with R - Jul 9, 2020.
Analyzing time series is such a useful resource for essentially any business, data scientists entering the field should bring with them a solid foundation in the technique. Here, we decompose the logical components of a time series using R to better understand how each plays a role in this type of analysis.
- Forecasting Stories 4: Time-series too, Causal too - Jun 1, 2020.
This article is about the story of taking effective business decisions basis a combined model. Let us together study how these components work hand in hand.
- Forecasting Stories 3: Each Time-series Component Sings a Different Song - May 8, 2020.
With time-series decomposition, we were able to infer that the consumers were waiting for the highest sale of the year rather than buying up-front.
- LSTM for time series prediction - Apr 27, 2020.
Learn how to develop a LSTM neural network with PyTorch on trading data to predict future prices by mimicking actual values of the time series data.
- Forecasting Stories 2: The Power of a Seasonality Index - Apr 14, 2020.
Read this second entry in a series on time series analysis and seasonality, and see how, through 2 simple use cases, the power of a seasonality index is uncovered.
- KDnuggets™ News 20:n13, Apr 1: Effective visualizations for pandemic storytelling; Machine learning for time series forecasting - Apr 1, 2020.
This week, read about the power of effective visualizations for pandemic storytelling; see how (not) to use machine learning for time series forecasting; learn about a deep learning breakthrough: a sub-linear deep learning algorithm that does not need a GPU?; familiarize yourself with how to painlessly analyze your time series; check out what can we learn from the latest coronavirus trends; and... KDnuggets topics?!? Also, much more.
- How (not) to use Machine Learning for time series forecasting: The sequel - Mar 30, 2020.
Developing machine learning predictive models from time series data is an important skill in Data Science. While the time element in the data provides valuable information for your model, it can also lead you down a path that could fool you into something that isn't real. Follow this example to learn how to spot trouble in time series data before it's too late.
- How To Painlessly Analyze Your Time Series - Mar 26, 2020.
The Matrix Profile is a powerful tool to help solve this dual problem of anomaly detection and motif discovery. Matrix Profile is robust, scalable, and largely parameter-free: we’ve seen it work for a wide range of metrics including website user data, order volume and other business-critical applications.
- KDnuggets™ News 20:n12, Mar 25: 24 Best (and Free) Books To Understand Machine Learning; Coronavirus Daily Change and Poll Analysis; 9 lessons learned during 1st year as a Data Scientist - Mar 25, 2020.
Read our analysis of coronavirus data and poll results; Use your time indoors to learn with 24 best and free books to understand Machine Learning; Study the 9 important lessons from the first year as a Data Scientist; Understand the SVM, a top ML algorithm; check a comprehensive list of AI resources for online learning; and more.
- Time Series Classification Synthetic vs Real Financial Time Series - Mar 18, 2020.
This article discusses distinguishing between real financial time series and synthetic time series using XGBoost.
- Forecasting Stories: Is it seasonality or not? - Mar 17, 2020.
Kicking off with a series of forecasting stories, starting with seasonality and its business applications. This first article speaks of course corrections that were based on weather and calendar driven seasonality.
- Introduction to Geographical Time Series Prediction with Crime Data in R, SQL, and Tableau - Feb 14, 2020.
When reviewing geographical data, it can be difficult to prepare the data for an analysis. This article helps by covering importing data into a SQL Server database; cleansing and grouping data into a map grid; adding time data points to the set of grid data and filling in the gaps where no crimes occurred; importing the data into R; running XGBoost model to determine where crimes will occur on a specific day
- Observability for Data Engineering - Feb 10, 2020.
Going beyond traditional monitoring techniques and goals, understanding if a system is working as intended requires a new concept in DevOps, called Observability. Learn more about this essential approach to bring more context to your system metrics.
- How to Get Started With Algorithmic Finance - Jan 23, 2020.
Algorithmic finance has been around for decades as a money-making tool, and it's not magic. Learn about some practical strategies along with and introduction to code you can use to get started.
- Stock Market Forecasting Using Time Series Analysis - Jan 9, 2020.
Time series analysis will be the best tool for forecasting the trend or even future. The trend chart will provide adequate guidance for the investor. So let us understand this concept in great detail and use a machine learning technique to forecast stocks.
- Predict Electricity Consumption Using Time Series Analysis - Jan 2, 2020.
Time series forecasting is a technique for the prediction of events through a sequence of time. In this post, we will be taking a small forecasting problem and try to solve it till the end learning time series forecasting alongside.
- AutoML for Temporal Relational Data: A New Frontier - Oct 30, 2019.
While AutoML started out as an automation approach to develop optimal machine learning pipelines, extensions of AutoML to Data Science embedded products can now enable the processing of much more, including temporal relational data.
- KDnuggets™ News 19:n41, Oct 30: Feature Selection: Beyond feature importance?; Time Series Analysis Using KNIME and Spark - Oct 30, 2019.
This week in KDnuggets: Feature Selection: Beyond feature importance?; Time Series Analysis: A Simple Example with KNIME and Spark; 5 Advanced Features of Pandas and How to Use Them; How to Measure Foot Traffic Using Data Analytics; Introduction to Natural Language Processing (NLP); and much, much more!
- Time Series Analysis: A Simple Example with KNIME and Spark - Oct 23, 2019.
The task: train and evaluate a simple time series model using a random forest of regression trees and the NYC Yellow taxi dataset.
- KDnuggets™ News 19:n37, Oct 2: The Future of Analytics & Data Science! Starting NLP with spaCy & Python - Oct 2, 2019.
This week, find out what the future of analytics and data science holds; get an introduction to spaCy for natural language processing; find out how to use time series analysis for baseball; get to know your data; read 6 bits of advice for data scientists; and much, much more!
- Using Time Series Encodings to Discover Baseball History’s Most Interesting Seasons - Sep 27, 2019.
Take me out to the ballgame! Take me out to the crowd! For the 2,829 seasons that have been played for 101 baseball teams since 1880, which seasons were unlike any others? Using SAX Encoding to recognize patterns in time series data, the most special years in baseball can be found.
- Detecting stationarity in time series data - Aug 20, 2019.
Explore how to determine if your time series data is generated by a stationary process and how to handle the necessary assumptions and potential interpretations of your result.
- Can we trust AutoML to go on full autopilot? - Jul 31, 2019.
We put an AutoML tool to the test on a real-world problem, and the results are surprising. Even with automatic machine learning, you still need expert data scientists.
- How to Use Python’s datetime - Jun 17, 2019.
Python's datetime package is a convenient set of tools for working with dates and times. With just the five tricks that I’m about to show you, you can handle most of your datetime processing needs.
- Separating signal from noise - Jun 4, 2019.
When we are building a model, we are making the assumption that our data has two parts, signal and noise. Signal is the real pattern, the repeatable process that we hope to capture and describe. The noise is everything else that gets in the way of that.
- Choosing Between Model Candidates - May 29, 2019.
Models are useful because they allow us to generalize from one situation to another. When we use a model, we’re working under the assumption that there is some underlying pattern we want to measure, but it has some error on top of it.
- DMIR Research Group at the University of Wurzburg: Postdoctoral Researcher in Machine Learning for Time Series Analysis [Wurzburg, Germany] - May 28, 2019.
The DMIR Research Group at the University of Würzburg offers a habilitation position for a postdoctoral researcher in the area of machine learning for temporal data.
- KDnuggets™ News 19:n19, May 15: Data Scientist – Best Job of the Year!; How (not) to use Machine Learning for time series forecasting - May 15, 2019.
"Please, explain." Interpretability of machine learning models; How to fix an Unbalanced Dataset; Data Science Poem; Customer Churn Prediction Using Machine Learning; A Complete Exploratory Data Analysis and Visualization for Text
- How (not) to use Machine Learning for time series forecasting: Avoiding the pitfalls - May 10, 2019.
We outline some of the common pitfalls of machine learning for time series forecasting, with a look at time delayed predictions, autocorrelations, stationarity, accuracy metrics, and more.
- KDnuggets™ News 19:n15, Apr 17: Time Series Forecasting with Neural Nets and LSTM; Why Data Scientists Need To Work In Groups - Apr 17, 2019.
Also: Why Data Scientists Need To Work In Groups; Data Science with Optimus - Intro; Make Your Own Job in Data Science; 2019 Best Masters in Data Science and Analytics - Europe Edition.
- Accelerating Time Series Analysis with Automated Machine Learning - Feb 14, 2019.
This IDC Solution Spotlight examines how automated machine learning tools can augment the analysis, modeling, and prediction of time series data to deliver easily understood and actionable insights for businesses in a simple and agile fashion. Get the report now.
- How To Fine Tune Your Machine Learning Models To Improve Forecasting Accuracy - Jan 23, 2019.
We explain how to retrieve estimates of a model's performance using scoring metrics, before taking a look at finding and diagnosing the potential problems of a machine learning algorithm.
- Sales Forecasting Using Facebook’s Prophet - Nov 28, 2018.
In this tutorial we’ll use Prophet, a package developed by Facebook to show how one can achieve this.
- KDnuggets™ News 18:n33, Sep 5: Practical Topic Modeling with Python; Classifying AI Technologies; Data Science Project Inspiration - Sep 5, 2018.
Also: An End-to-End Project on Time Series Analysis and Forecasting with Python; Financial Data Analysis - Data Processing 1: Loan Eligibility Prediction; OLAP queries in SQL: A Refresher; Word Vectors in Natural Language Processing: Global Vectors (GloVe)
- An End-to-End Project on Time Series Analysis and Forecasting with Python - Sep 3, 2018.
Time series are widely used for non-stationary data, like economic, weather, stock price, and retail sales in this post. We will demonstrate different approaches for forecasting retail sales time series.
- Autoregressive Models in TensorFlow - Aug 6, 2018.
This article investigates autoregressive models in TensorFlow, including autoregressive time series and predictions with the actual observations.
- Every time someone runs a correlation coefficient on two time series, an angel loses their wings - Jun 18, 2018.
We all know correlation doesn’t equal causality at this point, but when working with time series data, correlation can lead you to come to the wrong conclusion.
- Modelling Time Series Processes using GARCH - May 25, 2018.
To go into the turbulent seas of volatile data and analyze it in a time changing setting, ARCH models were developed.
Pages: 1 2
- Bitcoin Trade Signals - Apr 25, 2018.
This article covers the transformation of public emotions, big news and blockchain data into signals which can provide us with a better understanding as well as instructions for investing.
- How To Choose The Right Chart Type For Your Data - Apr 3, 2018.
The power of charts to assist in accurate interpretation is massive and that's why it is vital to select the correct type when you are trying to visualize data.
- Quick Feature Engineering with Dates Using fast.ai - Mar 16, 2018.
The fast.ai library is a collection of supplementary wrappers for a host of popular machine learning libraries, designed to remove the necessity of writing your own functions to take care of some repetitive tasks in a machine learning workflow.
- KDnuggets™ News 18:n10, Mar 7: Functional Programming in Python; Surviving Your Data Science Interview; Easy Image Recognition with Google Tensorflow - Mar 7, 2018.
- Time Series for Dummies – The 3 Step Process - Mar 5, 2018.
Time series forecasting is an easy to use, low-cost solution that can provide powerful insights. This post will walk through introduction to three fundamental steps of building a quality model.
- Survival Analysis for Business Analytics - Nov 27, 2017.
We compare survival analysis to other predictive techniques, and provide examples of how it can produce business value, with a focus on Kaplan-Meier and Cox Regression methods which have been underutilized in business analytics.
- Automated Feature Engineering for Time Series Data - Nov 20, 2017.
We introduce a general framework for developing time series models, generating features and preprocessing the data, and exploring the potential to automate this process in order to apply advanced machine learning algorithms to almost any time series problem.
- Top 6 errors novice machine learning engineers make - Oct 30, 2017.
What common mistakes beginners do when working on machine learning or data science projects? Here we present list of such most common errors.
- DeepSense: A unified deep learning framework for time-series mobile sensing data processing - Aug 2, 2017.
Compared to the state-of-art, DeepSense provides an estimator with far smaller tracking error on the car tracking problem, and outperforms state-of-the-art algorithms on the HHAR and biometric user identification tasks by a large margin.
Pages: 1 2
- What Data You Analyzed – KDnuggets Poll Results and Trends - Apr 26, 2017.
Image/video data analysis is surging, JSON replacing XML, anonymized data usage is growing in US and Europe (but not in Asia), itemsets and Twitter analysis is declining - some of the highlights of KDnuggets Poll on data types used.
- Time Series Analysis with Generalized Additive Models - Apr 18, 2017.
In this tutorial, we will see an example of how a Generative Additive Model (GAM) is used, learn how functions in a GAM are identified through backfitting, and learn how to validate a time series model.
- Introduction to Anomaly Detection - Apr 3, 2017.
This overview will cover several methods of detecting anomalies, as well as how to build a detector in Python using simple moving average (SMA) or low-pass filter.
- Visualizing Time-Series Change - Mar 9, 2017.
When creating time-series line charts, it’s important to consider which of the following messages you would like to communicate: Actual value of units? Change in absolute units? Percent change? Change from a specific point in time?
- Time Series Analysis: A Primer - Jan 17, 2017.
Time series analysis is a complex subject but, in short, when we use our usual cross-sectional techniques such as regression on time series data, variables can appear "more significant" than they really are and we are not taking advantage of the information the serial correlation in the data provides.
- Introduction to Forecasting with ARIMA in R - Jan 16, 2017.
ARIMA models are a popular and flexible class of forecasting model that utilize historical information to make predictions. In this tutorial, we walk through an example of examining time series for demand at a bike-sharing service, fitting an ARIMA model, and creating a basic forecast.
- Combining Different Methods to Create Advanced Time Series Prediction - Nov 16, 2016.
The results from combining methods for time series prediction have been quite promising. However, the degree of error for long-term predictions is still quite high. Sounds like a challenge, so some new experiments are forthcoming!
- The Great Algorithm Tutorial Roundup - Sep 20, 2016.
This is a collection of tutorials relating to the results of the recent KDnuggets algorithms poll. If you are interested in learning or brushing up on the most used algorithms, as per our readers, look here for suggestions on doing so!
- A simple approach to anomaly detection in periodic big data streams - Aug 24, 2016.
We describe a simple and scaling algorithm that can detect rare and potentially irregular behavior in a time series with periodic patterns. It performs similarly to Twitter's more complex approach.
- KxCon2016, International kdb+ programmer conference, May 19-22, Montauk, NY - Apr 22, 2016.
Kdb+ time-series database provides high performance analytics on very large-scale datasets. Kdb+ users and coders will gather for KxCon2016, 3 days of presentations and hands-on workshops.
- Deriving Better Insights from Time Series Data with Cycle Plots - Mar 9, 2016.
Visualization plays key role in analysis of time series data, to understand underlying trends. Here we are demonstrating the cycle plot which shows both the cycle or trend and the day-of-the-week or the month-of-the-year effect.
- Anomaly Detection in Predictive Maintenance with Time Series Analysis - Dec 9, 2015.
How can we predict something we have never seen, an event that is not in the historical data? This requires a shift in the analytics perspective! Understand how to standardization the time and perform time series analysis on sensory data.
- Data-Planet Statistical Datasets - Nov 4, 2015.
Data-Planet Statistical Datasets provides easy access to an extensive repository of standardized and structured statistical data, with more than 25 billion data points from more than 70 source organizations.
- Top KDnuggets tweets, Jul 21-27: Beginner Guide to Time Series Analysis; Free Deep Learning online course - Jul 28, 2015.
Beginner #Guide to #TimeSeries #Analysis; Nvidia free #online course: Intro to #DeepLearning ; To Code or Not to Code with @KNIME; Guide To Linear #Regression
- Top KDnuggets tweets, Mar 23-25: 24 free resources on Data Mining, Data Science; More Training Data or More Complex Models? - Mar 26, 2015.
24 free resources and online books on #DataMining, #DataScience, #MachineLearning; New R Online Tool for Seasonal Adjustment of time series; Key #DataScience question: More Training Data or More Complex Models?; Twitter #DataMining finds origins of ISIS support.
- Top stories for Feb 1-7: Avoiding a Common Mistake with Time Series; Top Big Data Influencers and Brands - Feb 8, 2015.
Avoiding a Common Mistake with Time Series; (Deep Learning Deep Flaws) Deep Flaws; Top Big Data Influencers and Brands; Two Most Important Trends in Analytics and Big Data.
- Top KDnuggets tweets, Feb 2-3: Avoiding a Common Mistake with Time Series; A New Year in Data Science, great overview - Feb 4, 2015.
Avoiding a Common Mistake with Time Series: use de-trending; A New Year in #DataScience, great overview of the #MachineLearning and #BigData; Data scientist memes - the 'hottest profession'; Top Big Data Influencers and Brands.
- KDnuggets™ News 15:n04, Feb 4: Top Big Data Influencers; A Common Mistake with Time Series; Ayasdi - Feb 4, 2015.
Top Big Data Influencers and Brands; K-means clustering is not a free lunch; Avoiding a Common Mistake with Time Series; Ayasdi: Managing Data Complexity through Topology; Big Data Could Revolutionize Healthcare.
- Avoiding a Common Mistake with Time Series - Feb 2, 2015.
We explore a common mistake in analyzing relationships between time series, and show how de-trending helps to avoid this error.
- “Vite fait, bien fait” – Averaging improves both accuracy and speed of time series classification - Dec 21, 2014.
Time series classification using k-nearest neighbors and dynamic time warping can be improved in many practical applications in both speed and accuracy using averaging.
- SPOTLIGHT: Can Data Science Save Humanity from Mosquitoes and other Deadly Insects? #2 - Oct 9, 2014.
KDnuggets launches Spotlight initiative to bring attention to academic research. The journey begins with Prof. Eamonn Keogh, UCR and his talented student, Yanping Chen, who are applying data mining to save us all from insect-vectored diseases.
- SPOTLIGHT: Can Data Science Save Humanity from Mosquitoes and other Deadly Insects? - Oct 8, 2014.
KDnuggets launches Spotlight initiative to bring attention to academic research. The journey begins with Prof. Eamonn Keogh and his student, Yanping Chen, who are applying data mining to save us all from insect-vectored diseases.
- Interview: Leo Meyerovich, Graphistry on Browser-based Interactive Big Data Visualization - Jul 24, 2014.