- The Essential Toolbox for Data Cleaning - Dec 5, 2019.
Increase your confidence to perform data cleaning with a broader perspective of what datasets typically look like, and follow this toolbox of code snipets to make your data cleaning process faster and more efficient.
Tags: Data Cleaning, Data Preparation
- The Rise of User-Generated Data Labeling - Dec 4, 2019.
Let’s say your project is humongous and needs data labeling to be done continuously - while you’re on-the-go, sleeping, or eating. I’m sure you’d appreciate User-generated Data Labeling. I’ve got 6 interesting examples to help you understand this, let’s dive right in!
Tags: Data Preparation, Data Science, User Generated Content
- Three Methods of Data Pre-Processing for Text Classification - Nov 21, 2019.
This blog shows how text data representations can be used to build a classifier to predict a developer’s deep learning framework of choice based on the code that they wrote, via examples of TensorFlow and PyTorch projects.
Tags: Data Preparation, IBM, Text Classification
- Pro Tips: How to deal with Class Imbalance and Missing Labels - Nov 20, 2019.
Your spectacularly-performing machine learning model could be subject to the common culprits of class imbalance and missing labels. Learn how to handle these challenges with techniques that remain open areas of new research for addressing real-world machine learning problems.
Tags: Balancing Classes, Data Preparation, Missing Values, Tips, Unbalanced
How to Speed up Pandas by 4x with one line of code - Nov 12, 2019.
While Pandas is the library for data processing in Python, it isn't really built for speed. Learn more about the new library, Modin, developed to distribute Pandas' computation to speedup your data prep.
Tags: Data Preparation, Data Preprocessing, Modin, Pandas, Python
- Set Operations Applied to Pandas DataFrames - Nov 7, 2019.
In this tutorial, we show how to apply mathematical set operations (union, intersection, and difference) to Pandas DataFrames with the goal of easing the task of comparing the rows of two datasets.
Tags: Data Preparation, Data Science, Pandas, Python
- How to Create a Vocabulary for NLP Tasks in Python - Nov 7, 2019.
This post will walkthrough a Python implementation of a vocabulary class for storing processed text data and related metadata in a manner useful for subsequently performing NLP tasks.
Tags: Data Preparation, Data Preprocessing, NLP, Python
- How Data Labeling Facilitates AI Models - Oct 31, 2019.
AI-based models are highly dependent on accurate, clean, well-labeled, and prepared data in order to produce the desired output and cognition. These models are fed with bulky datasets covering an array of probabilities and computations to make its functioning as smart and gifted as human intelligence.
Tags: AI, Data Preparation, Data Preprocessing
- 5 Advanced Features of Pandas and How to Use Them - Oct 25, 2019.
The pandas library offers core functionality when preparing your data using Python. But, many don't go beyond the basics, so learn about these lesser-known advanced methods that will make handling your data easier and cleaner.
Tags: Data Preparation, Pandas, Python
- Know Your Data: Part 2 - Oct 8, 2019.
To build an effective learning model, it is must to understand the quality issues exist in data & how to detect and deal with it. In general, data quality issues are categories in four major sets.
Tags: Beginners, Data Preparation, Data Preprocessing, Datasets
- Data Preparation for Machine learning 101: Why it’s important and how to do it - Oct 2, 2019.
As data scientists who are the brains behind the AI-based innovations, you need to understand the significance of data preparation to achieve the desired level of cognitive capability for your models. Let’s begin.
Tags: Data Preparation, Data Science, Machine Learning
- Data Mapping Using Machine Learning - Sep 27, 2019.
Data mapping is a way to organize various bits of data into a manageable and easy-to-understand system.
Tags: Data Cleaning, Data Preparation, Machine Learning
- KDnuggets™ News 19:n28, Jul 31: Top 13 Skills To Become a Rockstar Data Scientist; Best Podcasts on AI, Analytics, Data Science - Jul 31, 2019.
Learn the essential skills needed to become a Data Science rockstar; Understand CNNs with Python + Tensorflow + Keras tutorial; Discover the best podcasts about AI, Analytics, Data Science; and find out where you can get the best Certificates in the field
Tags: Convolutional Neural Networks, Data Preparation, Data Science Certificate, Data Science Skills, Podcast, Python, TensorFlow
Fantastic Four of Data Science Project Preparation - Jul 26, 2019.
This article takes a closer look at the four fantastic things we should keep in mind when approaching every new data science project.
Tags: Comic, Data Exploration, Data Preparation, Data Science, Domain Knowledge
- KDnuggets™ News 19:n24, Jun 26: Understand Cloud Services; Pandas Tips & Tricks; Master Data Preparation w/ Python - Jun 26, 2019.
Happy summer! This week on KDnuggets: Understanding Cloud Data Services; How to select rows and columns in Pandas using [ ], .loc, iloc, .at and .iat; 7 Steps to Mastering Data Preparation for Machine Learning with Python; Examining the Transformer Architecture: The OpenAI GPT-2 Controversy; Data Literacy: Using the Socratic Method; and much more!
Tags: Cloud, Data Preparation, Machine Learning, NLP, OpenAI, Pandas, Python
7 Steps to Mastering Data Preparation for Machine Learning with Python — 2019 Edition - Jun 24, 2019.
Interested in mastering data preparation with Python? Follow these 7 steps which cover the concepts, the individual tasks, as well as different approaches to tackling the entire process from within the Python ecosystem.
Tags: 7 Steps, Data Preparation, Data Preprocessing, Data Science, Data Wrangling, Machine Learning, Pandas, Python
How to select rows and columns in Pandas using [ ], .loc, iloc, .at and .iat - Jun 19, 2019.
Subset selection is one of the most frequently performed tasks while manipulating data. Pandas provides different ways to efficiently select subsets of data from your DataFrame.
Tags: Data Cleaning, Data Preparation, Jupyter, Pandas, Python
- Crowdsourcing vs. Managed Teams: A Study in Data Labeling Quality - Jun 12, 2019.
You need data labeled for ML. You can do it in-house, crowdsource it, or hire a managed service. If data quality matters, read this.
Tags: AI, Cloudfactory, Crowdsourcing, Data Preparation
- 5 Ways to Deal with the Lack of Data in Machine Learning - Jun 10, 2019.
Effective solutions exist when you don't have enough data for your models. While there is no perfect approach, five proven ways will get your model to production.
Tags: Data Preparation, Datasets, Machine Learning, Synthetic Data, Transfer Learning
- End-to-End Machine Learning: Making videos from images - May 23, 2019.
Video is a natural way for us to understand three dimensional and time varying information. Read this short post on how to achieve the creation of videos from still images.
Tags: Data Preparation, Image Processing, Machine Learning
- How to fix an Unbalanced Dataset - May 8, 2019.
We explain several alternative ways to handle imbalanced datasets, including different resampling and ensembling methods with code examples.
Tags: Balancing Classes, Data Preparation, Machine Learning, Unbalanced
- Top R Packages for Data Cleaning - Mar 15, 2019.
Data cleaning is one of the most important and time consuming task for data scientists. Here are the top R packages for data cleaning.
Tags: Data Cleaning, Data Preparation, Data Science, Machine Learning, R
- Preparing for the Unexpected - Feb 28, 2019.
In some domains, new values appear all the time, so it's crucial to handle them in a good way. Using deep learning, one can learn a special Out-of-Vocabulary embedding for these new values. But how can you train this embedding to generalize well to any unseen value? We explain one of the methods employed at Taboola.
Tags: Data Preparation, Data Science, Overfitting, Recommender Systems
- Acquiring Labeled Data to Train Your Models at Low Costs - Feb 27, 2019.
We discuss groundbreaking and unique methods to acquire labeled data at low cost, including 3rd-Party Plug-and-Play AI Model, Zero-Shot Learning, and Restructuring the Existing Data Set.
Tags: Data Preparation, Machine Learning, ParallelDots, Training Data, Transfer Learning
- Automatic Machine Learning is broken - Feb 19, 2019.
We take a look at the arguments against implementing a machine learning solution, and the occasions when the problems faced are not ML problems and can perhaps be solved using optimization, exploratory data analysis tasks or problems that can be solved with simple statistics.
Tags: Automated Machine Learning, AutoML, Data Preparation, Deployment
- Feature engineering, Explained - Dec 21, 2018.
A brief introduction to feature engineering, covering coordinate transformation, continuous data, categorical features, missing values, normalization, and more.
Tags: Data, Data Preparation, Data Processing, Feature Engineering, Normalization
- Six Steps to Master Machine Learning with Data Preparation - Dec 21, 2018.
To prepare data for both analytics and machine learning initiatives teams can accelerate machine learning and data science projects to deliver an immersive business consumer experience that accelerates and automates the data-to-insight pipeline by following six critical steps.
Tags: Data Preparation, Machine Learning
- Exploring the Data Jungle Free eBook - Dec 18, 2018.
This free eBook by Brian Godsey will provide you with real-world examples in Python, R, and other languages suitable for data science.
Tags: Data Preparation, Data Science, Data Visualization, Free ebook, Manning, Python, R
Common mistakes when carrying out machine learning and data science - Dec 6, 2018.
We examine typical mistakes in Data Science process, including wrong data visualization, incorrect processing of missing values, wrong transformation of categorical variables, and more. Learn what to avoid!
Tags: Collinearity, Data Preparation, Data Science, Data Visualization, Machine Learning, Missing Values, Mistakes
How to build a data science project from scratch - Dec 5, 2018.
A demonstration using an analysis of Berlin rental prices, covering how to extract data from the web and clean it, gaining deeper insights, engineering of features using external APIs, and more.
Tags: Berlin, Data Preparation, Data Science, Real Estate, Web Scraping
- Data Science Projects Employers Want To See: How To Show A Business Impact - Dec 4, 2018.
The best way to create better data science projects that employers want to see is to provide a business impact. This article highlights the process using customer churn prediction in R as a case-study.
Tags: Career Advice, Churn, Data Preparation, Data Science, R
- Text Preprocessing in Python: Steps, Tools, and Examples - Nov 6, 2018.
We outline the basic steps of text preprocessing, which are needed for transferring text from human language to machine-readable format for further processing. We will also discuss text preprocessing tools.
Pages: 1 2
Tags: Data Preparation, NLP, Python, Text Analysis, Text Mining, Tokenization
- Notes on Feature Preprocessing: The What, the Why, and the How - Oct 26, 2018.
This article covers a few important points related to the preprocessing of numeric data, focusing on the scaling of feature values, and the broad question of dealing with outliers.
Tags: Data Preparation, Data Preprocessing, numpy, Python, scikit-learn, SciPy
- Introduction to Active Learning - Oct 23, 2018.
An extensive overview of Active Learning, with an explanation into how it works and can assist with data labeling, as well as its performance and potential limitations.
Tags: Active Learning, Data Preparation, Figure Eight, Machine Learning
- ebook: Aggregating Data with Apache Spark™ - Sep 12, 2018.
Learn why cluster computing makes Spark the ideal processing engine for complex aggregations, the different types of aggregations that you can do with Spark, and more.
Tags: Apache Spark, Data Preparation, Databricks, ebook
- Self-Service Data Prep Tools vs Enterprise-Level Solutions? 6 Lessons Learned - Aug 30, 2018.
A detailed comparison between self-service data preparation tools and enterprise-level solutions, covering business strategy, accessible tools and solutions and more.
Tags: Data Preparation, Enterprise
Text Mining on the Command Line - Jul 13, 2018.
In this tutorial, I use raw bash commands and regex to process raw and messy JSON file and raw HTML page. The tutorial helps us understand the text processing mechanism under the hood.
Tags: Data Preparation, Data Preprocessing, NLP, Text Mining
- Data Retrieval and Cleaning: Tracking Migratory Patterns - Jul 3, 2018.
In this post, we walk through investigating, retrieving, and cleaning a real world data set. We will also describe the cost benefits and necessary tools involved in building your own data sets.
Tags: Data Preparation, Data Wrangling, Web Scraping
5 Data Science Projects That Will Get You Hired in 2018 - Jun 26, 2018.
A portfolio of real-world projects is the best way to break into data science. This article highlights the 5 types of projects that will help land you a job and improve your career.
Tags: Data Preparation, Data Science, Data Visualization, Hiring, Jupyter, Machine Learning
- Stagraph – a general purpose R GUI, for data import, wrangling, and visualization - Jun 25, 2018.
Stagraph is a new simple visual interface for R, which focuses on data import, data wrangling and data visualization.
Tags: Data Preparation, Data Visualization, R, Tidyverse
- Natural Language Processing Nuggets: Getting Started with NLP - Jun 19, 2018.
Check out this collection of NLP resources for beginners, starting from zero and slowly progressing to the point that readers should have an idea of where to go next.
Tags: Beginners, Data Preparation, NLP, Text Mining
- ioModel Machine Learning Research Platform – Open Source - Jun 5, 2018.
This article introduces ioModel, an open source research platform that ingests data and automatically generates descriptive statistics on that data.
Tags: Data Preparation, GitHub, Machine Learning, Open Source, Postgres, Python
- Virtual Training Events Without Leaving Your Desk - May 30, 2018.
Check out our lineup of upcoming virtual seminars, online learning courses, and customized training in your office. Space is limited, so reserve your seat early and score the best savings!
Tags: Agile, Business Analytics, Data Preparation, Data Science, Online Education, R, Visualization
- How to Organize Data Labeling for Machine Learning: Approaches and Tools - May 16, 2018.
The main challenge for a data science team is to decide who will be responsible for labeling, estimate how much time it will take, and what tools are better to use.
Pages: 1 2
Tags: Altexsoft, Crowdsourcing, Data Preparation, Image Recognition, Machine Learning, Training Data
- Data Augmentation: How to use Deep Learning when you have Limited Data - May 9, 2018.
This article is a comprehensive review of Data Augmentation techniques for Deep Learning, specific to images.
Tags: Data Preparation, Deep Learning
- 7 Useful Suggestions from Andrew Ng “Machine Learning Yearning” - May 8, 2018.
Machine Learning Yearning is a book by AI and Deep Learning guru Andrew Ng, focusing on how to make machine learning algorithms work and how to structure machine learning projects. Here we present 7 very useful suggestions from the book.
Tags: Andrew Ng, Book, Data Cleaning, Data Preparation, Free ebook, Machine Learning, Metrics
- Getting Started with spaCy for Natural Language Processing - May 2, 2018.
spaCy is a Python natural language processing library specifically designed with the goal of being a useful library for implementing production-ready systems. It is particularly fast and intuitive, making it a top contender for NLP tasks.
Tags: Data Preparation, Data Preprocessing, NLP, Python, Text Analytics, Text Mining
- Actionable Insights with Predictive Analytics for Marketers, May 9 - May 1, 2018.
Learn how your predictions can only be as good as your data, how to fix imperfect data, how to structure your customer data for optimal predictive power, and more.
Tags: Customer Analytics, Data Preparation, Looker, Marketing
- The Dirty Little Secret Every Data Scientist Knows (but won’t admit) - Apr 26, 2018.
Most people don’t realize, but the actual “fancy” machine learning algorithm is like the last mile of the marathon. There is so much that must be done before you get there!
Tags: Data Cleaning, Data Preparation, Data Science, Machine Learning
- Minimizing Model Risk with Automated Data Preparation & Machine Learning, Apr 19 - Apr 2, 2018.
Join DataRobot, Apr 19 at 2:00 pm ET/11:00 am PT, for a webinar on how to use Automated Data Preparation & Machine Learning to gain a competitive advantage, while quickly aligning your business operations to regulatory requirements.
Tags: Automated Data Science, Automated Machine Learning, Data Preparation, DataRobot
- Principles of Guided Analytics - Mar 27, 2018.
KNIME outline their guided analytics system and explain how this can assist data scientists to predict future outcomes.
Tags: Analytics, Data Preparation, Knime, Michael Berthold, Workflow
Text Data Preprocessing: A Walkthrough in Python - Mar 26, 2018.
This post will serve as a practical walkthrough of a text data preprocessing task using some common Python tools.
Tags: Data Preparation, Data Preprocessing, NLP, Python, Text Analytics, Text Mining
5 Things to Know About Machine Learning - Mar 7, 2018.
This post will point out 5 thing to know about machine learning, 5 things which you may not know, may not have been aware of, or may have once known and now forgotten.
Tags: Accuracy, CoLaboratory, Data Preparation, Ensemble Methods, Jupyter, Machine Learning, Validation
The Value of Semi-Supervised Machine Learning - Jan 17, 2018.
This post shows you how to label hundreds of thousands of images in an afternoon. You can use the same approach whether you are labeling images or labeling traditional tabular data (e.g, identifying cyber security atacks or potential part failures).
Tags: Data Preparation, Image Recognition, Machine Learning, SVM
- Governance in Data Science - Jan 16, 2018.
Governance roles for data science and analytics teams are becoming more common... One of the key functions of this role is to perform analysis and validation of data sets in order to build confidence in the underlying data sets.
Tags: Data Governance, Data Preparation, Data Science
- Webcasts: Finding analytic solutions to real problems - Jan 3, 2018.
The Technically Speaking webcast series provides real-word case studies with key insights on overcoming the challenges in data collection, preparation, and analysis - find the webcast that fits your current challenge.
Tags: Analytics, Data Analysis, Data Preparation, JMP
- A General Approach to Preprocessing Text Data - Dec 1, 2017.
Recently we had a look at a framework for textual data science tasks in their totality. Now we focus on putting together a generalized approach to attacking text data preprocessing, regardless of the specific textual data science task you have in mind.
Tags: Data Preparation, Data Preprocessing, NLP, Text Analytics, Text Mining, Tokenization
Automated Feature Engineering for Time Series Data - Nov 20, 2017.
We introduce a general framework for developing time series models, generating features and preprocessing the data, and exploring the potential to automate this process in order to apply advanced machine learning algorithms to almost any time series problem.
Tags: Automated Machine Learning, Data Preparation, Feature Engineering, Feature Selection, Time Series
- Webinar: Data Preparation Essentials for Automated Machine Learning, Nov 29 - Nov 16, 2017.
Jen Underwood will review how to organize data in a machine learning-friendly format that accurately reflects the business process and outcomes.
Tags: Automated Machine Learning, Data Preparation, DataRobot, Jen Underwood
- Social Media and Machine Learning Transform Self-service Data Prep - Oct 16, 2017.
Social media and machine learning concepts are transforming self-service data prep into a collaborative data marketplace.
Tags: Data Preparation, Datawatch, Social Media
- Python Data Preparation Case Files: Group-based Imputation - Sep 25, 2017.
The second part in this series addresses group-based imputation for dealing with missing data values. Check out why finding group means can be a more formidable action than overall means, and see how to accomplish it in Python.
Tags: Data Preparation, Pandas, Python
- A Solution to Missing Data: Imputation Using R - Sep 21, 2017.
Handling missing values is one of the worst nightmares a data analyst dreams of. In situations, a wise analyst ‘imputes’ the missing values instead of dropping them from the data.
Tags: Data Preparation, Missing Values, R
- Python Data Preparation Case Files: Removing Instances & Basic Imputation - Sep 14, 2017.
This is the first of 3 posts to cover imputing missing values in Python using Pandas. The slowest-moving of the series (out of necessity), this first installment lays out the task and data at the risk of boring you. The next 2 posts cover group- and regression-based imputation.
Tags: Data Preparation, Pandas, Python
42 Steps to Mastering Data Science - Aug 25, 2017.
This post is a collection of 6 separate posts of 7 steps a piece, each for mastering and better understanding a particular data science topic, with topics ranging from data preparation, to machine learning, to SQL databases, to NoSQL and beyond.
Tags: Data Preparation, Data Science, Deep Learning, Machine Learning, NoSQL, Python, SQL
- The Ultimate Guide to Basic Data Cleaning - Aug 24, 2017.
Data cleaning can seem intimidating, but it’s not hard if you know the basic steps. That’s why we’re excited to announce our newest ebook, “The Ultimate Guide to Basic Data Cleaning”!
Tags: Data Cleaning, Data Preparation, ebook, Free ebook
37 Reasons why your Neural Network is not working - Aug 22, 2017.
Over the course of many debugging sessions, I’ve compiled my experience along with the best ideas around in this handy list. I hope they would be useful to you.
Pages: 1 2
Tags: Data Engineering, Data Preparation, Gradient Descent, Neural Networks
- Data Version Control in Analytics DevOps Paradigm - Aug 14, 2017.
DevOps and DVC tools can help reduce time data scientists spend on mundane data preparation and achieve their dream of focusing on cool machine learning algorithms and interesting data analysis.
Tags: Analytics, Data Preparation, Data Science, DevOps, DVC, Open Source, Version Control
- How to squeeze the most from your training data - Jul 27, 2017.
In many cases, getting enough well-labelled training data is a huge hurdle for developing accurate prediction systems. Here is an innovative approach which uses SVM to get the most from training data.
Tags: Data Analysis, Data Preparation, Machine Learning, Support Vector Machines, SVM, Training Data
- Exploratory Data Analysis in Python - Jul 7, 2017.
We view EDA very much like a tree: there is a basic series of steps you perform every time you perform EDA (the main trunk of the tree) but at each step, observations will lead you down other avenues (branches) of exploration by raising questions you want to answer or hypotheses you want to test.
Tags: Data Analysis, Data Exploration, Data Preparation, Jupyter, Python, SVDS
- 7 Ways to Get High-Quality Labeled Training Data at Low Cost - Jun 13, 2017.
Having labeled training data is needed for machine learning, but getting such data is not simple or cheap. We review 7 approaches including repurposing, harvesting free sources, retrain models on progressively higher quality data, and more.
Tags: Crowdsourcing, Data Preparation, Gamification, Machine Learning, Training Data
- KDnuggets™ News 17:n22, Jun 7: 7 Steps to Mastering Data Preparation with Python; Why Does Deep Learning Not Have a Local Minimum? - Jun 7, 2017.
7 Steps to Mastering Data Preparation with Python; Why Does Deep Learning Not Have a Local Minimum?; 7 Techniques to Handle Imbalanced Data; Which Machine Learning Algorithm Should I Use?; Is Regression Analysis Really Machine Learning?
Tags: Data Preparation, Deep Learning, Machine Learning, Python, Regression, Unbalanced
7 Steps to Mastering Data Preparation with Python - Jun 2, 2017.
Follow these 7 steps for mastering data preparation, covering the concepts, the individual tasks, as well as different approaches to tackling the entire process from within the Python ecosystem.
Pages: 1 2
Tags: 7 Steps, Data Preparation, Data Preprocessing, Data Science, Data Wrangling, Machine Learning, Pandas, Python
7 Techniques to Handle Imbalanced Data - Jun 1, 2017.
This blog post introduces seven techniques that are commonly applied in domains like intrusion detection or real-time bidding, because the datasets are often extremely imbalanced.
Tags: Balancing Classes, Data Preparation, Data Science, Unbalanced
- KDnuggets™ News 17:n21, May 31: Python Machine Learning Workflows from Scratch; Machine Learning Crash Course - May 31, 2017.
Machine Learning Workflows in Python from Scratch Part 1: Data Preparation; Machine Learning Crash Course: Part 1; An Introduction to the MXNet Python API; How A Data Scientist Can Improve Productivity; Data science platforms are on the rise and IBM is leading the way
Tags: Data Preparation, Data Science, Data Science Platform, Data Scientist, Machine Learning, Python
- Data preprocessing for deep learning with nuts-ml - May 30, 2017.
Nuts-ml is a new data pre-processing library in Python for GPU-based deep learning in vision. It provides common pre-processing functions as independent, reusable units. These so called ‘nuts’ can be freely arranged to build data flows that are efficient, easy to read and modify.
Tags: Data Preparation, Deep Learning, IBM, Image Recognition, Python
Machine Learning Workflows in Python from Scratch Part 1: Data Preparation - May 29, 2017.
This post is the first in a series of tutorials for implementing machine learning workflows in Python from scratch, covering the coding of algorithms and related tools from the ground up. The end result will be a handcrafted ML toolkit. This post starts things off with data preparation.
Tags: Data Preparation, Machine Learning, Python, Workflow
- Data Preparation Strategies for Successful Machine Learning - May 18, 2017.
This upcoming 45-minute webinar explores efficient methods to explore and organize complex data, how to marry multiple datasets for feature engineering, and optimal target selection and how to address information leakage.
Tags: Data Preparation, DataRobot, Machine Learning, Strategy
- Technically Speaking – Analytic solutions to real-world problems - May 3, 2017.
Are you and your data "having issues?" JMP real-world case studies help you solve them with key insights on overcoming the challenges with data collection, preparation, and analysis.
Tags: Data Analysis, Data Preparation, Data Visualization, JMP
Pandas Cheat Sheet: Data Science and Data Wrangling in Python - Jan 27, 2017.
The Pandas library can seem very elaborate and it might be hard to find a single point of entry to the material: with other learning materials focusing on different aspects of this library, you can definitely use a reference sheet to help you get the hang of it.
Tags: Cheat Sheet, Data Preparation, DataCamp, Pandas, Python
- Data Exploration in Preparation for Modeling - Jan 13, 2017.
The most important traits for a good data analyst or data miner are curiosity, creativity and intuition for how to answer important questions using data. Read this white paper to learn more.
Tags: Data Preparation, JMP, Michael Berry, White Paper
- 6 Steps to Effective Data Preparation for Quality Conclusions - Jan 12, 2017.
Data preparation is usually the most time consuming part of a data analysis project. To get good results, follow the six steps here, starting with Understand the Business Needs, Get to Know the Data, and Wrangle, Munge, and Mash Up.
Tags: Data Preparation, Sisense
- Tidying Data in Python - Jan 4, 2017.
This post summarizes some tidying examples Hadley Wickham used in his 2014 paper on Tidy Data in R, but will demonstrate how to do so using the Python pandas library.
Tags: Data Cleaning, Data Preparation, Pandas, Python
5 Machine Learning Projects You Can No Longer Overlook, January - Jan 2, 2017.
There are a lot of popular machine learning projects out there, but many more that are not. Which of these are actively developed and worth checking out? Here is an offering of 5 such projects, the most recent in an ongoing series.
Tags: Boosting, C++, Data Preparation, Decision Trees, Machine Learning, Neural Networks, Optimization, Overlook, Pandas, Python, scikit-learn
- Interviews with Data Scientists: Claudia Perlich - Dec 2, 2016.
In this wide-ranging interview, Roberto Zicari talks to a leading Data Scientist Claudia Perlich about what they must know about Machine Learning and evaluation, domain knowledge, data blending, and more.
Pages: 1 2
Tags: Claudia Perlich, Data Preparation, Data Science, Kaggle, Machine Learning, Roberto Zicari
- Data Exploration in Preparation for Modeling - Nov 16, 2016.
What you don't know can hurt you, especially in predictive modeling. Read great examples how exploring your data before creating models will help you spot problems before your build incorrect models.
Tags: Data Preparation, JMP, Michael Berry, White Paper
- How to Choose a Data Format - Nov 3, 2016.
In any data analytics project, after business understanding phase, data understanding and selection of right data format as well as ETL tools is very important task. In this article, a very useful and practical set of guidelines is explained covering data format selection and ETL phases of project lifecycle.
Pages: 1 2
Tags: Data Cleaning, Data Engineering, Data Preparation, ETL, Hadoop, HDFS
- Data Preparation Tips, Tricks, and Tools: An Interview with the Insiders - Oct 14, 2016.
Data preparation and preprocessing tasks constitute a high percentage of any data-centric operation. In order to provide some insight, we have asked a pair of experts to answer a few questions on the subject.
Pages: 1 2
Tags: Advice, Data Preparation, Data Preprocessing, Sebastian Raschka, Tamr, Tips
- Behind the Dream of Data Work as it Could Be - Sep 13, 2016.
This post is an insider's overview of data.world, and their attempt to build the most meaningful, collaborative, and abundant data resource in the world.
Pages: 1 2
Tags: Data Analysis, Data Engineering, Data Preparation, Data Preprocessing, Data Science
- Automating Data Ingestion: 3 Important Parts - Sep 9, 2016.
In the day and age of ‘Big Data”, data ingestion has to be automated on some level. How best to automate it?
Tags: Automation, Claudia Perlich, Data Preparation, ETL, Quora
- Choosing Tools for Data ETLs - Aug 9, 2016.
Which tool should I use for my data pipelines? Get some advice from a data scientist recently having gone through this pipeline tool selection process.
Tags: AirBnB, Data Cleaning, Data Preparation, ETL
- Looker: Exploring the Census, Aug 11 Webinar - Aug 3, 2016.
We will dig into 20 years of Census voting data that we have loaded into Google BigQuery and modeled in Looker. You can ask anything you're interested and we will look it up, live.
Tags: BigQuery, Data Preparation, Looker, US Census
- Data Science for Beginners 2: Is your data ready? - Jul 28, 2016.
This second video and write-up in the Data Science for Beginners series discusses what is required of your data before it can be useful.
Tags: Beginners, Data Preparation, Data Science, Microsoft
- 5 More Machine Learning Projects You Can No Longer Overlook - Jun 28, 2016.
There are a lot of popular machine learning projects out there, but many more that are not. Which of these are actively developed and worth checking out? Here is an offering of 5 such projects.
Tags: Computer Vision, Data Preparation, Data Preprocessing, Javascript, Machine Learning, Natural Language Processing, NLP, Overlook, Python
- Infinite Data Overlap Detection Arrives to Speed Business Insights - Jun 8, 2016.
Infinite Data Overlap Detection(IDOD) is a new, Spark-based technology that empowers non-technical business users to automatically discover data patterns and blendany data type for any set of values from multiple sources – both inside and outside the enterprise.
Tags: Apache Spark, ClearStory Data, Data Cleaning, Data Preparation
- Doing Data Science: A Kaggle Walkthrough Part 3 – Cleaning Data - Jun 3, 2016.
This is part three in a fantastic 6 part series covering the process of data science, and the application of the process to a Kaggle competition. In this episode, data cleaning and preparation is covered.
Pages: 1 2
Tags: Data Cleaning, Data Preparation, Kaggle, Python
- How to Quantize Neural Networks with TensorFlow - May 4, 2016.
The simplest motivation for quantization is to shrink neural network representation by storing the min and max for each layer. Learn more how to perform quantization for deep neural networks.
Pages: 1 2
Tags: Data Preparation, Deep Learning, Neural Networks, TensorFlow
- How to Remove Duplicates in Large Datasets - Apr 27, 2016.
Dealing with huge datasets can be tricky, especially the data cleaning process. One of such processing is de-duplication, find out how you can solve this using the statistical techniques.
Tags: CleverTap, Data Cleaning, Data Preparation
- Doing Data Science: A Kaggle Walkthrough – Cleaning Data - Mar 23, 2016.
Gain insight into the process of cleaning data for a specific Kaggle competition, including a step by step overview.
Pages: 1 2
Tags: Data Cleaning, Data Preparation, Kaggle, Pandas, Python
- R Learning Path: From beginner to expert in R in 7 steps - Mar 23, 2016.
This learning path is mainly for novice R users that are just getting started but it will also cover some of the latest changes in the language that might appeal to more advanced R users.
Pages: 1 2 3
Tags: 7 Steps, Data Preparation, Data Science Education, Data Visualization, DataCamp, Hadley Wickham, Maps, R
- Prove Your Point with Data and a Fast Python Library - Jan 7, 2016.
Harness the power of Python and the command line to prove your point using data and a fast data-processing library.
Tags: Chess, Data Preparation, Python
- 5 Criteria To Determine If Your Data Is Ready For Serious Data Science - Dec 21, 2015.
If your data is a large, relevant, accurate, connected, and you also have a sharp question, you ready to do some serious data science. If you’re weak on 1-2 points, don’t worry. But if most criteria are not true, you need to do more preparation.
Tags: Data Preparation, Data Science, How to start
- Webinar: 5 tips to get more out of Data Lakes, Dec 16 - Dec 1, 2015.
Learn valuable tips to help optimize Big Data for agility and speed to insight; improve data accessibility, without the limitations of data warehouses, and prevent data sources from becoming data silos.
Tags: Data Lakes, Data Preparation, Data Warehouse, Lavastorm
- Lavastorm – 5 Tips to Get More From Tableau - Oct 20, 2015.
Tableau makes it easy for users to see the data, but data preparation for it is hard. This free ebook highlights how to overcome Tableau challenges with data access, data blending, advanced analytics, transparency and reusability.
Tags: Data Preparation, Free ebook, Lavastorm, Tableau
- Upcoming Webcasts on Analytics, Big Data, Data Science – Oct 20 and beyond - Oct 19, 2015.
Easier Data Prep and Analysis for Data Scientists, Measure and Enhance Analytics Maturity, Amazon QuickSight, Textual Healing, and more.
Tags: Amazon QuickSight, Analytic Maturity, Data Preparation, Hadoop, RapidInsight
- Upcoming Webcasts on Analytics, Big Data, Data Science – Oct 13 and beyond - Oct 12, 2015.
Optimizing Data Lake, Data Mining: Failure to Launch, Easier Data Prep and Analysis for Data Scientists, and more.
Tags: Data Lakes, Data Preparation, Failure to Launch, RapidInsight, TMA
- Easier Data Prep and Analysis for Data Scientists, Oct 20 Webinar - Oct 6, 2015.
Rapid Insight will show tools that make the data preparation and analysis process significantly faster, without losing the flexibility of advanced programming or SQL tools.
Tags: Data Preparation, RapidInsight, SQL
- Data Science Data Logic - Sep 17, 2015.
Even though participating in MOOCs and online competitions are good exercises to learn data science, but it is more than algorithms and accuracies. Understand how to formulate hypothesis, data creation, sampling, validation, etc. to become true data scientist.
Pages: 1 2 3
Tags: Data Preparation, Data Science, Forecasting, Olav Laudy
- Lavastorm Webinar, Aug 19: Leveraging Data for Effective Data Visualization - Aug 7, 2015.
Learn why data governance and a data framework are essential to visualization success; how to adopt an agile, self-service approach to data access, analytics and visualization; and tips for optimizing visualization tools through better data prep.
Tags: Data Preparation, Data Visualization, Lavastorm
- Data is Ugly – Tales of Data Cleaning - Aug 1, 2015.
Whether you want to do business analytics or build the deep learning models, getting correct data and cleansing it appropriately remains the major task. Find out experts opinions on how you can make efficient data cleansing and collection efforts.
Tags: Big Data, Data Cleaning, Data Preparation, Data-Driven Business
- How to properly present a Data Mining project? - Jul 14, 2015.
Building models and getting insights are job half done for the data scientist, presenting them to the audience is an art itself. See, how to approach the presentation after wrapping up the data science project.
Tags: Algolytics, Data Preparation, Presentation
- ParseHub gives Data Scientists a better, faster way to collect data - Jul 10, 2015.
ParseHub enables data professionals to easily collect, structure, combine and manipulate data, and speed up the Data Science process.
Tags: Data Cleaning, Data Preparation, Glassdoor, ParseHub
- 3 Things About Data Science You Won’t Find In Books - May 11, 2015.
There are many courses on Data Science that teach the latest logistic regression or deep learning methods, but what happens in practice? Data Scientist shares his main practical insights that are not taught in universities.
Pages: 1 2
Tags: Cross-validation, Data Preparation, Data Science, Feature Engineering, Feature Extraction, Overfitting
- Lavastorm Webinar: Demystify Your Data Flows for Better Regulatory Compliance, May 19 - May 7, 2015.
Complying with new financial regulations can be incredibly costly. Learn how Lavastorm helps firms aggregate and manage data, provide visibility into data sources, transformation and lineage, and reduce time and costs spent on compliance.
Tags: Compliance, Data Preparation, Lavastorm
- Top KDnuggets tweets, Mar 16-18: 87 Studies shown that accurate numbers aren’t more useful than the ones you make up (Dilbert) - Mar 19, 2015.
Also Sirius - a free, open-source version of Siri; #PI art: the first 13,689 digits of pi; Great tutorial + #Python code: 1-Layer Neural Networks.
Tags: Cartoon, Data Preparation, Deep Learning, Dilbert, Excel, Neural Networks, pi, Python, Siri
- Tamr Enterprise Platform for Scalable, End-to-End Data Unification - Feb 17, 2015.
The new Tamr Platform radically simplifies and speeds the availability of unified data for analytics and downstream application, with key new features: catalog, connect, and consume. Tamr also announced solutions for Pharma and Procurement.
Tags: Cambridge, Data Preparation, MA, Pharma, Procurement, Tamr
- Info Kit: Statistics, Predictive Modeling, Data Mining with JMP - Feb 10, 2015.
Find out how to challenge assumptions, spot patterns and reveal potential solutions to problems that otherwise would not be visible. Register for this complimentary info kit.
Tags: Data Mining, Data Preparation, JMP, Online Education, Statistics
- Shining Light on Dark Data - Jan 23, 2015.
Dark Data is the ever-present, relatively unknown, and unmanaged volumes of data that exist in every corner of one’s business. Here’s what to do with it.
Tags: Dark Data, Data Preparation, Pneuron, Tom Fountain
- Info Kit: Statistics, Predictive Modeling and Data Mining with JMP - Jan 13, 2015.
Find out how to challenge assumptions, spot patterns and reveal potential solutions to problems that otherwise would not be visible. Register for this complimentary info kit.
Tags: Data Preparation, JMP, Statistics
- DIY Crawlers vs. Crawlers as Service - Dec 22, 2014.
Crawling structured data from the web has been made easier with the choice between crawlers as a service, like webhose.io, and do-it-yourself, like import.io.
Tags: Big Data Services, Crawler, Data Preparation, import.io, Spider, Webhose.io
- Making Sense of Public Data – Wrangling Jeopardy – Part 2 - Oct 27, 2014.
Wrangling Jeopardy (Part 2) describes the remaining steps of the data transformation process, detailing how we used Trifacta to structure, clean, enrich and distill Jeopardy data for analysis.
Tags: Data Preparation, Data Processing, Jeopardy, Trifacta
- Top stories for Oct 5-11: Analyzing Ebola spread; Data science shows surveys may assess language more than attitudes - Oct 12, 2014.
Analyzing Ebola - Is it spreading at exponential rate?; Data science shows surveys may assess language more than attitudes; Making Sense of Public Data - Wrangling Jeopardy.
Tags: Data Preparation, Ebola, Jeopardy, Surveys, Top stories
- Salaries in IT – Scrape, refine, and plot case study - Oct 11, 2014.
Very good case study, showing how to scrape with import.io, refine with OpenRefine, and plot with Plot.ly. Also learn about salaries vs age in Belgium.
Tags: Belgium, Data Preparation, Data Visualization, import.io, OpenRefine, Plotly, Salary
- Top KDnuggets tweets, Oct 6-7: Great TED talk by @KnCukier “Big Data is better data”; Top 10 One-Person Startups - Oct 8, 2014.
Great TED talk by @KnCukier "Big Data is better data"; Top 10 One-Person Startups; 7 critical elements of effective dashboards and visualizations; Making Sense of Public Data - Wrangling Jeopardy.
Tags: Dashboard, Data Preparation, Data Wrangling, Jeopardy, Kenneth Cukier, startups, TED
- Making Sense of Public Data – Wrangling Jeopardy - Oct 7, 2014.
Trifacta’s Alon Bartur & Will Davis detail their process for transforming or “wrangling” publicly available Jeopardy data found on the web for downstream analysis.
Tags: Data Preparation, Data Processing, Data Science Platform, import.io, Jeopardy, Trifacta
- Dataiku Data Science Studio - Aug 26, 2014.
Data Science Studio (DSS) from Dataiku is a complete Data Science software tool for developers and analysts,
which significantly shortens the time-consuming load-clean-train-test-deploy cycles of building predictive applications.
A community edition and a free trial available.
Tags: Data Mining Software, Data Preparation, Data Science, Dataiku, Florian Douetteau, Prediction
- InnovAccer: Simplifying Research and Analysis - Jun 5, 2014.
Innovaccer cleans and prepares data for analysis by researchers to save time and improve confidence in the quality of the data.
Tags: Data Curation, Data Integration, Data Preparation, InnovAccer, Statistical Analysis
- Trifacta – Tackling Data Wrangling with Automation and Machine Learning - Mar 17, 2014.
Trifacta wants to solve the important problem of data cleaning and transformation by building better interfaces which use machine learning. It then aims to help enterprises make sense of their disparate data sources and cut down on time needed to prepare data for data science.
Tags: Data Preparation, Joseph Hellerstein, Paxata, Trifacta
- Paxata automates Data Preparation for Big Data Analytics - Mar 7, 2014.
Paxata wants to shorten and automate the data cleaning process, by augmenting data from a huge number of sources and by using machine learning to see statistical similarities between the data imported.
Tags: Accel Partners, Data Preparation, MDM, Paxata
