An Eight-Step Checklist for An Analytics Project
Follow these eight headings of an audit sheet that business analysts should address before submitting the results of their analytics project. One recommended approach is to rewrite each step as a question, answer it, and then attach it to your project.
By Mark Nadler, Ashland University.
Surgical teams that do not leave medical paraphernalia inside patients and airplane pilots who do not crash aircraft have one thing in common: they use checklists. As a business analyst if you want to avoid or minimize making comparable errors then use the below audit to check your work:
(1) What is the project’s problem? Is it descriptive, exploratory, explanatory, or predictive? Don’t expect help from your client or boss. He or she most likely doesn’t have the foggiest idea what the above terms mean. As an analyst, you must translate the fuzzy feelings of people into a project goal.
(2) Select proper statistical techniques. While it’s possible to use OLS on time series data, the assumptions necessary to generate meaningful results never hold in the real world. Always check your selected technique’s assumptions against the environment you are operating.
(3) Identify relevant variables. Excluding significant variables from your analysis biases your results. Including meaningless variables increase standard errors and weaken the analyst's ability to separate chance from fact. Of these two sins, bias is worse. Better to err on the side of having too many variables than too few.
(4) Evaluate your data generating process. Statistics has strict rules surrounding the proper generation of cross-sectional, time-series, experimental, and longitudinal data. With statistical standards of data generation often violated, analysts should be aware of their data’s limitations. To be theoretically correct if improperly generated data is analyzed, results should be limited to the data set examined.
(5) Clean your data. Data collection and data entry are fraught with corrupting influences. Checking data sets for mistakes is critical to the success of a successful analytics project. Recently a private consulting firm critiqued The National Highway Traffic Safety Administration’s (NHTSA) report supporting Tesla’s claim that its Autopilot reduces crashes by 40%. A problem in Tesla’s data was numerous missing values that NHTSA mishandled. After correcting for missing values, Tesla’s claim was reversed. Don’t treat data cleansing as a subsidiary activity.
(6) Describe and explore your data. Some analysts treat this step as generating the requisite table listing all variables with their various descriptive characteristics. Approaching data description and exploration in this manner is a mistake. Data exploration should entail exploring data -- this means getting your data into your bones and hopefully seeing something unexpected. Explore your data by exploring your data.
(7) Execute appropriate forms of statistical inference. For all of the rigmarole surrounding data science, it is based on mathematical statistics. While analysts frequently treat the selection of statistical tools “as if” on a shopping trip, incorrectly selected statistical procedures mean that your analysis will be wrong. View statistical procedures the same way you view a hammer, saw, or a screwdriver. Attempting to cut wood with a screwdriver will doom your project.
(8) Write a technical report. I have had students, who upon graduation, had second rate analytical skills but first-rate technical writing skills who have done very well as working analysts. If you cannot communicate the results of your work, then your efforts have been for naught. Better yet, if you can link your results to improving a practical decision that managers make, and if you can communicate all of this, then your last name becomes Gold. Learn to write a technical report.
- The Checklist Manifesto: How to Get Things Right by Atul Gawande
Bio: Mark Nadler is the A.L. Garber Family Chair in Economics in the Dauch School of Economics and Business at Ashland University in Ashland, Ohio. Mark has taught and consulted in the area of business analytics for ten years.
- The Machine Learning Project Checklist
- Your Checklist to Get Data Science Implemented in Production
- Four questions to help accurately scope analytics engineering project