Outliers by Karishma Rao, Lead Data Analyst, Yara Digital Farming
By | Karishma Rao | www.jobsforher.com
An outlier is a data point that differs significantly from other data points. Statistics has a very strange approach when it comes to Outliers, since they don’t help the model or skews data we prefer to take them out from our dataset (conventional belief). From a behaviour modeling perspective it makes sense; but what do we do when this data point represents a person or a group of people? Would it be ethical to simply take them out? OR Do we include them by broadening our minds?
After working in Market research for 5 Years, I started getting the feeling that I was getting obsolete. My skill set was becoming redundant and I was aware that there was a parallel universe where data and user behaviour was taking a different form. It was extremely frustrating because the world around me had changed, there were multiple courses and specialisations available. Navigating through all those courses and options was overwhelming. After investing 2 months into the research, LinkedIn webinars and TED talks, I concluded that I wanted to become a Big Data Analyst. But where do you start & how does one become a Big Data Analyst? After some more research, I zeroed on 4 institutes. Entrance exams – Check. Personal Interview – Check. Acceptance letter – Check. Now came the tough part – quitting the job I was holding at the time. It meant no steady income for 2 years, relying heavily on my savings and moving to a different state (Hyderabad) where I knew no one. EXCITING!
I was very keen on pursuing statistics – that was the one thing I was confident about (I was so wrong), until I met my Statistics professor (Late Professor Bhimasankaram Pochiraju, he was a Clinical Professor of Statistics, Executive Director at the Applied Statistics and Computing Lab (ASCL) and the Faculty Director for Business Analytics at the Indian School of Business). Our first class was on how to handle missing values and outliers. When we came to the topic of outliers, my professor explained who is an outlier, the impact outliers have on behaviour modeling and ways of treating an outlier. As we delved deeper on the subject of “Treating outliers”, the professor turned around and asked, “How many of you come from a non-programming (Java and C++) background?”. I raised my hand quickly to find out that out of a strength of 150 students only 3 of us were raising our hands. OH NO! What does this mean? We were anomalies in the professor’s dataset! Then he turns around and asks the class “What do we do with these outliers? Should we ask them to leave the room?” At this point, I turn around to all my fellow classmates begging them to keep us in the class. Fortunately, the class was kind and decided to keep us. With Just 2 questions our professor was able to demonstrate how taking out outliers can have a real-life impact, we can be happy that our model would work well, but we have to be responsible about the real-life implication on people.
The above situation left a long-lasting impression on me. We constantly try to “fit in” but what if we were not designed to “fit in”, what if we were designed to be an outlier? In that case, the best thing to do is get to know yourself, embrace your individuality and someday you will find people who are like-minded OR if you are lucky, you will meet a group of outliers.
Having worked with boutique agencies, corporates, multinationals, NGOs and educational institutes, it’s been evident that business success is based on team diversity and inclusion. We see a strong infusion of diversity & inclusion in Yara DAS. Here, everyday employees are encouraged to think outside the box. We are constantly challenged and challenge others to move away from the status quo. We keep our ears, mind and hearts open to learning something new about each other’s culture, environment and life experience.
As the first generation female, born in a business family, pursuing a serious career in behaviour change and product analytics, needless to say, in every sense of the word I was an outlier in my environment. But how was it a good thing? Being an outlier led me to Yara. And now I do not feel like an outlier anymore.
Republished with permission and originally published at www.jobsforher.com