The Hackathon Guide for Aspiring Data Scientists
This article is an overview of how to prepare for a hackathon as an aspiring data scientist, highlighting the 4 reasons why you should take part in one, along with a series of tips for participation.
By Jiwon Jeong, Data Science Researcher
A hackathon is a competition where various people, including programmers, developers and graphic designers, collaborate to design software projects intensively in a short period of times. The goal of a hackathon is to create a functional product at the end of the event.
Last month, I had an opportunity to participate in the JunctionX Seoul Hackathon held by Junction Ltd. Our team made an iOS application that brings an optimized shopping service experience for Korean beauty product powered by image search engines. This application can help travelers who want to explore Korean beauty products but not knowing what to try due to an obvious language barrier. Users can find useful data about a product just by scanning it and the information is translated into their local languages. There is also a searching history page which could be utilized for further customized service. You can find more about it here as well.
The app, Skana
Rather than talking about what I made, however, I’d like to share what I learned in the process. So today, I’m going to talk about the 4 reasons why you should apply for a place in a Hackathon (especially if you’re an aspiring data scientist), how you should prepare and some of my tips. I hope this article can motivate you to go out of your comfort zone and get some profound inspiration just as I did.
1. Why Hackathons
If you haven’t tried a hackathon or a programming competition before, and say you ask me if it’s worth trying, I’ll nod my head without hesitation. Here are the 4 reasons why you should go to a hackathon.
We, data science learners, tend to work alone or study alone. We tend to invest most of our time in data preprocessing and wangling. And we pour most of our energy into studying machine learning or deep learning algorithms. Our project usually starts with importing data (or sometimes with building data by ourselves) and ends with the evaluation of the predictions.
Almost every real-world project in the world sadly doesn’t work that way. We don’t work alone. We work with other team members. From database to deployment and product management, all the processes should be done through collaboration. Having toolkits beyond the “modeling” territory can be a great plus for aspirants. (Many articles are pointing out this and you can find one here as well) Now the important question, where can we learn and experience this?
“Unlike competitions, nobody gives you two .csv files called train and test and a nicely written evaluation metric. Almost 80% of the efforts go into defining the problem and getting and processing data. Remaining 20% effort goes into pure modeling and deployment.” - AnalyticsVidhya
A hackathon can be a great chance to collaborate with others and make a real-world project. Although there are some limitations, still you can learn how to communicate with others (who do not have much of the knowledge in machine learning as you do) and understand the overall workflow for releasing an actual product. And if you get a chance of being on the team with someone who knows a lot better than yourself in machine learning, I believe it’ll be such a great time to push your limits as well.
Business thinking is more crucial than you think
The first step of the project will be deciding what to build. And I can tell half of the winning chance is determined here. You should get off on the right foot. Implementation is also a critical point but the business value plays an essential role. If it’s neglected, your work can be nothing but just showing off how technically competent you are.
“Likewise, data scientists are hired to create business value, not just to build models. Ask yourself: How will the outcome of my work influence company decisions? What do I have to do to maximize this effect? With this entrepreneurial spirit, the third wave data scientist does not only produce actionable insights, but also seeks that they bring about real change.” - Towards Data Science
You’d have seen many people pointing out the importance of business mindset already. Finding a hidden need of users and observing a problem in the market. Suggesting a solution with your technical knowledge. Developing a service that can become profitable. At a hackathon, you are asked to think and discuss deeply with this viewpoint.
I heard of a guy who had won programming competitions more than 15 times. He retired in his ‘prize hunting’ career after he won one of Google’s competitions. Do you know what his secret was? It was just business sense and a little bit of wit. By making a clever and creative product, he made people shout “Brilliant!” which definitely leaves an impression on panels. So if you’re looking for an additional tip for winning, having business values with some wit can be a great bonus.
Great chance to handle extreme stress
The goal of a hackathon is creating a functional product during a short period (commonly 2 or 3 days). You are asked to present a prototype or demo of your product during these days. It’s all about limited time, limited resources and limited energy problems. Sometimes, you could have situations where things go wrong and you’d have to instantly find solutions. Sometimes you have to prioritize your work, which means what to come first and when to stop. You could feel intense pressure to do all your tasks and not to be a burden on the team. And as time passes, becoming completely famished and exhausted is literally inevitable! (More like energy drain or depletion until finally “falling” asleep).
If you’re someone like me, who tend to freeze at an extremely stressful moment, you’d be overwhelmed even by the air at the competition hall. It was my first time in a Hackathon so I experienced my brain being paralyzed several times. I had to get some fresh air and relax to ease off the tension. But this also became a great chance for me to learn how to dealing with stress. Because it’s unavoidable meeting those unexpected problems in the workplace and we have to manage those demanding moments.
The real-time of ”Hello, world”
Meeting new people is also a great pleasure, especially if they are like-minded people. At a hackathon, you can meet various people from different regions and with different specialties but with the same interest. They can be web developers, app developers or designers but they are people who’re willing to invest their weekend in developing something cool.
I met people who came from Russia to Korea just for this event. I also met a Ukrainian lady studying Computer Science in Korea. There were people from Denmark, French, China, and Japan. Meeting new people itself is a great joy. But more than that, I could hear what kind of pain points they have and what they are really passionate about. My horizon could become broadened in programming by chatting with fellow participants. And this is a reason why I think you should try a Hackathon with generous topics rather than heavily focused on the data science fields.
Also, it’d be good to care about what others do as well. Although you’ll be too busy to take extra care of others, this will help you understand the whole process as I mentioned above. We’re here to learn new things, anyway!
Photos by JunctionXSeoul and Me giving a presentation (on the right bottom)
2. How to prepare
So now does a Hackathon sound worth the try? If so, then your next question could be how you should prepare. Actually, nothing. Just go for it. Whatever skillsets you have, don’t hesitate to apply for. Hackathons are not a stage only for professionals. But if you’d like to prepare something before the day, then I’d like to list some of the skills I find useful and applicable.
Backend and Deployment
This is not a project that starts with importing the data and ends with some evaluation plots. You need to collect the data and build a database according to your team project. After you finish with data analysis or modeling part, you have to pass your model to frontend. Or you might need to deploy your work on a web by yourself. Therefore having database and Flask in your toolkits can be a huge asset for this event.
If you’re totally new to database, you can start your learning from understanding the difference between the SQL and NoSQL. Here is a compact SQL course for beginners but there are tons of tutorials on databases besides this. So go and pick according to your taste.
Flask is a microframework written in Python which is easy to learn and very useful for the model deployment. Here is an excellent tutorial on how to deploy a Keras deep learning models with Flask by Ben Weber. You can also find a series of video courses for Flask here.
API and Cloud Service
Conditions could be different for each hackathon, but the one I took part in highly depended on API and Cloud Services. There were several sub tracks and the participants were asked to use particular API to participate in a given track. But besides this condition, you’d need to collect data by yourself anyway. Where will you get the data? And what about the computing power for training? You’d have probably left your “super-computer” with GPUs at home.
Therefore it’d be better to have an experience of getting data from API and handling data in a JSON format. If you’re new to API, this article can explain to you what API is all about.
Ability to use a cloud service like AWS or Google Cloud GPU is also necessary. Here is an official guide for AWS EC2 but here is a friendly video by School of AI. You can also find a more detailed tutorial for beginners by Michael Galarnyk.
Fully charged energy of yourself
Get enough sleep. Take control of your condition. This is not a joke. During the competition, I slept for 4 hours in total. Sleeping less can be a good strategy but you need to be in good condition when you enter the hall. Also, a neck pillow and a blanket for a catnap can be as much useful item as “Flask” or “AWS.” ????????
Photos by JunctionXSeoul
3. My small last tips
Lastly, here are some of my small tips.
How to form your team
If you’re a solo applicant, you have to build your team from the bottom. There can be a social network platform like Slack where you can get the information from other applicants and make a team before the event. You might get a chance to join a premade team with one opening.
If you have an idea on what you want to develop, divide it into parts and look for crew members who can take on each part. You need to find someone who has the skill different from us (like data modeling). The common list of members you’d need to have is a backend engineer, a web/app developer, and a UI/UX designer.
Be honest with what you can do
You might think your ability is not enough for doing this. So you might want to hide your level or pretend knowing more than you do. But you really don’t have to. Hackathons are open to everyone. And no matter how much you know about programming, there is a portion that can be filled only by you! Is the person next to you know much more than you do? Awesome! You get more chances to learn. Asking for help is a better way to “help” your team than struggling alone and wasting precious time.
Enjoy, Enjoy and Enjoy!
It’s more than winning. Although you should do your best during the time, don’t lose a smile on your face. Enjoy the time for learning, meeting new people and working with them. Enjoy the time for being creative and developing a new thing that doesn’t exist in the world. Enjoy the time for being stressed and going beyond your limits. It will be extremely demanding. At the end of the event, however, you’ll go back to your home with new thoughts and lessons on your mind even if you don’t win a prize. And you’d find yourself looking forward to the next Hackathon like me. ????
Did this story resonate with you? Please share your insight with us. I’m always open to talk, so feel free to leave comments below and share your thoughts. I also share interesting and useful resources on LinkedIn so feel free to follow and reach out to me. I’ll be back with another interesting story, next time. As always, stay tuned!
Bio: Jiwon Jeong is a Data Science Researcher, data geek, bookworm, and travel lover.
Original. Reposted with permission.
- Building a Flask API to Automatically Extract Named Entities Using SpaCy
- Deploy your PyTorch model to Production
- Core Principles of Sustainable Data Science, Machine Learning and AI Product Development: Research as a core driver