I have been going through a series of interviews for the past few months as an interviewer as well as an interviewee. It all started back when my supervisor, who was a senior principal analyst, submitted his resignation after the original department has achieved all three main objectives. After his resignation, I had to take on the role as a technical interviewer to assess the technical capability of the interviewees.

However, for the same reason, I had suddenly lost a good supervisor and were no longer guided and mentored by an experienced IT professional. I no longer could receive constructive feedback on my technical and professional development. I no longer know what I know and know what I don’t know. Concurrently, I was trying to figure out where is the next direction for my own career.

Technical interviewer

As a technical interviewer, I knew what I was looking for to be part of the data science team. The candidate had to display a very strong interest as well as understanding of data science (machine learning at the very least). This included understanding the intuition behind ML models such as random forest, ensemble methods and support vector machine.

It would be helpful for the candidate to be able to present their ML/DL projects and able to defend it as we asked in-depth questions regarding it. It would demonstrate how much the candidate has done and how much they have researched to get the best performing results. At the same time, we would be able to find out where are their strengths and weaknesses and figure out how we could complement each other.

Besides that, we were looking at the fit of the person to the existing team as well. There were strong candidates that we would have loved to hire but due to the misfit to the team that we decided not to hire. So to those out that who are technically strong, don’t feel dejected! You are valued somewhere, you just need to find a good fit.

Technical interviewee

As a technical interviewee, I was trying to figure out where are the gaps in my technical skills and I realized that I am missing out on a lot of skills and experiences despite the fact that I am a senior data scientist and a technical lead in a newly formed data science team! After going through multiple interviews, there are three main area that I am lacking or I could continue to improve on.

Cloud experiences

I often get asked about my experience with cloud technologies (AWS, GCP, Azure) and my answer is “Sorry, I don’t have any experience with that at the moment. My company does not allow such technologies at the moment due to IT security and data governance concerns. However, I am learning it on the side as part of my own career development.”.

It is a sad truth. However, I realize that it shouldn’t limit my own career growth. I am now studying for the AWS Cloud Practitioner certificate before I move on to AWS Solutions Architect Associate certificate. Hopefully I would be able to get these two by Jun 2023.

Data science knowledge

The interviewers drilled down on my experience especially on NLP and sequential data. I have always been interested in NLP and sequential data (and also spatial data to a lesser extent). I have a good general idea of them but what I am lacking is, again, the practical experience.

Another interviewer asked why did I choose to evaluate my ML models using precision. I shared that it is dependent on context and in healthcare, it is more important to correctly detect the true positive as undetected true positive may result in death. However, I mixed up precision with recall as this answer is more appropriate for ‘recall’.

There is still much for me to work on in terms of my basics. My current plan is to write blog posts about NLP every 2 weeks for the entire 2023 starting from Feb. Ideally there should be a portion on the theory and another portion on the implementation using some toy dataset. I would have to find other resources to read up especially on basic statistics.

Data engineering skill

One of the interviews tested my knowledge on data structure and SQL. I have to admit that I have very little knowledge and exposure to it other than the random courses I took a while back. Even if I did learn about it, I wouldn’t have much opportunities to apply it at work given the IT security and data governance in place.

I would imagine that this would be an important skill I would have to apply them to my own personal projects. However, I do foresee concerns with it as there is no experienced person to guide and mentor me except for Google and stackoverflow. At the moment, I do not know exactly what needs to be done except that I need to work on my data engineering skills. I would have to ask ChatGPT on it and also find out the books to read.

I do remember that one of the interviewers asked me about my experience working with APIs and I honestly am not sure what exactly are APIs. Based on my limited knowledge, I knew that it was some interface programming but what constitutes as APIs? Are packages APIs? I am not sure. I will have to read up about it for my own better understanding.

Conclusion and follow up actions

In conclusion, I am still lacking a lot of skills and experiences as a data scientist. To address my gaps, these are my plans:

Study and complete AWS Cloud Practitioner by Jun 2023
Study and pass Solutions Architect Associate certificate by Dec 2023.
Complete Stanford CS224n Natural Language Processing with Deep Learning.