AI Interviewer Service Development Log

Hello. I'm Youngjun Kim from the Web Front Development team in the Service Development Office.

At a time when generative AI is in the spotlight, our dedicated team at Genesis Lab, including myself, has put in significant effort to develop a service called 'AI Interviewer'. I want to take this opportunity to share with you how, why, and what we were thinking about during the development process. 

👨🏻‍💻 What is video interviewing?

Video interviews are also known as video interviews (non-face-to-face interviews). Like a traditional interview, the interviewer and the candidate interact remotely and face-to-face, and many organizations now conduct interviews this way, using a microphone, camera, etc. Examples include interviews conducted via Zoom and Google Meet.

So, what's the difference between AI video interviewing and video interviewing?

🤖 AI Video Interview?

Since the coronavirus pandemic, the use of AI video interviews for hiring has increased dramatically in addition to in-person interviews. 

AI interviews, a revolutionary advancement in the field, are similar to video interviews. However, unlike traditional video interviews, AI takes over the role of the interviewer: conducting, scoring, and detecting fraud. This innovative approach brings numerous benefits, which I'm excited to share with you.

There are many types of interviews, but the three most common are in-person, video, and AI video interviews. Each has its characteristics, but our focus is AI video interviews.

The interview's time- and location-independent nature is something anyone would expect. However, having AI as the judge is a tricky area. For reasons we won't go into in this article, we'll cover in more detail in our following content. Still, human biases can be present, such as the condition of the interviewer, which interviewer they are facing, etc. There are advantages to using AI to reduce these biases (though a detailed pros/cons discussion would likely lead to a much longer paper...)

So, if you're applying to companies that have adopted AI video interviews and are using them for hiring, how can you practice?

🔥 Hey AI, help me interview...

As anyone who's used an AI video interview service like Viewer knows, the experience you get from an interview service depends on how the candidate responds to canned questions. It's a process of practising answering pre-formulated questions and thinking about how you'll react.

We set out to develop an AI interviewer that would not just answer canned questions but would use AI to create questions tailored to the candidate's experience, analyze their answers, and probe for more information, creating an interview experience that feels like a real-time human conversation.

There are five capabilities we've built.

Here's an example of what happens when you select the "Teamwork" competency out of these five.

• Select the 'Teamwork' competency from the five competencies.
• The interview begins with questions about the competency 'Teamwork'.
• At the end of the interview, the candidate's answers are analyzed to determine whether they demonstrate 'Teamwork' competency.

If you look at it from the company's perspective, you can create (customize) the competencies you want, like Genesis Labs, and hire candidates who excel in the competencies you've customized. (Viewer HR)

From the candidate's perspective, you can practice the competencies you have in place. Depending on the candidate's answers, you'll ask them again for clarification. After all the questions, you'll provide feedback by summarizing their answers, identifying gaps, and generating an overall opinion. Based on your feedback, the candidate will have a better interview experience.

🦾 How did you create it?

Our development journey with ChatGPT involved prompt engineering to ensure the desired results. The key to this engineering was to extract context from the candidate's answers. If the context was insufficient, we iterated on creating questions to elicit the necessary elements for validation, making the experience feel like an honest conversation with the candidate. However, it's important to note a typical challenge with ChatGPT, which we call 'hallucination ', and how we addressed it.

One thing to watch out for when using ChatGPT is a phenomenon everyone experiences: hallucination. Hallucination is when the GPT answers non-existent or decontextualized answers as if they were the truth. Reducing errors by cross-validating candidate answers with AI analysis is essential.

Sensitive and anomalous answers are also validated. When you answer the first question, you can have a realistic experience like an actual interview because it shows you the specific situation, role, and behaviour that the interviewer would ask in an actual interview.

Here's an example of the question-and-answer process

```

🤖 Q: Have you ever responded to a customer's needs while working part-time?

😀 When I was in college, I had an allergy customer, so I changed the menu to accommodate her.

```

🤖 Q: Please elaborate on what allergies the customer had and how you responded.

😀 When working in a restaurant, I once had a customer allergic to peaches. I explained that other alternatives didn't contain peaches and asked if she was allergic to different foods so I could help her avoid them.

```

🤖 Q: What have you learned from dealing with customers with allergies during your part-time job?

😀 {...Answer}

😹 (Epilogue) Development experiences and impressions

To conclude this post, I'd like to share my thoughts on the experience of developing a generative AI-based service.

The first is that you need to be careful and prevent hallucinations.

As conversations become more prolonged and prompts overlap, GPTs become less comprehensible and begin to pick up on slurring. This leads to inaccurate content and communication, which could be a better service experience. AI Interviewer was developed with the help of Genesis Labs, which helped us preserve the conversation's content and prevent halos. We would like to take this opportunity to thank them again~ 👍🏻

Second, sophistication.

Every service needs to be upgraded for a better service. We had to proceed to the next step once a particular performance was met to meet the service launch time. I am determined to upgrade the following update with better performance 🔥

Third, a balance between speed and accuracy

I've been using GPT3.5 and GPT4 for a while now, and I can see the difference between the two. This is from a developer's perspective, so it may not resonate with everyone, but here's what I noticed.

GPT3.5 Turbo: Fast. Understanding the context of complex and long conversations is not as good as in GPT4, but understanding simple discussions (2-3 sentences) is fast.

GPT4: Slower. Understands the context of complex and long conversations and can converse in various ways.

I use a mix of GPT3.5 and GPT4 and have found it necessary to strike a balance between speed and accuracy of analysis. I'm interested to see how this will change with GPT4-Turbo and GPT4!