• Advertisement
Sign in to follow this  

Is this a fair questions to discard a junior candidate?

Recommended Posts

Something happened recently, and I wasn't exactly sure whether it was fair, so I'm asking here to get other programmers' perspective on the matter.

In interviewing junior candidates for a junior role on a team, the key interviewer asked the person to explain why, in general, GPUs used 4x4 matrices for (complex) transformations. The question was basically thrown to see whether the candidate knew anything about homogeneous transformations (basically adding the w axis which is the origin to consider the offset of a translation, etc.)

The problem, in my perspective, was that the role the candidate was applying for was a junior role, with a specific engine in mind (Unity in this case), and that the engine itself would abstract that whole idea to a point where it would never be relevant for the developer. However, I can appreciate that gauging whether the potential developer had any idea what was happening 'behind the scenes' could have value, and that it can be a concept that's hard to grasp AFTER having started working in the industry, but it left a sour taste in my mouth that the otherwise savvy candidate would be discarded based on his ignorance of the maths behind it, given he had demonstrated results on a variety of 'complex' fronts for a junior (A*/Dijkstra, complex AI systems, etc.)

Note: the position was labeled as 'Junior Programmer' strictly, and none of the descriptive items listed any requirement of computer graphics-related experience per se (there was a mention of knowing basics such as 'draw calls' though, so up to the reader's interpretation whether the position entailed gfx-heavy computation skills).

I'll admit I'm on the fence on this one and hoping to get perspective on the matter (realistically, it isn't about whether homogeneous transformations are easy/hard to pick-up as much as it is whether this is a valid interview question to vet a junior candidate expected to work 'in-engine').

 

(@mods: not sure whether this belongs in Game Industry Job Advice (discussing the validity of the question in an interview and the resulting process) or one of the programming sub-threads (as it hinges on what it important/relevant for a developer), so feel free to move accordingly).

 

Thanks!

Share this post


Link to post
Share on other sites
Advertisement

Fairness is always subjective. Personally it's hard for me to imagine rejecting any candidate on the basis of a single technical question.

I would have failed with the question above, too. I don't think it's a particularly important concept; but then I'm not applying for graphics jobs.

Share this post


Link to post
Share on other sites

Hmm, personally this was something I was taught in 2nd year of Uni as far as I remember, may even have been first year. Though my course was specifically Games Programming. So to me it seems like a perfectly fair question to ask of a Junior programmer.
Though, I can see that there are likely the majority of candidates that would have done a Computer Science degree that likely wouldn't have covered anything like that.

You do mention though that he had demonstrated knowledge of several other areas such as pathfinding / AI. So realistically, I think the question is fair to ask a junior candidate, though I dont think it should be grounds for instantly discarding them from the job, especially if they have shown knowledge in other areas.

Share this post


Link to post
Share on other sites

In interviewing junior candidates for a junior role on a team, the key interviewer asked the person to explain why, in general, GPUs used 4x4 matrices for (complex) transformations. The question was basically thrown to see whether the candidate knew anything about homogeneous transformations (basically adding the w axis which is the origin to consider the offset of a translation, etc.)


Speaking as a junior, myself - yes, I would expect another junior dev to get this. In fact, I would expect a reasonably bright university student who had ever done anything related to 3D graphics to be able to answer this question. Homogeneous transformations are pretty much the first thing you learn in your first uni-level computer graphics course. I wouldn't expect a candidate to be able to throw together a homogeneous transformation matrix manually, on the spot, but I would hope that they at least know what they are. Even if you don't need someone with graphics experience right at that second, how do you know you won't a few months down the line? Given the choice between two junior candidates who both fulfill the requirements - one who can answer the math questions and understands what is actually happening when they use the math and one who can't - who would you choose?

That being said, I'm not sure I would fail a candidate solely on these grounds unless I had future graphics work in mind for the candidate. Edited by Oberon_Command

Share this post


Link to post
Share on other sites
The problem, in my perspective, was that the role the candidate was applying for was a junior role, with a specific engine in mind (Unity in this case), and that the engine itself would abstract that whole idea to a point where it would never be relevant for the developer.

I'd expect anybody who works near graphics (that is, including using something like Unity to put the graphics onto the screen, not just down in the D3D trenches) to have a basic idea of why 4x4 matrices might be used. I would not reject them as a candidate for lacking that knowledge though; if they don't, I would at least expect them to be able to talk intelligently about their experience with such transforms, perhaps speculate on why 4x4 is used, et cetera. So I think the question is fair, but that (in the general sense) discarding a candidate over a "wrong answer" to it is not. It is possible to not know the "correct" answer to a question and still perform well in an interview context when responding to that question.

Do you know the candidate was actually rejected by the asker over this question alone? A poor answer to one question generally should not disqualify somebody, but poor answers to many might. It's also possible the asker simply thought, based on the rest of the interview, the candidate might know this information was trying to probe the boundaries of the candidate's experience.

What's interesting to me is that you didn't know this was going to be a question asked by your interview team. To me that suggests a lack of pre-planning or structure to your interview sessions; I see this a lot in the industry. Interviews consist primarily of a handful of team members more or less randomly selected tossed into a room with the candidate to ask whatever random questions they come up with on the spot, or whatever pet questions those developers have, regardless of the position in question or the candidates background. This is an indication, in my mind, of a flawed interview process that does disservice to everybody involved.

I'd be more concerned about that than the fairness of one question in a vacuum. 

Edited by Josh Petrie

Share this post


Link to post
Share on other sites

I treated matrices as a black box until I had to debug a d3d9 -> opengl es 2 porting bug involving them, at which point I built myself a solid geometric understanding of 3x3 and 4x3 matrices, and at least enough of an understanding of 4x4 matrices to debug them.  (The bug was not realizing glUniformMatrix4fv ignored the transpose parameter in ES2.)

As a single reject reason, it seems rather picky for a junior position.  If you've got a flood of well qualified candidates, it might be reasonable to be this picky (especially if they're expected to start working on graphics code right away).  If you've got a trickle and several slots urgently in need of filling, it may be *very* worthwhile to give them a shot - you can always try and mitigate some of your risk by trying them out as a contractor first.  If they work out, you can always offer them an upgrade to full time employment.

Share this post


Link to post
Share on other sites

Like those above, it depends on context.

it isn't about whether homogeneous transformations are easy/hard to pick-up as much as it is whether this is a valid interview question to vet a junior candidate expected to work 'in-engine'

On its face I think the question is a good one. It asks for some general knowledge on a topic that MOST experienced game developers know, and that ALL experienced graphics developers know. It is a valid question, although it may have limited value.

Questions like that are also about the answer. If the person knows, I'd want to see how they describe it. If the person doesn't know the answer I'd love to hear how they think about it. I expect someone who is comfortable with a topic to be able to explain it easily, and to create a description that is easily understood. When I ask some technical questions and the person doesn't know, I ask them to take some guesses and describe what they think. If they show some sign of thoughtfulness and reasoning then it is still a good answer.

 

When I interview people one question alone is typically not enough to 'fail' a job interview. A series of questionable or factually wrong answers will get the candidate on the 'no hire' stack, but one usually won't. Most interviewers will allow for some questions to go badly, especially because people are often nervous and excited. The interview is a discussion, a way to learn about someone and their skills and experience and background. It isn't supposed to be a high-stakes trivia show.  That question is somewhat trivia-like, but still gives insights to experience and background.

Share this post


Link to post
Share on other sites

Doesn't sound like a graphics specific role reading the post, so what should be evaluated is his general capability around the CS domain, not the graphics domain.

Failing based on one graphics question seems to be over the top, even though the question is a basic one that is taught in entry graphics classes. Previous studios I have worked in definitely hired programmers that have never touched graphics in their life to work as junior gameplay programmers, and we had an in house engine that didn't abstract the maths as nicely as Unity as well.

So I think it's a bit tough to fail a person solely based on that.

Share this post


Link to post
Share on other sites

I think the question whether this is fair or not is misleading. If you are interviewing a candidate the job is to hire the best person for the position you can find. A candidate which cannot answer a simple question about affine transformation is definitely not the strongest candidate. I run into the same issue with white board coding interviews all the time. People claim it is unfair and doesn't bring out their strengths. I agree with all of this, but in my opinion a false positive is more expansive for a company than a false negative and as I know for sure that there are people exceeding at white board interviews and I am pretty sure that there are junior candidates that will be able to answer simple questions about affine transformation. These are the people you are looking for and the job of the interviewer is to find the best candidate. Personal drama has no room here.

So, in my opinion this was the right decision as there a potentially better candidates which are a better fit for that role and it is very legit to aim for this. Also, failing in an interview is not the end of the world. It happened to me as well! The important thing for the candidate is that he receives feedback and can take something out of the interview. So the next time he is asked about affine transformation he will be able to give good answers.

Edited by Dirk Gregorius

Share this post


Link to post
Share on other sites

The important thing for the candidate is that he receives feedback and can take something out of the interview.

Assuming, of course, that you are in a position to give such feedback.

Many legal departments won't give you clearance to discuss the content of the interview with someone who your company you did *not* hire.

Share this post


Link to post
Share on other sites
I never heard of any legal issues, but I do ask if the candidate wants feedback before I provide it. In my personal opinion this is only fair and the least you can do. I strongly believe that mistakes and failure are a good thing if we can learn and grow from them!

Share this post


Link to post
Share on other sites

Usually not giving feedback ensures that the candidate can't call for discrimination or try to prove he/she actually does the knowledge/experience. It's similar to the line in regards to references where employees are asked to only to give the dates of when the candidate was at the company.

In regards to the question, I don't see it as a deal breaker unless a lot of the work is based around the knowledge. I see the question (at least at face value) as part of a larger of group of question of seeing what knowledge the candidate as a whole.

Share this post


Link to post
Share on other sites

I agree with all of this, but in my opinion a false positive is more expansive for a company than a false negative and as I know for sure that there are people exceeding at white board interviews and I am pretty sure that there are junior candidates that will be able to answer simple questions about affine transformation. These are the people you are looking for and the job of the interviewer is to find the best candidate. Personal drama has no room here.

This assumes knowledge over what results in a false positive or false negative, which is data we don't have. You might deem it more likely to become a false positive, but I'd argue other factors are more important.

Especially for a junior position, I'd be far more interested in the candidate's thirst for knowledge, personal growth, the way they approach problems and their social/communication skills.

Share this post


Link to post
Share on other sites

 

 

Especially for a junior position, I'd be far more interested in the candidate's thirst for knowledge, personal growth, the way they approach problems and their social/communication skills.

All true, but I am surprised how someone with these traits would not know about affine transformations when applying for a role as game developer. I did practice with people from DigiPen at the whiteboard here at the office and also did go for many lunches to answer their questions. One of the guys just landed a senior position at Amazon essentially after a couple of months out of college. This is what I compare this junior candidate against and this is a guy with dedication and good social skills. There is competition there and there are excellent people out there at all kind of levels.

The point I want to make is not to only see this from the perspective of a candidate, but also from the position of the interviewer. As the interviewer you need to get the best person out of the pool of people applying for the role.

Share this post


Link to post
Share on other sites

There are many unknowns here. If there's literally only one role and you know you have other candidates are good, then sure, you might reject them based on one single question if you felt it was a super important one. But for many roles, it's not a really important question. And in many studios, you're not just hiring one position, you're hiring anyone that is good enough and paying them commensurately. So I still find the idea of rejecting a junior entirely because they didn't answer that question to be odd.

There was probably a time back in university when I could explain in detail what every part of the matrix meant and all these other details, but I've been in the industry a decade and I can't remember that stuff now because it doesn't matter. I don't think I've heard the term homogeneous transformation matrix for years except for this thread. It doesn't come up in practice because most of us can just use whatever functionality the engine provides to build and manipulate the matrices. e.g. If I need one in UE4, boom, FRotationTranslationMatrix::Make to the rescue. Like Hodgman said, the matrix can essentially be a black box unless you want to dig a bit deeper. Personally, I've always seen far too many people getting bogged down on rendering minutiae while neglecting gameplay, networking, and AI, so I specialised in those instead. No regrets.

Share this post


Link to post
Share on other sites

Thanks everyone. Haven't been ghosting this thread, but really wanted to see different opinions before making my mind about it. 

I'm afraid that, I too lack pieces of the puzzle and really can't rule one way or the other (and admittedly, this is less about trying to guess whether it's black or white than to gauge the shades of grey).

I do agree with several views shared here:

  • Whether it's 'fair' from the recruitment perspective is hardly relevant: recruiters are in for the best candidate, and if they had already met someone better, it was a criteria just as good as any to cutting the interview short as they had 'heard enough' (I can only assume earlier questions built up to that moment, even though they were not explicitly referred to).
  • There may have been unspoken requirements which may have impacted applicants negatively. I can remember a similar case where I had personally applied for a job about half a decade ago, and I was exceeding every requirement listed on the job ad by a wide margin, just to get on the 2 minutes call with the recruiter who looked up my resume and asked me 3 questions before saying: 'you're not nearly experienced enough' and, 'your resumé is not detailed enough, I should've known you hadn't done X before' (which the ad did not specify as a requirement to begin with, and objectively was not an information that could've been inferred without knowing the actual (confidential) projects the organization was taking on).
  • Authoring tools and 'friendly engines' such as Unity have made development more accessible to a lot of people, but in so doing have also contributed to the dilution of low-level developers that understand what happens behind the scene, making the 'real coder' a much rarer find. Similar to a post-apocalypse where no one could build anything without a power drill or even replicate any form of AC/DC converter for said tool, it feels like an organization is weakened by potential threats (changing techs for example).

The bottom line is that it's likely fair that the candidate didn't get the job, but that it wasn't necessarily 'his fault' in that he had no indication prior to the interview that his skillset did not match the requirements, and that, I believe can be attributed to unclear requirements from the recruiter, which is unfortunately all too commonplace where job ads are written in silos with next to no communication with devs. 

Thinking back, I can distinctly remember a recruiter at a studio I worked in that had created an advert calling for Unity developers familiar with either C++, JAVA or Python... I mean, really? JAVA != Javascript && C++ != C#, lady! At least we caught this one on time...

 

Thanks again for all the valuable insight!

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this  

  • Advertisement