Recent Posts


« | Main | »

Interview questions for SE/As (system engineers/administrators) – Unix/Linux

By Dale Reagan | June 22, 2010

I am presenting this post for reference by anyone who might interview a Unix/Linux systems engineer – thoughtful feedback is welcome; it might also be helpful for a systems engineer (SE) a few days before that next interview…  As a  group, SEs tend to have broad experience and expertise with both OS tools and perhaps hardware; questions with specific contexts and clear semantics should provide both the SE and the interviewers with better opportunities to determine a good ‘fit’.  Avoid  allowing bias into your thinking (both sides of the table) – there are no ‘right’ answers to IT questions – there are simply options and which options are selected as solutions for a given problem will be influenced by experience and/or dictated by the environment (i.e. start-up, big business, government, etc.) or other factors beyond your control.   Expect some level of variation in answers/approaches as you interview SE candidates but also expect an eagerness to learn new solutions and technologies… 🙂

A little Semantics – is it Systems Administrator? System Engineer? Systems Programmer?  Something else?

Language provides a wide range of choices in conveying and exchanging ideas; for the most part computing technology is still binary.  From my chair (and of course your view might be different) I see these roles as related but somewhat different:

  1. systems administrator (SA) – tends to switches, dials, monitoring, user related requests, general ‘care and feeding’ of systems/servers, spends more time using GUI than command line, over time develops extensive OS/product level knowledge (may specialize in a specific OS version, i.e. AIX, Solaris, RHEL, etc);
  2. systems engineer (SE) – does the above AND solves system problems, creates solutions for users, works primarily at the command line;
  3. systems programmer (SE+) – does the above AND creates system-level (i.e. new services and customized environments) solutions.

Some background – non-tech

Bloom’s Taxonomy breaks learning down in several types and then provides additional levels. First the types:

  1. Cognitive: mental skills (Knowledge),
  2. Affective: growth in feelings or emotional areas (Attitude), and
  3. Psychomotor: manual or physical skills (Skills).

There are six levels in the cognitive domain (very quick summary):

  1. Knowledge – simple data recall
  2. Comprehension – explain the information
  3. Application – apply the knowledge
  4. Analysis – break things down
  5. Synthesis – create something new from knowledge
  6. Evaluation – judge the value of ideas or real goods

For the cognitive domains above where would you expect system folks to perform?

A quick story: on day one a new SE was given the task of creating a tool to do X.  X was relatively simple.  There were no restrictions on which tools/solution paths he could use.  At the end of the day (after spending most of the day on the project) he returned with his carefully crafted script written in language Y using ~100 lines of code. The tool was demonstrated/tested and all was well.  A senior SE was asked to provide the same solution on the spot (i.e. needed in two minutes) and he produced a one-line solution using pipes and tools available on any standard Unix/Linux system.  Another SE (who operated at guru level with language Y) provided the same solution in three lines using language Y.  A good discussion ensued and all involved expanded their skill sets.  Most of the time KISS rules; the rest of the time you use the next closest solution and, if that means installing tool Z along with modules N, M and O, then that is what you use – all the while keeping in mind that your non-KISS solution is less portable, introduces new system dependencies, gotchas, and security concerns; a KISS solution is almost always better.

Interviews are partly about getting a feel for the organization, the position & the people and partly about basic fact finding:  is there a match between needs and skill sets?  a match with personalities? are there opportunities to expand knowledge and skill sets?  will this candidate be able to grow into the position?  will they be able to expand into other areas?  what is the level of domain knowledge (both sides of the table) and what are the concept acquisition skills like?

Concept knowledge

This is the ‘teach a man to fish’ vs ‘feed a man a fish’ arena.  You may need to feed him on day one but once he has the concepts he should be ok on his own. Read about it;  aware of it; could use it (or something similar) to solve problem X.  The actual implementation would reflect the environment, i.e.  doing X on a Windows server versus doing X on a Linux server versus doing X on an AIX server.   Concept knowledge can lead to new, novel applications/solutions to problems provided there is adequate domain knowledge (existing or obtainable) to implement the solution.  The original approach/philosophy behind the Unix OS (and related OS’s) is to combine small tools to accomplish tasks – keep it simple.

Another story: A Java programmer has written a tool that processes input files.  It was supposed to be a one-time ‘run’.  It wound up being needed for multiple runs.  He did not have a quick solution to process all of the files since system level variables needed to be adjusted for each run.  His time frame was ‘now’.   He had no shell scripting experience so he asked for assistance.  An SE was assigned to assist – he was not a Java programmer:  he provided a knowledge bridge (shared how to accomplish the task using system tools with the Java programmer) as well creating a simple shell script that changed the system variables as needed and ran the Java code as many times as needed.  Unix/Linux system engineers generally have high level concept knowledge covering the IT spectrum – they will generally be able to provide the ‘glue’ to connect new and old.

Some things to keep in mind

SE duties can cover any aspect of a system; some activities occur more frequently than others so I would expect expertise to bubble up as needed.  For instance, if the SE had recently spent significant time work with Apache, Samba, NFS, etc. then I would expect top-line responses to queries.  If it has been weeks/months since the last firewall change then I would expect that the base knowledge was there but that it might need refreshing for a detailed discussion.  Keep in mind that on a well managed system it might be months (or longer) before there is any need to change/tweak/adjust lower level system configurations or services; I usually review my custom notes and might even review current documentation for OS items that I have not touched in several weeks/months.   A simple example:  I recently installed mod_geoip along with related data sets.  Other than updating the data sets there is little to do until some update occurs with mod_geoip.  In a few months I probably won’t recall all the details that may have been involved in installing/configuring/testing mod_geoip – but I know where to find them…

Yet another story

I am leaving the office of a local client when the CIO requests to see me.  I drop by his office and he explains a data problem – might I have a solution? The problem: he has a multi-megabyte data set of fixed length records (one extremely long sequence of ASCII characters with an end of file marker) and needs to ‘convert’ the file to one record per line.  This is data generated by a mainframe export/report and now the data needs to be transferred to an external vendor who requires a file with one record per line.  While he could have his programming team create a new report/export program it would take several days to work through their coding process (requirements, coding, Q&A, etc.)  Do I have a tool (or can I create one) that will break long lines of text into one record per line?   Of course I did have a simple MS-DOS program to accomplish this – it was inspired by the many fine tools you find on Unix/Linux systems that do small tasks extremely well – and I just happened to have my floppy disk (remember those!) tool set with me.  I wrote the program when I had a need to do something similar when moving data between Unix/MS-DOS.  The program required 3 inputs:  1) number of characters per line, 2) an input file name and 3) an output file name.  It would simply add a linefeed character after reading X number of characters.

Concept & Domain Knowledge

Real world been there done that –  If you want to know what real level an SE is operating at then I would ask these types of questions.   Some questions would have only one answer.  Many would have multiple solutions.  The difference between an experienced SE and a less experienced SE would be reflected in the depth of knowledge (direct or in-direct; in your head or know how to find it) along with the number of solutions presented (my expectation would be that a more experienced SE would provide more than one path to explore to solve the problem or resolve an issue.)  Some sample questions (with a Unix/Linux context) with qualifiers follow.

Questions about standard Unix/Linux tools – do not use/ask unless you know some of the possible answers – all items may not be relevant to all Unix/Linux versions…

  1. Name five (5) two letter Linux/Unix tools.
  2. Name five (5) other Unix/Linux tools.
  3. There are file system problems – what should you consider doing to resolve them?
  4. What are the tables in a Linux firewall for?
  5. What are RFCs?  Describe one.
  6. What is SOA – context is DNS.
  7. What is a ‘smart host’ – use any context that you are aware of.
  8. What is a ‘proxy’? name one.
  9. How can you control network access to a Linux system?  Which files might you adjust?
  10. Where are most configuration files found on a Linux system?
  11. Where are most log files found on a Linux system?
  12. How might you setup remote logging from server X to server Y?
  13. Who needs the ‘root’ password?
  14. You ‘lost’ root access to a system – how do you resolve this?
  15. What is ‘lsof’ and what can you do with it?
  16. What standard system tool can be used to read/convert EBCDIC data?
  17. What, if any concerns might you have with using Open Source solutions in a production environment?
  18. What is your scripting tool of choice? Why?
  19. What is your preferred shell? Why?
  20. Which common tools/system components should you remove/disable (or not install) on a production server?  Why?
  21. How might you use more than one NIC on a system?
  22. The output from the command ‘ ps aux’ contains a process with a  ‘Z’ status flag – explain.
  23. Describe the sequence that occurs for an incoming or out-going email message.
  24. Describe the sequence that occurs during an HTTP session on a web server.
  25. What special considerations might you need to resolve when creating running a script from cron?

Using the list below – For the items that you recognize, select your own Unix/Linux context and describe what these items are used for and how and when you might use them or where you recall seeing them:

What does this line do?

What does the line below do – describe in detail.  List three ways to execute the output of the line.

What does this line do and how might you shorten it and get the same results?

Some General Questions

Given context/condition X how would you do/accomplish Y? (X = your problem de jeur…)

For the sample file extracts below that you recognize:  what might each extract be used for or be a part of?  Describe what the entries do – what is their purpose?  Where (on the file system) might you find these files?

File #1

<IfModule mod_negotiation.c>
<IfModule mod_include.c>
 <Directory "/var/www/error">
 AllowOverride None
 Options IncludesNoExec
 AddOutputFilter Includes html
 AddHandler type-map var
 Order allow,deny
 Allow from all
 LanguagePriority en es de fr
 ForceLanguagePriority Prefer Fallback

File #2

# Log cron stuff
cron.*                                                  /var/log/cron
# Everybody gets emergency messages
*.emerg                                                 *
# Save news errors of level crit and higher in a special file.
uucp,news.crit                                          /var/log/spooler
# Save boot messages also to boot.log
local7.*                                                /var/log/boot.log

File #3

# default: off
# description: The rsync server is a good addition to an ftp server, as it \
#       allows crc checksumming etc.
service rsync
 disable = yes
 flags           = IPv6
 socket_type     = stream
 wait            = no
 user            = root
 server          = /usr/bin/rsync
 server_args     = --daemon
 log_on_failure  += USERID

File #4

-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 53 -j ACCEPT
-A INPUT -m state --state NEW -m udp -p udp --dport 53 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 443 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited

File #5

title Fedora (
 root (hd1,10)
 kernel /vmlinuz- ro root=UUID=87c47f1c-aaca-4a55-a92b-0aceb7df22f5 nomodeset rhgb quiet
 initrd /initrd-
title Other
 rootnoverify (hd0,0)
 chainloader +1

What tool(s) might you use for:

General info

Note – probably a good idea to use questions like these with an open mind – you may expand your domain knowledge (unless, of course, you already know every possible answer – which is not likely; also avoid your personal biases – it is unlikely that the tool you used or that the way you did something is the only way to do it) in addition to quickly getting a good idea of the skill sets and domain knowledge of your potential hire.

Good luck on both sides of the table! 🙂

Topics: Problem Solving, Unix-Linux-Os | Comments Off on Interview questions for SE/As (system engineers/administrators) – Unix/Linux

Comments are closed.

YOUR GeoIP Data | Ip:
Continent: NA | Country Code: US | Country Name: United States
Region: | State/Region Name: | City:
(US only) Area Code: 0 | Postal code/Zip:
Latitude: 38.000000 | Longitude: -97.000000
Note - if using a mobile device your physical location may NOT be accurate...

Georgia-USA.Com - Web Hosting for Business