The grumpy whining about young engineers continues with Part 2 of my commentary on the skills of the fresh-faced in our industry. For context, please make sure you’ve read the original post and Part 1 of this response.
The second reason I raised the topic of skills for younger engineers?
2. Because Abstraction
See how I used the current fad of saying “Because <noun>”? I’m trying to bridge the generation gap, so stick with me for a moment. Let’s assume that this Software Defined Networking (SDN) thing goes full steam ahead, and we end up running automated networks using Python, APIs, controllers, and Single Panes of Glass. What happens when something doesn’t work through the clicky GUI? Is the software going to be smart enough to diagnose and self-repair a problem? Maybe sometimes, but when it’s not, what do we do?
Interestingly, Steven Iveson stole some of my “thought thunder” with his comment on my original post, although it echoes a theme raised on my previous post about the future of networking jobs in an SDN world. It seems that we’re may have a potential problem in the future, which is that the engineers in Tier 1 will be trained to handle the automation and control systems that are managing the network. Tier 3 (the “old guys” mentioned by Steven) are there for when things go wrong, but what’s left for a nominal Tier 2? And how do engineers make the jump from Tier 1 to Tier 3 successfully? I proposed that perhaps we end up in a situation where – since we’ll undoubtedly have a mixture of SDN and non-SDN or partial-SDN networks – Tier 1 may start in a pure SDN environment but would have to step out into a more legacy environment in order to earn their spurs on non-SDN network which would then allow them to learn the skills that would allow them to become that sour old guy in the corner who goes on about how kids today don’t know they’re born and anyway things were cheaper back in my day.
Abstraction?
“You mentioned abstraction,” you say. Yes, I did. To clarify, what I’m talking about here applies to a couple of areas. Firstly, that through the push towards fabrics and the software defined data center, we are hiding the underlying network from view. For example, we are abstracting a complex interconnection of switches and links as a “fabric”. We might not know exactly how the fabric works, or perhaps even how it makes its decisions. Does it matter if you know how ISIS works in order to understand TRILL or SPB? The whole concept is that you don’t need to know about it; it’s plug and play networking. And how are you going to manage it? Not node by node for sure; you’re going to manage it like a black box: data goes in, data comes out and what happens in the middle doesn’t matter (so long as it works).
The second area of abstraction – if abstraction is the right word – is that thing that happens to all technologies as they mature. Newcomers to IP don’t worry about how IP works; they just assume it does. And perhaps that’s why they don’t understand ARP; they’ve never encountered a situation where IP just didn’t work! Similarly, I don’t understand the voltages used in Ethernet, because it just works. I’ve abstracted all of that complicated detail to the level of a line on a diagram connecting two devices; I just see the bandwidth it provides and I don’t worry about the underlying technology, and just hope it works when I plug it in.
Design
What about network design? At Networking Field Day 7 I saw two companies – Dell and Juniper – showing us an automated network design solution of sorts. Juniper’s tool was a command line effort, and heaven help me I found myself saying “they need to put a GUI on that”. Don’t tell anybody, please! Dell’s offering, in contrast, was impressively GUified; tell it how many servers, at what speed they will connect, and the tool pretty much spits out a bill of materials, a full cable matrix and the base configurations to put on all the devices so that the network will come up as designed. When the manufacturer goes to that length to present an architecture to you, do you need to even think any more? We’ve abstracted design! I remember reading about the idea of the robotically managed data center last year if not before, and my gut reaction was “how ridiculous”. On the other hand, you do begin to wonder if you can get to the point of going to a data center provider and saying “I need 1000 servers”, and it gets provisioned and built physically at the click of a button. Ok maybe that’s a bit far-fetched, but just this week AT&T discussed their ambitious vision of their own software defined network, where “Customers will be able to self-provision networking services on the fly, as needed, just as they now do with compute services and storage from cloud providers.” Maybe the robots won’t play a part, but seriously, how cool is that?
Getting Back to the Point (Young Engineers)
Abstraction. Maybe it’s a necessary evil in order to spend less time on the parts of the network that are highly reliable and probably never need your skills, and more time on the more complex higher layer protocols running over the top. Truly the commoditization of the network, and something that the lovely folks at Microsoft and VMWare would like us to continue thinking (dumb network, smart overlays).
On the other hand, SDN ain’t here yet. The products are out there but very few data centers are built using them, and the software to make networks pointy clicky isn’t mature enough yet to hand over the data center to Tier 1. Until we get there, do we still have a need for a good understanding of what’s underneath the hood, so to speak?
More to come.
We’ve had to deal with this at the application layer for years. How many times have you had snoop a wire to figure out how an application behaved because the developer didn’t have a clue? Probably more than once. And why did the developer not know? Because he dragged a 3rd party widget into his development environment…a 3rd party widget he had no clue about how it worked internally.
Very true. The question may be, is that something we’re all guilty of to some degree, perhaps by necessity? We’re encouraged to reuse code because it’s a more efficient (sic) way to develop. Scientists do research with other people’s research/conclusions taken as a given and used as a basis for the hypothesis being evaluated, yet many times it turns out that the original research was flawed, but it was so old that it was assumed to be right and nobody had gone back and double checked it. Same deal – reuse research because it’s more “efficient” and lets you focus on the next level, rather than re-doing what has already been done, and is “known” to work.
My favorite developer trick, by the way, is to develop on the LAN and not once consider how an application will work 1) on the WAN, and 2) when 5000 people log in as they get to work in the morning. In fact, I should probably do a post on that topic as I have a couple of annoying examples.
Back to your point, the problem with nobody understanding what sits within that application module is that when something does go wrong, you’re screwed, and MTTR goes through the roof. Add to this that when you ask “who are the users of your application?” i.e. what source IPs should be talking to your server, the application owners often are unable to answer. It always makes me wonder, if there are no known users, who asked you to build the server, and why?
Very nice post John! I’m one of the young guys whose just starting out in Networking and I’ve loved your posts so far. I can say the talk of SDNs makes me a little nervous, as someone whose not focused heavily in the coding area. I enjoy working with and learning about all the protocols involved and would love to be “an old tier 3 guy” one day. I just hope that SDNs don’t come in too quickly, I feel that I wouldn’t enjoy that kind of work as much.
Keep up the good work with the posts, love the tone and attitude as much as the information!
I don’t think SDNs have to mean coding necessarily, but they will certainly mean a change in how we manage those networks, and on the trust we have to put in the vendor’s/vendors’ tools to diagnose and resolve problems. I’m actually pretty excited that SDN could remove so much of the drudgery-type work from network engineering, and let us get on with more interesting things. But abstraction can have consequences, as we’ve seen in many other areas.
I honestly think that you are over analyzing this a bit. The examples you’ve brought up like ARP and CAM/MAC table vs. ARP cache are explained in college textbooks and industry certs like the CCNA and CCNP. People being lazy and/or not motivated to learn is an issue at the personal level, but I don’t think it means anything beyond that.
Over-analysis comes with the territory 😉 I’ve given examples, but they are simply that, and they represent part of a potentially bigger problem – if it’s a problem, that is. That fact that they’re supposedly taught in textbooks and certifications isn’t really relevant; I don’t care where one picks the knowledge up, but it’s kind of useful. Today, most engineers know that TCP connections start with a three-way handshake. Tomorrow will engineers they know that, or will that detail have been obscured too? By the way, an engineer being lazy and/or not motivated is more than a personal issue; it’s likely a career-impacting issue.
I do see a situation happening, as I mentioned above, where for better or worse, we end up with a wide gap between the lower level engineers and the higher end engineers, and that gap may be a understanding of what actually happens rather than just knowing that the magic cloud does stuff to make it work. Whether that turns out to be an issue, we’ll have to see. But if we look ahead and see there’s a potential problem there, that gives us the opportunity to seek solutions that might help us avoid the problem in the first place. And wouldn’t that be cool, especially if you’re one of those Tier1 engineers?
The difficulty with networking is that you can’t “touch” most of it so it’s not real. Take geometry as an example. There are a ton of concepts in there and they are all rather abstract. But if they are presented in a format, such as building a house, WHOA they become real! Just like your physics class example. The students had the knowledge but it was never applied, so it wasn’t real. Once applied, it became real. Networking is the same way is the same way. You can read all you want to about, even setup a home lab, but it isn’t real until you are mentored into it by someone in the tier above you.
As for the younger vs. older concept. Not true. Age has nothing to do with it. Practical experience is what matters. I started with thin Ethernet (don’t miss that) and my career path took me away from networking, now I’m looping back toward it and trying to catch up on some things.
I know a Unix engineer who has been computing since 1965. He is currently “training” his grandson in the ways of *nix. He said this is the only way to ensure a replacement. To close the gap you mentioned, find some one with the base skill set and mentor him/her and make it “real.”