Packet Pushers talks about emerging trends in optics with Robert Coenen
Understanding Emerging Trends in Optics
Audio Transcript
Greg Ferro: Welcome to Packet Pushers heavy networking. The deep dive into the glory of data networking where the nerdy is the normal. In todays sponsored podcast, InterOptic returns to give us an update on small form-factor pluggable modules, you know, the modules that go into your switches and based on the feedback we get and with talking to you, we know that people aren’t too happy about the prices they are being charged by the brand vendors when they get their genuine modules. When you look around you see there are plenty of alternatives, but you don’t know what is the difference. There’s a wide variety of suppliers making exactly the same products but it’s hard to tell what the difference is about these products and you end up with decision paralysis. How do you know what to buy if you don’t know what the difference is between them? Well InterOptic has been focused on making SFP modules for over a decade now. That’s what they do, and they can really help you cut cost and give you confidence that the modules are going to work. Pretty simple business model. Joining us today is Rob Coenen. He is the VP of Business Development at InterOptic and Tim Doiron, Principle Analyst, Intelligent Networking at ACG Research. We are going to get deeper into what drives the optic market, what drives the small form-factor pluggables so you can be better informed about what’s going on, so you can be better informed about how that works, what goes on into that technology and what the difference is between branded and unbranded objects and help you make some decisions in that light. Let’s get straight into the thing. Let’s start by quickly reminding people about the InterOptic story. What is it that InterOptic does?
Rob Coenen: So, InterOptic offers large enterprise IT groups a new, simple, and risk-free way to save on IT networking hardware. Our guaranteed, 100% compatible optical transceivers and cables have exactly the same reliability as the OEM brand of products you trust because they are from the same manufacturers but at a significantly lower cost to IT professionals and network architects. I don’t know if you knew this but to fully install and populate a network switch in your data center up to 60% of the cost can be in the optical transceivers.
Greg Ferro: Oh, we know! (laughs). The weird part about it is that quite often you buy the switch and then you put four or five of them in to start, put the uplinks in and then you do the first two servers and then you don’t actually realize how much it costs because you slot them in one at a time. It’s like a hidden trick there.
Rob Coenen: Oh, it is! Yeah, and typically when you ask for a price on the network switch, they will bundle the transceivers in with the price so you can’t really see how much they cost. It’s like buying a car and finding out that the tires cost you $12,000. So yeah,
we save companies a lot of money on these as much as 50%.
So you work that out and you can save a dollar for every 4 dollars that you spend on your networking.
Greg Ferro: So the obvious question to ask here is
what is the difference between InterOptic brand SFP modules and brand vendors
, like when you go out to buy your switch from Cisco, Juniper, or Arista. They have got their optics and you got yours. What’s the difference between the two? Is there a physical difference? Is there special magic powers in there?
Rob Coenen: No, we pride ourselves on the fact that there is very little physical difference as possible. They come from the same manufacturers, they are coded so that when you plug them into an OEM box, it looks exactly the same. The OEM box thinks it’s exactly the same as an OEM branded device. The big difference is return on investment.
Tim Doiron: So, drill into that word coded. For people that don’t understand what a coding means when it comes to an optic. Explain that to them.
Rob Coenen: Oh ok. So these little transceivers, they do have a micro controller in them, they do have firmware, hard coded software that makes them run and they have registers and memory that you can look at with your digital monitoring and when OEM’s buy these devices from the manufacturers, because of course like everything else, they don’t actually make these things themselves.
These devices have to be coded up with code so that when you plug it into say a Cisco box, it plays nicely in a Cisco box
and the Cisco box recognizes it and says.. “oh, ok yes, that is a good device, that’s not a counterfeit device.”They are looking for counterfeits, not necessarily for third-party devices but they want to make sure none of the devices that you plug into them are either from somebody else from a different OEM or that simply just look suspicious. So, you have to be careful to make sure they are coded properly.
Greg Ferro: So those little micro processors in there, do they actually do magic? Like are they actually doing something value added or are they coded at the factory with silicon with a piece of firmware. Like the switch doesn’t actually download the firmware to them?
Rob Coenen: No, no. These devices are hardcoded at the factory and they are never upgraded in the field or anything like that. The switch box just interrogates them to see what their status is and that they look legit.
Greg Ferro: Gets a part number a serial number, version code, things like that so you’ve got some details of what’s happening in the SFP and there is also an interface in there to report things like signal strength and quality and stuff like that.
Rob Coenen: Exactly yeah, you can look to see what’s the optical power coming in, going out, what’s the temperature. You know, little things like that.
Greg Ferro: They are all standardized though right? They are not vendor specific?
Rob Coenen: No
these are all built to industry standards
So they call them multi-source agreements so the top ten or so manufacturers will agree on this is exactly how the device will be shaped and how it will fit into somebody’s OEM box and that way if you have different manufacturers at each end of the link they will play together nicely or you can pull one manufacturer out and put another one in and it works seamlessly.
Greg Ferro: That’s also important from the point of view that there are factories that make these and then there are vendors who sell them and the vendors who sell them want to buy from any of the factories. They don’t want to have to assemble their own factory just to make them. That’s what the standards are which is also important from that point of view.
Rob Coenen: Yeah, well, it can also play one against the other in terms of pricing. There is a lot of price pressure obviously because of this and somebody can replace your SFP with another manufacturer’s SFP and they work exactly the same.
Greg Ferro: So, if you’re going with non OEM, in this context, OEM is original equipment manufacturer. That’s your brand name, your Cisco, your Arista, your Juniper and so forth. If you go down the path of non-OEM, so non-branded, what are the issues?
There is a lot of variety out there between InterOptic and no name branded stuff, how we tell the difference between that and what are the factors that differentiate?
Rob Coenen: Well, you certainly have a whole continuum in this industry and the big things that differentiate them are reliability and compatibility. Compatibility meaning if you initially plug the thing in, does it work or do you have to play some games with it to get it to work inside your OEM switch. You have to tell the switch to ignore the fact that it doesn’t look legitimate or fear that if you upgrade your firmware, not firmware your operating system, your switch operating system software, that it will stop working with that device you just plugged into it and the other is reliability. Does it keep on working? Maybe it works when you first plug it in but will it still keep working in a month? Or three months? Or a year?
Tim Doiron: You said reliability. Is it really that big of a problem in the industry? I mean occasionally you hear about bad batches of SFP’s that come through, but does it really matter that much from vendor to vendor who you would buy from?
Rob Coenen: Oh certainly. Yeah, I mean there is a big difference in cost between these manufacturers and the primary differences are sort of short term reliability and long term reliability and these devices are of course are expensive to engineer and manufacture, but even more so they are very expensive to test because you have to take every single one of them and burn them in for sometimes as long as three weeks. You have to have them powered up and in a hot temperature oven, and keep an eye on it for a while to see if these lasers start to die on you after a while. We’ve seen examples of a customer who bought, let’s say cut rate devices and they showed us two one gallon zip lock bags full of dead SFP’s that they’ve pulled out of their switches for the past couple of months and just slowly accumulated them in these bags because these devices hadn’t been properly burned in so they started dying on them in the first couple of months.
Greg Ferro: Hadn’t they heard of rubbish bins? Why would you collect them?
Rob Coenen: They had intended on returning them or something at some point in time but now it became like more of a joke. Let’s see how many of these things I can accumulate.
Greg Ferro: Ok, so there is a gap. I guess what you’re trying to say is that there is a massive gap between cheap as possible price and there’s a middle of the market where there is a sweet point between not paying branded vendor pricing and paying the cheap as possible price and there is a service level difference between the two.
Rob Coenen: Exactly and there are things that could go wrong on that as well if the coding is not done exactly right inside these transceivers. We have a horror story from a major US retailer and in this example they installed hundreds of non-brand transceivers in their boxes in their data center and it worked fine there for awhile and then they did a patch on their operating system and their switches and all of these ports went down all at once and they had to scramble to try and get them replaced.
Greg Ferro: That’s brutal. Automation can get you a hell of a long way. I am just thinking of all those people running out automation scripts and they’ve upgraded all their switches and then it’s like.. oh my gosh! So, one of the things here, I have seen some people saying that you can reprogram the SFP’s on the fly and they’ll sell you a reprogrammer to do this. Do you recommend that as a practice?
Rob Coenen: We do not.
It’s impossible to properly reset the firmware in the coding in one of these optical transceivers on premise.
The coding can be quite sophisticated as the OEM’s obviously want to make sure they’re not counterfeited so in the OEM switches they are looking for signs of “counterfeiting” and this can happen when you use the on-site premise programming. For example, the on premise programmers, they tend to give all of the devices the same serial number and a lot of the switches their operating systems look for that now so when you plug this in, one or two might work but then after a while the operating system and switch will notice that they all have the same serial number that means they must be counterfeit and it shuts the boards down on you.
Greg Ferro: So, I always had hoped that I would have boxes and boxes of spares and I could just reprogram them to be run in any switch so if I had a multi-vendor network and you’re sort of saying that your experience has been that this is not necessarily the right way out. It might work for some people but not going forward as things change.
Rob Coenen: Exactly. There’s probably a lot of coding you would have to do at the switch level in order to get that to play nicely. And if you’re only deploying a few of these that’s ok, but for major enterprises they have better things to do than spending weeks trying to get their switches to work with these devices.
Greg Ferro: So Tim, you’ve got a lot of experience there with markets and the trends because you’re in the research space for this. I guess one of the questions here are
what are the optical networking trends that are impacting the market and why should customers start thinking about cutting down on the cost of these modules.
Tim Doiron: You mentioned it earlier as the price of the optics or the cost of the optics, and certainly as the electronics have gotten cheaper, open sourced, white boxed, those types of capabilities the percentage of the investment or the capital that you’re having to spend in optics inside of the data center and interconnecting switches, routers, and devices has certainly gone up. With that pressure is also bandwidth growth and bandwidth demand that we know is pervasive and unrelenting.
Greg Ferro: So, do the modules get a lot more expensive? So as we go from, so I know a lot of people out there are still running 1G/10G in their data center, they are looking at going to 10/40G, but the 40G is pretty much dead so you really want to be going 10G/100G wouldn’t you?
Tim Doiron: Right, absolutely, most of the connectivity today that you’d see especially if looked in a hyper scaled data center would be 25/100G and 100G has been dominate over 40G for some number of years already. And the next transition that is happening is the move to 400G. We are certainly early in that process and early in the delivery of modules for that. That is definitely true. And 100G is certainly the work horse no doubt about it but
400G is emerging.
Greg Ferro: Yeah, but you can buy, so what’s the pricing differential between the 40G to 100G optics these days? Pretty much the same I thought because 40G optics are very expensive to make because they have 4 channels at 10G and 100G is 4 channels at 25G.
Tim Doiron: Yeah, I mean 40G has been declining for a long time here or at least a number of years and 100G is undoubtedly the dominate path.
When people see the need to move beyond 10G today, in general, they are moving to 100G.
Ethan Banks: What are the economics of the 400G look like? Is the pricing going to be 4X of 100G? Does it get to a point where it starts to make sense to move that direction or am I sticking with 100G right now because 400G is just way out there?
Tim Doiron: I think it would certainly depend upon your needs as a service provider or as an enterprise but if you look at our past trends in the space, initial new generations of optics have been expensive and then they tend to come down rapidly over time as volume and as compatibility emerges and so certainly a volume centric business and 100G is economically the sweet spot today,
400G is leading edge and would be expensive verses 100G but we would certainly see that go down over time as the volume of 400G goes up.
Greg Ferro: What I would say to people, one of the challenges with buying 100G switches today so you buy 100G uplinks and then maybe use 10G for the service is that the difference between 40G and 100G is not too bad. I mean Rob you were saying that the pricing difference is quite slim.
Rob Coenen: It is. Yeah, nowadays you can get 100G for as little as only 3X the price of 10G yet you’re getting like 10 times the bandwidth so you’re definitely future proofing yourself with 100G.
Greg Ferro: So, if you bought a 32 port switch with 100G, put in a 400G ports into an uplink and then put 32/10’s, you wouldn’t have to do any costs because you’re not oversubscribing, you have got plenty of bandwidth going up.
Rob Coenen: Exactly
Greg Ferro: Yeah, people need to think about those things when you are doing the designs with that sort of stuff. Tim I want to come back to something here. I also know that InterOptic does some stuff around DWDM and selling optics into DWDM modules. What does the market for DWDM look like?
Tim Doiron: So, the dense weight division multiplexing market is generally used in the transport infrastructure where you’re looking at long distances because you got to optimize the use of the fiber that’s in the ground and so when you think of metro networks, you think of long haul networks, you think of submarine networks. All of those are part of that DWDM networking system and from a size of market perspective, in 2018 that market was about a 15 billion dollar market and we see single digit growth for that market in the coming years going up as high as 17, 18 billion dollars in 2022.
Greg Ferro: Well, you’ve seen the emergence of these optical edge boxes where we are seeing the routers that actually have optical modules that go in them so you can drive a datacenter interconnect between two sites very cheaply, like you say, run some doc 5 or put some DWDM modules on it and all of a sudden you have about ½ TB of bandwidth between the two instead of 10G or something and at a much cheaper price. Are you seeing something happening in this wave division multiplexing where customers are starting to do their own DWDM instead of turning to the service, right? Is that a grace market for you?
Tim Doiron: Yeah, one of the things that has occurred is we used to have vertically integrated platforms that were large and multiple blades or modules stuck into those platforms. Today there is certainly a move toward disaggregation and modularity and so we have pizza box type devices or appliances that are like the servers for compute platforms but that are actually for dense wave division multiplexing purposes. We also see coherent technology moving closer to the data center, closer to the access network in the form of even pluggable modules. The OIF today is Optical Internetworking Forum is working on something that is called a 400G ZR Standard and that is creating a QSFPDD module that supports dense wave division multiplexing and is interoperable in the industry through a defined MSA collaboration.
Greg Ferro: Alright, so you’re saying the DWDM modules are the same as the SFP’s in principle, same multi-source agreements, same standardized designs for the steel casings, the heat dissipation, power consumption and the launch signals and all of that sort of stuff. So, Rob, does that mean that InterOptic is going to be selling those types of products to customers as well?
Rob Coenen: Correct. Yes we’ve been selling pluggable DWDM optics to customers for several years now and also supporting them on a technical side. They will come back to us with questions. What type of module is best for this 40 Km of dark fiber that I have or this 80 Km length that I have. They will be given loss budgets from the people who are leasing these fibers to them and they will come to us and say “hey can we support this with brand X, model Y transceiver?”
Tim Doiron: I would like to jump in there. Coherent DWDM systems have tended to be less industry compatible and more vendor specific and the reason is because, the vendors are turning the knobs of modulation and forward air correction and baud rates and they’re working on their digital signal processing algorithms to maximize the amount of through put and performance they can get versus the two pairs of fibers that they have that they need to run many kilometers, hundreds of kilometers, or even thousands of kilometers. And so traditionally the DWDM market has not been interoperable as it relates to beyond 40km or really beyond 10km types of networking but like with the 400 G ZR activity and other things in the industry other pressures we are seeing the application of more industry compatible DWDM types of systems emerge.
Ethan Banks: Yeah, because if it’s a standard, it’s a standard that everyone has got to align to versus if you’re doing DWDM and maybe you’ve got a little bit of flexibility with your brand you can do what you’re describing. Tweak the knobs and tune things to push the max out there and have something that differentiates you a bit.
Tim Doiron: Yeah that’s absolutely correct and in order to have inoperability amongst the vendor community and again when you think about the metropolitan 600 km networks or multi thousand km networks or even hundreds of km networks, squeezing out that performance is key and to get interoperability, a lot of times you end up having to operate at the least common denominator of capabilities and the industry hasn’t wanted to squash that innovation, it wants to encourage the innovation in digital signal processing and then the ability for the vendor community to come up with advanced algorithms that can continue to improve performance.
Greg Ferro: But at the end of the day, all of the pluggable optics and all the pluggable copper, we use optics to try and talk about copper and talk about fiber in case you’re missing that piece of the lingo. They are all going to the same mechanical footprint, right? They are all going to the same physical connector,
whether it’s SFP, QSFP, QSFPDD, or OSFP. They are all physically the same mechanical packaging and also the signaling is all standardized.
Tim Doiron: And again, on the transceiver side that is definitely true when we are talking about something that’s inside the datacenter, some number of meters, or even some number of kilometers. Absolutely what you said was true. But, in the coherent DWDM systems, what has started is a combination of optics and it’s digital signal processing and so in general the DSP’s have started being on the boards that are the modules that slide into these chassis based systems and then the optics have been pluggable and you’ve seen sizes like CFP or CFP 2. Those sizes have come down over time, but we’ve started with DSP’s in the modules and optical pluggable. Today we can get a 200G CFP2DCO which is the digital signal processor married with the optics in a pluggable form factor that is a reasonable size at 200 G per second and again those would tend to be vendor specific, but the mechanical packaging and housing would be compliant with the CFP2DCO specifications.
Greg Ferro: That’s good for vendors because they can buy the mounting frames and the driver chips they are all standardized, commoditized components they don’t have to get too far. So, let me ask you this question, we’ve recently seen some switches which use something that is called a QSFPDD module and then most of the other vendors have gone with an OSFP if I remember rightly. What’s the difference between those two? Now they’re both 400G if I remember right and why would we pursue two different form factors for one line speed.
Tim Doiron: There certainly is different form factors and mechanical specifications being promoted. Arista has been a leader in the OSFP MSA activity while Cisco has been promoting the QSFPDD specification.
Greg Ferro: So, Tim, what’s the difference between those two and why would I care?
Tim Doiron: So, right now as people look at QSFPDD and OSFP and all of these sizes, they’re really trying to deal with an equation that I kind of keep in my head, which is the price of this optics is directly proportional to, what do I want for performance or distance, what do I want for speed and inversely proportional to the footprint or the power that the module is occupying and so kind of
the competition, if you will, between QSFPDD and OSFP is
that QSFPDD has the backward compatibility capability with the existing QSFP and QSFP28 modules that are out there in terms of that specification while the OSFP is thinking a little more down the road at the 400G moving to 800G and beyond so they are proposing a slightly larger package and size.
Greg Ferro: Yeah, the package and sometimes the larger packages are there just to get rid of heat as you’re pushing so much power into those to run the processor.
Tim Doiron: Absolutely heat dissipation is one of the number one issue that all of these vendors absolutely struggle with and if you look at OSFP versus QSFPDD, the QSFPDD can get 36 ports on a single rack unit faceplate, while the OSFP can get 32 ports on the faceplate of a single rack unit.
Greg Ferro: Yeah I don’t see that as a big deal, I know that’s counted as a big deal but the point here is that most of the 100g/400g chip sets are only doing 32 ports.
Tim Doiron: That is true.
Greg Ferro: Right? So, it’s kind of like if you’re Barefoot Tofino 2 which is a 4tb chip set but it does 32/400g ports. Redundant, there is no point in having 36 ports on this.
Tim Doiron: The purpose could be with some form of over subscription, or if you had some lower speed interfaces or something that you also wanted to put on the product.
Greg Ferro: So, Rob you are living through some of this. What does that mean to customers? Does that just mean that InterOptic sells both types of modules with the same customer experience with those? Or is there some more insight in there?
Rob Coenen: Well, from our point of view, we are agnostic as to which of these two solutions will end up being the customer favorite but you can kind of liken this situation to back at the 10g level that was the battle between the X2 and the ZENPAK transceivers which were virtually the same but did not interchange with each other. You couldn’t plug one into the other one’s port and over time they each tried to vie for the winning solution, although in the end neither one of did. They were both replaced with a much smaller SFP+ form factor. So, sometimes it has nothing to do with which one is technically superior like BETA max.. now I am really dating myself.. versus VHS. Everyone agreed that the BETA max was superior but none the less the VHS won.
Greg Ferro: It’s hard to keep your metaphors updated in this era of technology.
Tim Doiron: From an industry perspective though, the reality I think is we are going to see both of these happen and where the volume ends up going is where the economics will go too.
Ethan Banks: Rob, let me ask you another question relating to about packaging and that’s about breakout cables. One of the questions that comes up with the Packet Pushers audience slack is “Ok so I’ve got this particular kind of an optic and I need to break it out like this. What’s my breakout cable assembly?” How are you guys doing the patching? Because then it ends up being kind of a tangled mess sometimes. Is there a trend that you are seeing with customers that they are standardizing on or considerations that people should be thinking about when going to a breakout cable?
Rob Coenen: Well previously, before the breakout cables, if you had a top of rack switch every single server required it’s own port at the top of rack switch. So if you had 32 ports on your top of rack switch, you could feed 32 servers and what the breakout cables enabled, and really it was at 40/10g, that this was the first time you could use one “40g” port to feed 4 10g servers and this saved you a lot of space in terms of the switch. You could now serve 4 times as many servers from one switch. Which doesn’t really seem like a big deal, but it actually is in terms of the density. It greatly increases the density of your rack, greatly decreased the amount of switch hardware that you required and so people instantly jumped on this and it became very popular and it’s dominate form factor at 10/40g and this is now evolving into 100g where you can get 100g switches that have 100g uplink and then 4/25g downlinks down to the servers because you can now get servers that have 25G.
Greg Ferro: You’re saying that your switch has got a 100g port and then the breakout cable gives you 4/25g port because 100g is made up of 4/25g channels in reality.
Rob Coenen: Exactly, when the 100g standard was created it was done so that you could break this out nicely. 40g is a little more haphazard, it sort of came not accidentally it was certainly by design, but not by design when 40g first proposed. It was sort of tacked on afterwards at the 100g level people realized that this was going to be popular, so they roped it in right from the beginning.
Ethan Banks: So, it’s right in the spec. So in other words, if I order a 100g optic, if I had the right cable assembly, I will be able to break that out to 4/25g or is there is a special optic I need to be paying attention to order?
Rob Coenen: Yeah, no it has its own part number with these breakout cables 1-4.
Greg Ferro: Well I am sure you could advise them, maybe they need to contact you and ask some more questions and dive in because I think there’s also some requirements in the switches themselves too?
Rob Coenen: There is, yeah, you have to make sure you have the right switch that supports this kind of thing because otherwise it’s just, 40g ports to feed 40g servers or something like that. You have to make sure it’s broken out to 4/25g.
Ethan Banks: The questions then are we buying and optic and a cable that’s breakout? Or is it all one assembly where it’s the optic with the fan out all as one big part?
Rob Coenen: It’s all one thing, yeah. And these do tend to be copper as well. There are optical breakouts certainly, but a lot of the breakout cables are what they call passive copper.
Greg Ferro: Good for service because you can put a switch in the middle of the rack and then you use the breakout cables to get to the service.
Rob Coenen: Exactly
Greg Ferro: But you have the 100g switch there just in case you wanted to put a 100g nic in your service. (laughs)
Rob Coenen: It will happen.
Greg Ferro: It will happen. I want to take a bit of a deviation here, this is one of my pet peeves is I have seen a lot of cabling companies talk about MTP and MPO cabling which is this cabling sort of that has 12 or 8, 12 or 16 fibers in a single sheath and they have a special connector on the end that allows you to present 8, 12, 16 or even more sometimes, fiber cores directly so you can actually use it as a replacement for horizonal cabling in the data center because it has got so many cores in it. Are you seeing that popular? Or do you get involved with customers in that sort of that design stuff sometimes?
Rob Coenen: Oh certainly, yeah, this has become I would say slightly more popular over the years and it’s slowly gained traction with customers. They certainly shied away from it in the beginning because, “oh no, the connector is different, that means we need new protocols for cleaning them and we need new little devices to reach inside the ports and clean them” and that sort of thing. And then there were some concerns in the beginning about the reliability of these connectors. Are they more lossy than regular LC or SC connectors but they are in terms of density far superior to just two fiber cables you’re getting 8 or 12 fibers almost exactly for the price of 2. And it’s definitely becoming more popular at 100g as well because you know, even though, say it’s only three times the cost of a 10g, 100g is no matter how you look at it a lot more expensive than a 10g. So people are becoming a lot more sensitive to “what are some of my options at 100g? How can I reduce the cost of these optics?” and these parallel fibers is definitely one way to do that in the fact that the fiber itself might be slightly more expensive but the optics you plug in at each end are vastly cheaper.
Greg Ferro: I think also it’s a way of shifting costs because a lot of people continue to insist in the data center using a big rack of fiber optic cabling all the way around, on every rack and then it does nothing for decades. It’s an empty PMBO cabling solutions just clapped together. They click together literally and you can unplug them so if you decommission a rack, you can actually just pull out the fiber and roll it up and put it back in the cubby and run it out again. So, it gives you a lot of flexibility in an enterprise data center to add or remove fibers from a rack because you just plug it straight into a patch panel and boom, you’ve got 24 cores or something. I just wanted to know if the trend was there or was it actually getting adopted and stuff like that?
Rob Coenen: Oh, for sure.
Greg Ferro: I wanted to ask about the optics for 100g and 400g. Have they got new technology? Is there like some special magic going on inside the box? Or is it literally like we took the 10g signal and turned in up 10 times?
Tim Doiron: From a technology perspective, the move to 400g and 100g, you mentioned it earlier that in your transceiver process you will typically have a parallel fiber cable that has 4/25 is the way we achieve 100g. As you get bigger than that and
as you take a look at the move to 400g, it’s harder and harder to just keep cranking up the speed
and so what you see people playing with is a combination of moving away from what we would call a non-return-to-zero or just a one zero bit stream and go to some level of modulation like Pam 4 which gives you two bits per symbol and that’s one way you can double the through put and then the other is, as we talked about the OSFP cable, we talked about QSFPDD is a move to more parallel fibers in that link. So, crank up the speed, crank up the modulation and get away from just a non-return-to-zero and also crank up the amount of parallelism is how in the transceiver space, that is happening.
Rob Coenen: Oh, for sure. At 10g there was a relatively minimal number of types of link you could have but at 100g there has definitely been a lot more innovation, almost an explosion of different kinds of networking options. Which is great in some sense but also confusing for some of our customers as they are faced instead of two or maybe three options, they now have almost a dozen options. I mean I could list them off right here. You have passive copper, active copper, active optical cables, SR4, SWDM4, BiDi, CWDM4, CWDM4 Lite, 4WDM as opposed to the other two CWDM’s, PSM4 and finally the LR4. Those can all coexist inside of a data center.
Greg Ferro: And we always had a lot of different standards for the higher rate speeds. Which one is actually popular? Like which are the ones that people actually use?
Rob Coenen: At 100g we are definitely seeing CWDM4, SR4, and LR4 are proving to be the most popular. Those are probably the most similar to the equivalence to the 40g and 10g. Some of the newer types, the BiDi and PSM4 that are less popular but certainly they have their proponents as well.
Greg Ferro: So, if we have a look at the photonics, now sometimes inside of the SFP modules, we’ve actually got lasers inside of it and
we’re actually seeing a lot of discussion around silicon photonics as an alternative to lasers.
Is that something that we’re actually seeing move ahead? Because they have been talking about silicon photonics for a decade or so or maybe two and we’ve never really seen it come out but we’ve seen a lot of companies make acquisitions around the silicon photonics. So, is that something that we should see? If silicon photonics makes it out into the world is it something we should care about?
Tim Doiron: So silicon photonics, you’re absolutely right, is something that’s been through the hype cycle at least once and maybe twice over the past 20 years.
Greg Ferro: I can remember going into an Intel conference and they are going to use silicon photonics to replace copper buses on the motherboards at one point.
Rob Coenen: I was at that Intel. I worked for Intel at that time and yeah I remember us hyping that one.
Greg Ferro: We were going to have optical channels in the motherboard instead of having copper signals.
Tim Doiron: The one thing I would correct in your statement is that it’s not silicon photonics replacing lasers you still need lasers. Todays application of silicon photonics is about how can I take the optical components that today have high touch in terms of human interaction and less automation. I have to put them together, I have to align them, and we just have seen less investment into optics and putting optics pieces together and packaging them than say the integrated circuit technology like your micro processors and your switches and those which have tons of automation and tons of investments around how to make your electronics faster and cheaper and so, the application of silicon photonics today is about how can I apply some of those same manufacturing methods that I see in integrated circuits and electronics and how can I bring that to the optical component and assembly side of the equation.
Ethan Banks: Tim, does that mean the science challenges of silicon photonics are largely behind us? Because I kept hearing, “we can’t quite get it right. It doesn’t work consistently” and there is science that has got to get done before we can really rely on silicon photonics as a technology.
Tim Doiron: Yeah, that’s a great question and I wouldn’t say it’s all behind us. But if you look at Cisco recently made the acquisition of Luxtera I think it actually closed yesterday. Juniper made an acquisition previously in the silicon photonics arena with Arion as a company that was working in that space too. I’ll just say,
I think progress has been made and the proof will be in the pudding in terms of the ability to squeeze the economics and improve the price points through silicon photonics capabilities.
All of that said, generally speaking you still see lasers off boarded from the silicon photonics because they just haven’t been able to get lasers on board but the other components they have been able to put there so you predominantly see silicon photonics in the short reach, kind of inside the data center transceiver arena. And in the market, just to give a market size to that, people are kind of starting to pay attention to silicon photonics and how much of what we are all selling is silicon photonics based. We’ll start to see more and more analysts take a look at that, but I think those numbers are still in the single digits kind of percentages of what of the market is actually being achieved there but we’ll start to see more numbers and more tracking from the industry.
Ethan Banks: But ultimately, we have to go this direction as I understand it because we are reaching some physic limits within the existing package we have.
We have to go with silicon photonics as speeds ever increase.
Is that a fair statement?
Tim Doiron: Well, yeah, we have to do something and I will just kind of talk about the somethings. So, yes the one area people are pushing is silicon photonics. The other area is, there are companies out there like Infinera and others who are still doing Indian phosphate based integrated photonic circuits or picks they might refer to it. So there certainly is a parallel track I would just say that is going on in the Indian phosphate side of things and kind of parallel to the silicon photonics. But you hit on really one of the biggest challenges which is as you think about the Barefoot Tofino switch chip, as you think about Tomahawk 3 from Broadcom, ultimately those ports have to make their way to the optics on the front faceplate of the switch and the way we do that today is electrical and there is a bunch of parallel electrical interfaces that we are just cranking up the speed on to 25g we are going to 50g and we are going to 100g but you still have an electrical to optical conversion that is going on there and so there are organizations like Kobo, the consortium for on-board optics, and there’s other folks who are looking at how they can put the optics co-packaging with the switch chip. So, there is a lot of innovation that is going in and going in to move the optics closer to the switch chips and whether that results in silicon photonics or on-board optics, all of that is in process right now.
Greg Ferro: And that’s part of the shrinkage process. So, 10 years ago when you looked and the line card that came out of a Nexus 7000, it looked like a kids Lego block collection that exploded on top of it. There were so many chips on it and there would be like 3 or 4 motherboards all stacked on top of each other. Where as today when you look at an equivalent capacity, there is like 1 big switching ASIC, and half a dozen SerDes’ and bang you’re done. It’s a very different sort of board to me.
Rob Coenen: Interesting you make that analogy, and that’s something that people have been talking about with silicon photonics and it’s inevitability if you want to think of it that way because they look at it in the same way that happened with integrated circuits where we went from individual devices to one’s that are hundreds and thousands and millions of them on a chip that are all shrunk down to the size of a fingernail. Silicon photonics is very similar to that in the fact that this is an automated way of making these devices and so for high volumes and high numbers could be very attractive from a cost point of view, but it is still just a manufacturing technology. It hasn’t offered any new types of devices.
Greg Ferro: It doesn’t suddenly boost the lasers by a hundred times or something, it’s literally a way to build a laser on the ASIC die at the time of manufacturing.
Rob Coenen: Exactly, and you can only make these devices so small. They are not like the integrated circuits that can now shrink down now to almost atomic sizes. Light has a certain wavelength, you can’t make wave guides only 7 nanometers wide or what have you. They have to be a certain width or otherwise, they don’t funnel light down them anymore.
Greg Ferro: You need a certain amount of surface area to actually laze and generate a signal as well. So, it’s not like they’re going to get to 100 nanometer type of scale because you need enough material.
Rob Coenen: Exactly, and they are actually pretty big and there are actually not that many devices on a silicon photonics chip. It’s different for integrated circuits and transceivers. Very quickly for these computers that back in day they wanted first a hundred vacuum tubes, then a thousand, then 10 thousand, but for silicon photonics the number of actual components has increased from 4 to 16.
Greg Ferro: So, the reason I wanted to bring up silicon photonics is because for a long time we were told that it was going to revolutionize the SFP module market by dramatically changing the cost. I guess what I learned from this discussion is that it probably isn’t going to happen in the foreseeable future so I could take silicon photonics and take it off my radar and wait for the next hype cycle to come around again in the next few years’ time and take another look.
Rob Coenen: Exactly
Greg Ferro: So, Tim, do you agree with those statements I just made?
Tim Doiron: I don’t agree with those statements. I think you absolutely should be paying attention to what’s going on in the arena of silicon photonics and it’s not that everything has been solved, or that everything is solved for every distance and every reach of optic that’s out there. However, Cisco just paid $660 million dollars for Luxtera. Luxtera has been working in this space and has been one of the pioneers for over 10 years. Intel has been investing in this space for a very long time and again we’re starting to see silicon photonic based implementations enter the market place and with all of that
I think that silicon photonics is something that your listeners should be paying attention to
with the potential to alter the economics and the price points of the optical components that they’re buying and using in their networks.
Greg Ferro: And you’re saying it’s going to happen sooner rather than later so say within the next two to three years?
Tim Doiron: A portion of the market will definitely and is definitely beginning to benefit from silicon photonics. When it reaches a tipping point and when it becomes vast majority. I am not going to make that crystal ball guess.
Greg Ferro: No, no. So don’t take it off the agenda and keep an eye on it, it might actually change your buying strategy. And those types of discussions, if you’re buying several hundred optical modules, silicon photonics might change your future strategy around that. So, maybe you want to focus your supplier around a shorter time frame. That may be the actual point of that. Ok, we are getting towards the end of the show. Rob, what do you think about pricing pressures? Like optics have sort of gotten cheaper over time and there are various pressures that define how the prices go down because obviously it still costs money to manufacture, there’s research and development.
What are the factors that drive pricing pressure on the market today?
Rob Coenen: Well there has always been a lot of pressure on this market and you know traditionally transceivers have dropped in price, from when they were first introduced, they have dropped in price about 15% per year until it hits some sort of minimal price where it tends to be where the cost per bit about to where it was significantly cheaper than the previous generation. This works great for customers of course lower prices means better value for them and oddly enough it has kind of worked in favor of our company in the fact that it has put a lot of pressure on system integrator and resellers in terms of very diminished margins for them. So, we have had them come to us and partner with us so they could actually get better margins on the devices that they sell.
Greg Ferro: So Tim, some final thoughts there on pricing pressures?
Tim Doiron: Certainly the pressure on pricing is something that is causing the vendor community to invest and invest in alternative technologies. We’re looking at speed increases. We’re looking at more parallel lanes. We’re also looking at things like where we used to use things like binary signaling. We’re using a modulated signaling like Pam 4. We’re investing in silicon photonics and we’re investing in and looking at things like the consortium for on board optics, moving the optics closer to the electronics and the switch chips.
Greg Ferro: Thanks Tim. So, Rob, if people wanted to get in contact with InterOptics and get more information and maybe talk to you about getting some of your optics in for a bit of a trial or to work in their network where should they go?
Rob Coenen: Just come to our website www.interoptic.com.
Greg Ferro: Thanks very much for InterOptic for joining us today. They have been a longtime sponsor of the Packet Pushers and of course make it possible for us to bring you these discussions. We’ve aimed here today to find you the balance between useful information as well as product information so that there is something there for both. So, please do get over to InterOptic.com and check them out and if you find something there you like, maybe you can do a little business with them and tell them that Packet Pushers sent you. As always you can always find this and many more, fine free technical podcasts along with our community blogs at packetpushers.net. You can follow us on twitter under Packet Pushers and find us on LinkedIn or like us on Facebook if you must and rate us on Apple podcast. It really really helps us if you could tell more people about us. It helps us to stay here bringing you more great discussions like that we have had today and last but not least, remember that too much networking would never be enough.
