What is all that %sys time ?  “I never know what she’s _doing_ in there…” Ha!

12:01:35 PM CPU %usr %nice %sys %iowait %irq %soft %idle
12:01:36 PM all 0.08 0.00  3.33 0.00    0.00 5.00  91.59
12:01:36 PM 0   0.00 0.00 40.59 0.00    0.00 59.41  0.00

...

You can instantly find out with ‘perf top’.  In this case (netperf), the kernel is spending time copying skb’s around, mediating between kernel and userspace.  I wrote a bit about this in a previous blog post; the traditional protection ring.

All that copying takes time…precious, precious time.  And CPU cycles; also precious.  And memory bandwidth…etc.

Continue reading “Battle Plan for RDMA over Converged Ethernet (RoCE)” →