What are ‘No Operation’ (NOP) PDUs?

In the world of iSCSI, it was necessary to implement a way for initiators and targets to verify that connections and/or sessions are still active and functioning by design. This is what NOP is, it is basically a “ping” or “echo” request at the iSCSI layer to do just that; ensure everything is in a healthy state. This operation can be performed by either the target or the initiator as it is important for both to be up-to-date on the status of connectivity.

There are two different types of NOP requests:

NOP-In: A No Operation request sent from the target to the initiator.

NOP-Out: A No Operation request sent from the initiator to the target.

Each of the above can be either a request or a response to the others request. Remember, either a target or initiator can send these requests and each must be responded to. So if a target sends a NOP-In request then it expects a NOP-Out response to ensure all is well and the same with initiator to target.

The next logical questions here seem to be, “how often can a NOP request be issued,” “how long does the responder have to reply to the request,” and “what happens if the requestor doesn’t get a response?”

Let’s start with the first question, how often can a target or initiator issue a NOP request? While there are other reasons for NOP requests to be issued, it is often timer based when a request is issued. More specifically, when the link is considered “idle”. Again, this is not a hard and fast rule but is generally what you will see. Within ESXi this time is controlled by the “NoopOutInterval” and is set to 15 seconds by default.

Next, how long does the responder have to reply? This is controlled by NOP-Out Timeout value (NoopOutTimeout) in ESXi and is 10 seconds by default. This means that once the NOP-Out request is sent to the target, it will wait 10 seconds for the response before any action is taken. 10 seconds is a generous time in the world these days and is more than ample time to wait for a response.

This leaves us with the next question: what happens if the requestor doesn’t get a response? If no response is received to this request, then the initiator (or target) believes something to be wrong with their current session, and thus, is torn down to try and recover. Upon successful tear down, iSCSI will attempt to recover the session and if recovery is unsuccessful, reports this back to the SCSI layer where it will work in parallel with the Native Multipathing Plugin (NMP) to determine where to retry the outstanding I/O requests.

An iSCSI session recovery is deemed as failed after 10 seconds by default in ESXi. This value can be modified to be as low as 1 second or as high as 120 seconds.