OIT Network Systems: Device Blocking Policies

The primary mission of the OIT Network Systems group is to ensure the correct and efficient operation of the campus network for the benefit of Princeton University faculty, staff, and students.

Sometimes our mission makes it necessary to deny network service for a customer's device. For example, we block devices that disrupt or degrade network service. We also block a device when its activity creates a security risk for others. At times we are directed to block service because a customer's network activity places the customer or the University at legal risk, or the activity is in serious violation of University policies.

This document provides an overview of the most common reasons we block devices, how customers are notified of these blocks, and the timeframe for removing such blocks once the customer has resolved the issue.

Why Are Devices Blocked by OIT Network Systems?

These are the most common reasons that lead to OIT Network Systems blocking service for a device:

Disrupting Service

When a device disrupts network service, we take appropriate action to restore normal service for the campus as expeditiously as possible. Usually this includes immediately blocking network service for the disruptive device.

Some common examples include devices that act as rogue DHCP servers, or that steal another's IP address.

Degrading Service

Sometimes a device's activity isn't actually disrupting network service, but instead is only degrading network service. We take appropriate action to restore the quality of network service to an acceptable level, or to prevent the service quality from falling below an acceptable level.

Examples include devices that flood campus DNS service with excessive requests, keep asking campus DHCP service for new leases, or send erroneously-addressed traffic that must be blocked by campus routers on an ongoing basis.

Depending on the degree of service degradation, it may be necessary to block the device immediately. Other times it may possible to give the customer some time to address the issue without blocking the device; if the problem continues, however, we will block the device eventually.

Impeding the Ability of OIT to Manage Network Service

On occasion a device is neither disrupting nor degrading network service, but is operating in such a way as to impede the ability of OIT to manage network service. Ultimately this can affect the quality of network service, because when OIT's ability to manage network service is impeded, we cannot respond expeditiously to detect and correct network problems.

Usually it possible for us to give the customer some time to try to fix the problem, or to disconnect the device until s/he can fix it. If the issue continues (the customer neither fixes the problem nor disconnects the device), we may block the device.

While uncommon, examples include devices that cause an excessive frequency of certain ordinary network events to be logged by network infrastructure. The logging of some ordinary is necessary to allow OIT to perform diagnosis and management of the network. A device that is responsible for an excessive frequency of these events can impede our ability to maintain usable logs. An example is a device with a Ethernet interface that is unable to maintain a stable Ethernet link, causing the Ethernet link to oscillate rapidly between "up" and "down".

Attacking or Compromising Other Devices

A device that appears to be the source of an attack or attempted attack on other devices will be blocked. Typically we block as soon as is practical.

Some examples include devices scanning the network to find targets for potential attacks, devices attempting to guess passwords, and devices attempting to flood targets with excessive traffic.

Persistent Attachment to Campus Network without Host Database Registration

Devices attached to the campus network must be registered in the Princeton University Host Database. (That document notes there are several exceptions, most notably devices properly using Temporary Visitor Wireless Network Access (TVWNA).)

When a device is persistently attached without being registered, OIT Network Systems will request that someone responsible for the device register the device or disconnect the device. If the device remains attached and unregistered, we block network service for the device after a suitable period.

Often these situations also come under the "degrading network service" category. This is because the device often is configured to obtain an IP address via DHCP or BootP. The device's ongoing DHCP or BootP attempts (which fail because the device is not registered) place unnecessary load on campus DHCP/BootP servers, and add unnecessary broadcast traffic to the network.

Violations

Various violations may lead to a device being blocked:

We may block a device operating in such a way that it violates the University's core IT policies, defined in Princeton University Information Technology Resources and Internet Access -- Guidelines for Use.
We will block a device at the direction of OIT's Information Technology Policy Advisor. Common causes include reports of intellectual property infringement (for example, copyright violations), violations of the Information Technology policy document cited above, or violations of the University's Rights, Rules, Responsibilities.
We will block a device at the direction of University disciplinary staff or Office of General Counsel. (Such requests normally come through OIT's Information Technology Policy Advisor.)
We will block all network service for a customer who has earned "three strikes" as per OIT's Three Strikes Policy for Network Disruptions.
Individuals are not permitted to attach to the campus network any device operating as a Wireless Access Point in those University dormitories and apartment buildings where OIT Wireless Service is installed.
The first time that we become aware of a problem involving a customer attaching to the campus network (in a dormitory or apartment building where OIT Wireless Service is available) a device operating as a Wireless Access Point, we will block network service for the device and require that the customer disconnect the device. The second time this happens, we will block all network service for the customer.

While these are the most common reasons we block network service for a device, it is not intended to represent a complete list. There may be less common situations that also are best addressed by blocking network service for a device. Each incident is handled individually, so we may take the most appropriate action as the circumstances warrant.

How Long Do We Wait Before Blocking a Device?

When a device is disrupting service, we usually must block the device immediately. The same is true when a device is attacking or compromising other devices. And we block immediately when there is a violation involved.

Often we do not need to block the device immediately. For example, if the device is only degrading network service (rather than disrupting it), and the degree of service degradation is not severe, it usually is possible to give the customer some time so s/he may fix the problem, or to disconnect the device until s/he can fix it.

For example, if the network can continue to perform acceptably while there are few devices degrading service in a particular way at any one time, we may be able to wait several days for the customer(s) to take action.

In these cases, the problem description OIT Network Systems provides often says something to the effect that "If the customer cannot correct the problem at this time, the customer should disconnect the device from the campus network until s/he can correct the problem."

If the issue continues (the customer neither fixes the problem nor disconnects the device), we block network service for the device. We do so because if we were to leave such matters unresolved, the number of such incidents would grow until their cumulative effect would be to degrade network service below acceptable levels.

How long we wait for the customer to address the problem before the device is blocked varies depending on the problem's nature. For many problems, we wait three business days from the time that a reasonable attempt was made to notify a responsible person of the problem.

When the only issue is that the device is attached to the campus network without being registered in the Princeton University Host Database, we typically wait one calendar week (or five business days) before blocking. (If there are additional issues, such as excessive DHCP or BootP traffic, or use of inappropriate IP addresses, the device may be blocked sooner.)

For some "service is degraded" problems we permit the problem to continue only for a few hours before we must block the device. In such cases, we usually say so in the problem description. For example, we may say: "If the customer cannot correct the problem at this time, the customer should disconnect the device from the campus network until s/he can correct the problem. If the problem is still continuing at the end of the business day, network service for the device will be blocked." We recognize that it is unrealistic to expect most customers will be able to take action in so short a timeframe; we take this approach so that those customers who can take swift action may do so rather than be blocked.

How long we wait before blocking also may be influenced by whether the device has been the source of the same problem repeatedly in the past. For example, after a device has caused the same problem three times, further incidents may lead to the device being blocked promptly rather than giving the customer several business days to address the problem. Similarly, if a device caused the same problem in the past leading to it being blocked, was then unblocked because the customer informed OIT that the problem was resolved, and then the problem resumes quickly enough show that it had never been resolved, OIT will block the device promptly.

How Do We Notify Customers?

When OIT Network Systems makes the decision to block the device, we ask the OIT Help Desk to notify someone responsible for the device.

We normally do so using OPM, OIT's incident tracking system. This assigns a unique "ticket ID" to the incident, and ensures that information about the incident is available round-the-clock to information technology support staff within OIT and in other University departments. This also makes it possible to refer the incident to information technology support staff in other University departments when appropriate.

How the OIT Help Desk determines who best to notify can vary depending on the nature of the incident:

If the incident involves a device that is registered in the Princeton University Host Database, the OIT Help Desk may use the Technical Contact fields in the Host Database entry. Or they may use the Department field. In some case, the Help Desk may need to resort to the Building and Room fields.
The person the Help Desk choose to notify may vary depending on whether the device is registered as a Dormnet subscription (e.g., a student's personal machine), as opposed to an office device.
For an incident where no Host Database entry is known, but only an Ethernet port in a specific location is known, the OIT Help Desk may use the information that appears in the OIT Network Atlas to determine who requested that Ethernet port to be activated; they may choose to contact that person. Or they may look up the location in the University's Campus Directory to determine who may reside or work in that location. For locations likely to be in departmental space, the Help Desk may instead contact that department's information technology support staff, if such staff exist.

Once the OIT Help Desk determines who should be notified, they determine how best to notify that person:

In many cases, the Help Desk will try to contact that person via telephone or email. Sometimes they may try both.
If the customer is a staff or faculty member of a department with information technology support staff (e.g., SCAD or DCS), the Help Desk may instead (or additionally) refer the incident to that department's IT support staff. The Help Desk does so using the OPM incident tracking system. SCAD and DCS members have access to OPM.
OPM sends email to addresses specified by the department's information technology support staff to notify those staff of the incident. OPM allows departmental SCAD and DCS members to handle the incident ticket, annotate it, and to route it further as appropriate.
In some cases the department's IT support staff will address the issue themselves. Other times they will contact the customer to ask the customer to address the issue. Sometimes the department's IT support staff will refer the matter back to the OIT Help Desk, if for example, the issue is one that would be better handled by OIT.
This allow each department flexibility in handling these matters in a way that is suited to the individual department.

Exceptions to this notification approach include:

When OIT Network Systems blocks a device at the instruction of some other authorized party, we do not arrange for customer notification. Instead, the person who directed us to block the device arranges for appropriate customer notification.
When OIT Network Systems blocks a device and there is no information that would allow OIT to reasonably identify someone responsible for the device, we do not attempt any notification. For example, if the device is not registered in the Princeton University Host Database and is attached via Temporary Visitor Wireless Network Access (TVWNA), customer identification is not practical.

How Long Does it Take to Be Unblocked After the Customer Addresses the Issue?

We recognize that customers want to have their devices unblocked as quickly as possible. OIT Network Systems strives to handle "unblock" requests within one business day of the time we receive the request.

The problem description provided by OIT Network Systems usually states how the person handling the incident should advise Network Systems once the issue has been addressed, so that the block may be removed. For example, the OPM incident ticket we create may say: "Once the problem is fixed (or the device is disconnected from the network), please transfer this ticket to the OPM NET-INCOMING queue to have the block(s) removed." So if the person responsible for the device notifies the OIT Help Desk that the problem is fixed, the OIT Help Desk will transfer the OPM incident ticket to OIT Network Systems and request that the block(s) be removed. Alternatively, if the incident is being handled by departmental IT support staff who have OPM access, they may transfer the OPM incident ticket to OIT Network Systems and request that the block(s) be removed.

Once OIT Network Systems has received an unblock request, we strive to handle it within one business day. In support of this goal, we prioritize the handling of such "unblock" requests above many of the other tasks we perform.

Be aware that:

The "one business day" starts when the request reaches OIT Network Systems, not when the request reaches some other OIT group or departmental IT support staff. OIT Network Systems can't act on a request that we've not yet received.
After we remove the block, it may take additional time for the removal to be fully effective. This may extend beyond the "one business day" turnaround.
Some kinds of unblocks are effective as soon as we perform them, for example, re-enabling an Ethernet port or removing hardware address filters.
Others unblocks aren't effective until database updates have reached a variety of network servers. Examples include removing marks that make a Host Database entry ineligible for services (e.g., OIT Mobile IP Service or wireless services provided by OIT). When an unblock will take some time before it is fully effective, Network Systems indicates that as we handle the request. For example, we may write in the OPM incident ticket "This will be effective in an hour."
Although "handling an unblock request" usually implies "unblocking the device", that's not always the case.
Sometime the request we receive get doesn't satisfy the conditions necessary for us to unblock the device. For example, consider a device that was blocked because it was attacking others on the network; the problem description may state that we will unblock the device upon being notified that the cause was determined and corrected. If the unblock request we receive says "I rebooted the device; please unblock it," but there is no indication the problem issue has been resolved, OIT Network Systems will likely "handle" such a request by declining to unblock the device, and routing the request back to the OIT Help Desk.
Other times the block was installed by OIT Network Systems at the direction of someone else authorized to tells us to block devices. In such cases, we will "handle" the unblock request by referring it to the person who initiated the block request.
Unblock requests that are misrouted may not be handled within the "one business day" turnaround.
For example, the problem description when the device was blocked may have instructed the person handling the OPM incident ticket: "Once the problem is fixed, please transfer this ticket to the OPM NET-INCOMING queue to have the block(s) removed."
If departmental or OIT support staff instead create a new OPM incident ticket asking that OIT Network Systems remove the block, handling may be delayed. OIT Network Systems will need to locate the original incident ticket, copy the new request into the original ticket, and close the unnecessary new ticket. This unnecessary additional "paperwork" adds processing time.
Another form of misrouting occurs when customers email or phone OIT Network Systems staff members to request unblocks, rather than contacting the OIT Help Desk (or departmental IT support staff when appropriate). Similar misrouting happens when departmental IT support staff email or phone OIT Network Systems staff members rather than following the instructions in the OPM ticket explaining where to transfer the OPM ticket to have the block(s) removed.
Misrouting the unblock request can delay handling of the unblock request beyond the normal "one business day" turnaround.
Requests that do not include the necessary information for us to identify the incident may not be handled as promptly, or may be routed back to the customer for clarification.

The "one business day turnaround" is specific to unblock requests for those blocks initiated by OIT Network Systems. It doesn't apply to other kinds of requests submitted to OIT Network Systems. We prioritize "unblock" requests above many other kinds of requests.

Appendix: Why Don't Network Systems Staff Contact Customers Directly?

Customers sometimes inquire why OIT Network Systems asks the OIT Help Desk to notify customers when their devices are causing a problem, rather than having OIT Network Systems contact customers directly:

Customers have pointed out that having OIT Network Systems contact customers and departmental IT support staff directly could reduce the time between blocking and customer notification.
Customers have said that if OIT Network Systems were to notify customers directly when they are about to block a device instead of blocking a device, sometimes the customer would be able to take immediate action to correct problems without the need to have their devices blocked.
Customers sometimes say that having OIT Network Systems notify customers directly would be preferable, so that the customers could obtain live troubleshooting services from Network Systems staff while the problem is happening, instead of having the customer obtain support from the OIT Help Desk or departmental IT support.

We recognize that there is some validity to the observations above, however, there are reasons that OIT Network Systems asks the OIT Help Desk to notify customers when their devices are causing a problem. These reasons include:

OIT Network Systems is a small group whose primary mission is to ensure the correct and efficient operation of the campus internet. If the group were to engage in customer notification as well, its mission to operate the network would be impeded.
The OIT Help Desk's mission includes providing telephone and online assistance for campus computing and network-related problems. The Help Desk is staffed around the clock nearly every day of the year, and is the primary point of contact OIT provides to customers.
Over the course of a typical week, OIT Network Systems may block dozens of devices. And we may identify dozens more as degrading network service or requiring some action to avoid being blocked.
Each time that a device is blocked (or identified as needing action to avoid being blocked), it is necessary to determine who best to contact, and how best to contact that person. Actual contact may involve sending email or making telephone calls, often repeatedly. The Help Desk is best positioned to do these things; in fact, customer contact is a central part of its mission.
If Network Systems were to try to attempt to track down customers and contact them prior to blocking a disruptive device, it would prolong the disruption experienced by the rest of the University community. Given a choice between restoring network service for the University as a whole as quickly as possible, or avoiding inconvenience to the person responsible for the disruptive device, we choose the former.
Even having Network Systems perform customer contact after performing the block is not practical; operating the network already requires the full attention of the Network Systems staff.
Network disruptions occur round-the-clock. When they happen outside Univerity business hours, off-duty Network Systems staff may be paged to diagnose the problem and restore service.
Expecting these off-duty staff to also attempt to locate someone responsible for the disruptive device and alert that person to the problem at that time seems particularly wasteful, given that the OIT Help Desk is staffed round-the-clock nearly every day of the year, and is OIT's primary point of contact OIT for customers.
When a device is blocked because it is degrading (rather than disrupting) network service, by the time it is blocked OIT has almost always tried to contact someone responsible for the device.
When the Help Desk notifies a customer (or departmental IT support staff) about a blocked device, very often the customer will need someone to spend significant time explaining the issue. The customer will normally expect that whoever contacts the customer will be able to provide that service.
OIT Network Systems does not have the resources do to this. Instead, we provide enough technical information in the trouble ticket we create to allow other IT professionals (for example, OIT Help Desk staff) provide this level of customer support.
The customer may need significant assistance to address the problem, even if the customer understands the problem we have described. The customer will often expect that whoever contacts the customer will be able to provide that service.
OIT Network Systems does not have the expertise in operating systems and applications needed to provide that assistance; this is something that OIT Help Desk is better prepared to provide. Typically Network Systems staff can characterize the problem from the point of view of the network (e.g., "the device is stealing the IP address of the network router") and identify the device that is the source of the problem. The OIT Help Desk and other IT support staff are better able to assist the customer in diagnosing the cause and correcting it.

For these reasons, OIT Network Systems relies on the OIT Help Desk to notify customers when we discover devices that are causing problems.