1.877.833.1202

Windows Server Tuning - Expanding TCP/IP socket/port connection pool

This support article provides solutions on what to do when you see an error in the Windows Server event viewer that contains:

Error Message : Only one usage of each socket address (protocol/network address/port) is normally permitted.

The full error message is at the bottom of this article.

Explanation of Issue

On Windows Server with ASP, .NET, IIS, etc - there is  normally only one (1) permitted usage of each socket address (protocol/network address/port). When a client initiates a TCP/IP socket connection to a server the client typically connects to a specific port on the server and requests that the server respond to the client on a TCP or UDP port chosen from an available pool of ports. Under certain conditions it is possible that the available ports in the default range will be exhausted. A new client connection will be refused if all ports are used and a message will be written to the event log.

Why this happens   

TCP/IP port exhaustion can occur on a client computer if the client computer is engaging in an unusually high number of TCIP/IP socket connections with a Server application. When a FileHold Server application is very busy with client requests, each FileHold client is making repeated, authenticated calls to the FileHold server application, and consequently there is repeated and continuous open and closing of connections. A client connection is comprised of 5 parts ( Protocol, Local IP, Local Port, Remote IP, Remote Port).

By default there are 4076 ports available in Windows 2003 because the usable port range is between port 1025 and port 5000. The FileHold web services communicate with the authenticated client repeatedly. On a very busy server the FileHold application can run into these port bottlenecks within the Windows server.

By default on Windows 2008 servers the operating system allows socket connections to be established between the ports 49152 – 65535; this equates to a little over 13,000 user ports. This is a larger amount than the default of Windows 2003, but it may need to be increased on a busy server.

If all of the available ports are allocated to FileHold client sessions then new clients experience a condition known as TCP/IP port exhaustion. When TCP/IP port exhaustion occurs client port reservations cannot be made and errors will occur in client applications that attempt to connect to a server via TCP/IP sockets. TCP/IP port exhaustion is more likely to occur under high load conditions than under normal load conditions. It can also be an indicator that the server hardware is underpowered and not adequate for the job.

At the bottom of this support article is a listing of event ID's that can be an indicator that this is occurring.  Each of these events will display a similar error:  TCP Provider, error: 0 - An existing connection was forcibly closed by the remote host. 

EventID's:  100, 141, 177, 223, 288, 291, 293, 312, 323

Resolution for Windows 2003 Server and Windows 2008 Server

Adjust the web server TCP/IP socket parameters to provide the IIS/ASP.NET FileHold Windows Server environment with adequate network sockets at a sufficient reusable rate. To do this, perform the following tasks in the Windows registry: 

IMPORTANT: Incorrect use of the Windows registry editor for Windows 2003 may prevent the operating system from functioning properly. Great care should be taken when making changes to a Windows registry.  Registry modifications should only be carried-out by persons experienced in the use of the registry editor application. It is recommended that a complete backup of the registry be made prior to making any registry changes.

Windows 2008 Server

  • On Windows 2008 servers, by default the operating system allows socket connections to be established between the ports 49152 – 65535; this equates to a little over 13,000 user ports.  This should be increased if you see the warning signs in the Windows event viewer logs.
  • To display the current ports for the TCP protocol use the netsh command
  • Open command prompt as administrator
  • Then type

netsh int ipv4 show dynamicport tcp

-To increase the ports use the net shell command

netsh int ipv4 set dynamicport tcp start=1500 num=63000

This will provide 63,000 TCP user ports.

  • When increasing the MaxUserPort, Microsoft recommends that port 1434 be reserved for use by the SQL Server Browser service (sqlbrowser.exe). 
  • On a Windows 2008 server we simply start the port allocation at 1500 as shown in the previous Windows 2008 netsh command.

Windows 2003 Server

1.  On the FileHold Application Server please locate the following key in the Windows registry: 

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters

2.  Update the following values, or create them if they do not already exist: 

Name

Type

Default

Recommended (decimal)

MaxUserPort

DWORD

5,000

64,512

TCPTimedWaitDelay

DWORD

240

120

MaxFreeTcbs

DWORD

2,000

65,536

MaxHashTableSize

DWORD

512

16,384

 

TIP:  
•When increasing the MaxUserPort, Microsoft recommends that port 1434 be reserved for use by the SQL Server Browser service (sqlbrowser.exe).

•On a Windows 2003 server add the following registry key

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters

Add a Multi-string Value key.

Rename the new key to ReservedPorts

Add the values: 1434-1434

3.  For changes to take effect, REBOOT the server. 

 

Sample Snippet from a Windows Event viewer

MachineName : FILEHOLD1

TimeStamp : 6/30/2011 5:57:05 AM

FullName : Microsoft.Practices.EnterpriseLibrary.ExceptionHandling,

Version=2.0.0.0, Culture=neutral, PublicKeyToken=89d498f6b8d04fcc

AppDomainName :

/LM/w3svc/1/root/FH/FileHold/LibraryManager-1-129538815002500000

ThreadIdentity : Joe Tester

WindowsIdentity : TESTDOMAIN\FH_Service

      Inner Exception

      ---------------

      Type : System.Net.Sockets.SocketException, System, Version=2.0.0.0,

Culture=neutral, PublicKeyToken=b77a5c561934e089

      Message : Only one usage of each socket address (protocol/network

address/port) is normally permitted 10.53.17.10:80

      Source : System

      Help link :

      ErrorCode : 10048

      SocketErrorCode : AddressAlreadyInUse

      NativeErrorCode : 10048

      Data : System.Collections.ListDictionaryInternal

      TargetSite : Void DoConnect(System.Net.EndPoint,

System.Net.SocketAddress)

      Stack Trace :    at System.Net.Sockets.Socket.DoConnect(EndPoint

endPointSnapshot, SocketAddress socketAddress)

         at System.Net.Sockets.Socket.InternalConnect(EndPoint remoteEP)

         at System.Net.ServicePoint.ConnectSocketInternal(Boolean

connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address,

ConnectSocketState state, IAsyncResult asyncResult, Int32 timeout,

Exception& exception)


Listing of Event ID's

Source

Event ID

Value

199, 100

Description

Only one usage of each socket address (protocol/network address/port) is normally permitted

 

Source

Event ID

Value

5000

Description

P1 acceleratorservice.exe, P2 7.5.2.0, P3 475edacb, P4 mscorlib, P5 2.0.0.0, P6 471ebc5b, P7 416b, P8 27, P9 system.runtime.remoting.remoting

 

Source

Event ID

Value

293

Description

TCP Provider, error: 0 - An existing connection was forcibly closed by the remote host.

 

Source

Event ID

Value

288

Description

TCP Provider, error: 0 - An existing connection was forcibly closed by the remote host.

 

Source

Event ID

Value

323

Description

TCP Provider, error: 0 - An existing connection was forcibly closed by the remote host.

 

Source

Event ID

Value

312

Description

TCP Provider, error: 0 - An existing connection was forcibly closed by the remote host.

 

Source

Event ID

Value

223

Description

TCP Provider, error: 0 - An existing connection was forcibly closed by the remote host.

 

Source

Event ID

Value

291

Description

TCP Provider, error: 0 - An existing connection was forcibly closed by the remote host.

 

Source

Event ID

Value

141

Description

TCP Provider, error: 0 - An existing connection was forcibly closed by the remote host.

 

Source

Event ID

Value

177

Description

TCP Provider, error: 0 - An existing connection was forcibly closed by the remote host.