When you are done with your parallel computations, close any open remote kernel connections. This frees the resources occupied on the remote machines and closes the open network connections.

Note that exiting the local master kernel may or may not close the open connections cleanly. Always use CloseSlaves[] before exiting the master kernel.