The first time Linux started swapping it was doing so really extensively even though only a small amount of additional memory was needed. Depending on the disk speed and size of the swap space this can take a while.
The fix is included in
- RHEL 5.7.z update 2.6.18-274.11.1
- RHEL 6.2 base 2.6.32-220
- SLES 10 SP4 kernel update 2.6.16.60-0.91.1
- SLES 11 SP1 kernel update 2.6.32.45-0.3.1
and will be available for the other distros in the future as well.
As the system becomes totally unresponsive during that period I really recommend to get this fix installed if your system has more than a minimal swap attached and you are experiencing heavy first time swapping now and then. Contact your distribution partner or service provider in this case to get a temp fix.