URLConnection.SendSync Is a Performance Nightmare, and What You Can Do About It
By Thom McGrath on
In my app called Beacon, I've made use of URLConnection.SendSync in a thread to make code flow easier to manage, as the code is mostly a chain of steps. For the feature in question, the user would select a number of servers for Beacon to make a series of HTTP requests. The average user might have one or two servers, though it's not uncommon for a user to have eight to ten. There's technically no limit. For each server, Beacon would run a thread with a chain of URLConnection.SendSync calls.
I previously used my HTTPClientSocket class for these requests, but I just couldn't squeeze the right kind of performance out of it for this purpose. I don't actually need more throughput, I need more time for the main thread to do its work, and that hasn't been possible. So despite my dislike for URLConnection's platform-specific inconsistencies, I had to switch back.
So imagine my surprise and confusion when switching from HTTPClientSocket back to URLConnection didn't actually improve anything.
The problem is URLConnection.SendSync still eats up a lot of cycles while working. I had a user with eight servers tell me it took about nine minutes for one of them to finish. All the threads were fighting for resources and making everything take far longer than it should. I suspect Xojo needs to make the method sleep longer, but that's just a guess.
So here's the gist of what I did to solve the problem. I switched (again) to a subclass of URLConnection called SychronousHTTPSocket. I've used this class in the past, and it used to be a subclass of Xojo.Net.HTTPSocket, which never had a way to execute synchronously. So the Send method was overridden to make the request as normal, but then immediately pause the current thread. The pause is important, as we really don't want the thread doing anything until complete. Calling this on the main thread would logically be an exception.
Then in the PageReceived and Error events, I set some properties for the response to later be retrieved by the calling thread, then resume the thread. This is accomplished by storing a reference to the original calling thread, otherwise we'd always try to resume the main thread. Once the thread resumes, it moves out of the Send method where the calling code can read the results from the properties.
This works great because with the thread paused, it's not even considered during the event loop, and the URLConnection polling happens on the main thread exactly as normal. I made a special build for the user to confirm, and the timing could be measured in seconds instead of minutes!
So if you're using URLConnection.SendSync, seriously consider NOT using it. There are better ways. I believe Xojo was right when Xojo.Net.HTTPSocket didn't include the possibility. And if you're using URLConnection.SendSync on the main thread... you should know better and this post probably isn't for you.
Since Beacon is open source, my SynchronousHTTPSocket is available as part of that repository: SynchronousHTTPSocket. Though I recommend writing your own, because this one has legacy behaviors such as the LastHTTPStatus method that URLConnection now has a built-in replacement for.