login about faq

Is this true or did i miss something?

I mean what is this API worth anyhow if there is no shared background thread? I can start a thread with a few lines of code myself so where is the benefit from it. I have to open thousand connections to satisfy the 100MBit connection of my web spider. This is eating up a lot of resources and for small GET with failed IF-modified requests i'm sure there is a lot of overhead.

I know i should have checked this during 30days trial but honestly i'm very very disappointed if i'm right. Is there any way to implement real background thread/asynchronous processing myself?

It raises the overall question in my mind: Do you see performance as a serious design goal or is it just easy useability?

asked May 08 '15 at 11:21

llothar's gravatar image


edited May 08 '15 at 11:54

The purpose of the async background method calls is to satisfy the needs of those that might be developing applications in certain programming languages where it is difficult to create a background thread, and the need is simply to do an HTTP request in the background without blocking the UI. It's not necessarily a feature for a high-performance web spider. For every developer trying to create a high-performance web spider, there are a thousand others with a far simpler needs.

It sounds like what you really need is to decouple the request-sending from the response-receiving. This is the only way a single thread can have multiple requests ongoing at once. The Chilkat HTTP API methods both send and receive the response, and the entire request-response may be more complicated than a single request followed by a single response. There could be authentication (NTLM for example which requires a 3-step process), or 100-continue, or redirects, etc. These things are all handled automatically.

Given your needs, you probably want to use the Socket API. GET requests are simple to form. You can create your own thread and then implement some scheme where you send requests and get responses. HTTP responses, unless chunked and/or gzipped, are MIME. Your request could indicate that you don't want a gzipped response. I can't recall, but it may be possible to indicate you don't want chunked responses. In that case, given that HTTP requests and responses are simply a start line followed by MIME, you could use Chilkat MIME to parse each response.


answered May 08 '15 at 11:53

chilkat's gravatar image

chilkat ♦♦

Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here



Answers and Comments

Markdown Basics

  • *italic* or __italic__
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported



Asked: May 08 '15 at 11:21

Seen: 1,264 times

Last updated: May 08 '15 at 11:54

powered by OSQA