Archived Forum Post

Index of archived forum posts

Question:

Ckhttp Content-Type Header always has double quotes in charset

Feb 07 '13 at 10:15

It always has double quotes in charset, even using the SetRequestHeader For example, Content-Type: text/xml; charset="utf-8"

It causes some issues when calling the Microsoft web service, For example, ---- Received ---- HTTP/1.1 415 Cannot process the message because the content type 'text/xml; charset="utf-8"' was not the expected type 'text/xml; charset=utf-8'. Cache-Control: private Server: Microsoft-IIS/7.5 RequestId: c2ee717c-e608-4ac0-a8f7-2191293be87f X-AspNet-Version: 2.0.50727

According to the http spec, it seems no "double quotes" in Content-Type charset field


Answer

The Chilkat.HttpRequest object has a "SendCharset" boolean property that may be set to true/false. It defaults to true. Set it equal to false (0) if you don't want the charset attribute added to the Content-Type header field.


Answer

We already add custom header manually without using the SendCharset or ContentType VC 2008 with Chilkat 9.4.0

    req.put_SendCharset(false);
    req.AddHeader("Content-Type", "text/xml; charset=utf-8");

The header sent and response received shown below. Always extra double quotes add in charset attribute. The response clearly stating the error.

---- Sending ----
POST /Autodiscover/Autodiscover.xml HTTP/1.1
User-Agent: ExchangeServicesClient/15.00.0516.014
Content-Type: text/xml; charset="utf-8"
Host: autodiscover-s.outlook.com
Authorization: Basic GZzb2Z0Lm9ubWljcm9zb2Z0LmNvbTpuZW83NjVFeCE=
Content-Length: 1153

---- skip -----

---- Received ----
HTTP/1.1 415 Cannot process the message because the content type 'text/xml; charset="utf-8"' was not the expected type 'text/xml; charset=utf-8'.
Cache-Control: private
Server: Microsoft-IIS/7.5
RequestId: 7436fb51-200f-4f1b-9197-f1579fe9ed5e
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET
X-DiagInfo: HKXPRD0210CA016
Date: Tue, 05 Feb 2013 17:58:37 GMT
Content-Length: 0

Testing with WinInet with

Content-Type: text/xml; charset=utf-8
wlll get sucess response


Answer

The last error log:

 ChilkatLog:

SynchronousRequest:

DllDate: Dec 12 2012

UnlockPrefix: XEXSXHttp

Username: ASUSLAPTOP:eric

Architecture: Little Endian; 32-bit

Language: Visual C++ 9.0

VerboseLogging: 0

domain: autodiscover-s.outlook.com

port: 443

ssl: 1

RequestData:

HttpVersion: 1.1

Verb: POST

Path: /Autodiscover/Autodiscover.xml

Charset: utf-8

SendCharset: 0

MimeHeader: User-Agent: ExchangeServicesClient/15.00.0516.014

Content-Type: text/xml; charset="utf-8"

--RequestData

ReadTimeout: 20

ConnectTimeout: 10

httpConnect:

hostname: autodiscover-s.outlook.com

port: 443

ssl: 1

Need to establish connection to the HTTP server...

ConnectTimeoutMs_1: 10000

calling ConnectSocket2

IPV6 enabled connect with NO heartbeat.

connectingTo: autodiscover-s.outlook.com

resolveHostname1:

dnsCacheLookup: autodiscover-s.outlook.com

Resolving domain name (IPV4)

--resolveHostname1

GetHostByNameHB_ipv4: Elapsed time: 0 millisec

myIP_1: 192.168.9.187

myPort_1: 51687

connect successful (1)

clientHelloMajorMinorVersion: 3.1

buildClientHello:

majorVersion: 3

minorVersion: 1

numRandomBytes: 32

sessionIdSize: 0

numCipherSuites: 10

numCompressionMethods: 1

--buildClientHello

readIncomingTls_serverHello:

processTlsRecord:

processHandshake:

handshakeMessageType: ServerHello

handshakeMessageLen: 0x46

processHandshakeMessage:

MessageType: ServerHello

Processing ServerHello...

ServerHello:

MajorVersion: 3

MinorVersion: 1

SessionIdLen: 32

CipherSuite: RSA_WITH_RC4_128_MD5

CipherSuite: 00,04

CompressionMethod: 0

Queueing ServerHello message.

ServerHello is OK.

--ServerHello

--processHandshakeMessage

handshakeMessageType: Certificate

handshakeMessageLen: 0x1447

processHandshakeMessage:

MessageType: Certificate

ProcessCertificates:

Certificate:

derSize: 1709

certSubjectCN: outlook.com

certSerial: 4E10E630000800027CC3

certIssuerCN: Microsoft Secure Server Authority

--Certificate

Certificate:

derSize: 1559

certSubjectCN: Microsoft Secure Server Authority

certSerial: 61033336000500000030

certIssuerCN: Microsoft Internet Authority

--Certificate

Certificate:

derSize: 1302

certSubjectCN: Microsoft Internet Authority

certSerial: 07276202

certIssuerCN: GTE CyberTrust Global Root

--Certificate

Certificate:

derSize: 606

certSubjectCN: GTE CyberTrust Global Root

certSerial: 01A5

certIssuerCN: GTE CyberTrust Global Root

--Certificate

NumCertificates: 4

Queueing Certificates message...

--ProcessCertificates

--processHandshakeMessage

handshakeMessageType: ServerHelloDone

handshakeMessageLen: 0x0

processHandshakeMessage:

MessageType: ServerHelloDone

Queueing HelloDone message.

--processHandshakeMessage

--processHandshake

--processTlsRecord

--readIncomingTls_serverHello

HandshakeQueue:

MessageType: ServerHello

MessageType: Certificate

MessageType: ServerHelloDone

--HandshakeQueue

Dequeued ServerHello message.

Dequeued Certificate message.

DequeuedMessageType: ServerHelloDone

OK to ServerHelloDone!

No client certificate required by the server.

Encrypted pre-master secret with server certificate RSA public key is OK.

Sending ClientKeyExchange...

Sent ClientKeyExchange message.

Sending ChangeCipherSpec...

Sent ChangeCipherSpec message.

Derived keys.

Installed new outgoing security params.

Sending FINISHED message..

algorithm: arc4

keyLength: 128

Sent FINISHED message..

readIncomingTls_changeCipherSpec2:

processTlsRecord:

processChangeCipherSpec:

ccsProtocolType: 1

--processChangeCipherSpec

--processTlsRecord

--readIncomingTls_changeCipherSpec2

readIncomingTls_handshakeFinished2:

processTlsRecord:

processHandshake:

handshakeMessageType: HandshakeFinished

handshakeMessageLen: 0xc

processHandshakeMessage:

MessageType: HandshakeFinished

FinishedMsgLen: 12

Queueing Finished message.

--processHandshakeMessage

--processHandshake

--processTlsRecord

--readIncomingTls_handshakeFinished2

Dequeue the FINISHED message...

Dequeued Finished message.

Handshake completed successfully.

Secure Channel Established.

Connected

--httpConnect

connectTime1: Elapsed time: 344 millisec

sendRequestGetResponse_1:

sendRequest:

Adding Host header...

host: autodiscover-s.outlook.com

port: 443

Not auto-adding cookies.

Adding Basic Authentication Header..

--sendRequest

sendRequestTime: Elapsed time: 62 millisec

---- Reading HTTP Response ----

readResponse2_4:

No transfer-encoding header field.

sslContentLength: 0

extraLen: 0

Already have entire response.

readResponseTime: Elapsed time: 2200 millisec

--readResponse2_4

processResponse_3:

responseStatus: 415

--processResponse_3

--sendRequestGetResponse_1

totalTime: Elapsed time: 2606 millisec

Success.

--SynchronousRequest

--ChilkatLog

Answer

Here's a new build that should solve the problem:

http://www.chilkatsoft.com/preRelease/chilkat-9.4.1-x86-vc9.zip

However, technically the problem is with the web service. Let me explain:

HTTP requests and responses are MIME, which is a standard use extensively in Internet technologies. For example, all email's are also MIME. An HTTP client or server should be able to parse MIME according to all the standards that are commonly used. One such standard involves header fields that have name/value attributes, such as the charset in this case. The attribute value may be enclosed in double-quotes. This would make sense if the attribute value contains whitespace characters. Even if it doesn't, the MIME parser should be able to handle attribute name/value pairs with or without double-quotes.

Internal to Chilkat, decisions are made about whether or not to use double quotes around an attribute value. If the value contains whitespace chars, or any of a number of other special chars, then quotes are used. One such char is "-". This is why the quotes were used for "utf-8" -- because of the "-" character. There were notes in the internal Chilkat source indicating that some other MIME parsers, which could've been email or HTTP related, had trouble when double quotes were NOT used. Therefore, we now have a damned-if-we-do and damned-if-we-don't situation: One server requires the double-quotes, another requires NO double-quotes. What a complete F** mess. If only the servers were properly implemented, there would be no trouble.

Anyway, Chilkat works around it -- but I doubt Chilkat can keep contorting it's insides to make up for the piss-poor implementation of many of these server-side applications..


Answer

We suggest you may either

  1. When using CkHttp SetRequestHeader or CkHttpRequest AddHeader to manually add Content-Type header, just keep the header string as the programmer set. When using put_ContentType, you may automatically check it to add double quotes if necessary.

or

  1. Just add a boolean flag property to choose whether automatically add double quotes or just keep the header string as it is set by the programmer.

This shall provide better backward compatibility with reasonable flexibility. Since in some cases, developers may want to just set the header as it is originally set. If needing double quotes or even single quote, we can just set the header as we want.