HttpClient.Get and URL Encoded RequestUri Bug?

Topics: Web Api
Apr 21, 2011 at 8:02 PM
Edited Apr 21, 2011 at 8:03 PM

Hey guys,

It appears to me that HttpClient is not correctly URL encoding the request URI right before getting on the wire.  Internally, the use of System.Uri is causing any pre-encoded URIs to be decoded (which is fine).  I'm having to double encode my querystring content to work around this problem.

E.g., if you have a service with the following signature:

	[WebGet(UriTemplate = "{nameIdentifier}")]
	public string GetUser(string nameIdentifier)

and you set up a service route like:

	routes.MapServiceRoute<UsersService>("users");

and you write code like this:

	using (var httpClient = new HttpClient("http://localhost/"))
	{
		using (HttpResponseMessage httpResponseMessage = httpClient.Get(String.Format("users/{0}", HttpUtility.UrlEncode("this/does/not/work"))))
		{
			return httpResponseMessage.Content.ReadAsString();
		}
	}

you end up sending the request like (which obviously returns a 404):

	http://localhost/users/this/does/not/work

instead of like:

	http://localhost/users/this%2fdoes%2fnot%2fwork

Is this a bug or do I need to resort to using actual query string parameters?

Apr 21, 2011 at 9:34 PM
Edited Apr 21, 2011 at 9:36 PM

I believe the problem is with the System.Uri class.  Unfortunately, the following test passes:

 

        [TestMethod]
        public void TestUrl() {
            var url = new Uri("http://localhost:8000/" + HttpUtility.UrlEncode("this/does/not/work"));
            Assert.AreEqual(url.AbsoluteUri, "http://localhost:8000/this/does/not/work");
        }

 

I'm not sure if there is any way to change this behaviour.

Using the query string it does work,

	[TestMethod]
        public void TestUrlUsingEscapedQueryString() {
            var url = new Uri("http://localhost:8000/foo?" + HttpUtility.UrlEncode("this/does/not/work"));
            Assert.AreEqual(url.AbsoluteUri, "http://localhost:8000/foo?this%2fdoes%2fnot%2fwork");
        }

Apr 21, 2011 at 9:40 PM
DarrelMiller wrote:

I believe the problem is with the System.Uri class.  Unfortunately, the following test passes:

 

        [TestMethod]
        public void TestUrl() {
            var url = new Uri("http://localhost:8000/" + HttpUtility.UrlEncode("this/does/not/work"));
            Assert.AreEqual(url.AbsoluteUri, "http://localhost:8000/this/does/not/work");
        }

 

I'm not sure if there is any way to change this behaviour.


I agree that System.Uri is the source of the "problem".  However, I don't think that System.Uri is broken nor does Microsoft.  I'm not really sure that this problem can be reliably be solved with the current API that HttpClient exposes.  As I mentioned, I can work around this by double URL encoding but that just feels dirty.  I guess the real issue is that "pretty URLs" are at odds with the HTTP specification and in this case are not a viable option.

Apr 21, 2011 at 9:46 PM
DarrelMiller wrote:

Using the query string it does work,

 

	[TestMethod]
        public void TestUrlUsingEscapedQueryString() {
            var url = new Uri("http://localhost:8000/foo?" + HttpUtility.UrlEncode("this/does/not/work"));
            Assert.AreEqual(url.AbsoluteUri, "http://localhost:8000/foo?this%2fdoes%2fnot%2fwork");
        }

 


Right, as expected.  Would it be possible to create a Get overload that accepts an HttpContent like Post (but obviously with HttpMethod.Get) so that FormUrlEncodedContent could be leveraged?

Apr 22, 2011 at 1:07 PM
davidpeden3 wrote:
I agree that System.Uri is the source of the "problem".  However, I don't think that System.Uri is broken nor does Microsoft.  I'm not really sure that this problem can be reliably be solved with the current API that HttpClient exposes.  As I mentioned, I can work around this by double URL encoding but that just feels dirty.  I guess the real issue is that "pretty URLs" are at odds with the HTTP specification and in this case are not a viable option.

That Connect bug report you linked to was giving an example of percent-encoding in the query string.  That is allowed by System.URI and does work.  It's in the path segment where it auto decodes incorrectly.  As per the the URI spec RFC3986, the path segments are supposed to support percent-encoding.

Interestingly, Microsoft have created a workaround for the issue on the server side when using HttpListener (see http://msdn.microsoft.com/en-us/library/system.net.configuration.httplistenerelement.unescaperequesturl.aspx)  However, I'm not sure how you would be able to call this from .Net (unless you double encode) as I did the most basic test I could with HttpWebRequest and it still decodes the URI path before sending it over the wire.