Is HttpRequestMessage<T> a good thing?

Topics: Web Api
Jan 18, 2012 at 2:53 PM
Edited Jan 18, 2012 at 2:55 PM

I'm looking at how access to the request context is provided in WebApi and I do have an issue with it; I wondered what others thought.

Background:

It's possible to provide a parameter of type HttpRequestMessage OR to "wrap" a parameter in HttpRequestMessage<T> to get access to the request context in the body of your service implemenation. Similarly, for responses, you can either return a void or object which is turned into an HttpResponseMessage or HttpResponseMessage<T> respectively.

The features blurb highlights the provision this feature over something like WebOperationContext.Current for reasons of testability, and to extent I support this. I certainly agree that static functions are horrible for testability.

BUT.... :)

It does pollute your API. In our case this is particularly problematic because our current system generates documentation for its API by parsing the signatures. Now, it's not a great leap to modify the parsing to understand how to deal with HttpRequestMessage/<T> and HttpRequestResponse/<T> when it sees them, but it does make me think that something doesn't sit right about the approach.

For example, let's say sit down and design my API - like a good architect, I start with the interfaces with a view to handing them off to my developers for implementation.

I might have something like:

[ServiceContract]
public interface ICalculator
{
    int Add(int x, int y);
}

But I just don't know at this stage whether my developer is going to need access to the context. With WebApi as it stands, I'm encouraged to wrap my parameters  - but it's ambigous. I could do any of:

int Add(HttpRequestMessage requestMessage, int x, int y);

int Add(HttpRequestMessage requestMessage<int> x, int y);

int Add(x, HttpRequestMessage requestMessage<int> y);

On top of this, I could elect to change the return type to HttpResponseMessage<int>.

All of these are changing the C# interface - even though they don't change the interface of the web service consumer. From a process point of view, it makes it hard to understand whether or not you are making breaking Api changes and to enforce consistency.

I wonder if it would be better to inject an object into my service implementation that I can call to get access to the context like so (this is illustrative and won't compile):

public interface IWebOperationContextProvider
{
    WebOperationContext GetCurrent();
}

public class WebOperationContextProvider : IWebOperationContextProvider
{
    public WebOperationContext { get { return WebOperationContext.Current; } }
}

public class CalculatorImpl : ICalculator
{
   WebOperationContext OperationContext { get { return _contextProvider.GetCurrent(); } }

   public CalculatorImpl(IWebOperationContextProvider contextProvider)
   {
         _contextProvider = contextProvider;
   }
}

Neither is totally ideal, but at least the latter doesn't pollute my API. What do you think?

Jan 18, 2012 at 4:02 PM

I'm having the exact same issues with context to the current user.

I did the same thing. I made a base class that implements User { get; set; } that returns an IUser built off Thread.CurrentPrincipal.

Still working on how to test it, because I can't seem to get injection working well with StructureMap.

Coordinator
Jan 18, 2012 at 11:42 PM

As we integrate Web API with MVC one pattern we are looking at is using a property on a base class that you derive from to give you access to the HttpRequestMessage instance, so that you can write code like this:

public int Add(int x, int y) 
{
    var host = this.Request.Headers.Host;
}

The nice thing about this model is that you don’t have to change your signatures. You do have to derive from a base type though, similar to how you derive from Controller with MVC.

Daniel Roth

Jan 18, 2012 at 11:44 PM

Problem is this won't work for testing. I need injection so our unit tests can build out a fake Request object because there is no HTTP context.

Coordinator
Jan 19, 2012 at 12:36 AM

You can still do unit testing like this:

var mywebapi = new MyWebApi();

mywebapi.Request = new HttpRequestMessage(…) {…};

mywebapi.MyOp(…);

Daniel Roth

Jan 19, 2012 at 1:39 AM

Sounds good, I can live with property injection.

Coordinator
Jan 19, 2012 at 5:51 AM
Edited Jan 19, 2012 at 10:42 AM

<HTTP_RANT>

It really depends on what you are using HTTP for and how you design your apis. In your above example you are using HTTP in an RPC fashion to expose a calculator contract. You are taking your implementation and using HTTP as a transport protocol to expose that contract to be invoked. If you are trying to do  that, then I agree that HttpRequestMessage<T> / HttpResponseMessage<T> gets in the way and you may view it as a violation of concerns.

However.....

HTTP is not transport protocol[1], and it is not RPC[2].

I talked about this in more depth at my QCon London talk which you can access here. Mike Amundsen also has a great article on the problems of trying to use HTTP for other than a transfer protocol  here.

HTTP was designed to enable long lived evolvable systems that run for many years. It reduces coupling between clients by exposing resources in a variety of formats to a range of clients and using the uniform interface i.e. standard HTTP methods, headers, etc. It is an application layer protocol that offers a lot of rich information which clients, intermediaries and servers can use to process the message. From an HTTP perspective that uniform interface IS the contract. Because that contract is fixed and does not change, we get evolvability. New resources can be added without breaking backward compat. In your calculator example if the signature of the contract changes, all existing clients are broken. If however you stick strictly to the uniform interface that contract will never change.

Back to web apis, they are a bridge between the HTTP world and your business domain, they are not the business domain itself. At that bridge there are a whole set of HTTP concerns that need to get addressed like working with HTTP Status Codes, Headers like ETags, and dealing with the content. It is that philosophy that drove the design of HttpRequestMessage<T> / HttpResponseMessage<T>. At that boundary where you use them they are not a separate concern, they are a primary concern. 

If we go with the grain of HTTP then we expose our system via the uniform interface rather than bleeding our implementation. SOAP for example went against that grain and that is why we have SOAP Contracts, WSDL etc.

That means instead of exposing an ICalculator, exposing a calculator resource. That resource can expose sub-resources which allow access to different aspects of the application. For example doing a GET with form url encoding in the query string with this uri ".../calculator/adder?value1=5&value2=6" results in a response of 11. 

The API implementation

public class CalculatorApi {
  private ICalculator _calculator;
  public CalculatorApi(ICalculator calculator) {
    _calculator = calculator;
  }

  [WebGet(UriTemplate="adder?value1={value1}&value2={value2}")] 
  public HttpResponseMessage<AdderResult> GetAdder(double value1, double value2)
  {
    var result = new AdderResult();
    result.sum = _calculator.Add(value1, value2);
    var response = new HttpResponseMessage<AdderResult>(result);

    //set browsers to cache this forever as it will never change
    response.Headers.CacheControl = new CacheControlHeaderValue() { MaxAge = new TimeSpan(999999999, 0, 0, 0) };
    return response;
  }
}

public class AdderResult {
  public double sum {get;set;}
}

The implementation of the CalculatorApi.GetAdder method delegates to the calculator implementation to do the work. Now a calculator result never changes, so the Cache-Control header is set to a very large MaxAge thus telling clients they can cache it forever. HttpResponseMessage<T> works perfectly here to allow returning the Sum as well as setting the headers.

This is just one simple example, but a realistic use case where accessing the headers allows the server to give the client special instructions on how to handle the response.

One could go further and decide to embed link headers instructing the client about other resources they can access related to the Adder,  like Subtractor, Multiplier, etc. 

These concerns as well as many others are completely valid from an HTTP standpoint and do not pollute the API as long as the purpose of my API is to be that bridge. 

[1] - http://www.ics.uci.edu/~fielding/pubs/dissertation/evaluation.htm - 6.5.3

[2] - http://www.ics.uci.edu/~fielding/pubs/dissertation/evaluation.htm - 6.5.2

</HTTP_RANT>

Jan 19, 2012 at 6:16 AM

Nice explanation. :)

 

Coordinator
Jan 19, 2012 at 6:55 AM

Thanks :-) 

Jan 19, 2012 at 9:42 AM
Edited Jan 19, 2012 at 10:04 AM

Thanks Glen. I agree completely with what you are saying. In the real case rather than the simplified example I presented here, the concrete implementation is barely more than a protocol transition layer that passes requests through to ActiveMQ and on to the real business logic. I left that detail out of my sample code for simplicity; I agree you wouldn't want business logic in here.

In this case, I think the root of the problem is that I've inherited a system that generates documentation, jschema and SMD by parsing the XML compiled from the triple slash comments - which is tantamount to reflecting over the interfaces. I can either fix the documentation generator to account for HttpMessageRequest<T> and friends, or follow the wrap approach you suggested. In the later case this means duplicating all the interfaces just to get a handle on the HttpMessageRequest/Response, which seems painful in this case as I am effectively wrapping wrappers.

We also have to handle the possibility of flash/javascript clients running in browsers that intercept non-200 status codes - exactly as you say this is a transfer protocol consideration unrelated to the underlying api that needs to be explicitly handled at the HTTP level. [Practically speaking this can be handled in a WebApi error handler. We suppress non-200 status codes for clients that indicate that preference, much like the twitter API and others.]

I still think there is a problem with HttpRequestMessage coming in as a parameter to the method call and HttpResponseMessage being arbitrarily omittable. Particularly in light of what you have described, is it not hurting uniformity because I have so many different ways of describing my intent? As I initially described, I can write several different method signatures that are all effectively the same. I think there should be a consistent mechanism for accessing HttpRequest/Response - like the base class approach danroth suggested in this thread. In any case, shouldn't the WebApi mandate HttpResponseMessage as a return type given what you've said? Otherwise it's just encouraging bad behaviour. :)

Thanks again.

Coordinator
Jan 19, 2012 at 10:02 AM

Hi Whirly

Glad to help. I hear you on the challenges of an existing system, those constraints can be very real.

Another alternative approach that is possible today is to use operation handlers to handle HTTP concerns. This way your API method signatures don't have the HTTP messages and can implement the contract. As Dan mentioned, once the MVC integration comes you'll be able to use controllers which expose the messages as properties.

Cheers

Glenn

Jan 19, 2012 at 10:22 AM

Glenn, sorry for nitpicking, but wouldn't you expose the calculator as "sum" instead of "adder" since the sum is really the resource you expose?

Whirly, you can already abstract the HttpRequestMessage into a base class like they mention above, it's actually quite easy.  See my example here:
http://wcf.codeplex.com/discussions/242055#post723590

Coordinator
Jan 19, 2012 at 10:37 AM
Edited Jan 19, 2012 at 10:38 AM

I wouldn't, because Adder expresses intent on the role of the resource. It is an Adder. When I access the resource I am sending it data it can use (via the url) to perform the add.

That doesn't mean sum is wrong. I just preferred modeling the role of the resource rather than it's output.

For that matter you could do something like calculator/4,5/sum i.e. give me the sum resource that is a child of the 4,5 resource. That may look odd, but it is valid :-)

Jan 19, 2012 at 10:37 AM

@Glenn - The GetAdder method in your example takes in value1 and value2 but when you call the calculator's add method, you reference adder.value1 and adder.value2. Is this a mistake? Also where does HttpRequestMessage<T> fit in?

Coordinator
Jan 19, 2012 at 10:41 AM

That approach works as long as you are per-request and not singleton which hopefully you are not since singleton creates a ton of problems.

Coordinator
Jan 19, 2012 at 10:43 AM

@devtrends

Yes it was a mistake though I have now fixed it, thanks for pointing that out.

I don't show HttpRequestMessage<T> in that example, but I could for example looking at headers, etc. However using HttpResponseMessage<T> illustrates the same problem as it is part of the operation signature which is the issue that was raised.

Coordinator
Jan 19, 2012 at 10:44 AM
Edited Jan 19, 2012 at 10:45 AM

@Siggi, that example there would be a pretty nice blog post, or even contrib. In particular for cases where there are legacy constraints it would be useful.

Jan 19, 2012 at 10:47 AM

Very interesting, this expressing intent/roles of a resource is new to me.  I've always been thinking about modelling the result of an operation and calculator/4,5/sum is exactly the kind of URL I've been going with.  I would be worried that people would just wrap their RPC ways into "intents", like /TheMethodRunner?method=CalculateSum&value1=5&value2=42.

Can you recommend any reading to further explain this?

Coordinator
Jan 19, 2012 at 11:05 AM
Edited Jan 19, 2012 at 11:06 AM

The intent I am expressing is to the server not to the client, as ideally the uris would be opaque to the client. Meaning I am using resources that make sense for the domain of the application. Roy actually does recommend this if you read his thesis.

I understand your concern (your example) , and abuses like that can happen. However I am not doing that :-) I am creating a resource for a specific purpose and that resource does not violate the semantics of HTTP/REST.

Having a resource that performs something is fine as long as it obeys the semantics of HTTP. For example posting to Order/1/Approver could result in the order getting approved.

As far as reading, I'm not sure I have anything specific, but I'll hunt for you. What i can say is if you read Roy's dissertation it makes it very clear what the constraints are for REST and as long as you are not violating it you are in the clear.

Which constraint of REST does this violate? Well I know one :) which is cachability as some servers don't cache query strings so really it should be /calculator/adder/value1=5&value2=6 so that it is cachable.

Jan 19, 2012 at 11:33 AM

Agree with you on cachability, I generally only use query params for filtering or input into an algorithm like search (an example that would be hard to model as only "results").

In your order example if you would post to Order/1/Approver to change an order, that wouldn't use the proxy cache invalidation that can happen if you would instead do a PUT on Order/1/ with the approved bit set to true.

Jan 19, 2012 at 12:34 PM

Going back to the original theme of the post, I have found the biggest hurdle to Web API adoption is an understanding that the API layer is just that - another layer.  As Glenn said, the responsibility of that layer is to provide the API that consumers can access via HTTP.  The layer delegates to the domain layer after handling all of the HTTP concerns and does the same on the way out.  I actually had to build an application without the API layer (using unit tests) to drive this point home.  Once they saw the application was fully functional, then put the API layer on top to expose it, I could see they all got it.

I actually love the idea of a base class that exposes Request and Response properties instead of putting HttpRequestMessage/HttpResponseMessage in the signature.  However, this sounds like we are simply going back to implementing our services (API methods) as MVC actions.  What about those of us that host in a Windows Service and/or don't want the overhead of MVC? Since so much of MVC, like routing, has been moved lower into the stack, will we really be dependent upon MVC or simply leverage the same features from base class assemblies?

I do hope that there is a lot of documentation provided around these changes as they sound pretty extensive - especially for those of us that have really embraced the Web API as it is today (such as having Message, Request and Response handlers, which I understand is also changing!).

 

Coordinator
Jan 19, 2012 at 5:06 PM

You will still be able to self-host in a Windows Service or your own process. In fact, the new stack is at its core a simple message handler pipeline that you can host wherever you want. Parts of MVC that are not relevant to Web API development (like views for example) are not pulled in, so what you end up with is a light-weight rehostable HTTP server. With the upcoming Beta we will have a complete set of docs and samples to help get you going on the new bits.

Daniel Roth

From: SonOfPirate [email removed]
Sent: Thursday, January 19, 2012 5:35 AM
To: Daniel Roth
Subject: Re: Is HttpRequestMessage<T> a good thing? [wcf:286557]

From: SonOfPirate

Going back to the original theme of the post, I have found the biggest hurdle to Web API adoption is an understanding that the API layer is just that - another layer. As Glenn said, the responsibility of that layer is to provide the API that consumers can access via HTTP. The layer delegates to the domain layer after handling all of the HTTP concerns and does the same on the way out. I actually had to build an application without the API layer (using unit tests) to drive this point home. Once they saw the application was fully functional, then put the API layer on top to expose it, I could see they all got it.

I actually love the idea of a base class that exposes Request and Response properties instead of putting HttpRequestMessage/HttpResponseMessage in the signature. However, this sounds like we are simply going back to implementing our services (API methods) as MVC actions. What about those of us that host in a Windows Service and/or don't want the overhead of MVC? Since so much of MVC, like routing, has been moved lower into the stack, will we really be dependent upon MVC or simply leverage the same features from base class assemblies?

I do hope that there is a lot of documentation provided around these changes as they sound pretty extensive - especially for those of us that have really embraced the Web API as it is today (such as having Message, Request and Response handlers, which I understand is also changing!).

Jan 20, 2012 at 1:27 PM
gblock wrote:
Which constraint of REST does this violate? Well I know one :) which is cachability as some servers don't cache query strings so really it should be /calculator/adder/value1=5&value2=6 so that it is cachable.

Ouch!  I did not know that!

Am assessing the implications of this.... one of which is that my "?format=xml" vs "?format=json" has gone out the window, and I'll go back to the Accept header.

A question: if values for the OData system query options always give deterministic, reliable results (when used to query a service's collections) then presumably, wrt caching at least, they shouldn't be query strings?!

Jan 20, 2012 at 2:29 PM

Andrew how about using ".xml" and ".json"?

Jan 20, 2012 at 2:57 PM

Hi SiggiG, thanks for the suggestion, but I've implement the Accept header now, via a MediaTypeMapping and it's working well.

My idea re. "?format=xml | json" was that a client would specify this in the entry-point / bookmark URI, and not have to do anything else after that.  My JSON representations would include outgoing URIs with "?format=json" and XML representations would have "?format=xml".  Less work for the client.  But Glenn's remark was an eye opener... potentially every resource representation would be uncached!

Now my clients can do nothing if they want XML, and set the Accept header on every request if they want JSON (may switch that around, not sure).

Reading around the subject, the conclusion is unescapable: the Accept header is the right thing to use.  I believe Mark Masse's REST book argues against ".xml" and ".json" (or I may have seen it elsewhere).

I had never seen Glenn's technique of setting property values in the path and not the query string (/calculator/adder/value1=5&value2=6) but if the result is always going to be the same, and query strings bust cache, then it makes sense.  Hence my question about OData system query options: if they always give the same answers in your service, better to have them in the path and not as query strings, no? (from the pov of caching).  Is that right, or have I missed something?

 

Jan 20, 2012 at 3:09 PM

I think most people prefer to use the query string for filtering and, um, queries :) I use it for paging and filtering, but don't really have any queries.

Good to see you coming around to the Accept header, as per another discussion we've had ;)

Jan 20, 2012 at 3:27 PM

>> I think most people prefer to use the query string for filtering and, um, queries

Yeah, but if "?$filter=PortfolioName eq 'FTSE100'" always gave the same results, couldn't / shouldn't that result be cached?  That's what I'm getting at.  But I probably will stick with having them as query strings.

 

>> Good to see you coming around to the Accept header, as per another discussion we've had ;)

Heh - you were right on that one!  :)

Here's a question: if you have a number of different media types of the form "application/vnd.mycompany.myapp.XYZ+json" where XYZ varies, do you 'accept' the more generic "application/vnd.mycompany.myapp+json" (or even "application/json") to match all of them?

Jan 20, 2012 at 3:33 PM

I think we're starting to go a "bit" off topic, but here goes :)

For my project I accept application/json, */* or even an empty header, and I chose the +json formatter to be used as default in those cases.  This is more to cater to clients that haven't bothered to include a specific media type (and they will probably break when I have to offer v2 of the resource).
However if you do specify a certain media type I require you to specify the full name with a version.

Jan 20, 2012 at 3:58 PM

Thanks Siggi!  No more off-topic questions.  Have a good w/e.