Enterprise Integration Zone is brought to you in partnership with:

Ted Neward is the Principal at Neward & Associates, a developer services company. He consults, mentors, writes and speaks worldwide on a variety of subjects, including Java, .NET, XML services, programming languages, and virtual machine/execution engine environments. He resides in the Pacific Northwest. Ted is a DZone MVB and is not an employee of DZone and has posted 50 posts at DZone. You can read more from them at their website. View Full User Profile

REST != HTTP

11.07.2008
| 27459 views |
  • submit to reddit

Roy Fielding has weighed in on the recent "buzzwordiness" (hey, if Colbert can make up "truthiness", then I can make up "buzzwordiness") of calling everything a "REST API", a tactic that has become more en vogue of late as vendors discover that the general programming population is finding the WSDL-based XML services stack too complex to navigate successfully for all but the simplest of projects. Contrary to what many RESTafarians may be hoping, Roy doesn't gather all these wayward children to his breast and praise their anti-vendor/anti-corporate/anti-proprietary efforts, but instead, blasts them pretty seriously for mangling his term:

I am getting frustrated by the number of people calling any HTTP-based interface a REST API. Today’s example is the SocialSite REST API. That is RPC. It screams RPC. There is so much coupling on display that it should be given an X rating.

Ouch. "So much coupling on display that it should be given an X rating." I have to remember that phrase--that's a keeper. And I'm shocked that Roy even knows what an X rating is; he's such a mellow guy with such an innocent-looking face, I would've bet money he'd never run into one before. (Yes, people, that's a joke.)

What needs to be done to make the REST architectural style clear on the notion that hypertext is a constraint? In other words, if the engine of application state (and hence the API) is not being driven by hypertext, then it cannot be RESTful and cannot be a REST API. Period. Is there some broken manual somewhere that needs to be fixed?

Go Roy!

For those of you who've not read Roy's thesis, and are thinking that this is some kind of betrayal or trick, let's first of all point out that at no point is Roy saying that your nifty HTTP-based API is not useful or simple. He's simply saying that it isn't RESTful. That's a key differentiation. REST has a specific set of goals and constraints it was trying to meet, and as such prescribes a particular kind of architectural style to fit within those constraints. (Yes, REST is essentially an architectural pattern: a solution to a problem within a certain context that yields certain consequences.)

Assuming you haven't tuned me out completely already, allow me to elucidate. In Chapter 5 of Roy's thesis, Roy begins to build up the style that will ultimately be considered REST. I'm not going to quote each and every step here--that's what the hyperlink above is for--but simply call out certain parts. For example, in section 5.1.3, "Stateless", he suggests that this architectural style should be stateless in nature, and explains why; the emphasis/italics are mine:

We next add a constraint to the client-server interaction: communication must be stateless in nature, as in the client-stateless-server (CSS) style of Section 3.4.3 (Figure 5-3), such that each request from client to server must contain all of the information necessary to understand the request, and cannot take advantage of any stored context on the server. Session state is therefore kept entirely on the client.

This constraint induces the properties of visibility, reliability, and scalability. Visibility is improved because a monitoring system does not have to look beyond a single request datum in order to determine the full nature of the request. Reliability is improved because it eases the task of recovering from partial failures [133]. Scalability is improved because not having to store state between requests allows the server component to quickly free resources, and further simplifies implementation because the server doesn't have to manage resource usage across requests.

Like most architectural choices, the stateless constraint reflects a design trade-off. The disadvantage is that it may decrease network performance by increasing the repetitive data (per-interaction overhead) sent in a series of requests, since that data cannot be left on the server in a shared context. In addition, placing the application state on the client-side reduces the server's control over consistent application behavior, since the application becomes dependent on the correct implementation of semantics across multiple client versions.

In the HTTP case, the state is contained entirely in the document itself, the hypertext. This has a couple of implications for those of us building "distributed applications", such as the very real consideration that there's a lot of state we don't necessarily want to be sending back to the client, such as voluminous information (the user's e-commerce shopping cart contents) or sensitive information (the user's credentials or single-signon authentication/authorization token). This is a bitter pill to swallow for the application development world, because much of the applications we develop have some pretty hefty notions of server-based state management that we want or need to preserve, either for legacy support reasons, for legitimate concerns (network bandwidth or security), or just for ease-of-understanding. Fielding isn't apologetic about it, though--look at the third paragraph above. "[T]he stateless constraint reflects a design trade-off."

In other words, if you don't like it, fine, don't follow it, but understand that if you're not leaving all the application state on the client, you're not doing REST.

By the way, note that technically, HTTP is not tied to HTML, since the document sent back and forth could easily be a PDF document, too, particularly since PDF supports hyperlinks to other PDF documents. Nowhere in the thesis do we see the idea that it has to be HTML flying back and forth.

Roy's thesis continues on in the same vein; in section 5.1.4 he describes how "client-cache-stateless-server" provides some additional reliability and performance, but only if the data in the cache is consistent and not stale, which was fine for static documents, but not for dynamic content such as image maps. Extensions were necessary in order to accomodate the new ideas.

In section 5.1.5 ("Uniform Interface") we get to another stinging rebuke of REST as a generalized distributed application scheme; again, the emphasis is mine:

The central feature that distinguishes the REST architectural style from other network-based styles is its emphasis on a uniform interface between components (Figure 5-6). By applying the software engineering principle of generality to the component interface, the overall system architecture is simplified and the visibility of interactions is improved. Implementations are decoupled from the services they provide, which encourages independent evolvability. The trade-off, though, is that a uniform interface degrades efficiency, since information is transferred in a standardized form rather than one which is specific to an application's needs. The REST interface is designed to be efficient for large-grain hypermedia data transfer, optimizing for the common case of the Web, but resulting in an interface that is not optimal for other forms of architectural interaction.

In order to obtain a uniform interface, multiple architectural constraints are needed to guide the behavior of components. REST is defined by four interface constraints: identification of resources; manipulation of resources through representations; self-descriptive messages; and, hypermedia as the engine of application state. These constraints will be discussed in Section 5.2.

In other words, in order to be doing something that Fielding considers RESTful, you have to be using hypermedia (that is to say, hypertext documents of some form) as the core of your application state. It might seem like this implies that you have to be building a Web application in order to be considered building something RESTful, so therefore all Web apps are RESTful by nature, but pay close attention to the wording: hypermedia must be the core of your application state. The way most Web apps are built today, HTML is clearly not the core of the state, but merely a way to render it. This is the accidental consequence of treating Web applications and desktop client applications as just pale reflections of one another.

The next section, 5.1.6 ("Layered System") again builds on the notion of stateless-server architecture to provide additional flexibility and power:

In order to further improve behavior for Internet-scale requirements, we add layered system constraints (Figure 5-7). As described in Section 3.4.2, the layered system style allows an architecture to be composed of hierarchical layers by constraining component behavior such that each component cannot "see" beyond the immediate layer with which they are interacting. By restricting knowledge of the system to a single layer, we place a bound on the overall system complexity and promote substrate independence. Layers can be used to encapsulate legacy services and to protect new services from legacy clients, simplifying components by moving infrequently used functionality to a shared intermediary. Intermediaries can also be used to improve system scalability by enabling load balancing of services across multiple networks and processors.

The primary disadvantage of layered systems is that they add overhead and latency to the processing of data, reducing user-perceived performance [32]. For a network-based system that supports cache constraints, this can be offset by the benefits of shared caching at intermediaries. Placing shared caches at the boundaries of an organizational domain can result in significant performance benefits [136]. Such layers also allow security policies to be enforced on data crossing the organizational boundary, as is required by firewalls [79].

The combination of layered system and uniform interface constraints induces architectural properties similar to those of the uniform pipe-and-filter style (Section 3.2.2). Although REST interaction is two-way, the large-grain data flows of hypermedia interaction can each be processed like a data-flow network, with filter components selectively applied to the data stream in order to transform the content as it passes [26]. Within REST, intermediary components can actively transform the content of messages because the messages are self-descriptive and their semantics are visible to intermediaries.

The potential of layered systems (itself not something that people building RESTful approaches seem to think much about) is only realized if the entirety of the state being transferred is self-descriptive and visible to the intermediaries--in other words, intermediaries can only be helpful and/or non-performance-inhibitive if they have free reign to make decisions based on the state they see being transferred. If something isn't present in the state being transferred, usually because there is server-side state being maintained, then they have to be concerned about silently changing the semantics of what is happening in the interaction, and intermediaries--and layers as a whole--become a liability. (Which is probably why so few systems seem to do it.)

And if the notion of visible, transported state is not yet made clear in his dissertation, Fielding dissects the discussion even further in section 5.2.1, "Data Elements". It's too long to reprint here in its entirety, and frankly, reading the whole thing is necessary to see the point of hypermedia and its place in the whole system. (The same could be said of the entire chapter, in fact.) But it's pretty clear, once you read the dissertation, that hypermedia/hypertext is a core, critical piece to the whole REST construction. Clients are expected, in a RESTful system, to have no preconceived notions of structure or relationship between resources, and discover all of that through the state of the hypertext documents that are sent back to them. In the HTML case, that discovery occurs inside the human brain; in the SOA/services case, that discovery is much harder to define and describe. RDF and Semantic Web ideas may be of some help here, but JSON can't, and simple XML can't, unless the client has some preconceived notion of what the XML structure looks like, which violates Fielding's rules:

A REST API should be entered with no prior knowledge beyond the initial URI (bookmark) and set of standardized media types that are appropriate for the intended audience (i.e., expected to be understood by any client that might use the API). From that point on, all application state transitions must be driven by client selection of server-provided choices that are present in the received representations or implied by the user’s manipulation of those representations. The transitions may be determined (or limited by) the client’s knowledge of media types and resource communication mechanisms, both of which may be improved on-the-fly (e.g., code-on-demand). [Failure here implies that out-of-band information is driving interaction instead of hypertext.]

An interesting "fuzzy gray area" here is whether or not the client's knowledge of a variant or schematic structure of XML could be considered to be a "standardized media type", but I'm willing to bet that Fielding will argue against it on the grounds that your application's XML schema is not "standardized" (unless, of course, it is, through a national/international/industry standardization effort).

But in case you'd missed it, let me summarize the past twenty or so paragraphs: hypermedia is a core requirement to being RESTful. If you ain't slinging all of your application state back and forth in hypertext, you ain't REST. Period. Fielding said it, he defined it, and that settles it.

 

Before the hate mail comes a-flyin', let me reiterate one vitally important point: if you're not doing REST, it doesn't mean that your API sucks. Fielding may have his definition of what REST is, and the idealist in me wants to remain true to his definitions of it (after all, if we can't agree on a common set of definitions, a common lexicon, then we can't really make much progress as an industry), but...

... the pragmatist in me keeps saying, "so what"?

Look, at the end of the day, if your system wants to misuse HTTP, abuse HTML, and carnally violate the principles of loose coupling and resource representation that underlie REST, who cares? Do you get special bonus points from the Apache Foundation if you use HTTP in the way Fielding intended? Will Microsoft and Oracle and Sun and IBM offer you discounts on your next software purchases if you create a REST-faithful system? Will the partisan politics in Washington, or the tribal conflicts in the Middle East, or even the widely-misnamed "REST-vs-SOAP" debates come to an end if you only figure out a way to make hypermedia the core engine of your application state?

Yeah, I didn't think so, either.

Point is, REST is just an architectural style. It is nothing more than another entry alongside such things as client-server, n-tier, distributed objects, service-oriented, and embedded systems. REST is just a tool for thinking about how to build an application, and it's high time we kick it off the pedastal on which we've placed it and let it come back down to earth with the rest of us mortals. HTTP is useful, but not sufficient, so solve our problems. REST is as well.

And at the end of the day, when we put one tool from our tool belt "above all others", we end up building some truly horrendous crap.

Published at DZone with permission of Ted Neward, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Tags:

Comments

Francisco Peredo replied on Fri, 2008/11/07 - 8:34pm

The thing I do not get is why nobody realizes that Rest Is Just Sql Reinvented,

Take a look at this definitions:

    * The REST hypothesis is that the semantics of HTTP constitutes a coordination language which is is sufficiently general and complete to encompass any desired computational communication pattern. I.e., the hypothesis is that GET / PUT / POST / etc. can provide all coordination semantics between discretely modeled units of programs

    * The SQL hypothesis is that the semantics of RelationalAlgebra constitute a language which is is sufficiently general and complete to encompass any desired computational communication pattern. I.e., the hypothesis is that INSERT / SELECT/ UPDATE / DELETE. can provide all coordination semantics between discretely modeled units of programs

Is the similarity just a coincidence?, Read more here.

 Why is people so exited about REST if SQL has been able to do everything REST does now, but more than 10 years ago? I just do not get it...

Jochen Bedersdorfer replied on Fri, 2008/11/07 - 9:43pm in response to: Francisco Peredo

Well, SQL is not serving hypertexts last time I looked at it. ;o)

But more important (and related): There is no equivalent to the URL!

 

aaaaa aaaaaaaa replied on Sat, 2008/11/08 - 8:36am

thanks, this is a clear explanation of the REST pattern!

Solomon Duskis replied on Sat, 2008/11/08 - 6:46pm in response to: Jochen Bedersdorfer

 Well, SQL is not serving hypertexts last time I looked at it. ;o)

But more important (and related): There is no equivalent to the URL!

Playing devil's advocate, relational databass do indeed have an equivalent to hypertext: foreign keys.

Henry Story replied on Tue, 2008/11/11 - 4:59am

yes, there is a very close similarity with SQL, and of course some serious innovations. The innovations is of course what is important, otherwise you could just say: what is the point of SQL is that not just so much like say paper card indexing, or COBOL?

 The big difference is of course the URL. That is you have a system of distributed data. In a Relational Database (RDB) the data can only be contained and controlled by one organisation, or when multiple organisations are involved a lot of interaction overhead is required. RDF makes it possible to link data in a distributed way across the internet.  It also has a query language that is very close to SQL: SPARQL. So yes you are right in many many ways this is very similar to Relational Database. The difference is that what is being worked on is a global, distributed, one with no central point of control.

Now of course you can break REST principles. And a huge number of applications do. They may indeed have many other benefits, such as tying people into your system. But what they don't get are the benefits of REST, which of course is not minor. It is related to metcalf's law, and the benefits one gets by participating in a huge system whose value grows exponentially.

 So what readers of this excellent article should be wondering about, is what are the benefits of REST that they may be missing by not following its principles?

Mike Kelly replied on Tue, 2008/11/11 - 10:53am

REST is not 'SQL reinvented' - that is utter nonsense. Actually reading the dissertation might help you 'get why nobody has realized this yet'. It is fair to say that the 2 share some similarities, but they are complementary to one another - I fail to see how this is a criticism of any kind, quite the opposite.

Mr Neward, I completely disagree with the final part of your article.

REST could be viewed as just a means to an ends from a technical perspective, but practically speaking - the benefits to business of the RESTful approach outweigh those of any other alternatives, in most cases. Better technical interoperability begets more business interoperation, and this can only be a good thing for SOA.

 Why is it wrong to suggest that the 'perfect' SOA adheres to REST principals, particularly when all the evidence seems to support that suggestion?

 It is, to some extent, worth making the point that it is not always necessary or appropriate to follow every one of these principals, but also to make the point that careful consideration should be given in making that kind of decision because the opportunity cost of doing so may be large.

Perhaps you could give an example of where RESTful principals require breaking?

Tony Morris replied on Tue, 2008/11/11 - 11:03pm

"The thing I do not get is why nobody realizes that Rest Is Just Sql Reinvented,"

luxspes, REST is a non-concept. It does not exist in any logically consistent form. It is important to the survival of non-concepts  that they be ill-defined and masquerade otherwise. They often perpetuate by repetition of myth with peer reinforcement and pseudo-science. You will no doubt attract the wrath of a cult (or member of) within the REST proponents who will accuse you of making a misrepresentation of REST with the presupposition of your statement. Indeed, cults within the cult war with each other over this definition. Some call it religion. These people are not malicious, but ignorant and scientifically illiterate. You should understand this to resolve your dilemma.

 

If you can understand why non-concepts exist and how they survive, you will have no problem understanding why nobody realizes...

Mike Kelly replied on Wed, 2008/11/12 - 7:41am in response to: Tony Morris

pointless points are one to look out for aswell - defensive garbage which contains lots of blind flailing and finger pointing.

 Apparently sharing passion and conviction for something makes you part of a cult, does that mean SOA is a cult? I'd say yes, according to Tony's ridiculous definition.

So assuming that baseless and pointless accusation it true, does it actually negate the value of the arguments made in favour of SOA and REST? Clearly not,

 It's worth considering how much collusion and misrepresentation is brought to the table by individuals and groups who have invested interests in keeping development methodologies overly complex and drawn out - money, money, money; you have to keep up cash flow and barriers to entry.

Tony Morris replied on Wed, 2008/11/12 - 5:05pm

Hi MikeK, No SOA is not a cult and not under my given definition. A cult is made up of members who propagate a non-idea. SOA is a non-idea. It is given unwarranted legitimacy, since it also (like REST) does not exist in a logically consistent form. I've no doubt it looks differently to an observer who lacks rigour in their formal reasoning, but this is aside and one that is uninteresting to argue (it often degenerates quickly yielding no useful conclusion).

 

It is possible, though I am not necessarily suggesting it, that proponents of SOA may form a cult, but SOA is itself not a cult. As a non-existent concept, it certainly exhibits all the required traits for it to follow.

Mike Kelly replied on Thu, 2008/11/13 - 10:01am

No offence, but you're coming across a tad pretentious.

I don't really understand the point you're trying to make..

Francisco Peredo replied on Fri, 2008/11/14 - 7:23am

Well in a way we could say that RDMBs serve HyperText: think about it, most non trivial web applications store pretty much all its information in RDBMS, so just because a WebServer transforms it in to XML it suddenly becomes such a different buzword compliant thing? Take a look at Microsoft Astoria, what else its but a thin layer on top of a RDMBS that translates urls in to SQL queries?

 And the URL... the URL is just there to name a resource, if you have an url like:

 http://company/payrollsystem/employees?salary=5000

 what is the advantage of that vs:

 select * from company.payrollsystem.employees as e where e.salary=5000?

 they both do the job: specify that we want, all the employees with salary of 5000. But what if we want to do a Join? (or many Joins, with Unions, and Exists and other relational stuff) if we use the SQL query we are fine (I would prefer to use a D but that is another story), and if we use an URL? well, then we are in trouble, because AFAIK there is no standard way to deal with this things in URLs. Is SPARQL for that? If that so it is so sad they have chosen a syntax so terribly flawed.

And about REST being for "global, distributed, one with no central point of control [stuff]" that idea certainly looks good in paper, but think about it,if in practical terms SPARQL is using HyperText datasources, that means they do not come indexed, or even in a nice binary optimized format, they are just a huge XML/XHTML file, if we currently have performance issue when we link an Oracle database with an Ms-SqlServer database and try to do a join between a table in one of them an a table if another (oh, and please explain me why that is not global, distributed, with no one central point of control stuff)... Just thinking about the performance hit of a Join in  SPARQL makes me think that something is really flawed with that approach...

 REST and SOA are like SQL expression and Stored procedure, REST is SQL for XML/Http, and SOA/RPC is stored procedures for XML/Http, but in the end, they are the same thing, they are just new clothes for the old emperor. They are a new, perhaps nice layer on top of already existing technology,but, somehow manage to attract all the hype, and in the process they even find time to say that their way of working is so revolutionary that it makes old technology obsolete, when in reality they are just thin extensions on top of that "old technology" they claim to replace. I guess that is why some times those of us that do not fully get what is so great about REST feel like a REST proponents can seem like...well, fanatics (members of a cult)...

 

Henry Story replied on Fri, 2008/11/14 - 8:31am

REST versus SQL? These debates clearly show that what is needed is not more dicsussion but some exploring of real RESTful services that play in the linked data world. Try out the Open Calais service from Reuters, the global news agency.

Mike Kelly replied on Fri, 2008/11/14 - 9:06am in response to: Francisco Peredo

"REST is SQL for XML/Http" - rubbish. Sorry, but that's just plain wrong. For a start, what does XML have to do with anything?

Your claim that it is merely 'new clothes for the old emperor' is based on a lack of understanding at your end. RESTful applications completely change the significance of a URI - In that, unlike RPC, a URI is used to... indicate a resource.. Uniform Resource Indicator.. does that make any sense to you? No? Oh well then, nevermind - you must be right.

This is fundamental change to the way in which a UA interacts with your application; namely that it actually makes proper use of URIs (and the HTTP protocol) instead of just using HTTP as a convenient transport for tunnelling through firewalls. HTTP is not a transport protocol, it's a transfer protocol.

In all seriousness, how much do you actually know about REST?

Francisco Peredo replied on Fri, 2008/11/14 - 9:32am

Maybe I didn't explaing myself I am not saying REST is wrong, I am just saying that this idea of using GET / PUT / POST / it is just too similar to what SQL provides, we could say that GET is SELECT, PUT is INSERT, POST is UPDATE... yes I know they are not exactly the same but they have the same philosophy, why this similarity does not seem to have (buzzword) name? or do you know the name for it?

Also can you explain me why this:

http://company/payrollsystem/employees?salary=5000

 is (or is not) so much better than:

 select * from company.payrollsystem.employees as e where e.salary=5000?

 or why the most exciting thing about something like:

 select * from http://company/payrollsystem.employees as e where e.salary=5000?

it is that it is using an uri and not that relational algebra is still the most useful way to manipulate data?

Mike Kelly replied on Fri, 2008/11/14 - 11:10am in response to: Francisco Peredo

You're talking about HTTP specifically, not REST. All you're doing is giving one example where a URI is constructed and reflects a SQL query, why does that mean that every URI is representative in this way? That's an assumption on your part.

The significance of the HTTP verbs is to do with safety and idempotency, rather than 'CRUD':

GET = safe and idempotent

DELETE = not safe + idempotent

PUT = not safe + idempotent

POST = not safe + not idempotent

There's plenty of reading material available for you - Fielding's dissertation is a good start (since that's where REST came from).

Francisco Peredo replied on Fri, 2008/11/14 - 10:42pm

Of course I am giving an example where a URI is constructed and reflects a SQL query, that is want URIs do, they indicate a resource.. URI is Uniform Resource Indicator, it is there to indicate you where is a resource... no? and that is what SQL queries do too, don't they, you write an URI and you get the resource it is pointing to, you write an SQL query and you get the resource it is pointing to. So, why we do not write SQL query like stuff in browsers? Plain luck, plain bad luck, because the guy that invented URIs didnt remember that SQL only needed an small extension to do what URIs do.

If the significance of the HTTP verbs is to do with safety and idempotency then it would I be right to say that:

Safe means a request doesn’t cause any side effects. A safe request just grabs data from a database and display it. Static pages, browsing source code, reading your email online — these are all “safe” requests

Idempotent means that doing the request 10 times has the same effect as doing it once. An idempotent request might create something in a database the first time, but it won’t do it again.

Then:

SELECT: safe and idempotent

DELETE = not safe + idempotent

UPDATE = not safe + not idempotent

INSERT = not safe + not idempotent

So we could say that SQL is missing an operation that is not safe + idempotent, a kind of UPSERT or MERGE operation that can be called many times without risk, if we add that, then SQL becomes RESTful?and if it does not, what is the name of SQL+UPSERT?

 

 

 

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.