We read this article recently:

Curtis has been tremendously influential in the software process improvement field, and this Curtis paper was his most highly cited: Google Scholar lists 763 papers which cite it. Of those papers which cite this one, more than fifty of them have been cited more than 50 times, themselves. Of his other papers, 11 that I can find have been cited more than 50 times and six more than 150 times (again according to Google Scholar). Those papers were all written between 1979 and 1995, in the golden years of software methodology resarch.

Other parts of Curtis' work is familiar to us, even if we didn't know he was involved. He is a former director of the Software Process Program at the Software Engineering Institute (SEI) at Carnegie Mellon University, the creators of CMM and CMMI. He co-authored the Capability Maturity Model for Software, and was the chief architect of the People Capability Maturity Model (PCMM). PCMM reminds me a lot of a practitioner's approach to achieving a learning organziation.

PCMM reflects what he learned in this paper (as far as I can tell from the summaries I've found):

• The People CMM systematically infuses the required competencies, leading to empowerment. At level 3, the manager begins to trust the process and steps a little back. At level 4, the manager takes advantage of the frame-work installed till level 3, and steps further back. At level 5, interactions of workgroups with those in other units are aligned. Performance is aligned to optimize the system. The model helps systematically manage performance. “The issue is not evaluating individuals. But to bring in continuing discussion of how work is performed.” [1]

Currently, he a founding partner of Capability Measurement (couldn't find anything out about this company) and was until recently the chief process officer of Borland Software Corporation (he remains their chief process advisor). He was the cofounder and chief scientist of TeraQuest Metrics, Inc (acquired in 2005 by Borland), which helped large companies manage and take control of their software development processes [2].  

[More]

Summary of: 

Graves, S; Ramachandran, R; Keiser, K; Maskey, M; Lynnes, C; "Deployable Suite of Data Mining Web Services for Online Science Data Repositories", 23rd Conference on IIPS, (2007).
 

Background 

I read this article because I wanted to find a example of someone who wanted to use web services for something that was (a) specific and real (b) compelling (to me), and (c) includes of legacy systems and data stores. Most of my reading about web services has been generic, hypothetical, or simply uninteresting (for example, "retrieve purchase orders", or "process inventory". Yawn.).  

Summary

NASA is focused on developing a distributed service-oriented architecture to enable remote data processing of its huge data stores. This paper describes a NASA ACCESS project called “Deployable Suite of Data Mining Web Services for Online Science Data Repositories”, which is aimed at developing web service based technology to allow scientists to locally define analysis workflows that can directly use data residing in online repositories. Furthermore, it aims to allow scientists both within and outside of NASA to "combine persistent distributed data processing services and make them available to other users over the internet" (p. 1).

This project has chosen web service technology because it "provides a standard way to remotely interface with programmatic components and to orchestrate the chaining of services through standardized service descriptions" (p. 2). The web services solution being developed by this project uses the standard XML/SOAP/WSDL group, plus BPEL4WS for process orchestration. The project will be using an existing XML schema for data transfer (ESML, the Earth Science Markup Language) and will adapt an existing data mining toolkit (the ADaM (Algorithm Development and Mining) Toolkit) to allow existing data mining algorithms to be adapted and new ones to be developed, and will encapsulate existing data access and analysis applications (such as the Simple, Scalable Script-based Science Processor for Measurements (S4PM)).

Much of the goal of this project is to expose to scientists worldwide the both legacy data sets and databases, to future data sets and databases, and to share data mining and analysis solutions. These previously were inaccessible outside of NASA computers, and required researchers to request access, and either go to the NASA site, or to transport huge quantities of data from NASA to their own site for processing. Any analysis solution developed by that researcher would typically remain isolated at that researcher's site and would not be accessible to other scientists.

The article also describes a prototype architecture, and relates a description of its first application.

Issues

WSDL records

This paper contains the first description I've found of in-situ use of WSDL, as well as workflows:

• "A workflow is generated within the Workflow Composer [part of ActiveBPEL], based on experimentation in the Sand Box. This workflow is then deployed to a BPEL engine, which returns a URL pointing to the WSDL for that workflow. This URL is then transmitted to the GES DISC via a Web Services request along with a specification of the data to be mined, such as the dataset to be mined and temporal or spatial constraints" (p. 6).

NASA ACCESS

NASA ACCESS seems to be a NASA sponsored project to create computational infrastructure which will allow scientists to more easily share data and processing resources. Other NASA ACCESS projects I found on the Internet are: the Data and Information Application Layer (DIAL) project, A Distributed Knowledge Extraction Framework Based on SemanticWeb Services (SKIF) and Modeling and On-the-fly Solutions for Solid Earth Sciences (MOSES).

[More]

Summary of:

Harris, C; "Building self-modifying multi-agent and P2P networks using WSDL and soap", Integration of Knowledge Intensive Multi-Agent Systems, 2005. International Conference on, (2005) pp. 71-75.

Summary

The author proposes building a distributed, evolutionary multi-agent system in which the middleware is web services. He then describes a system that he built which fulfills this. Each agent acts as a SOAP node in a peer to peer network, and can dynamically export its functionality via WSDL. Other inter-agent communication protocols may be used simultaneously. Using web services as middleware grants easy interoperability for agents of heterogeneous implementations and platforms, and allows agents to be incorporated into existing web service architectures.

A web service agent differs from a web service in that it "has the ability to receive or invent new computational capacities at run-time and automatically make them available to the Internet as Web services" (p. 77). Each agent uses a code-to-WSDL translator to translate new code to WSDL descriptions, writes its WSDL descriptions as files, and exports via an internal HTTP server. Agents can thus access each other's WSDL descriptions, parse them, and exchange the appropriate SOAP messages. An agent's WSDL description can change dynamically.

The agents that the paper describe are evolutionary in that there is functionality which applies "an appropriate transformation to the instantiated code in order to make it more suitable for the given task" (p. 79). Agents are expected to be able to modify their functionality, either in response to external inputs or commands, or via self-reflective learning or training. This may be accomplished via genetic programming, "a neural network, a rational algorithm, or an expert system-guided process" (p. 79).

The evolutionary multi-agent system that the author implemented consisted of agents written in Python. Agents used either a genetic programming algorithm or neural network to evolve their algorithms. They also accessed external WSDL services to do some of their work.

Issues

Genetic programming and training neural networks can be hard problems. In order for genetic programming to work, one must be able to generate programs, compare them and rank them somehow: we need to know the appropriate fitness criteria to apply for any given problem. As the author says "knowing beforehand exactly what evolutionary criteria to apply to a genetic programming system or what expert rules need to be fed to an expert system in order to locate a satisfactory goal state can often times be more difficult than simply hand-coding the solution" (p. 79).

Critique

The paper does not discuss how agents in such a peer to peer network are notified when an agent updates, adds or removes an interface, and the corresponding WSDL description.

Harris' definition of web services as client-server differs substantially from Erl's. Harris says: "current commercial software packages for working with WSDL and SOAP are typically constructed to satisfy a strict client-server model, usually occurring in a B2B or B2C context in which a single service is written and maintained by human developers" (p. 77), while Erl says "Web services do not fit into the classic client-server model" [1, p. 50].  

References

1. Erl, Thomas; Service-Oriented Architecture : Concepts, Technology, and Design, Prentice Hall PTR: None (2005).

[More]

After reading and re-reading Erl's sections on WSDL [1, p?-?] and UDDI [1, p?-?], I find that I understand why WSDL is useful, but that I don't understand how a web services architect or developer actually uses a WSDL record in a practical sense.

So I had two questions: where does a WSDL document live, and how is it used? Is the WSDL record processed by a tool automatically, or do humans simply read the record and use it to guide code development? Do web services themselves access WSDL records -- is there machine to machine communication and automatic setup of interfaces going on with WSDL at the center?

Where the WSDL document lives

Finding the interface to a web service is part of a process called service discovery: where is the service, and how do I talk to it?

One way to publish WSDL records is using a UDDI registry: users look in the registry for the service they want, and retrieve the WSDL record from the UDDI registry [1]. The abstract portion of the WSDL service description is published in the registry as a UDDI businessService, while the concrete prortion of it is published as a UDDI tModel [2]. Both the businessService and tModel entities are configured to refer to a URL which points at the relevant part of the WSDL document.

Without a UDDI registry, a WSDL record may simply live on the Internet on a URI, or even in the local filesystem. Pilgrim says, "A WSDL file is just that: a file. More specifically, it's an XML file. It usually lives on the same server you use to access the SOAP web services it describes, although there's nothing special about it" [3]. Presumably the developer who is writing code to interact with the service described a WSDL document so located must locate the WSDL document manually.

Using the WSDL document

It seems like developers who are writing code to interface with a web service download the WSDL document and either generate code from it manually, use a tool to generate it stub code which can be later fleshed out by the developer. Examples of such code generators are WSD2Java from the AXIS, the Apache web services project, wsdl2py from the Zolera Soap Infrastructure for python, or the wsdl.exe generator for C# from microsoft.com.

References

1. "WSDL and UDDI", www.w3schools.com, Jun 19, 2007.

2. Brittenham, Peter; Cubera, Francisco; Ehnebuske, Dave; Graham, Steve, "Understanding WSDL in a UDDI registry, Part 1", www.ibm.com/developerworks, Sep 1, 2001.

3. Pligrim, Mark, "Introducing WSDL" in Dive into Python, www.diveintopython.org, 2004.

 

[More]

his article is a continuation of SOAP vs REST.

In my previous post, I reported that the REST approach to web services, while largely equivalent to the XML/SOAP/WSDL/UDDI approach, differs from it in that it trades many SOAP messages for many URIs. REST advocates claim that the advantages of this approach are: a set of REST requests can be spread across many servers, which means that REST services are scalable; that REST obviates the need for special service discovery mechanisms like UDDI; that REST supports server side caching; and that less specific client side software is needed: one browser can access any REST service.

One large part of the debate in the web services community over SOAP vs REST has to do with business process choreography [1, p. 2] (more typically called workflow): coordination of long running business processes over many services and potentially more than one organization. A standard way of conducting choreography is critical because the complexity of coordinating API and data format specifications among many entities quickly becomes unruly.

Imagine that there is a a complicated action which requires many services or service components to be invoked in a particular order, such as placing an order and receiving an order confirmation.

In the REST paradigm, the initial requestor of the service is stepped through the complicated process by the REST server, which includes the URI for the next sub-task of the action in the response from the last sub-task. The REST server publishes a well-known starting URI to which the initial requestor first goes. HTTP GETing from this URI causes the process to start on the server side -- a context is set up on the server for the initial requestor -- and the REST server responds with the URI of the first sub-task. The initial requestor then HTTP POSTs its request to that URI, and the REST server responds appropriately, and includes the URI of the next sub-task in that response, and so on until the action is complete. It looks something like this:

Coordination of the XML syntax of the messages passed between requestor and REST server must be negotiated beforehand.

In the SOAP paradigm, the SOAP server presents a single URI facade to the initial requestor, through which all communications travel (this facade is itself a web service, and Erl calls it an orchestration [2, p. 364]). Choreography is handled via message types as defined by the WSDL records for the action. The SOAP server publishes a WSDL record for a BPEL4WS process which describes the different message types it will accept, and their syntax. The initial requestor interacts with the orchestration service via SOAP messages, and what message it will send next is dictated by the orchestration service.

The BPEL4WS web services orchestration language is one of the primary choreography mechanisms for SOAP/WSDL/UDDI based web services. It depends on the WS-Coordination and WS-Transaction standards to do its work.

 

References

1. Muehlen, Michael; Nickerson, Jeffrey; Swenson, Keith; "Developing web services choreography standards: the case of REST vs. SOAP", Decis. Support Syst., Vol. 40, No. 1 (2005) pp. 9-29.

2. Erl, Thomas; Service-Oriented Architecture : Concepts, Technology, and Design, Prentice Hall, (2005).

 

[More]

Chris – thanks for the great recap. After reading the paper, it was really nice to come here and read your very nicely written recap of the salient points.  Doris

[More]

The term "Web service" generally describes an architecture of large scale networked software which wraps chunks of functionality in standardized APIs which are accessed over a network.

The web service architecture that Erl [1] describes is based on a combination of XML, SOAP, WSDL and UDDI. In this scheme, a UDDI service is an XML based registry of web services which acts as a broker of metadata on those web services. A consumer would contact a UDDI service to search for a provider of a service based on metadata filtering. Once the consumer has chosen a provider from the UDDI service, they would discover the API for the corresponding web service by asking the service for its WSDL (which is also an XML format) record. They would then utilize the service by using the WSDL record to construct and send appropriate SOAP (another XML format) messages to the service, and interpret its SOAP replies.

An alternative web service architecture called REST is currently in use by entities such as Google, Yahoo, eBay, del.icio.us, Flikr, CiteULike, Twitter. REST was first described in a doctoral dissertation in 2000 by Roy Fielding, one of the principal authors of the Hypertext Transfer Protocol (HTTP) [2] and is based on several principles:

• A service is divided into one or more resources, each of which is uniquely addressable via a URI over a network [2].

• Clients and servers communicate via HTTP and exchange representations of the resource (documents which contain the actual data). This representation may be in any format, so long as both clients and servers understand it [2], although that format is typically XML [3].

• The client should need to know only the URI of the resource and what it wants to do (typically PUT, POST or GET) -- it should not care what lies between it and the service (firewalls, gateways, caches, etc.). This is known as layering [2].

• All communications between client and server should be stateless [2].

The REST approach differs from the XML/SOAP/WSDL/UDDI approach in that it trades many SOAP messages for many URIs. A SOAP based service operating on a single URI may offer a client many different potential messages or functions. A REST based service offers the same type of functionality by providing many different URIs each of which accept a limited set of functions (typically HTTP GET and PUT).

Advocates of REST claim, among other things that:

• REST provides equivalent functionality to XML/SOAP/WSDL/UDDI.
• Since REST is stateless, it is scalable: a set of REST requests can be spread across many servers.
• Since URIs can be used in hyperlinks, and thus in hypertext documents, REST obviates the need for special service discovery mechanisms like UDDI.
• Can reduce server load by supporting server side caching.
• Less client side software is needed: one browser can access any REST service

References

1. Erl, Thomas; Service-Oriented Architecture : Concepts, Technology, and Design, Prentice Hall PTR: None (2005).

2. Representational State Transfer @ Wikipedia, June 6, 2007.

3. What is service-oriented architecture @ xml.com.

[More]

Hi, Chris – you provide a lot of fodder for discussion! I think my expectations of WEB services must be way too low. In the Sara Grant interview, McAfee comments that the WEB services hype is unrealistic and that integrating applications is never going to be easy. Though over time he suspects Web services will become more complex, so far, he laments, “Web services are being used to automate simple business processes—transmitting an order, acknowledging a shipment, describing an item for sale, etc.” To me, that doesn’t sound too bad compared to capabilities of 5 to 10 years ago. Doris

[More]

[More]

Chapters 1 and 2 of ''Service-Oriented Architecture'' by Thomas Erl [1] introduces the idea of enterprise level applications built using web services. Web services are the basis of a strategy for doing distributed computing. The commercial sector showed great interest in web services from the start. But unlike Erl, who limits his book to applying web services to integration initiatives within the enterprise (where judging from the remainder of the book, complexity can still be found in great quantities), the "craze surrounding Web services revolv[ed] around this nirvana of inter-organizational distributed computing (supply chains being integrated across enterprises and across continents with applications built out of small parts that are supplied on demand by many distinct vendors)" [3, p. 2]. By 2005, researchers were beginning to see that that world had not yet arrived, and was not likely to do so for years. "Cross-company implementations [...] are still comparatively rare." [4] What happened?

Web services are encapsulated chunks of functionality (which may be small or large) which communicate over the web (using HTTP) using a standard messaging protocol (SOAP). SOAP, in turn, is built out of XML. Organizations can create new applications by integrating SOAP enabled web services, possibly using XSLT to translate data between different web services. For legacy applications that are not SOAP enabled, organizations can write web service wrappers which translate between legacy native data formats and SOAP/XML.

A stack of languages, services and protocols -- XML, SOAP, WSDL, and UDDI -- were first associated and termed "web services" by Microsoft in 2000. SOAP itself was developed "by Dave Winer, CEO of Userland Software, Microsoft engineers Bob Atkinson and Mohsen Al-Ghosein, and Don Box, co-founder of DevelopMentor Incorporated" in 1999 [2].

Web services make integrating applications easier, but even with web services, integration is still far from simple. Both technical and organizational issues slowed the advance of web services within the enterprise, and all but stopped them from leaving it. Technically, there are very real issues of "security, intellectual property exposure, and performance that follow from exposing web services outside the enterprise" [3, p. 8]. There is also fundamentally that "any two applications are virtually guaranteed to contain dissimilar data and execute dissimilar business processes. [...] Before any systems integration can take place, these dissimilarities need to be resolved. There is no magic bullet in the Web services toolkit that does this automatically or quickly." [4] Organizationally, service and data ownership, data definitions, access, acceptable use, business process definitions and more must be agreed upon among stakeholders within an enterprise -- this itself is difficult: "This might not seem like the kind of work that leads to disputes, but it is." [4] For two enterprises to agree upon these things is even more difficult.

As the difficulties of integrating applications once again revealed themselves, firms looked inward, focusing on wrapping legacy applications to free its data, and enabling integration across functional silos, and generally realigning business processes to enable web services internally. We are beginning to see the first steps towards true realization of cross-organizational computing as companies like Google, Yahoo, eBay and services like del.icio.us, Flikr, CiteULike, Twitter, etc. publish APIs (albeit based on REST instead of SOAP) for their services that are being used in mashups (combining data from several web services to produce something more than the sum of its parts) across the Internet. Will firms follow suit with their own private data?

[More]