Geeks With Blogs

News This is the *old* blog. The new one is at blog.sixeyed.com
Elton Stoneman
This is the *old* blog. The new one is at blog.sixeyed.com

[Source: http://geekswithblogs.net/EltonStoneman]

The combination of .NET Services and SQL Services in the cloud provide a very simple model for implementing massively distributed processing. It's an appealing idea – enterprises of any size have hundreds or thousands of workstations which can be idle for the majority of their time. Assuming a generous average 10% resource utilisation during working hours, workstations which are not powered down are averaging 3.3% utilisation over the working week.

Co-ordinating the available processing power of the workstation landscape becomes a lot simpler in the cloud. Work packages are published to the cloud, and workstations retrieve and process them. All that's required is a connected workstation capable of running .NET, a simple Windows service to fetch and execute work packages – and a business process that lends itself to being split into discrete parts, with minimal constraints on processing time.

Case Study: Document Generation

An obvious example is a document generation system. Typically this would be an event-driven front-end, publishing document requests which are retrieved and processed by a batch system. Simplified, the document generation workflow is: retrieve document template; retrieve business entity; mash entity data with template; publish result. In this architecture the hardware of the batch system is the definite limit to how many documents can be published in a set timeframe. The time taken to process each document is critical, so performance is a key consideration through design and implementation.

Move this to the cloud, and the limit to how many documents that can be processed is the number of workstations online processing work packages. With hundreds of boxes picking up requests simultaneously, the time taken for each process is far less significant. Assuming a complex document takes 15 seconds to render in the batch architecture, to cover 5,000 documents a day requires a server farm of 8 boxes, each with 2 quad-core processors. Even if the cloud model takes three times as long to render the same document, 160 workstations can process the same load in the same time. A lot more boxes, but they can be of a much lower spec and might be provided from the PCs already used in a single department.

Proof of Concept

I'm working on a POC which implements a simple distributed document generation system using Microsoft's currently-available CTP cloud services. Taking into account the considerations below, the POC works through a SQL Service used to store work package and document template data, and three workflow services for publishing and retrieving work packages, and flagging a package as complete. I'll publish a more detailed technical post when the POC is available, but at a high level the design looks like this:


Considerations

A distributed architecture has some of the same considerations you find in processing across a server farm, and some additional concerns:

  • Distributed intelligence – how much complex logic do we want on the workstations? Auto-update functionality can be used to ensure the workstation always has the latest code before processing a work package, but the more dependencies the logic has, the more involved this becomes;
  • Data access – workstations processing packages may be outside the enterprise network, and directly accessing data sources may not be possible. The work package may be constructed to contain all necessary data, but this increases the processing load for the publishing system, and raises issues on data duplication and currency;
  • Security – depending on the data contained in the work packages and the processing work needed, anonymous access may not be feasible; a Windows authentication model with role- or claim-based authorisation will enable fine-grained security, but this will need more complex setup of the processing service on the workstations;
  • ACIDity – workstations may go offline at any time, and there's no guarantee that a work package will be completed once it's picked up. Transaction control is likely to need a custom solution depending on the requirements of the work package – the publisher could define a time limit, after which the package is published again if the first consumer has not responded, provided there are no affects from processing the package twice.

Potential Applications

This architecture has a number of business attractions. Large enterprises could utilize their workstation landscape and ensure security by publishing work packages which are processed by their own domain machines. Smaller organizations could utilize workstation resources from a virtual processing centre in the cloud. Start-ups wouldn't need an up-front investment in their own servers . Service providers could provision processing power by recruiting home PCs on a pay-per-use basis. Operating system manufacturers could offer a free basic version in return for a commitment to providing processing power…

From a technical view, this also opens interesting possibilities. Distribute all services to internal or external processing centres via the cloud and separate application servers are no longer needed. Provide a virtual master database with SQL Services, with out-of-band integration to any LOB databases and separate database servers are no longer needed. You move to a 1+1-tier architecture of distributed smart clients and the cloud, with no server estate.

Posted on Tuesday, December 2, 2008 10:59 PM The Cloud | Back to top


Comments on this post: Cloud-Distributed Computing

# re: Cloud-Distributed Computing
Requesting Gravatar...

hi,The Motif of your blog is very good to me, I hope more exchanges with you this Motif.
Left by bridesmaid dresses on Jun 11, 2010 4:11 AM

# re: Cloud-Distributed Computing
Requesting Gravatar...
It is very interesting article and quite impressive and more informative and looking forward to read such article.Keep it ip.
Left by platinum protection123 on Jun 12, 2010 3:23 AM

# re: Cloud-Distributed Computing
Requesting Gravatar...
That’s pretty good article, I really like the tips you have given..Thanks a lot.Keep blogging.
Left by utah uterine fibroids treatment1 on Jun 24, 2010 8:35 AM

# re: Cloud-Distributed Computing
Requesting Gravatar...
Hi,this is one of the finest site and I really appreciate for this.Thanks for the blog.Will be referring a lot of friends about this.Looking forward to reading more from you.
Left by pain doctors59 on Jul 26, 2010 6:19 AM

# re: Cloud-Distributed Computing
Requesting Gravatar...
I am following your blog regularly and got great information.Having written articles that require this much work, I
Left by team national467 on Jul 27, 2010 8:46 AM

# re: Cloud-Distributed Computing
Requesting Gravatar...
The blog which you write is good.In future your blog provided us valuable information to work on.thanks for writing such a nice post.
Left by Agritourism Tuscany on Oct 02, 2010 10:20 AM

# re: Cloud-Distributed Computing
Requesting Gravatar...
Will follow this with close view..
Left by daily forex trading news on Dec 14, 2010 12:51 AM

# re: Cloud-Distributed Computing
Requesting Gravatar...
As always a great read, and food for thought
Left by compare exchange rates on Dec 14, 2010 12:52 AM

# re: Cloud-Distributed Computing
Requesting Gravatar...
This article concerns about a cloud provider shutting down for financial or legal reasons, which has happened in a number of cases. You present this topic in such a brilliant way that is easy to understand.
Left by platinum protection on Jan 29, 2011 6:07 AM

# re: Cloud-Distributed Computing
Requesting Gravatar...
Distributed computing is a field of computer science that studies distributed systems. A computer program that runs in a distributed system is called a distributed program. It also refers to the use of distributed systems to solve computational problems.
Left by submit your article on Mar 25, 2011 1:22 AM

# re: Cloud-Distributed Computing
Requesting Gravatar...
I'm working on POC with previous version of MS CTP. I feel quite difficult to work on it.
Left by ged schools on May 13, 2011 7:48 AM

# re: Cloud-Distributed Computing
Requesting Gravatar...
Great read and good luck, we have added it to our bookmarks and will wait for a follow-up to this topic hopefully more positive ;-)
Indian escorts
Left by London Escorts on Jun 05, 2011 12:02 AM

# re: Cloud-Distributed Computing
Requesting Gravatar...
Amazing write up! thank you for this information!
Left by kineski horoskop on Mar 20, 2012 12:41 PM

# re: Cloud-Distributed Computing
Requesting Gravatar...
Learning how to get out of credit card debt guides on the web is really tiring. I was simply delighted to feel replenished in reading through this blog.
Left by Craig Higginbotham on May 07, 2014 5:49 AM

# re: Cloud-Distributed Computing
Requesting Gravatar...
The psychology dissertation writing service is best to understand about individual sections or your full dissertation.
Left by norim on Dec 03, 2015 1:23 PM

Your comment:
 (will show your gravatar)


Copyright © Elton Stoneman | Powered by: GeeksWithBlogs.net