A workflow is a work procedure where workflow steps represent the activities, users must follow in order to achieve the desired outcome. A workflow consists of a sequence of concatenated (connected) steps, where each step follows the precedent without delay or gap and ends just before the subsequent step may begin. This concept is related to non overlapping tasks of single resources. The Simple Workflow Service of Amazon (SWF) is a service that deals with the flow of work that builds climbing and flexible applications. It makes it easy to build applications that coordinate work across distributed components. It is one of the Application services of Amazon Web Services (AWS).
Now to design an application using AWS’s SWF we need to understand the building blocks of the SWF. The Workflows and Tasks are the concepts in SWF whereas Workflow Starter, Activity Workers, Deciders are the actors in a workflow. Activity tasks, Decision Tasks are the types of tasks. Domain provides a way of scoping SWF resources. If multiple workflows need same resources they can be declared under one domain, they can share the resources among themselves. Workflow in different domain cannot communicate with each other.
Workflow in a SWF is the application we have to develop. Tasks are represented by activities. The Activities are the part of workflow in other words workflow is a set of activities. To develop application we need to form different activities. In a workflow we may need to perform an activity number of time. The single activity processing is an activity task. A workflow is a distributed application composed of coordination logic and tasks that run asynchronously on multiple computing devices. The order in which the activities should perform is decided in coordination logic. Now as we know are finished with paper work, we go to console of Amazon and register the workflow type and the activity type. During registration you specify name for activity, version and some default configuration values. Now as it is ready, execution can be done any time as many times as you want. For the purpose when the workflow already present and current workflow is to connect to existing it is to be done through workflow id. Workflow ID should be provided by us. In addition to this SWF attaches its own unique identifier known as run ID. So if we give same identifier in case the execution carries on by run id associated by SWF.
Workflow history is maintained by SWF. The history consists of complete and consistent record of every event that occurred since the workflow execution started. An event represents a discrete change in your workflow execution's state, such as a new activity being scheduled or a running activity being completed. Programmatically, the events in the workflow execution history are represented as JavaScript Object Notation (JSON) objects. The history itself is a JSON array of these objects. Each event has a type such as WorkflowExecutionStarted or ActivityTaskCompleted, a timestamp UNIX time format, and an ID which uniquely identifies event. Each type of event has a distinct set of descriptive attributes that are appropriate to that type. The workflow histories contains every event that causes the execution state of the workflow execution to change, such as scheduled and completed activities, task timeouts, and signals. The workflow execution uses this information to determine next steps. It provides the evidence for the process that work flow is going through. If any fault due to any activity it can be determine. Operations that do not change the state of the workflow execution do not typically appear in the workflow history. For example, the workflow history does not show poll attempts or the use of visibility operations. The workflow history has several key benefits:
- It enables applications to be stateless, because all information about a workflow execution is stored in its workflow history.
- For each workflow execution, the history provides a record of which activities were scheduled, their current status, and their results. The workflow execution uses this information to determine next steps.
- The history provides a detailed audit trail that you can use to monitor running workflow executions and verify completed workflow executions.
Amazon SWF stores the complete history of all workflow executions for a configurable number of days after the execution closes. This period, which is known as the workflow history retention period, is specified when you register a Domain for your workflow.
Actors in Workflow are the main builders of workflow. To perform operation we need actors. They can be developed in any language. These actors communicate with Amazon SWF through its web service API. We can develop these actors in programming language. Workflow Starter it can be anything that initiate workflow execution. E .g. website, mobile application, system. Deciders it is implementation of workflow logic. It controls the flow of activity tasks. Whenever there is a change in work flow it is stored in history now decision task arrives with that history .Amazon SWF gives the task to decider. It takes the necessary steps and gives the decision to the SWF. A decision is an Amazon SWF data type that can represent various next actions. Decisions are in JSON format. This continues until the decider finds that the work flow is completed. Activity Worker it is a process or a thread. Activity worker polls SWF for new tasks that are appropriate. It finishes with the given work and polls for next it does it until completion of work.
Data Exchange between Actors is part in which the actors communicate with each other to carry the workflow smoothly. Input data can be provided to a workflow execution when it is started. Similarly, input data can be provided to activity workers when they schedule activity tasks. Depending on the size and sensitivity of the data, you can pass data directly or pass a pointer to data stored on another system or service (e.g., Amazon Simple Storage Service or Amazon SimpleDB). The data passed directly and the pointers to other data stores are recorded in the workflow execution history; however, Amazon SWF does not copy or cache any of the data from external stores as part of the history. Because Amazon SWF maintains the complete execution state of each workflow execution, including the inputs and the results of tasks, all actors can be stateless.
Amazon SWF interacts with activity workers and deciders by providing them with known as tasks. There are two types of tasks in Amazon SWF: Activity tasks are the tasks associated with activity worker. To perform task activity worker needs some information that is there in activity tasks. Decision tasks it task tells a decider that the state of the workflow execution has changed so that the decider can determine the next activity that needs to be performed. The decision task contains the current workflow history.
Domain provides a way of scoping SWF resources. If multiple workflows need same resources they can be declared under one domain, they can share the resources among themselves. Workflow in different domain cannot communicate with each other.
Object identifiers are the objects of Amazon SWF. Following are the objects of Amazon SWF. Workflow Type: A registered workflow type is identified by its domain, name, and version. Activity Type: A registered activity type is identified by its domain, name, and version. Decision tasks and Activity tasks: Each decision task and activity task is identified by a unique task token. The task token is generated by Amazon SWF and is returned with other information about the task in the response from PollForDecisionTask or PollForActivityTask. Workflow execution: A single execution of a workflow is identified by the domain, workflow ID, and run ID. The first two are parameters that are passed to StartWorkflowExecution. The run ID is returned by StartWorkflowExecution.
Tasks Lists, is a way to provide all tasks associated with the workflow. Just like queues. When the task is scheduled it is to be added to queue (task list). There are two different list Activity Tasks and Decision tasks. Activity Tasks list, When an activity type is register default task list is created however, when the decider schedules an activity task the decider can optionally specify a different task list on which to schedule the task. Decision Task lists, Each workflow execution is associated with a specific decision task list. When a decider polls for a new decision task, the decider specifies a decision task list to draw from. A single decider could serve multiple workflow executions by calling PollForDecisionTask multiple times, using a different task list in each call, where each task list is specific to a particular workflow execution. The decider could poll a single decision task list that provides decision tasks for multiple workflow executions. Multiple deciders can serve a single workflow execution by having all of them poll the task list for that workflow execution.
Tasks routing, When an activity worker polls for a new task, it can specify an activity task list to draw from. If it does, the activity worker will accept tasks only from that list. You can ensure that certain tasks get assigned only to particular activity workers by this. Assigning particular tasks to particular activity workers is called task routing. Workflow execution closure, Once the workflow start it can be closed, as completed, canceled, fail – decider takes the decision accordingly. Polling for Tasks, decider and worker communicate with the SWF. If there is availability of tasks then response is given immediately however if there is no task then related to the worker then TCP connection is hold for 60 secs even if there is no response then empty response is send.
Timeouts are set to run workflow executions correctly. There are different types of timeouts. Workflow Start to Close (timeoutType: START_TO_CLOSE): This timeout specifies the maximum time that a workflow execution can take to complete. It is set as a default during workflow registration but it can be overridden with a different value when the workflow is started. If this timeout is exceeded, Amazon SWF closes the workflow execution and adds an event of type WorkflowExecutionTimedOut to the workflow execution history. Decision Task Start to Close (timeoutType: START_TO_CLOSE): This timeout specifies the maximum time that the corresponding decider can take to complete a decision task. It is set during workflow type registration. If this timeout is exceeded, the task is marked as timed out in the workflow execution history, and Amazon SWF adds an event of type DecisionTaskTimedOut to the workflow history. Activity Task Start to Close (timeoutType: START_TO_CLOSE): This timeout specifies the maximum time that an activity worker can take to process a task after the worker has received the task. Activity Task Heartbeat (timeoutType: HEARTBEAT): This timeout specifies the maximum time that a task can run before providing its progress. Activity Task Schedule to Start (timeoutType: SCHEDULE_TO_START): This timeout specifies how long Amazon SWF waits before timing out the activity task if no workers are available to perform the task. Once timed out, the expired task will not be assigned to another worker. Activity Task Schedule to Close (timeoutType: SCHEDULE_TO_CLOSE): This timeout specifies how long the task can take from the time it is scheduled to the time it is complete. As a best practice, this value should not be greater than the sum of the task schedule-to-start timeout and the task start-to-close timeout. These were some basics of workflow.
Now heading towards the advanced concepts, the concepts such as Versioning, Signals, Child workflow, Markers, and Tags are included in it. In practical we implement our project and application many times adding extra functionality, so versioning the same project plays a important role. In workflows and activity workers version is done so that if the task given to them is having update then versioning is used. Signals enable you to inject information into a running workflow execution. If we want to use the signals you need to program the decider to recognize the signal event. Now decider can take actions according to signals and data. Complicated workflows can be broken into small parts. These small parts are Child workflows. It increases the modularity of the program. To initiate a child workflow, the decider for the parent workflow uses the StartChildWorkflowExecution decision. Input data specified with this decision is made available to the child workflow through its history. The attributes for the StartChildWorkflowExecution decision also specify the child policy, that is, how Amazon SWF should handle the situation in which the parent workflow execution terminates before the child workflow execution, TERMINATE, REQUEST_CANCEL, ABANDON are the values. The child workflow are initiated by the decider, also alone can be started. Markers are used when we want to use the record information purposely for our convenience. When you want to differentiate the workflow execution u can tag them. As we have name plates in our houses is the same to be explained here.
We now start with using SWF. We have to first register Domain, then workflow, then activity. After registering we go for developing Activity Worker. Poll Amazon SWF for an activity task, PollForActivityTask is also a Action. The activity worker polls for a task .if it is not present then sends empty response in which the task Token is empty string and if available then it sends data specified by decider when it schedule the activity. Begin performing the task, In the ActivityTask’s input parameter content are present which are to be interpreted by the activity worker ,we have to program activity worker such that it interprets the content in the input parameter of ActivityTask. The content is data specified by the decider. Now program Activity Worker to process data and execute the logic. Periodically report heartbeat to Amazon SWF if the task is long-lived, it is the time out period for communication purpose. If the heartbeat timeout exceed the specified then it report error and new decision task is scheduled. If even after time out Activity worker attempts to connect Amazon SWF then error exception is seen. Report that the task completed or failed and return the results to Amazon SWF, After executing a task, the activity worker should report whether the activity task completed or failed. To complete an activity task, program the activity worker to call the RespondActivityTaskCompleted action after it successfully completes an activity task, specifying the task token. Launching Activity, to launch an activity we have to package the logic into executable E g java executable. We can create as many Activity Workers as u can. The decider schedules activity task to polling activity worker. Developing Deciders, the workflow history is stored in SWF. This enables you to dynamically add and remove deciders as necessary, which makes the processing of your workflows highly scalable. At a time only one decision task open in a workflow execution. Interprets the workflow, Apply the coordination logic as per history of workflow and makes decision what to do next, completes the decision task and provides a list of decision. Defining Coordination Logic; Polling for Decision Tasks; Applying the Coordination Logic; Responding with Decisions; Closing a Workflow Execution. All is the programming part we need to write logic to define logic, poll decision task, applying logic, responding with decision, finally closing execution
There are a number of different types of errors that can occur during the course of a workflow execution. Validation Errors, the request is done to register a domain for example and if the Amazon SWF does not respond then validation error occurs. If the request that failed is a decision, an appropriate event will be listed in the workflow execution history. For example, if the StartTimer decision failed, you would see a StartTimerFailed event in the history. Errors in Enacting Action or Decision, even if the request is properly formed, errors may occur when Amazon SWF attempts to carry out the request.
Deciders, activity workers, and workflow executions all operate within the constraints of time out periods in this type of error, a task or a child workflow timeout. An event will appear in the history that describes timeout. Errors Raised by user code, it is our errors manual error in code. Errors related to closing workflows. Events if attempt to close a workflow that has a pending decision task. Launching Deciders is similar to launching Activity making Executable and running it.
We have to start a workflow execution of a registered workflow type from any application using the StartWorkflowExecution action. When you start the execution you associate an identifier, called the workflow Id, with it. The workflow Id can be any string that is appropriate for your application, such as the order number in an order processing application. You cannot use the same workflow Id for multiple open workflow executions within the same domain however; reuse the workflow Id of a closed execution. If the StartWorkflowExecution action is successful, the service returns the runId for the workflow execution. The runId uniquely identifies this workflow execution from any other workflow executions currently running under Amazon SWF. Save the runId in case you later need to specify this workflow execution in a call to Amazon SWF.
Hence we can start using SWF by learning these concepts. AWS has provided API to run workflow. Using those program the Workflow according to your application.