Application tracing allows you to log information when a program is executing. The trace log can then be used for diagnostic purposes such as debugging failures in the program execution.
Tracing is used to help you identify issues with your application (performance of the app or simply debugging an issue) instead of going through stack traces. The use of tracing is particularly useful in a distributed microservice architecture environment where all the apps are instrumented by some kind of tracing framework and while the tracing framework runs in the background, you can monitor each trace in the UI. You can use that to track any abnormalities or issues to identify the location of the problem.
Some Considerations
Keep the following in mind when using the tracing capability in Flogo® Enterprise:
● At any given point of time only one tracer can be registered - if you try to register multiple tracers, then only the first one that you register is accepted and used at runtime to trace all the flow's activities.
● All the traces starts at the flow level. There are two relations between spans - a span is either the child of a parent span or the span is a span that follows (comes after) another span. You should be able to see all the operations and the traces for the flows and activities that are part of an
application.
● Tracing can be done across apps by passing the tracing context from one app to another. To trace across multiple apps, you must make sure that all apps are instrumented with similar sort of tracing frameworks, for example, OpenTracing semantics so that they understand the framework language, otherwise it will not be possible for you to get a holistic following of the entire trace through multiple services.
● When looping is enabled for an activity, each loop is considered one span since each loop calls the server which triggers a server flow.
● If a span is passed on to the trigger, then that span becomes the parent span. You should be able to see how much time is taken between the time the event is received by the trigger and the time the trigger replies back. This only works for triggers that support the extraction of the context from the underlying technology, for instance triggers that support HTTP headers.
The ReceiveHTTPMessage REST trigger and InvokeRESTService activity are supported for this release where the REST trigger is able to extract the context from the request and
InvokeRESTService activity is able to inject the context into the request. If two Flogo apps are both Opentracing-enabled, when one app calls the other, you can see the chain of events (invocation and how much time is taken by each invocation) in the Jaeger UI. If app A is calling app B, the total request time taken by app A is the cumulative of the time taken by all activities in app A plus the time taken by the service that it calls. If you open up each invocation separately, you can see the details of how much time was taken by each activity in that invocation.
● Triggers that support span (for instance the REST trigger) are always the parent, so any flows that are attached to that trigger are always the child of the trigger span. Trigger span is completed only after the request goes to the flow and the flow returns.
● A subflow becomes a child of the activity from which it is called.
OpenTracing
Flogo® Enterprise supports OpenTracing for application tracing.
Jaeger
Flogo® Enterprise provides an implementation of the OpenTracing framework using the Jaeger backend. In Flogo Enterprise, the Flogo® app binary is built with Jaeger implementation and can be
enabled by setting the FLOGO_APP_MONITORING_OT_JAEGER environment variable to true. You can track how the flow went through, execution time for each activity, or in case of failure, the cause of the failure.
Each app is displayed as a service in the Jaeger UI. In Flogo Enterprise, each flow is one operation (trace) and each activity in the flow is a span of the trace. A trace is the complete lifecycle of a group of spans. The flow is the root span and its activities are its child spans.
Pre-requisites The following pre-requisites must be met before using the tracing capability in Flogo® Enterprise:
● By default, Jaeger is not enabled in Flogo® Enterprise, hence tracing is not enabled. To enable Jaeger, set the FLOGO_APP_MONITORING_OT_JAEGER environment variable to true.
● Ensure that the Jaeger server is installed, running, and accessible to Flogo® Enterprise.
● If your Jaegar server is running on a machine other than the machine on which your app resides, be sure to set the JAEGER_AGENT_HOST and the JAEGER_AGENT_PORT environment variables. Refer to the https://github.com/jaegertracing/jaeger-client-go#environment-variables page for the
environment variables that you can set.
Flogo® Enterprise-Related Tags in Jaegar
In OpenTracing, each trace and span have their own tags. Tags are useful for filtering traces, for example if you want to search for a specific trace or time interval.
Adding your own custom tags for any one span (activity) only is currently not supported. Any custom tags that you create will be added to all spans and traces.
Flogo® Enterprise introduces the following Flogo-specific tags:
For flows
flow_name Name of the flow
flow_id Unique instance IDs that are generated by the
Flogo engine. They are used to identify specific instances of a flow (such as when the same flow is triggered multiple times)
For activities
flow_name Name of the flow
flow_id Unique instance IDs that are generated by the
Flogo engine. They are used to identify specific instances of a flow (such as when the same flow is triggered multiple times)
task_name Name of activity
taskInstance_id Unique instance ID that are generated by the Flogo engine. This identity is used to identify the specific instance of an activity when an activity is iterated multiple times. This ID is used in looping contructs such as iterator or Repeat while true.
For subflows
parent_flow Name of the parent flow
parent_flow_id Unique ID of the parent flow
flow_name Name of the subflow
flow_id Unique instance IDs that are generated by the
Flogo engine. They are used to identify specific instances of a flow (such as when the same flow is triggered multiple times)
The tag values are automatically generated by the Flogo® Enterprise runtime. You cannot override the default values. You have the option to set custom tags by setting them in the environment variable
JAEGER_TAGS as key/value pair. Keep in mind that these tags will be added to all spans and traces.
Refer to the https://github.com/jaegertracing/jaeger-client-go#environment-variables page for the environment variables that you can set.