The environment is the following:
-
Demo apps:
-
OpenTelemetry Collector
-
Jaeger
-
Prometheus
-
Apache Kafka (for async communication between apps cases)
You will need to compile this example first:
mvn compileAll the following docker-compose commands should be run from this directory.
You may want to remove any old containers to start cleaning:
docker rm -f kafka zookeeper prometheus jaeger otel-collectorWe suggest using multiple terminal windows to start the following components:
-
Infrastructure components
-
Demo apps
-
TripBooking
-
FlightBooking
-
HotelBooking
-
CarBooking
-
-
Testing for sending HTTP requests
Please use four different shells for each application:
You can use any HTTP client (web browser, curl, httpie, postman etc.) for testing purposes.
-
Prometheus UI: http://localhost:9090/graph
-
Jaeger UI: http://localhost:16686
MDC Logging is enabled, and tracing information printing into the logs to be able to find corresponding trace logs entries. For example:
...
11:52:18.923 INFO [d02a363f16e88d9f012a36563b5464f5,9a328d33319645ab] bookTrip-http - New book trip request with trace=00-d02a363f16e88d9f012a36563b5464f5-9570717e10d38afa-01
11:52:18.931 INFO [d02a363f16e88d9f012a36563b5464f5,9a328d33319645ab] bookTrip-http - Response: [{"bookingId":82,"car":"Volkswagen Jetta","startDate":"12-11-2018","endDate":"15-11-2018","price":152}, {"bookingId":907,"flight":"China Eastern Airlines 2782","startDate":"12-11-2018","endDate":"15-11-2018","price":133}, {"bookingId":926,"hotel":"Sheraton","startDate":"12-11-2018","endDate":"15-11-2018","price":200}]
...-
occlient installed (guide) -
already logged in into cluster (running
oc login) -
destination project already created (running
oc new-project otel-example)
Create kafka cluster named otel-cluster:
cat << EOF | oc apply -f -
kind: Kafka
apiVersion: kafka.strimzi.io/v1beta2
metadata:
name: otel-cluster
spec:
kafka:
version: 3.7.0
replicas: 3
listeners:
- name: plain
port: 9092
type: internal
tls: false
- name: tls
port: 9093
type: internal
tls: true
config:
offsets.topic.replication.factor: 3
transaction.state.log.replication.factor: 3
transaction.state.log.min.isr: 2
default.replication.factor: 3
min.insync.replicas: 2
inter.broker.protocol.version: '3.7'
storage:
type: ephemeral
zookeeper:
replicas: 3
storage:
type: ephemeral
entityOperator:
topicOperator: {}
userOperator: {}
EOFCreate tempo monolithic named monolitic-example as distributed tracing storage:
cat << EOF | oc apply -f -
apiVersion: tempo.grafana.com/v1alpha1
kind: TempoMonolithic
metadata:
name: monolitic-example
spec:
jaegerui:
enabled: true
resources:
limits:
cpu: '2'
memory: 2Gi
route:
enabled: true
resources:
limits:
cpu: '2'
memory: 2Gi
storage:
traces:
backend: memory
EOFCreate opentelemetry collector named otel-example used to collect everything coming from the agent on the applications and exporting the traces into the tempo storage and the metrics in a prometheus format:
cat << EOF | oc apply -f -
kind: OpenTelemetryCollector
apiVersion: opentelemetry.io/v1beta1
metadata:
name: otel-example
spec:
config:
exporters:
debug: {}
otlp/tempo:
endpoint: 'http://tempo-monolitic-example:4317'
tls:
insecure: true
prometheus:
endpoint: '0.0.0.0:8889'
metric_expiration: 180m
processors:
batch:
send_batch_size: 1000
timeout: 10s
receivers:
otlp:
protocols:
grpc: {}
http: {}
service:
pipelines:
traces:
exporters:
- debug
- otlp/tempo
processors:
- batch
receivers:
- otlp
metrics:
exporters:
- debug
- prometheus
processors:
- batch
receivers:
- otlp
replicas: 1
ports:
- name: promexporter
port: 8889
protocol: TCP
targetPort: 8889
EOFCreate the opentelementry instrumentation to allow to inject the agent parameter into the startup command in the application
cat << EOF | oc apply -f -
apiVersion: opentelemetry.io/v1alpha1
kind: Instrumentation
metadata:
name: camel-instrumentation
spec:
exporter:
endpoint: 'http://otel-example-collector-headless:4317'
java:
env:
- name: OTEL_SERVICE_NAME
valueFrom:
fieldRef:
fieldPath: 'metadata.labels[''app'']'
EOFCreate the service monitor to allow the prometheus metrics exposed by the opentelemetry collector be scraped by the OpenShift metrics storage
cat << EOF | oc apply -f -
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: otel-example-collector
spec:
endpoints:
- interval: 30s
port: promexporter
scheme: http
path: /metrics
selector:
matchLabels:
app.kubernetes.io/name: otel-example-collector
EOFmvn clean install -Popenshiftnow once the pods are ready it is possible to call the Trip Booking entry point
Sync communication (over HTTP):
curl http://$(oc get route trip-booking -o go-template --template='{{.spec.host}}')/camel/bookTripAsync communication (over Kafka):
curl http://$(oc get route trip-booking -o go-template --template='{{.spec.host}}')/camel/asyncBookTripThe Jaeger console is available at
echo https://$(oc get route tempo-monolitic-example-jaegerui -o go-template --template='{{.spec.host}}')To query the metrics it is possible to use the integrated OpenShift monitoring console at
echo $(oc whoami --show-console)/monitoring/query-browsera query like
sum(camel_exchanges_total{service="otel-example-collector-headless"}) by(exported_job, routeId)will show you the exchanges for each route on each application
If you hit any problem using Camel or have some feedback, then please let us know.
We also love contributors, so get involved :-)
The Camel riders!